Obstacle perception method for industrial vehicle ADAS based on large model

Abstract

The present invention relates to the field of industrial vehicle control technology, and specifically to an obstacle perception method of an industrial vehicle ADAS fused with a large model, comprising: when an obstacle is detected within the radar range of the industrial vehicle, obtaining the grayscale image of the obstacle and the height data and position data of the obstacle at each moment within a preset time period; obtaining a collision risk index based on the object contour distribution in the obstacle grayscale image at the initial moment within the preset time period, combined with the object attributes and height data of the obstacle at the initial moment; obtaining the degree of perceived influence of the obstacle on the industrial vehicle based on the difference between the position data of the obstacle at every two moments and the change between the height data at adjacent moments; dividing the obstacle warning level of the industrial vehicle according to the collision risk index and the degree of perceived influence, and obtaining the level perception result of the obstacle. The present invention effectively improves the reliability of the industrial vehicle ADAS system fused with a large model and the perception accuracy of the obstacle.

Description

Technical Field

[0001] The present invention relates to the technical field of industrial vehicle control, and in particular to an obstacle perception method of an industrial vehicle ADAS integrating a large model. Background Art

[0002] In modern industrial transportation and logistics operations, industrial vehicles (such as forklifts, stackers, mining forklifts, etc.) play a vital role in environments such as warehousing, production lines, and construction sites. With the continuous development of automation and intelligent technology, industrial vehicles are gradually moving towards autonomous driving to improve operational efficiency and safety. Advanced Driving Assistance System (ADAS) is a system that integrates multiple technologies such as sensors, computing platforms, and decision-making control algorithms. It mainly assists drivers or realizes the vehicle's autonomous driving function through perception, prediction, planning, and control. Among them, obstacle perception is one of the core technologies of the ADAS system. Especially in dynamic and complex industrial environments, obstacle perception plays an important role in avoiding collisions, ensuring safety, and improving work efficiency.

[0003] The current industrial vehicle ADAS system that integrates large models mainly relies on the comprehensive analysis and fusion of multiple sensor data. The large model can learn more accurate feature representation and perform multi-dimensional information fusion through training on a large amount of data. The current method of obstacle perception in the direction of industrial vehicles is to conduct risk analysis based on a single factor of obstacle and industrial vehicle positioning data, which makes the obstacle perception accuracy low. Summary of the invention

[0004] In order to solve the technical problem that the existing collision analysis is performed by a single factor of the obstacle and the positioning data of the industrial vehicle, resulting in low obstacle perception accuracy, the purpose of the present invention is to provide an obstacle perception method for industrial vehicle ADAS integrating a large model, and the technical solution adopted is as follows:

[0005] During the operation of industrial vehicles, when an obstacle is detected within the radar range of the industrial vehicle, the industrial vehicle ADAS system that integrates the large model obtains the grayscale image of the obstacle and the height data and position data of the obstacle at each moment within the preset time period;

[0006] According to the distribution regularity of the object contour information in the obstacle grayscale image at the initial moment within the preset time period, combined with the object attribute information corresponding to the obstacle at the initial moment and the obstacle height data at each moment, the obstacle collision risk index is obtained;

[0007] According to the difference between the position data of the obstacles corresponding to each two moments and the change distribution between the height data of the obstacles at adjacent moments, the degree of influence of the obstacles on the perception of the industrial vehicle is obtained;

[0008] According to the collision risk index and the perception impact degree, the obstacle warning level of the industrial vehicle is divided to obtain the level perception result of the obstacle.

[0009] Preferably, the obstacle collision risk index is obtained based on the regularity of the distribution of the object contour information in the obstacle grayscale image at the initial moment within the preset time period, combined with the object attribute information corresponding to the obstacle at the initial moment and the obstacle height data at each moment, specifically including:

[0010] Obtaining the obstacle contour complexity coefficient according to the edge distribution of adjacent pixel points on the edge of the obstacle grayscale image at the initial moment within a preset time period;

[0011] Obtain the object obstruction coefficient of the obstacle according to the object attribute information corresponding to the obstacle at the initial moment within the preset time period combined with the neural network in the fusion large model;

[0012] The maximum value of the obstacle height data at all times within the preset time period is taken as the obstacle height characteristic coefficient;

[0013] According to the contour complexity coefficient, the object obstruction coefficient and the height characteristic coefficient, a collision risk index of the obstacle is obtained in combination with the chassis height of the industrial vehicle.

[0014] Preferably, obtaining the obstacle contour complexity coefficient according to the edge distribution of adjacent pixel points on the edge of the obstacle grayscale image at the initial moment within the preset time period specifically includes:

[0015] Recording the obstacle grayscale image at the initial moment within the preset time period as the target obstacle image; obtaining edge pixel points of each edge line in the target obstacle image;

[0016] Calculate the slope value of each edge pixel point on the corresponding edge line in the target obstacle image;

[0017] Based on the average of the difference between the slope values ​​of every two adjacent edge pixel points on each edge line, the pixel complexity parameter of each edge line is determined; the cumulative sum of the pixel complexity parameters of all edge lines in the target obstacle image is used as the contour complexity coefficient of the obstacle.

[0018] Preferably, the object obstruction coefficient of the obstacle is obtained by combining the object attribute information corresponding to the obstacle at the initial moment within the preset time period with the neural network in the fusion large model, specifically including:

[0019] Obstacles with different object attributes are labeled in the form of manual labeling to obtain labeled data, wherein the labeled data includes object obstruction coefficients of obstacles with different object attributes; a neural network is trained using the labeled data; and object attribute information corresponding to the obstacle at an initial moment within a preset time period is input into the trained neural network to obtain the object obstruction coefficient of the obstacle.

[0020] Preferably, the obstacle collision risk index is obtained according to the contour complexity coefficient, the object obstruction coefficient and the height characteristic coefficient in combination with the chassis height of the industrial vehicle, specifically including:

[0021] The ratio between the height characteristic coefficient and the chassis height of the industrial vehicle is calculated, and the collision risk index of the obstacle is obtained according to the contour complexity coefficient, the object obstruction coefficient, the ratio and the volume data of the obstacle; the contour complexity coefficient, the object obstruction coefficient, the ratio and the volume data are all positively correlated with the collision risk index.

[0022] Preferably, the degree of influence of obstacles on the perception of industrial vehicles is obtained according to the difference between the position data of obstacles corresponding to every two moments and the change distribution between the height data of obstacles at adjacent moments, which specifically includes:

[0023] Obtain the dynamic characteristic index of the obstacle according to the difference between the position data of the obstacle corresponding to each two moments;

[0024] According to the data change between the height data of the obstacle at each moment and the adjacent moments, combined with the dynamic characteristic index, the degree of influence of the obstacle on the perception of the industrial vehicle is obtained.

[0025] Preferably, obtaining the dynamic characteristic index of the obstacle according to the difference between the position data of the obstacle corresponding to every two moments specifically includes:

[0026] For every two moments in a preset time period, the Euclidean distance between the position data of the obstacle is used as the distance parameter between every two moments; based on the mean value of all distance parameters in the preset time period, the dynamic characteristic index of the obstacle is determined.

[0027] Preferably, the data change between the height data of the obstacle at each moment and the adjacent moments is combined with the dynamic characteristic index to obtain the degree of influence of the obstacle on the perception of the industrial vehicle, which specifically includes:

[0028] In a preset period of time, the ratio between the height data of the obstacle at each moment and the adjacent previous moment is used as the height change parameter between each two adjacent moments, and the average value of all height change parameters in the preset period of time is calculated to obtain the comprehensive change factor;

[0029] The perceived impact degree of the obstacle on the industrial vehicle is obtained according to the dynamic characteristic index of the obstacle and the comprehensive change factor, and both the dynamic characteristic index and the comprehensive change factor are positively correlated with the perceived impact degree.

[0030] Preferably, the obstacle warning level of the industrial vehicle is divided according to the collision risk index and the perception impact degree, specifically including:

[0031] The normalized result of the Euclidean norm between the collision risk index of the obstacle and the perceived impact of the obstacle on the industrial vehicle is calculated to obtain the warning coefficient, and the obstacle warning level of the industrial vehicle is divided according to the warning coefficient.

[0032] Preferably, the use of warning coefficients to classify obstacle warning levels of industrial vehicles specifically includes:

[0033] When the warning coefficient is less than a preset first threshold, the obstacle warning level of the industrial vehicle is low; when the warning coefficient is greater than or equal to the first threshold and less than a preset second threshold, the obstacle warning level of the industrial vehicle is medium; when the warning coefficient is greater than or equal to the second threshold, the obstacle warning level of the industrial vehicle is high; the first threshold is less than the second threshold.

[0034] The embodiments of the present invention have at least the following beneficial effects:

[0035] The present invention first collects data through the industrial vehicle ADAS system integrated with a large model when the industrial vehicle senses an obstacle during its travel, providing a data basis for the subsequent analysis of the change in obstacle characteristics within a period of dynamic observation, that is, a preset period of time. Then, the first aspect quantifies the collision risk index of the obstacle, mainly considering the obstacle's own attributes and object contours when the obstacle is initially sensed, and simply analyzes the obstacle's contour type to characterize the possible risk coefficient size and collision situation of the obstacle. Further, the second aspect quantifies the degree of influence of the obstacle on the perception of the industrial vehicle, mainly considering the possible movement of the perceived obstacle, the position data of the obstacle with movement is quite different, and the height data also shows a certain trend of change. Finally, combined with the characteristic analysis results of the two aspects, a comprehensive analysis of the possible threat situation of the current obstacle is equivalent to that of the industrial vehicle, which can more accurately quantify the obstacle warning level, effectively improve the reliability of the industrial vehicle ADAS system integrated with a large model and the perception accuracy of the obstacle, and provide a more powerful guarantee for the safe operation of industrial vehicles in complex environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] In order to more clearly illustrate the technical solutions and advantages in the embodiments of the present invention or the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0037] Figure 1 It is a flowchart of the steps of the obstacle perception method of the industrial vehicle ADAS fused with a large model provided by the present invention;

[0038] Figure 2 is a flowchart of the steps of the method for obtaining the collision risk index of an obstacle provided by the present invention;

[0039] Figure 3 is a flowchart of the steps of a method for obtaining the degree of influence of obstacles on the perception of industrial vehicles provided by the present invention;

[0040] Figure 4 It is a module structure diagram of the obstacle sensing device of the industrial vehicle ADAS fused with a large model provided by the present invention;

[0041] Figure 5 It is a structural schematic diagram of a computer device provided by the present invention. DETAILED DESCRIPTION

[0042] In order to further explain the technical means and effects adopted by the present invention to achieve the predetermined invention purpose, the obstacle perception method of the industrial vehicle ADAS fused with a large model proposed by the present invention, its specific implementation, structure, features and effects are described in detail as follows in combination with the accompanying drawings and preferred embodiments. In the following description, different "one embodiment" or "another embodiment" does not necessarily refer to the same embodiment. In addition, specific features, structures or characteristics in one or more embodiments may be combined in any suitable form.

[0043] Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.

[0044] The specific scheme of the obstacle perception method of industrial vehicle ADAS integrating a large model provided by the present invention is described in detail below with reference to the accompanying drawings.

[0045] See also Figure 1 , which shows a flowchart of the obstacle perception method of an industrial vehicle ADAS integrating a large model provided by an embodiment of the present invention, the method comprising the following steps:

[0046] Step S100, during the operation of the industrial vehicle, when an obstacle is detected within the radar range of the industrial vehicle, the grayscale image of the obstacle and the height data and position data of the obstacle at each moment in a preset time period are obtained by integrating the large model of the industrial vehicle ADAS system.

[0047] During the operation of industrial vehicles, the laser radar of the ADAS system installed on the industrial vehicles can detect whether there are obstacles within the radar range of the industrial vehicles. Then, it is necessary to analyze and judge the various position information between the obstacle and the industrial vehicle and the obstacle's own attribute information to perceive whether the obstacle poses a safety hazard to the currently running industrial vehicle. In order to provide a data basis for subsequent data analysis, it is necessary to first collect data of various dimensions related to the corresponding obstacles when the laser radar detects an obstacle within the radar range during the operation of the industrial vehicle.

[0048] Specifically, the obstacle image is collected by the camera of the ADAS system of the industrial vehicle, and the obstacle image is grayed and processed to obtain the grayscale image of the obstacle. The height data of the obstacle is obtained through the point cloud data obtained by the laser radar in the ADAS system. The position data of the obstacle is frequently sensed and obtained within a preset time period in the perception direction of the obstacle. The sampling frequency of the laser radar can be set to 10Hz, wherein the position data can be the position coordinates of the obstacle, which is used to reflect the location information of the obstacle at the current moment. In this embodiment, the time interval between each two adjacent moments in the preset time period is equal, and the time length of the preset time period is 5s. In other embodiments, the implementer can set it according to the specific implementation scenario.

[0049] It should be noted that the ADAS system for industrial vehicles is a well-known technology, and the process of using the ADAS system to collect data is also a well-known technology. Here, only a brief introduction is made to the ADAS system. It can be understood that the ADAS system for industrial vehicles that integrates a large model mainly relies on the comprehensive analysis and fusion of multiple sensor data. The large model can learn more accurate feature representations and perform multi-dimensional information fusion through training on a large amount of data. In the ADAS system for industrial vehicles, the integrated large model is like a hub for information fusion and collaborative computing. It can process cross-modal data, that is, the collaboration between different types of sensor data. The integrated large model can be regarded as an organic combination of multiple intelligent computing models, including not only neural networks, but also other models. The method involved in this embodiment is based on the implementation of the integrated large model.

[0050] In the advanced driver assistance system (ADAS) and obstacle perception method of industrial vehicles that integrate large models, the deployment of edge computing devices and the acquisition of perception information during the working process are key links; common edge computing devices include industrial-grade computers, embedded systems and dedicated edge AI processors; edge computing devices are usually installed in the on-board system of industrial vehicles, and the installation location should ensure that the installation of the equipment does not affect the operation and safety of the industrial vehicle, and ensure that the equipment can easily connect to various sensors (such as cameras, lidar, etc.) and communication modules.

[0051] Industrial vehicles collect environmental information in real time through a variety of sensors, including: cameras, which are used for image acquisition and visual recognition; lidar, which provides three-dimensional spatial information to help identify the shape and position of objects; radar, which is used to measure the distance and speed of objects; the perception information is then fused, analyzed, and processed using large models, and the final results are fed back to the control system of the industrial vehicle in real time, improving the perception accuracy of the industrial vehicle in complex environments.

[0052] Step S200, according to the regularity of the distribution of the contour information of the object in the obstacle grayscale image at the initial moment within the preset time period, combined with the object attribute information corresponding to the obstacle at the initial moment and the obstacle height data at each moment, the collision risk index of the obstacle is obtained.

[0053] In the operating environment of industrial vehicles, when industrial vehicles are working according to the predetermined route, they may encounter some obstacles, including people, goods, and debris on the ground. However, different obstacles have different warning trigger levels for industrial vehicle ADAS. For example, for people on the route as obstacles, the warning level and risk factor are relatively high, while when the height and volume of the obstacle are low, the warning level is low, and the corresponding possibility of collision is also small. Based on this, we can first use machine vision methods to analyze the contour performance of obstacles in the direction of industrial vehicle travel, which can preliminarily reflect the complexity of the obstacle contour, and then combine other aspects of its own attributes to evaluate the risk of collision caused by obstacles.

[0054] In this embodiment, if Figure 2 As shown, the method for obtaining the collision risk index of the obstacle can be implemented by steps S201 to S204.

[0055] Step S201, obtaining the obstacle contour complexity coefficient according to the edge change of adjacent pixel points on the edge of the obstacle grayscale image at the initial moment within a preset time period.

[0056] Analyze the outline of the obstacle to identify the general type of obstacle. For example, if the obstacle is general cargo, the outline distribution of this type of obstacle is relatively uniform and regular. If the obstacle is a person or irregular debris piled on the ground or other irregular objects, the outline distribution of this type of obstacle is relatively irregular, showing a more complex distribution on the edge of the outline. Different types of obstacles will produce different collision situations. For cargo, there may be no collision due to its low height, and the risk factor is low. For walking people, there may be no collision due to different dynamic travel directions and times, and so on. Based on this, it is first necessary to analyze the outline distribution of the obstacle when an obstacle has been detected within the radar range of the industrial vehicle to obtain a preliminary understanding of the current obstacle information.

[0057] Specifically, the grayscale image of the obstacle at the initial moment in the preset period is recorded as the target obstacle image; it can be understood that the initial moment of the preset period is also the first moment of the preset period, which represents the moment when the obstacle is detected. In this embodiment, the target obstacle image is processed using an edge detection algorithm to obtain edge pixel points of each edge line in the target obstacle image.

[0058] Then, calculate the slope value of each edge pixel in the target obstacle image on the corresponding edge line; the slope value can reflect the contour change at the position of the edge pixel on each edge line in the image. Its calculation method is a well-known technology and is only briefly introduced here. Take the central pixel of the target obstacle image as the origin, the image row as the horizontal axis, and the image column as the vertical axis to construct a plane rectangular coordinate system, and the position coordinates of each pixel in the target obstacle image can be obtained. The horizontal coordinate of the pixel position coordinates is the row where the pixel is located, and the vertical coordinate is the column where the pixel is located. Then, the slope value of each edge pixel can be calculated through the position coordinates of two adjacent edge pixels on the same edge line.

[0059] Furthermore, based on the average of the difference between the slope values ​​of each two adjacent edge pixel points on each edge line, the pixel complexity parameter of each edge line is determined; the cumulative sum of the pixel complexity parameters of all edge lines in the target obstacle image is used as the contour complexity coefficient of the obstacle. As a specific example, the calculation method of the contour complexity coefficient of the obstacle can be expressed as:

[0060]

[0061] in, Represents the complexity coefficient of the obstacle contour, Indicates the total number of edge lines contained in the target obstacle image. Represents the total number of edge pixels contained on the nth edge line in the target obstacle image. Represents the slope value of the mth edge pixel on the nth edge line in the target obstacle image. Represents the slope value of the m+1th edge pixel on the nth edge line in the target obstacle image.

[0062] Represents the pixel complex parameters of the nth edge line, It reflects the difference in edge distribution between adjacent edge pixels. The pixel complexity parameter reflects the regularity of the shape of an edge line. The larger the value, the greater the difference in edge distribution between edge pixels, and the more irregular the shape of the corresponding edge line. The shape distribution of all edges in the image is synthesized by summing. The larger the value of the contour complexity coefficient, the more irregular the edge of the obstacle, the more likely it is a person or an irregularly shaped object, and the greater the contour complexity of the corresponding object.

[0063] Step S202, the object obstruction coefficient of the obstacle is obtained based on the object attribute information corresponding to the obstacle at the initial moment within the preset time period combined with the neural network in the fusion large model.

[0064] By integrating the large model, the ADAS system of the industrial vehicle can obtain the relevant attribute information of the detected obstacles, such as the size information, shape information, object category, etc. The ADAS system can identify some common types of obstacles, such as pedestrians, other vehicles, shelves, goods, etc. Obstacles of different types or with different attributes have different degrees of obstruction to industrial vehicles. For example, more regular goods have a smaller degree of obstruction, while pedestrians or other vehicles have a relatively larger degree of obstruction, and the corresponding risk factor is also higher.

[0065] This embodiment uses the neural network in the fusion large model to obtain the object obstruction coefficient of the obstacle, which is used to characterize the obstruction situation of the obstacle. The larger the value of the object obstruction coefficient, the greater the obstruction caused by the obstacle, and the greater the corresponding collision risk.

[0066] Specifically, obstacles with different object attributes are annotated in the form of manual annotation to obtain annotation data, wherein the annotation data includes object obstruction coefficients of obstacles with different object attributes; the annotated data is used to train a neural network; the object attribute information corresponding to the obstacle at the initial moment within a preset time period is input into the trained neural network to obtain the object obstruction coefficient of the obstacle. Among them, this embodiment uses a fully connected neural network in a fusion large model for training, and the loss function of the network is a mean square error loss.

[0067] More specifically, the object attribute data corresponding to all obstacle types in the working environment of the industrial vehicle is obtained, and the relevant staff mark each different obstacle according to experience. The result of the marking is the object obstruction coefficient, which represents the degree of obstruction set according to experience. The larger the value, the greater the degree of obstruction, and the smaller the value, the smaller the degree of obstruction. Among them, the object attribute data includes the obstacle size, shape and object category recognized by the ADAS system of the industrial vehicle that integrates the large model.

[0068] In other embodiments, the implementer can set a corresponding object obstruction coefficient according to the identified obstacle object category, for example, 10 for people, 9 for other vehicles, 8 for shelves, 3 for regular goods, etc. Relevant staff in the current industrial environment can set it based on experience. The larger the value of the object obstruction coefficient, the greater the degree of obstruction of the corresponding obstacle, and the smaller the value of the object obstruction coefficient, the smaller the degree of obstruction of the corresponding obstacle.

[0069] Step S203: taking the maximum value of the obstacle height data at all times within the preset time period as the obstacle height characteristic coefficient.

[0070] In analyzing the direction of industrial vehicle travel, the initial moment within the preset time period is the initial stage of obstacle detection. During the preset time period for dynamic observation of the obstacle, if the obstacle presents dynamic characteristics, that is, the obstacle is moving, then the height of the obstacle during the preset time period of dynamic observation may change to a certain extent. Different heights at different times have different degrees of dangerous impact and obstruction on industrial vehicles. The higher the height of the obstacle, the greater the corresponding obstruction and the higher the risk coefficient. The lower the height of the obstacle, the smaller the corresponding obstruction and the lower the risk coefficient.

[0071] In this embodiment, the maximum value of the obstacle's height data at all times within a preset time period is used as the obstacle's height characteristic coefficient, and the height characteristic coefficient reflects the maximum height representation value that the obstacle may have through movement within the preset time period for dynamic observation.

[0072] Step S204 , obtaining a collision risk index of an obstacle according to the contour complexity coefficient, the object obstruction coefficient and the height characteristic coefficient in combination with the chassis height of the industrial vehicle.

[0073] In the ADAS system of industrial vehicles in the current industrial environment, the application of fusion large models can significantly improve the ability to perceive obstacles, and thus effectively evaluate the collision risk of obstacles during the operation of industrial vehicles. On the one hand, when the maximum height characteristic of the obstacle is lower than the chassis height of the industrial vehicle, the industrial vehicle can pass through the obstacle directly, and the corresponding obstacle has a lower degree of obstruction and a lower risk factor. On the other hand, combined with the performance of the obstacle's own attributes, the degree of obstruction quantitatively represented by the obstacle's own attributes is used to characterize the possible collision situations. Based on this, combined with the data feature analysis results of the two aspects, a comprehensive assessment of the possible collision risks of the current obstacles is conducted, providing a data basis for subsequent collision risk warnings for obstacles.

[0074] Specifically, the ratio of the height characteristic coefficient to the chassis height of the industrial vehicle is calculated, and the collision risk index of the obstacle is obtained according to the contour complexity coefficient, the object obstruction coefficient, the ratio and the volume data of the obstacle; the contour complexity coefficient, the object obstruction coefficient, the ratio and the volume data are all positively correlated with the collision risk index.

[0075] In this embodiment, the product of the contour complexity coefficient, the object obstruction coefficient, the ratio, and the volume data of the obstacle is used as the collision risk indicator of the obstacle. It can be understood that the chassis height of the industrial vehicle can be directly extracted from the fused large model. Each industrial vehicle has a corresponding vehicle body height and chassis height, and these data are inherent attributes of the industrial vehicle. At the same time, the volume data of the obstacle is the data obtained after normalizing the volume of the obstacle. The volume of the obstacle can be calculated in the cloud by fusing the large model with the point cloud data of the obstacle collected by the ADAS system of the industrial vehicle. The method of obtaining the volume of the obstacle belongs to the well-known technology in the field of the vehicle's ADAS system, and will not be introduced in detail here.

[0076] In this embodiment, as a specific example, the calculation formula of the collision risk index of the obstacle can be expressed as: ,in represents the collision risk index of the obstacle, Represents the complexity coefficient of the obstacle contour, The object obstruction coefficient representing the obstacle, Represents the volume data of the obstacle, represents the height characteristic coefficient of the obstacle, Indicates the chassis height of an industrial vehicle.

[0077] The larger the product of , the more complex the contour information of the object is perceived in the direction of the industrial vehicle, the greater the degree of obstruction, and the larger the volume of the object. For industrial vehicles, large objects also hinder the movement of industrial vehicles, and the corresponding risk level is greater. The product reflects the obstruction and risk level of the obstacle from the characteristic performance of the obstacle's own attributes. The larger the value is, the higher the obstacle height during the movement of the industrial vehicle has exceeded the safe height value from the ground during the movement of the industrial vehicle. The higher the degree of obstruction of the corresponding obstacle, the higher the risk coefficient.

[0078] Step S300, according to the difference between the position data of the obstacle corresponding to every two moments and the change distribution between the height data of the obstacle at adjacent moments, the perception influence degree of the obstacle on the industrial vehicle is obtained.

[0079] When the vehicle is working along the predetermined route, the obstacles perceived by the industrial vehicle ADAS system integrated with the large model may be in a state of motion. For the industrial vehicle in motion, if the moving trajectory of the moving obstacle intersects with the forward route of the industrial vehicle, the possibility of collision is higher and the corresponding risk factor is also higher.

[0080] After the laser radar in the ADAS system of the industrial vehicle integrated with the large model senses an obstacle, the laser radar in the sensing direction of the obstacle will frequently sense the position changes of the obstacle within a preset time period. By analyzing the movement of the obstacle during the dynamic observation period, the movement of the obstacle can be quantitatively characterized, and it can be analyzed whether the movement will pose a safety threat to the current behavior of the industrial vehicle.

[0081] Based on this, the first aspect is to analyze the position change of obstacles within the dynamic observation time range, and the second aspect is to analyze the height change of obstacles within the dynamic observation time. Combining the analysis results of the two aspects, the possible impact of obstacles with dynamic characteristics on industrial vehicles is evaluated. Specifically, Figure 3 As shown, the method for obtaining the degree of influence of obstacles on the perception of industrial vehicles can be implemented by step S301 and step S302.

[0082] Step S301, obtaining a dynamic characteristic index of an obstacle according to the difference between the position data of the obstacle corresponding to every two moments.

[0083] For every two moments in a preset time period, the Euclidean distance between the position data of the obstacle is used as the distance parameter between every two moments; based on the mean value of all distance parameters in the preset time period, the dynamic characteristic index of the obstacle is determined.

[0084] At any two moments in a preset time period, there is a distance parameter between the locations of the obstacles, which reflects the position change of the obstacles at any two moments. The average value of all distance parameters in the preset time period is used as the dynamic characteristic index of the obstacle. The larger the value, the greater the possibility that the obstacle is in motion when the ADAS system of the industrial vehicle perceives the movement of the obstacle, that is, the greater the dynamic nature of the obstacle.

[0085] Step S302: according to the data change between the height data of the obstacle at each moment and the adjacent moments, combined with the dynamic characteristic index, the degree of the perceived influence of the obstacle on the industrial vehicle is obtained.

[0086] During the dynamic feature observation period, the height of the obstacle perceived by the industrial vehicle gradually increases with time. In terms of data, the height data of the obstacle shows a trend of gradually increasing. This is a visual effect caused by the obstacle getting closer and closer to the edge of the industrial vehicle. When the obstacle itself is more dynamic, the obstacle moves relative to the industrial vehicle, indicating that the obstacle has a greater impact on the industrial vehicle and the higher the safety threat.

[0087] Specifically, within a preset time period, the ratio between the height data of the obstacle at each moment and the adjacent previous moment is used as the height change parameter between each two adjacent moments, and the average value of all height change parameters within the preset time period is calculated to obtain a comprehensive change factor; the perceived impact of the obstacle on the industrial vehicle is obtained according to the dynamic characteristic index of the obstacle and the comprehensive change factor, and both the dynamic characteristic index and the comprehensive change factor are positively correlated with the perceived impact degree.

[0088] As a specific example, the calculation formula for the degree of influence of obstacles on the perception of industrial vehicles can be expressed as: ,in Indicates the degree of impact of obstacles on the perception of industrial vehicles, Indicators representing the dynamic characteristics of obstacles, Indicates the total number of moments included in the preset period. Indicates the height data of the obstacle at the t+1th moment within the preset time period, Indicates the height data of the obstacle at the tth moment within the preset time period.

[0089] is the height change parameter between two adjacent moments, It is a comprehensive change factor, which reflects the height change of obstacles in the process of laser radar perception in the ADAS system of industrial vehicles. When the dynamic characteristic index of the obstacle is larger, it means that the obstacle has stronger dynamic characteristics during the current dynamic observation time. Then the value of the comprehensive change factor is larger, which means that the obstacle shows a trend of getting higher and higher in the perception process with the change of time, which means that the distance between the obstacle and the industrial vehicle is getting closer and closer with the change of time. Then the corresponding perception influence value is larger, which means that the possibility and influence of the obstacle on the industrial vehicle are greater.

[0090] It is understandable that if there is a driver in an industrial vehicle, obstacles with dynamic behavior may affect the driver's line of sight, thus posing a potential safety problem. If the obstacle is a plastic bag on the ground, if the plastic bag is stationary or fixed to the ground, the safety threat posed by the plastic bag to the industrial vehicle is small. However, if the plastic bag may have dynamic behavior over time, such as floating, it may block the direction of travel of the industrial vehicle during the floating process, thereby increasing the safety threat and risk factor.

[0091] Step S400: classify obstacle warning levels of industrial vehicles according to the collision risk index and the perception impact degree, and obtain a level perception result of the obstacle.

[0092] During the movement of industrial vehicles, after the industrial vehicle ADAS system integrated with the large model senses an obstacle, it uses the data analysis results of the edge computing device and the recognition results of the integrated large model, and uses the obstacle collision risk index to reflect the possible risk of obstacles sensed within the radar range of the current industrial vehicle from the obstacle's own attributes and height distribution. The degree of influence of obstacles on the perception of industrial vehicles is used to reflect the impact of obstacles sensed within the radar range of the current industrial vehicle on the safety threat of the industrial vehicle from the dynamic characteristics of the obstacles and the height change trend under the dynamic state. Combining the data analysis results of these two aspects, it is possible to more comprehensively characterize the distribution level of perceived obstacles relative to industrial vehicles that require warnings.

[0093] Based on this, the warning coefficient corresponding to the industrial vehicle is first quantified, and then the obstacle warning level of the industrial vehicle is divided using the warning coefficient. Specifically, the normalized result of the Euclidean norm between the collision risk index of the obstacle and the perceived impact of the obstacle on the industrial vehicle is calculated to obtain the warning coefficient.

[0094] More specifically, the early warning coefficient can be expressed as: ,in represents the warning coefficient, represents the collision risk index of the obstacle, Indicates the degree of impact of obstacles on the perception of industrial vehicles, Represents the hyperbolic tangent function, which is used to obtain the normalized result of the Euclidean norm. The obstacle characteristics in two aspects are analyzed comprehensively to quantitatively characterize the extent to which the current obstacle requires early warning.

[0095] For industrial vehicles, the higher the level of warning required, the greater the threat posed by the current obstacles to the industrial vehicles, and the greater the risk factor. The warning coefficient can then be used to analyze the distribution of the level of warning required for the current industrial vehicles.

[0096] Specifically, when the warning coefficient is less than a preset first threshold, the obstacle warning level of the industrial vehicle is low; when the warning coefficient is greater than or equal to the first threshold and less than a preset second threshold, the obstacle warning level of the industrial vehicle is medium; when the warning coefficient is greater than or equal to the second threshold, the obstacle warning level of the industrial vehicle is high; the first threshold is less than the second threshold.

[0097] In this embodiment, the value of the first threshold is 0.3, and the value of the second threshold is 0.7. The implementer can set it according to the specific implementation scenario. In the current industrial environment, corresponding solutions can be provided according to different obstacle warning levels. For example, when the obstacle warning level of the industrial vehicle is low, it means that the current obstacle is a potential obstacle, but the impact of the obstacle on the industrial vehicle is low and is within a safe range. When the obstacle warning level of the industrial vehicle is medium, it means that the obstacle is approaching, and it is recommended to slow down or change the route. When the obstacle warning level of the industrial vehicle is high, it means that a collision may occur and braking or avoidance measures must be taken immediately. This embodiment is only an exemplary description. In other embodiments, the implementer needs to provide appropriate solutions based on the specific implementation scenario and the experience of relevant staff. The ultimate goal is to ensure that industrial vehicles can operate safely and efficiently in complex industrial environments, and effectively improve the reliability of the industrial vehicle ADAS system and the perception accuracy of obstacles.

[0098] like Figure 4 As shown, the present invention provides an obstacle sensing device for industrial vehicle ADAS integrating a large model, comprising:

[0099] The data acquisition module is used to acquire the grayscale image of the obstacle and the height data and position data of the obstacle at each moment in a preset period of time through the industrial vehicle ADAS system fused with the large model when an obstacle is detected within the radar range of the industrial vehicle during the operation of the industrial vehicle;

[0100] The risk analysis module is used to obtain the collision risk index of the obstacle according to the distribution regularity of the object contour information in the obstacle grayscale image at the initial moment within the preset time period, combined with the object attribute information corresponding to the obstacle at the initial moment and the obstacle height data at each moment;

[0101] The impact analysis module is used to obtain the perceived impact of obstacles on industrial vehicles based on the difference between the position data of obstacles corresponding to every two moments and the change distribution between the height data of obstacles at adjacent moments;

[0102] The level analysis module is used to classify the obstacle warning level of the industrial vehicle according to the collision risk index and the perception impact degree, and obtain the level perception result of the obstacle.

[0103] The present application also provides a computer device. Figure 5 , which shows a schematic diagram of the structure of a computer device provided by an embodiment of the present invention, the computer device includes a memory 501, a processor 502, and a computer program 503 stored in the memory 501 and running on the processor 502, wherein when the processor 502 executes the computer program 503, the computer device can execute the obstacle perception method of the industrial vehicle ADAS integrating the large model introduced above.

[0104] The embodiment of the present application also provides a computer program product. When the computer program product is run on a computer device, the computer device can execute the obstacle perception method of industrial vehicle ADAS integrating a large model introduced above.

[0105] An embodiment of the present application also provides a computer-readable storage medium, in which a computer program code is stored. When the computer program code is executed on a computer device, the computer device can execute the obstacle perception method of the industrial vehicle ADAS integrating the large model introduced above.

[0106] In the embodiments provided in the present application, it should be understood that the provided computer device, computer program product and computer-readable storage medium are all used to execute the corresponding methods provided above. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the methods provided above and will not be repeated here.

[0107] The embodiments described above are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. An obstacle perception method for industrial vehicle ADAS integrating a large model, characterized in that: The method comprises the following steps: During the operation of industrial vehicles, when an obstacle is detected within the radar range of the industrial vehicle, the industrial vehicle ADAS system that integrates the large model obtains the grayscale image of the obstacle and the height data and position data of the obstacle at each moment within the preset time period; According to the distribution regularity of the object contour information in the obstacle grayscale image at the initial moment within the preset time period, combined with the object attribute information corresponding to the obstacle at the initial moment and the obstacle height data at each moment, the obstacle collision risk index is obtained; According to the difference between the position data of the obstacles corresponding to each two moments and the change distribution between the height data of the obstacles at adjacent moments, the degree of influence of the obstacles on the perception of the industrial vehicle is obtained; According to the collision risk index and the perception impact degree, the obstacle warning level of the industrial vehicle is divided to obtain the level perception result of the obstacle; The obstacle collision risk index is obtained based on the distribution regularity of the object contour information in the obstacle grayscale image at the initial moment within the preset time period, combined with the object attribute information corresponding to the obstacle at the initial moment and the obstacle height data at each moment, specifically including: Obtaining the obstacle contour complexity coefficient according to the edge distribution of adjacent pixel points on the edge of the obstacle grayscale image at the initial moment within a preset time period; Obtain the object obstruction coefficient of the obstacle according to the object attribute information corresponding to the obstacle at the initial moment within the preset time period combined with the neural network in the fusion large model; The maximum value of the obstacle height data at all times within the preset time period is taken as the obstacle height characteristic coefficient; Obtaining a collision risk index of an obstacle according to the contour complexity coefficient, the object obstruction coefficient and the height characteristic coefficient in combination with the chassis height of the industrial vehicle; The step of obtaining the obstacle contour complexity coefficient according to the edge distribution of adjacent pixel points on the edge of the obstacle grayscale image at the initial moment within the preset time period specifically includes: Recording the obstacle grayscale image at the initial moment within the preset time period as the target obstacle image; obtaining edge pixel points of each edge line in the target obstacle image; Calculate the slope value of each edge pixel point on the corresponding edge line in the target obstacle image; Based on the average of the difference between the slope values ​​of each two adjacent edge pixel points on each edge line, the pixel complexity parameter of each edge line is determined; the cumulative sum of the pixel complexity parameters of all edge lines in the target obstacle image is used as the contour complexity coefficient of the obstacle; The object obstruction coefficient of the obstacle is obtained based on the object attribute information corresponding to the obstacle at the initial moment within the preset time period and the neural network in the fusion large model, specifically including: Obstacles with different object attributes are annotated in the form of manual annotation to obtain annotation data, wherein the annotation data includes object obstruction coefficients of obstacles with different object attributes; a neural network is trained using the annotation data; object attribute information corresponding to the obstacle at an initial moment within a preset time period is input into the trained neural network to obtain the object obstruction coefficient of the obstacle; the object obstruction coefficient of the obstacle is used to characterize the obstruction situation of the obstacle, and the larger the value of the object obstruction coefficient, the greater the obstruction caused by the obstacle, and the greater the corresponding collision risk; The obstacle collision risk index is obtained according to the contour complexity coefficient, the object obstruction coefficient and the height characteristic coefficient in combination with the chassis height of the industrial vehicle, specifically including: The ratio between the height characteristic coefficient and the chassis height of the industrial vehicle is calculated, and the collision risk index of the obstacle is obtained according to the contour complexity coefficient, the object obstruction coefficient, the ratio and the volume data of the obstacle; the contour complexity coefficient, the object obstruction coefficient, the ratio and the volume data are all positively correlated with the collision risk index.

2. The obstacle perception method of industrial vehicle ADAS integrating a large model according to claim 1 is characterized in that: The influence degree of the obstacle on the perception of the industrial vehicle is obtained according to the difference between the position data of the obstacle corresponding to each two moments and the change distribution between the height data of the obstacle at adjacent moments, which specifically includes: Obtain the dynamic characteristic index of the obstacle according to the difference between the position data of the obstacle corresponding to each two moments; According to the data change between the height data of the obstacle at each moment and the adjacent moments, combined with the dynamic characteristic index, the degree of the perceived influence of the obstacle on the industrial vehicle is obtained.

3. The obstacle perception method of industrial vehicle ADAS integrating a large model according to claim 2 is characterized in that: The dynamic characteristic index of the obstacle is obtained according to the difference between the position data of the obstacle corresponding to each two moments, specifically including: For every two moments in a preset time period, the Euclidean distance between the position data of the obstacle is used as the distance parameter between every two moments; based on the mean value of all distance parameters in the preset time period, the dynamic characteristic index of the obstacle is determined.

4. The obstacle perception method of industrial vehicle ADAS integrating a large model according to claim 2 is characterized in that: The data change between the height data of the obstacle at each moment and the adjacent moments, combined with the dynamic characteristic index, obtains the degree of influence of the obstacle on the perception of the industrial vehicle, specifically including: In a preset period of time, the ratio between the height data of the obstacle at each moment and the adjacent previous moment is used as the height change parameter between each two adjacent moments, and the average value of all height change parameters in the preset period of time is calculated to obtain the comprehensive change factor; The perceived impact degree of the obstacle on the industrial vehicle is obtained according to the dynamic characteristic index of the obstacle and the comprehensive change factor, and both the dynamic characteristic index and the comprehensive change factor are positively correlated with the perceived impact degree.

5. The obstacle perception method of industrial vehicle ADAS integrating a large model according to claim 1 is characterized in that: The obstacle warning level of the industrial vehicle is divided according to the collision risk index and the perception impact degree, specifically including: The normalized result of the Euclidean norm between the collision risk index of the obstacle and the perceived impact of the obstacle on the industrial vehicle is calculated to obtain the warning coefficient, and the obstacle warning level of the industrial vehicle is divided according to the warning coefficient.

6. The obstacle perception method of industrial vehicle ADAS integrating a large model according to claim 5 is characterized in that: The obstacle warning levels of industrial vehicles are divided by using the warning coefficient, specifically including: When the warning coefficient is less than a preset first threshold, the obstacle warning level of the industrial vehicle is low; when the warning coefficient is greater than or equal to the first threshold and less than a preset second threshold, the obstacle warning level of the industrial vehicle is medium; when the warning coefficient is greater than or equal to the second threshold, the obstacle warning level of the industrial vehicle is high; the first threshold is less than the second threshold.

Images