Road target detection method, equipment, medium, product and unmanned vehicle

Abstract

The present invention discloses a road target detection method, equipment, medium, product and unmanned vehicle, which are applied to the field of autonomous driving technology. The method includes rasterizing the road surface area to be detected in front of the driving direction to generate a bird's-eye view grid area including multiple grids with spatial voxels; determining the elevation difference between the road surface where the spatial voxels of each grid are located and the horizontal plane of the autonomous driving vehicle coordinate system based on the inclination data of the road surface area to be detected and the horizontal plane of the vehicle coordinate system and the position information of each grid; based on the horizontal plane elevation difference corresponding to each grid, determining whether there is a target object in the road surface area to be detected through the spatial position correspondence between the bird's-eye view grid area and the road surface area to be detected. The present invention can solve the problem of low accuracy of road target area detection in related technologies, can accurately estimate the depth or elevation of potholes / obstacle areas, and effectively improve the accuracy of road target area detection.

Description

Technical Field

[0001] The present invention relates to the field of autonomous driving technology, and in particular to a road target detection method, electronic equipment, non-volatile storage medium, computer program product and unmanned vehicle. Background Art

[0002] With the rapid development of artificial intelligence and Internet of Things technologies, autonomous driving technology has also developed accordingly. The smoothness of the road not only affects the normal operation of autonomous vehicles and the comfort of passengers, but also affects the life of the road and its infrastructure.

[0003] When monitoring the presence of potholes and low raised obstacles on the road, the relevant technology is unable to accurately estimate the depth or elevation of the pothole / obstacle area, resulting in low detection accuracy of potholes and low raised obstacles on the road.

[0004] In view of this, accurately identifying road potholes and low raised obstacle areas on the road is a technical problem that needs to be solved by technical personnel in this field.

[0005] It should be noted that the information disclosed in the above background technology section is only used to enhance the understanding of the background of the present application, and therefore may include information that does not constitute the prior art known to ordinary technicians in the field. Summary of the invention

[0006] The present invention provides a road surface object detection method, electronic device, non-volatile storage medium, computer program product and unmanned vehicle, which can accurately estimate the depth or elevation of pothole / obstacle areas and effectively improve the detection accuracy of road potholes and raised low-raised obstacles.

[0007] In order to solve the above technical problems, the present invention provides the following technical solutions:

[0008] In one aspect, the present invention provides a method for detecting a road target, comprising:

[0009] The road surface area to be detected located in front of the driving direction is rasterized to generate a bird's-eye view grid area including multiple grids, each grid having spatial voxels in a horizontal direction perpendicular to the road surface; based on the inclination data of the horizontal plane of the road surface area to be detected and the autonomous driving vehicle coordinate system and the position information of each grid, the elevation difference between the road surface where the spatial voxels of each grid in the bird's-eye view grid area are located and the horizontal plane of the autonomous driving vehicle coordinate system is determined; based on the horizontal plane elevation difference corresponding to each grid, it is determined whether there is a target object in the road surface area to be detected through the spatial position correspondence between the bird's-eye view grid area and the road surface area to be detected.

[0010] In a first exemplary embodiment, determining the elevation difference between the road surface where the spatial voxels of each grid of the bird's-eye view grid area are located and the horizontal plane of the autonomous driving vehicle coordinate system according to the inclination data of the road surface area to be detected and the horizontal plane of the autonomous driving vehicle coordinate system includes:

[0011] According to the inclination degree of the road surface corresponding to each grid, the road surface area to be detected is divided into multiple types of road surface areas, and the inclination degree of the same type of road surface areas meets the preset same or similar inclination conditions; according to the inclination data of each road surface area and the horizontal plane of the autonomous driving vehicle coordinate system and the position information of each grid, the elevation difference between the road surface where the spatial voxels of each grid in the bird's-eye view grid area are located and the horizontal plane of the autonomous driving vehicle coordinate system is determined.

[0012] In a second exemplary implementation, the road surface area to be detected is divided into multiple types of road surface areas according to the inclination of the road surface corresponding to each grid, including:

[0013] According to the three-dimensional position information of the point cloud data inside each grid, the three-dimensional space plane equation of the corresponding grid area is fitted; based on the three-dimensional space plane equation of each grid, the average normal vector within each grid is determined; according to the clustering result of the average normal vector of each grid, the road surface area to be detected is segmented through the spatial position correspondence between the bird's-eye view grid area and the road surface area to be detected, so as to obtain multiple types of road surface areas.

[0014] In a third exemplary embodiment, before the inclination data of each road surface area and the horizontal plane of the autonomous driving vehicle coordinate system is determined, the method further includes:

[0015] According to the three-dimensional space plane equation of each grid, the plane angle between the road surface area corresponding to each grid and the horizontal plane of the autonomous driving vehicle coordinate system is determined; the autonomous driving vehicle coordinate system adopts the laser radar coordinate system; the coordinate system vertical distance between the origin of the laser radar coordinate system and the road surface is determined according to the laser radar installation position and the vehicle result parameters, and according to the coordinate system vertical distance and the plane angle, the position information of the intersection of the horizontal coordinate axis of the autonomous driving vehicle coordinate system and the road surface corresponding to each grid is determined.

[0016] In a fourth exemplary embodiment, determining the elevation difference between a road surface where a spatial voxel of each grid of the bird's-eye view grid area is located and a horizontal plane of the autonomous driving vehicle coordinate system includes:

[0017] For each grid, determine the position information of the intersection of the horizontal axis of the autonomous driving vehicle coordinate system and the road surface corresponding to the current grid on the horizontal axis; based on the plane angle, calculate the elevation difference between the road surface where the spatial voxels of each grid in the bird's-eye view grid area are located and the horizontal plane of the autonomous driving vehicle coordinate system according to the position information of each grid on the horizontal axis and the position information of the corresponding intersection on the horizontal axis.

[0018] In a fifth exemplary implementation, determining the plane angle between the road surface area corresponding to each grid and the horizontal plane of the autonomous driving vehicle coordinate system according to the three-dimensional space plane equation of each grid includes:

[0019] Perform plane fitting processing on the point cloud data in each grid to obtain the corresponding three-dimensional space plane equation; use the angle between the normal vector of the three-dimensional space plane equation of each grid and the normal vector of the horizontal plane of the autonomous driving vehicle coordinate system as the plane angle between the road surface area of ​​the corresponding grid and the horizontal plane of the autonomous driving vehicle coordinate system.

[0020] In a sixth exemplary embodiment, the determining whether there is a target object in the road surface area to be detected based on the horizontal plane elevation difference corresponding to each grid and through the spatial position correspondence between the bird's-eye view grid area and the road surface area to be detected includes:

[0021] If the horizontal plane elevation difference corresponding to the current grid is a positive value, then there is a raised obstacle at the corresponding position of the current grid in the road surface area to be detected, and the height of the raised obstacle is the horizontal plane elevation difference; if the horizontal plane elevation difference corresponding to the current grid is a negative value, then there is a pothole area at the corresponding position of the current grid in the road surface area to be detected, and the depth of the pothole area is the horizontal plane elevation difference.

[0022] In a seventh exemplary implementation, the process of rasterizing the road surface area to be detected in front of the driving direction to generate a bird's-eye view grid area including a plurality of grids, each grid having spatial voxels in a horizontal direction perpendicular to the road surface includes:

[0023] The number of laser radar lines installed on the autonomous driving vehicle is obtained; the number of grids and the grid size are determined according to the number of laser radar lines, and the spatial size of the road surface area to be detected is determined according to the number of grids and the grid size; and the road surface area to be detected located within the corresponding spatial size in front of the driving direction is rasterized according to the number of grids and the grid size.

[0024] In an eighth exemplary implementation, a road surface area to be detected in front of the driving direction is subjected to a rasterization process to generate a bird's-eye view grid area including a plurality of grids, each grid having spatial voxels in a horizontal direction perpendicular to the road surface, including:

[0025] Obtain the resolution of the laser radar installed on the autonomous driving vehicle; determine the number and size of grids according to the resolution of the laser radar, and determine the spatial size of the road surface area to be detected according to the number and size of grids; and perform rasterization processing on the road surface area to be detected within the corresponding spatial size in front of the driving direction according to the number and size of grids.

[0026] In a ninth exemplary implementation, before rasterizing the road surface area to be detected in front of the driving direction, the method further includes:

[0027] The maximum elevation of the target object in the road area to be detected is obtained; the maximum elevation of the target object is divided according to a preset accuracy threshold to determine the total number of spatial cubes contained in the spatial voxels of each grid; according to the maximum elevation of the target object, the total number of spatial cubes and the preset plane size of the spatial cube, the real physical space size corresponding to the spatial cube of the spatial voxel of each grid is determined.

[0028] In a tenth exemplary implementation, determining the real physical space size corresponding to the space cube of the space voxel of each grid according to the maximum elevation of the target object, the total number of space cubes and the preset plane size of the space cube includes:

[0029] The total number of space cubes is taken as a multiple of the number of segments according to the maximum elevation of the target object, and the real physical space height corresponding to the space cube is taken as the ratio of the maximum elevation of the target object to the total number of space cubes; the real physical space size corresponding to the space cube of the space voxel of each grid is determined according to the real physical space height and the grid size.

[0030] The present invention also provides an electronic device, comprising a processor, wherein the processor is used to implement the steps of the road surface target detection method as described in any of the preceding items when executing a computer program stored in a memory.

[0031] The present invention also provides a non-volatile storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the road surface target detection method as described in any of the preceding items are implemented.

[0032] The present invention also provides a computer program product, comprising a computer program / instruction, which, when executed by a processor, implements the steps of the road surface target detection method as described in any of the preceding items.

[0033] Finally, the present invention also provides an unmanned vehicle, including an automatic driving perception system, wherein the automatic driving perception system is used to implement the steps of the road target detection method as described in any of the preceding items when executing a computer program stored in a memory.

[0034] The technical solution provided by the present invention has the advantages that the road ahead is divided into multiple grids, each grid is provided with a spatial voxel perpendicular to the road direction, and the elevation information of the low raised obstacle / pothole area can be accurately calculated by the elevation difference between the road surface where the spatial voxel is located and the horizontal plane of the vehicle coordinate system. In order to avoid the situation where there is an angle between the road surface and the plane of the vehicle coordinate system, which leads to errors in the elevation calculation process of the low raised obstacle / pothole area, when calculating the elevation information of the low raised obstacle / pothole area, the inclination data of the road surface and the plane of the vehicle coordinate system are also added, so as to further improve the elevation calculation accuracy of the low raised obstacle / pothole area, avoid the misjudgment of the low raised obstacle / pothole area with a small elevation, and accurately estimate the depth or elevation of the pothole / obstacle area, so as to identify whether there are such targets as low raised obstacles / pothole areas on the road surface, thereby effectively improving the detection accuracy of road potholes and raised low raised obstacles, which is not only conducive to ensuring the safe and stable operation of the autonomous driving vehicle, effectively improving the riding experience of the passengers, but also conducive to rapid road maintenance and extending the life of the road and its infrastructure.

[0035] In addition, the present invention also provides corresponding electronic devices, non-volatile storage media, computer program products and unmanned vehicles for implementing the road target detection method, further making the method more practical. The electronic devices, non-volatile storage media, computer program products and unmanned vehicles have corresponding advantages.

[0036] The technical features mentioned above, the technical features to be mentioned below, and the technical features shown separately in the drawings can be arbitrarily combined with each other, as long as the combined technical features are not contradictory. All feasible feature combinations are technical contents clearly recorded in this article. Any of the multiple sub-features contained in the same sentence can be applied independently, and does not have to be applied together with other sub-features. It should be understood that the above general description and the detailed description below are only exemplary and cannot limit the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] In order to more clearly illustrate the technical solutions of the present invention or related technologies, the following briefly introduces the drawings required for use in the embodiments or related technical descriptions. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0038] Figure 1 A schematic diagram of a process flow of a road target detection method provided by the present invention;

[0039] Figure 2 A schematic diagram of a road surface area to be detected after rasterization provided by the present invention;

[0040] Figure 3 A schematic diagram of spatial voxels on a longitudinal section of a road surface provided by the present invention;

[0041] Figure 4 A side view schematic diagram of elevation estimation provided by the present invention;

[0042] Figure 5 A structural diagram of a specific implementation of the road target detection device provided by the present invention;

[0043] Figure 6 A structural diagram of a specific implementation of an electronic device provided by the present invention;

[0044] Figure 7 A schematic diagram of obstacle recognition for an unmanned vehicle provided by the present invention. DETAILED DESCRIPTION

[0045] In order to enable those skilled in the art to better understand the technical solution of the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. The terms "including" and "having" in the specification and the above-mentioned drawings, as well as any variations thereof, are intended to cover non-exclusive inclusions. The term "exemplary" means "used as an example, embodiment or illustrative". Any embodiment described herein as "exemplary" need not be interpreted as being superior or better than other embodiments.

[0046] It is understandable that potholes or low raised obstacles on the road can cause the autonomous vehicle to drive unsteadily and then bump, which will not only affect the comfort of the passengers, but also cause the vehicle to suddenly lose control and cause traffic accidents. If the vehicle is driven on potholes and low raised obstacles for a long time, it will damage the vehicle's suspension system, tires, and sensors, which will affect the vehicle's performance and safety. In addition, potholes on the road will damage the road material and cause the settlement of the roadbed. If they are not repaired in time, the potholes will easily expand and deepen, accelerating the damage to the road infrastructure and affecting the service life and carrying capacity of the road. In order to avoid the above problems, during the driving process of the vehicle, it will monitor in real time whether there are potholes or raised obstacle areas on the road ahead, remind the driver of the road detection situation, and issue a warning to slow down the speed of the car, or absorb and offset the impact and vibration caused by potholes / low raised obstacles on the road through the intelligent suspension system.

[0047] At present, the traditional way to monitor whether there are potholes or low raised obstacles on the road is manual visual inspection, where structural engineers and certified inspectors regularly check whether there are potholes or low raised obstacles on the road surface and report the location of such target areas. This method relies on the personal experience of inspectors and engineers, which not only cannot guarantee the accuracy of the detection results, but is also inefficient, costly, and dangerous. In order to solve the drawbacks of manual visual inspection, related technologies use computer vision technology to achieve more efficient and safer detection of road target areas. The target areas here include road obstacle areas that affect the normal operation of vehicles, including but not limited to potholes and low raised obstacle areas.

[0048] Related technologies use computer vision algorithms to achieve three-dimensional road imaging through 3D point cloud modeling, and then detect road obstacle areas based on segmentation methods to improve the accuracy of road target area detection. This method uses some algorithms, such as RANSAC (Random Sample Consensus) to improve the robustness of road point cloud modeling and DCNN (Deep Convolution Neural Network) to improve image classification accuracy. However, this type of method requires a small field of view and is based on a single-frame 3D road point cloud that is a plane or quadratic surface. The detection accuracy and scene adaptability are still difficult to achieve practicality, and it is difficult to obtain the width, depth, volume and other geometric characteristics of road pits with high precision. For example, a related technology realizes the detection of potholes on the road surface of unmanned vehicles through multi-sensor fusion. This method uses sound waves to scan the road surface ahead. When a pothole is detected, the camera is used to capture the road image, and then the pre-processed image is threshold segmented to obtain a binary image. The number of white pixels in the binary image is counted to determine whether there is water in the pothole. The image information captured by the camera is used to calculate the width of the pothole edge from the road edge on both sides, and the minimum torque for the unmanned vehicle to pass through the pothole is calculated. Finally, the relationship between the minimum torque and the maximum torque is used to determine whether the unmanned vehicle can pass safely. This method is greatly affected by the threshold value for image binarization, is difficult to adapt to various lighting conditions, and cannot provide information such as the depth of the pothole. Another related technology uses an RGB-D (Red Green Blue-Depth, a color image with depth information) depth camera to detect low raised obstacles and potholes on the road. This method uses the original point cloud of the RGB-D depth camera for preprocessing to obtain the corresponding plane point cloud, extracts the raised low raised obstacle point cloud data from the plane point cloud, and projects the extracted raised low raised obstacle point cloud onto the plane model parameter plane, downsamples the raised low raised obstacle point cloud on the model parameter plane, performs convex hull segmentation, transforms the convex hull into the / map (map) coordinate system and publishes it, merges the projected raised low raised obstacle point cloud into the ground plane point cloud to form a new plane point cloud, transforms the new plane point cloud into the base_footprint (robot body coordinate system) into a ground plane two-dimensional grid map, performs pseudo pothole filtering on the ground plane two-dimensional grid map, and extracts potholes using an edge extraction method. Since the effective working distance of RGB-D cameras is very limited, usually within 10 meters, and they are basically ineffective under strong outdoor light, the accuracy of the point cloud generated by the RGB-D camera is poor due to the influence of the projection density of the generator. The effectiveness of the actual use process needs to be verified, the usage scenarios are limited, and the final detection accuracy cannot be guaranteed.Another related technology is to detect potholes on the road based on neural networks and binocular stereo vision. This method uses a binocular camera installed in front of the vehicle to obtain left and right views of the road scene; and uses a target detection neural network to detect the left view. When a pothole is detected, the left and right views are stereo matched to obtain a depth map. The pothole position information predicted by the neural network is used to cut out the depth map of the pothole and the road surface around the pothole. Finally, a combination of RANSAC and genetic algorithms is used to fit the road plane around the pothole, so as to calculate the distance from the pothole point cloud to the fitting plane, and the distance is summed and averaged to obtain the pothole depth. The minimum circumscribed rectangle of the projection point is determined by projecting the pothole point cloud onto the fitting road plane, and the area of ​​the pothole is obtained by determining the area of ​​the minimum circumscribed rectangle. However, due to the existence of void points in the parallax, this method has a large estimation error in the depth direction, and it is difficult to obtain high-precision pothole depth and bump height, resulting in low detection accuracy of road target areas. Another related technology uses low-cost BEV (bird eye's view) perception to achieve road obstacle detection in self-driving systems, such as the LSS (Lift Splat Shoot) method, which sets a discrete depth distribution along the light for each pixel, turning the depth distribution estimation into a classification problem. For example, BEVheight (a visual three-dimensional object detection framework with height information designed based on roadside scenes) is combined with the LSS method to estimate the elevation distribution to improve the target recognition accuracy. However, for low raised obstacles and potholes on roads with a small range of elevation information, since it does not take into account the inclination and collapse of the road surface, the elevation information recognition accuracy of low raised obstacle areas and pothole areas is poor.

[0049] It can be seen that the relevant technology fails to accurately estimate the depth or elevation of the pothole / obstacle area, resulting in low detection accuracy of road potholes and low raised obstacles on the road. The present invention divides the road surface ahead into multiple grids, and each grid is provided with spatial voxels perpendicular to the road surface direction. The elevation information of the low raised obstacle / pothole area can be accurately calculated by the elevation difference between the road surface where the spatial voxels are located and the horizontal plane of the vehicle coordinate system. In order to avoid the situation where there is an angle between the road surface and the plane of the vehicle coordinate system, which leads to errors in the elevation calculation process of the low raised obstacle / pothole area, when calculating the elevation information of the low raised obstacle / pothole area, the inclination data of the road surface and the plane of the vehicle coordinate system are also added to further improve the elevation calculation accuracy of the low raised obstacle / pothole area. After introducing the technical solution of the present invention, various non-limiting embodiments of the present invention are described in detail below. In order to better illustrate the present invention, many specific details are given in the specific embodiments below. Those skilled in the art should understand that the present invention can also be implemented without these specific details. In some other instances, methods, means, components and circuits well known to those skilled in the art have not been described in detail in order to obscure the subject matter of the present invention.

[0050] First see Figure 1 , Figure 1 A schematic flow chart of a road surface object detection method provided in this embodiment, this embodiment may include the following contents:

[0051] S101: rasterizing a road surface area to be detected in front of the driving direction to generate a bird's-eye view grid area including a plurality of grids, each grid having spatial voxels in a horizontal direction perpendicular to the road surface.

[0052] In this embodiment, the road surface area to be detected is located in front of the vehicle, and is a space area extending forward from the position of the front of the vehicle. Its size can be flexibly selected according to actual conditions. The vehicle in this embodiment can be any vehicle with an automatic driving function, and is not limited to an automatic driving vehicle or an unmanned vehicle. The vehicles referred to in this embodiment are all vehicles with automatic driving functions, and the corresponding automatic driving vehicle coordinate system is the coordinate system where the vehicle with automatic driving functions is located. The vehicle can also be called an ego vehicle (the current automatic driving vehicle). Similarly, the ego vehicle refers to a vehicle with automatic driving functions. In the field of computer vision, a network structure is used to predict elevation information to replace monocular depth estimation, such as BEVheight and RoadBEV (road surface reconstruction in a bird's eye view). However, the error of the predicted elevation of this method can reach about 2 cm, and this error makes it impossible to accurately judge low raised obstacles and potholes. The present invention divides the road surface area to be detected into multiple grids under the target perspective, that is, rasterization processing. The target perspective can be, for example, bev (bird's eye view), such as Figure 2 As shown in the figure, the road area to be detected in front of the vehicle is divided into multiple grids. Since the laser radar installed on the vehicle will emit laser light during the vehicle's driving process, Figure 2 The black area in the figure represents the projection point of the laser radar point cloud on the road surface, and the white area represents no data. The grid size and the number of grids can be flexibly selected according to the actual situation, such as the amount of calculation, the target detection accuracy and the configuration parameters of the laser radar installed on the vehicle, which does not affect the implementation of the present invention. Among them, the target object is an object that the present invention needs to detect in the road surface area to be detected. The object has a small height in the direction perpendicular to the horizontal plane of the road surface, including but not limited to low raised obstacles and potholes. For the convenience of distinction, the road surface area to be detected after rasterization is defined as the bird's-eye view grid area.

[0053] Taking into account that there may be a certain angle between the road surface and the xy plane of the vehicle coordinate system, if this situation is not taken into account, the error will be introduced into the elevation calculation. For areas with smaller elevations, that is, obstacles such as low raised obstacles and potholes, misjudgment may be caused. Therefore, the present invention also provides each grid with a spatial voxel in a horizontal direction perpendicular to the road surface. The spatial voxel is a plurality of spatial cubes arranged in a horizontal direction perpendicular to the road surface. The spatial voxel can reflect the ups and downs of the ground, which is used in the subsequent accurate calculation of the elevation information of low raised obstacles and potholes, so as to be used for roadside perception, smart transportation, V2X (vehicle to everything, information exchange between vehicles and the outside world), smart security, collaborative perception and other fields for obstacle avoidance, environmental perception, etc.

[0054] S102: Determine the elevation difference between the road surface where the spatial voxels of each grid in the bird's-eye view grid area are located and the horizontal plane of the autonomous driving vehicle coordinate system based on the inclination data of the road surface area to be detected and the horizontal plane of the autonomous driving vehicle coordinate system and the position information of each grid.

[0055] In this embodiment, since the bird's-eye view grid area is located on the road surface, when constructing the local world coordinate system on the road surface , the spatial voxels are set in the Z-axis direction of the local world coordinate system in each grid. If the road surface is When the coordinate system has a certain inclination angle, if the height difference caused by the inclination angle is not considered, the subsequent height calculation of low obstacles and potholes will have errors, such as Figure 3 As shown, Figure 3 The left side is a schematic diagram of the space voxel, which sets 2t space cubes, and the right side is the longitudinal section of the road surface. is the elevation difference between the road surface where the spatial voxel is located and the coordinate system of the autonomous driving vehicle, that is, the horizontal plane of the vehicle coordinate system. When the road surface is horizontal or the pitch angle between the vehicle coordinate system and the road surface is 0, 0. However, since the road surface itself may be inclined or when the vehicle passes through a low raised obstacle or a pothole area, there is a certain pitch angle between the vehicle coordinate system and the ground. In order to more accurately obtain the elevation of the low raised obstacle or pothole area, Values ​​such as Figure 3 As shown in the right half of Take the elevation difference between the road surface where the spatial voxel is located and the horizontal plane of the vehicle coordinate system.

[0056] S103: Based on the horizontal plane elevation difference corresponding to each grid, determine whether there is a target object in the road surface area to be detected through the spatial position correspondence between the bird's-eye view grid area and the road surface area to be detected.

[0057] Among them, the spatial position correspondence between the bird's-eye view grid area and the road surface area to be detected refers to the position of each grid in the bird's-eye view grid area corresponding to the road surface area to be detected, that is, the correspondence between the bird's-eye view grid area and the road surface area to be detected. After the horizontal plane elevation difference of each grid is determined through the previous step, based on the position of the grid corresponding to the road surface area to be detected, it can be determined that the corresponding area of ​​the road surface area to be detected has a height in the direction perpendicular to the road surface, that is, the area belongs to the area where the target object to be detected by the present invention is located, such as low raised obstacles such as speed bumps, raised cracks, or potholes with puddles, deep pits, and shallow pits.

[0058] In the technical solution provided in this embodiment, the road ahead is divided into multiple grids, each grid is provided with a spatial voxel perpendicular to the road direction, and the elevation information of the low raised obstacle / pothole area can be accurately calculated by the elevation difference between the road surface where the spatial voxel is located and the horizontal plane of the vehicle coordinate system. In order to avoid the situation where there is an angle between the road surface and the plane of the vehicle coordinate system, which leads to errors in the elevation calculation process of the low raised obstacle / pothole area, when calculating the elevation information of the low raised obstacle / pothole area, the inclination data of the road surface and the plane of the vehicle coordinate system are also added to further improve the elevation calculation accuracy of the low raised obstacle / pothole area, avoid the misjudgment of the low raised obstacle / pothole area with a small elevation, and accurately estimate the depth or elevation of the pothole / obstacle area, and then identify whether there are such targets as low raised obstacles / pothole areas on the road surface, thereby effectively improving the detection accuracy of road potholes and raised low raised obstacles, which is not only conducive to ensuring the safe and stable operation of the autonomous driving vehicle, effectively improving the riding experience of the passengers, but also conducive to rapid road maintenance and extending the life of the road and its infrastructure.

[0059] The above embodiment does not limit how to perform rasterization processing on the road surface area to be detected. Based on the above embodiment, the present invention also provides an exemplary rasterization processing implementation method of the road surface area to be detected, which may include the following contents:

[0060] As an exemplary rasterization method, the number of laser radar lines installed on the autonomous driving vehicle can be obtained; the number of grids and the grid size are determined according to the number of laser radar lines, and the spatial size of the road surface area to be detected is determined according to the number of grids and the grid size; according to the number of grids and the grid size, the road surface area to be detected located within the corresponding spatial size in front of the driving direction is rasterized.

[0061] As another exemplary rasterization method, the resolution of the laser radar installed on the autonomous driving vehicle can be obtained; the number of grids and the grid size are determined according to the resolution of the laser radar, and the spatial size of the road surface area to be detected is determined according to the number of grids and the grid size; according to the number of grids and the grid size, the road surface area to be detected within the corresponding spatial size in front of the driving direction is rasterized.

[0062] In this embodiment, the number of grids in the bird's-eye view grid area and the size of the space corresponding to each grid on the road to be detected can be flexibly adjusted according to the number of laser radar lines or resolution. For example, the vehicle is equipped with a 32-line mechanical laser radar. Considering the accuracy and calculation amount, the number of grids in the bird's-eye view grid area is 2000×2000, and the space size corresponding to each grid is 0.01 meter×0.01 meter. The length and width of the space area corresponding to the bird's-eye view grid area are 20 meters and 20 meters respectively. For more than 20 meters, it is a far-field detection area. If the vehicle is equipped with a 128-line mechanical laser radar, the size of the space corresponding to the bird's-eye view grid area can be increased to 40 meters×30 meters. In order to improve the detection and recognition accuracy, the size of the space corresponding to each grid can be set to 0.01 meter×0.01 meter.

[0063] As can be seen from the above, this embodiment sets the parameters of the bird's-eye view grid area according to the laser radar parameters of the vehicle, which is beneficial to improving the measurement accuracy of road targets.

[0064] In order to further improve the recognition accuracy of road surface objects, based on the above embodiments, the present invention further provides a method for determining parameters of spatial voxels of a grid, which may include the following contents:

[0065] The maximum elevation of the target object in the road area to be detected is obtained; the maximum elevation of the target object is divided according to the preset accuracy threshold to determine the total number of spatial cubes contained in the spatial voxels of each grid; according to the maximum elevation of the target object, the total number of spatial cubes and the preset plane size of the spatial cube, the real physical space size corresponding to the spatial cube of the spatial voxel of each grid is determined.

[0066] In this embodiment, the number of space cubes contained in the space voxel is determined by the maximum elevation of the target object to be identified, such as a low raised obstacle / pothole area. For example, the total number of space cubes can be determined by the multiple of the number of segments of the maximum elevation of the target object, and the real physical space height corresponding to the space cube can be determined by the ratio of the maximum elevation of the target object to the total number of space cubes; the real physical space size corresponding to the space cube of the space voxel of each grid is determined according to the real physical space height and the grid size. For example, the space voxel may include A space cube, The maximum elevation of the low raised obstacle / pothole area to be identified is set. For example, the maximum elevation of the low raised obstacle / pothole area to be identified is 0.15 meters. In order to accurately identify the elevation information of each target object such as a low raised obstacle, the maximum elevation, that is, 15 centimeters, can be divided according to the preset accuracy threshold. The preset accuracy threshold can be selected as 1 cm, so 15 centimeters is divided into 1 cm, and the corresponding , the total number of space cubes contained in the space voxel is 2×15=30. For example, in order to facilitate data processing, the length and width of the space cube correspond to the same space size as the length and width of the unit grid, such as 0.01 meters × 0.01 meters. Correspondingly, the height of the space cube corresponds to the real space size Meters, so the real space size of each stereo frame is 0.01×0.01×0.005, in meters.

[0067] As can be seen from the above, in this embodiment, the spatial voxel parameters are set according to the maximum elevation of the target object. Since the spatial voxels are used to calculate the elevation information of the target object, it is beneficial to improve the recognition accuracy of the target object.

[0068] In the above embodiment, there is no limitation on how to determine the horizontal plane elevation difference. This embodiment also provides an exemplary method for determining the horizontal plane elevation difference, which may include the following contents:

[0069] According to the inclination degree of the road surface corresponding to each grid, the road surface area to be detected is divided into multiple types of road surface areas, and the inclination degree of the same type of road surface areas meets the preset same or similar inclination conditions; according to the inclination data of the horizontal plane of each road surface area and the autonomous driving vehicle coordinate system and the position information of each grid, the elevation difference between the road surface where the spatial voxels of each grid in the bird's-eye view grid area are located and the horizontal plane of the autonomous driving vehicle coordinate system is determined.

[0070] Considering that there may be one or more sections of the road surface area to be detected that have a certain inclination angle relative to the vehicle coordinate system, in order to further improve the accuracy of target recognition, this embodiment divides the areas with the same or similar inclination angles in the road surface area to be detected into one category. The so-called preset same or similar inclination conditions are the same or similar inclination angles, and then calculates the corresponding horizontal plane elevation difference for each type of road surface area. When dividing the road surface area to be detected, compared with the rasterization processing of step S101, the road surface area to be detected can be rasterized according to a larger grid size. A relatively large range can ensure that there are multiple point cloud data inside to determine the accuracy of subsequent clustering. For example, the grid size of S101 is 0.01 meters × 0.01 meters. In this embodiment, the road surface area to be detected can be rasterized according to a grid size of 0.2 meters × 0.2 meters.

[0071] Exemplarily, this embodiment also uses normal vector consistency as a criterion to realize the division of the road surface area to be detected into multiple types of road surface areas, which may include the following contents: fitting the three-dimensional space plane equation of the corresponding grid area according to the three-dimensional position information of the point cloud data inside each grid; determining the average normal vector in each grid based on the three-dimensional space plane equation of each grid; according to the clustering result of the average normal vector of each grid, the road surface area to be detected is segmented through the spatial position correspondence between the bird's-eye view grid area and the road surface area to be detected, so as to obtain multiple types of road surface areas. In this embodiment, after the dense point cloud of the road surface is segmented into multiple segments, the average normal vector calculation method of the local area is as follows: obtaining the point cloud data inside each grid; fitting the three-dimensional plane of the local area according to the three-dimensional coordinates of these point clouds, and further calculating its normal vector through the three-dimensional plane equation. Considering that the road surface corresponding to the current grid may have multiple road surface conditions with different inclination angles, or the vehicle passes through an obstacle, such as Figure 4 There is a phenomenon that the rear wheel of the vehicle presses on the obstacle, causing the vehicle to tilt. After the average normal vector of each grid is calculated, the average normal vector of each grid can be clustered by any clustering algorithm, such as mean shift and k-mean. The area is further segmented according to the clustering result to obtain multiple road surface areas, such as different uphill and downhill sections. The corresponding horizontal elevation difference is calculated for all road surface areas. For a road surface without an inclination, the corresponding horizontal elevation difference is 0.

[0072] Exemplarily, this embodiment also provides a method for determining position information: according to the three-dimensional space plane equation of each grid, the plane angle between the road area corresponding to each grid and the horizontal plane of the autonomous driving vehicle coordinate system is determined; the autonomous driving vehicle coordinate system adopts the laser radar coordinate system. The vertical distance between the origin of the laser radar coordinate system and the road surface is determined according to the laser radar installation position and the vehicle result parameters, and the position information of the intersection of the horizontal coordinate axis of the autonomous driving vehicle coordinate system and the road surface corresponding to each grid is determined according to the vertical distance of the coordinate system and the plane angle. Correspondingly, the calculation process of the horizontal plane elevation difference can be: for each grid, determine the position information of the intersection of the horizontal axis of the autonomous driving vehicle coordinate system and the road surface corresponding to the current grid on the horizontal axis; based on the plane angle, according to the position information of each grid on the horizontal axis and the position information of the corresponding intersection on the horizontal axis, calculate the horizontal plane elevation difference between the road surface where the spatial voxels of each grid in the bird's-eye view grid area are located and the autonomous driving vehicle coordinate system.

[0073] In this embodiment, after each road surface area is obtained, all point cloud data projected in the road surface area can be obtained. Each point includes xyz coordinates. Based on the principle of solid geometry, if a number of points in a three-dimensional plane are obtained, the least squares method can be used to perform plane fitting to obtain the plane equation. Here, all point cloud data coordinates are in the laser radar coordinate system, so the fitted road surface plane equation is also in the laser radar coordinate system. Based on the common knowledge of solid geometry, the plane angle between the road surface area corresponding to each grid and the horizontal plane of the autonomous driving vehicle coordinate system can be obtained. Exemplarily, this embodiment also provides a method for determining the plane angle: plane fitting processing is performed on the point cloud data in each grid to obtain the corresponding three-dimensional space plane equation; the angle between the normal vector of the three-dimensional space plane equation of each grid and the normal vector of the horizontal plane of the autonomous driving vehicle coordinate system is used as the plane angle between the road surface area of ​​the corresponding grid and the horizontal plane of the autonomous driving vehicle coordinate system. Figure 4 As shown, the laser radar coordinate system is usually As the ego-vehicle coordinate system, Figure 4 This is a side view, so there is no labeling. . Plane Angle The calculation process is: Get the kth road area, the point cloud data in the area , is the total number of point clouds in the kth road surface, and m represents the mth point cloud. Perform the least squares plane fitting to obtain the three-dimensional plane equation of the road surface , Ak, Bk, Ck, Dk represent the parameters of the plane equation, so its normal vector According to the principle of solid geometry, the road surface plane is Plane Angle Its normal vector is and The plane normal vector angle. According to the installation position of the laser radar and the vehicle structure parameters, the distance H from the origin of the laser radar coordinate system to the ground can be obtained. Based on the angle and H, according to the principle of trigonometric geometry, the horizontal coordinate of the intersection point x of the vehicle coordinate system X axis and the road surface can be calculated .

[0074] As an efficient calculation method, the elevation calculation formula can be stored locally in advance, and the elevation information of the spatial voxel corresponding to each grid (i, j) of the road section can be calculated by calling the elevation calculation formula. , the elevation calculation formula can be expressed as:

[0075] ;

[0076] Among them, i and j represent the row and column positions of the bird's-eye view grid area, such as (2, 2) means located at Figure 2 The grid in the second row and second column of the Bird's Eye View grid area in .

[0077] As can be seen from the above, this embodiment divides the road surface according to the inclination angle, and determines the road surface inclination angle in a simple manner, thereby effectively improving the determination accuracy of the elevation information of the target object.

[0078] It can be understood that the coordinate values ​​of the pothole area and the low raised obstacle in the coordinate system are different in positive and negative, so the type of the target obstacle can be further identified based on the positive and negative of the horizontal plane elevation difference. If the horizontal plane elevation difference corresponding to the current grid is a positive value, then there is a raised obstacle at the corresponding position of the current grid in the road surface area to be detected, and the height of the raised obstacle is the horizontal plane elevation difference; if the horizontal plane elevation difference corresponding to the current grid is a negative value, then there is a pothole area at the corresponding position of the current grid in the road surface area to be detected, and the depth of the pothole area is the horizontal plane elevation difference.

[0079] For an autonomous driving system, if it can accurately identify low raised obstacles / potholes, accurately assess their harmfulness, and make reasonable plans, it can effectively improve the safety and comfort of the autonomous driving system. Based on this, the present invention also provides an embodiment for accurately assessing the harmfulness of low raised obstacles and potholes. The vehicle in this embodiment takes an unmanned vehicle as an example. Of course, the unmanned vehicle can be replaced by any vehicle with an autonomous driving function, which will not affect the implementation of the present invention. This embodiment may include the following contents:

[0080] After the above embodiment obtains the elevation information of each target object, the maximum elevation is calculated based on the elevation information. , mean elevation , elevation mean square error , where max is used to identify the maximum elevation, mean is used to identify the mean of the elevation, and sd is used to identify the mean square error of the elevation. Then, the size information of the road area to be detected, the vehicle parameters of the unmanned vehicle, and the length of the target i are obtained. ,width ,area Angle between lane line , as the three-dimensional shape data of the target object. According to the three-dimensional shape data of the target object and the vehicle parameters, the hazard situation of the target object to the unmanned vehicle and the hazard situation of the driving state are determined, and the target object hazard assessment result is obtained; based on the target object hazard assessment result corresponding to each road convex obstacle and / or pothole area contained in the road surface area to be detected, the vehicle driving route of the unmanned vehicle is generated.

[0081] For example, the hazard of the target object to the unmanned vehicle may be chassis scratches, and the chassis scratch hazard assessment process may be: determine the chassis scratch probability of the target object to the unmanned vehicle based on the maximum elevation of the target object and the chassis height of the vehicle; determine the chassis scratch hazard of the target object to the unmanned vehicle based on the chassis scratch probability and vehicle parameters. The maximum elevation can be obtained from the three-dimensional shape data of the target object, and the chassis scratch probability is used to evaluate the chassis scratch probability, which can be obtained based on the maximum elevation of the target object and the chassis height of the vehicle. Vehicle chassis height The value of chassis is determined by the numerical relationship between the target and the vehicle chassis. If the maximum elevation of the target is greater than or equal to the height of the vehicle chassis, the chassis of the unmanned vehicle will inevitably be scratched. If the maximum elevation of the target is less than the height of the vehicle chassis, the probability of the chassis of the unmanned vehicle being scratched can be determined based on the ratio between the maximum elevation and the height of the vehicle chassis. The following relationship (1) can be used to be directly called to calculate the probability of the target object scratching the chassis of the unmanned vehicle . After determining the chassis scratch probability of the target object on the unmanned vehicle, the chassis scratch hazard can be determined based on the chassis scratch probability and maximum scratch time of the target object on the unmanned vehicle. Exemplarily, the vehicle length and vehicle movement speed are obtained from the vehicle parameters, and the maximum scratch time of the unmanned vehicle is determined based on the vehicle length and vehicle movement speed; the chassis scratch hazard value of the target object to the unmanned vehicle is obtained based on the product of the chassis scratch probability and the maximum scratch time. As an efficient method for determining the chassis scratch hazard, the following relationship (2) can be directly called to calculate the chassis scratch hazard value of the target object to the unmanned vehicle. Chassis scratch probability of the i-th target object The relationship is:

[0082] . (1)

[0083] The maximum scratch hazard of this embodiment for The product of the maximum scratch time and the maximum scratch length is the vehicle length , maximum scratch time Therefore, the calculation formula of chassis scratch hazard value can be expressed as:

[0084] . (2)

[0085] Exemplarily, the hazard of the target object to the unmanned vehicle may be a entrapment hazard, and the entrapment hazard assessment process may be: determining the entrapment probability of the unmanned vehicle when passing through the pothole area based on the angle between the driving direction of the unmanned vehicle and the pothole area, and the tire width in the vehicle parameters; determining the entrapment hazard of the unmanned vehicle based on the three-dimensional morphology data of the pothole area, vehicle parameters, entrapment probability and time passing through the entrapment area.

[0086] In this embodiment, there may be an angle between the vehicle's driving direction and the pothole area. In order to accurately determine whether the vehicle is trapped in the target object, a line segment AB can be defined. Line segment AB is the projection length of the pothole area in the tire width direction at the current tire-ground contact point. Correspondingly, the entrapment hazard is the numerical relationship between the length of line segment AB and the width of the vehicle tire. For example, the calculation process of the entrapment probability can be: according to the width of the pothole area , the angle between the horizontal plane of the pothole area and the lane line , determine the projection length of the pothole area in the tire width direction. For example, the ratio between the two is the projection length. If the projection length is less than or equal to the projection length, the driverless vehicle is trapped in the pothole area; if the tire width of the driverless vehicle is greater than the projection length, the probability of being trapped is determined according to the ratio of the projection length to the tire width. As an efficient way to determine the probability of being trapped, the following relationship (3) can be directly used to calculate the probability of being trapped in the pothole :

[0087] . (3)

[0088] When the probability of being trapped is determined, the hazard value of being trapped can be determined based on the probability of being trapped, the degree of being trapped, and the time of being trapped in the area. Among them, the degree of being trapped can be determined based on the numerical relationship between the average elevation of the target object and the height of the vehicle chassis. If the target object's elevation is greater than or equal to the vehicle chassis height, it must be trapped. Accordingly, the trapped hazard value can be set to the maximum value, such as 1. If the target object's elevation mean is less than the vehicle chassis height, the trapped hazard degree is determined based on the ratio of the target object's elevation mean to the vehicle chassis height. As an efficient way to determine the trapped hazard degree, the following relationship (4) can be directly used to calculate the trapped hazard degree of the unmanned vehicle: For example, the degree of danger of the driverless vehicle being trapped is determined based on the average elevation of the pothole area and the vehicle chassis height in the vehicle parameters; the angle between the driving direction of the driverless vehicle and the pothole area is determined based on the angle between the driving direction of the driverless vehicle and the pothole area. , length of the pothole area Determine the entrapment length of the unmanned vehicle. The entrapment length can be The vehicle speed according to the length of the trap and the vehicle parameters , determine the time when the driverless vehicle is trapped The entrapment time can be expressed as According to the probability of entrapment, the degree of entrapment and the time of entrapment, the entrapment hazard value of the unmanned vehicle is determined. As an efficient method for determining the entrapment hazard value, the following relationship (5) can be directly used to calculate the entrapment hazard value of the unmanned vehicle: :

[0089] ; (4)

[0090] . (5)

[0091] For example, the hazard of the target object to the unmanned vehicle may be a bump hazard, and the bump hazard assessment process may be as follows: according to the angle between the driving direction of the unmanned vehicle and the pothole area and the length of the target object, the length of the bump area of ​​the unmanned vehicle is determined; according to the length of the bump area and the vehicle running speed of the unmanned vehicle, the bump area time is determined; according to the mean square error of the elevation in the three-dimensional shape data of the target object, the bump area length is determined; and bumpy area time to determine the bumpy hazard value of the driverless vehicle during driving.

[0092] In this embodiment, when the vehicle is driving along a straight line on the road to be tested, the bumps experienced by the vehicle are based on the driving time of the vehicle in the low obstacle / pothole area and the degree of ups and downs of the low raised obstacle / pothole area itself. In this embodiment, the bump hazard can be determined based on the bump degree and the time it takes to pass through the bump area. For example, the bump hazard is defined as the product of the bump degree and the time it takes to pass through the bump area, where the length of the bump area can be expressed as , time to pass through the bumpy area As an efficient method for determining the turbulence hazard value, the following equation (6) can be directly used to calculate the turbulence hazard value of the unmanned vehicle: : . (6)

[0093] Exemplarily, in order to accurately assess the hazard of the target object to the unmanned vehicle, the present embodiment further combines the proportion of the target object in the entire road area for hazard assessment, which may include the following contents: according to the number of lanes and lane widths, the road surface to be inspected is divided into multiple areas to be analyzed, and the area ratio hazard level value is determined according to the size of the area to be analyzed, the number of lanes occupied by the target object and the three-dimensional morphology data.

[0094] In this embodiment, the area ratio hazard level is used to evaluate the area of ​​a low raised obstacle / pothole area in the road to be detected, that is, the area of ​​the target object i. The ratio of the area of ​​the entire road surface to be detected to the area where the road surface is located is harmful to the vehicle. For the convenience of statistics, the road driving area can be divided along the road direction. For the convenience of processing and improving the evaluation efficiency, the road surface to be detected can be divided at equal distances. Exemplarily, the road surface to be detected is divided into multiple areas to be analyzed at equal distances along the road direction according to the product of the preset area length, the number of lanes and the lane width as the area width; the road area area is determined according to the number of lanes occupied by the target object, the preset area length and the lane width; the area ratio hazard degree value is determined according to the ratio of the obstacle area area and the road area area in the three-dimensional morphology data of the target object. For example, the width of the road surface to be detected is: , n is the number of lanes, is the lane width and the length is , such as 2 meters. As an efficient way to determine the area ratio hazard value, the following relationship (7) can be directly used to calculate the area ratio hazard value of the unmanned vehicle: :

[0095] ; (7)

[0096] Here, m is the number of lanes occupied by the target object, such as m is 2.

[0097] The above embodiment is to perform a quantitative assessment on the hazard situation of a single target object. This embodiment further determines the hazard level of a single target object based on the above quantitative assessment value, which may include the following contents:

[0098] Obtain the chassis scratch hazard value, entrapment hazard value, bump hazard value and area ratio hazard value of the target object; determine the single obstacle vehicle hazard value based on the chassis scratch hazard value, entrapment hazard value and bump hazard value of the target object; determine the target object hazard assessment result based on the numerical relationship between the product of the single obstacle vehicle hazard value and the area ratio hazard value and a first preset hazard threshold.

[0099] In this embodiment, the first preset hazard threshold is a preset empirical value, and its numerical relationship with the target object hazard assessment value is used to determine the hazard level of a single target object. The target object hazard assessment value is determined based on the chassis scratch hazard value, the sinking hazard value, the bump hazard value, and the area ratio hazard level value of the target object. As an efficient method for determining the target object hazard assessment value, the following relationship (8) can be directly called to calculate the target object hazard assessment value of the unmanned vehicle:

[0100] . (8)

[0101] Single target The value range is , taking the first preset hazard threshold of 0.5 as an example, the following relationship shows that when the target object hazard assessment value is greater than the first preset hazard threshold, the hazard level is high hazard; when the target object hazard assessment value is less than or equal to the first preset hazard threshold, the hazard level is low hazard:

[0102] . (9)

[0103] Of course, if the degree of harm to the target object is to be distinguished more finely, multiple harm thresholds can be set, and the harm degree corresponding to each harm threshold interval can be defined. For example, the first preset harm threshold is less than the third preset harm threshold. If it is greater than the third preset harm threshold, it is considered to be the maximum harm. If it is between the third preset harm threshold and the first preset harm threshold, it is considered to be medium harm. If it is less than the first preset harm threshold, it is considered to be low harm. After determining the harm degree of the target object, the target harm assessment result can be jointly generated based on the target harm assessment value and the target harm degree of the target object, so as to more clearly and accurately assess the harm of the target object.

[0104] The above-mentioned embodiment is to quantitatively evaluate the hazard situation of a single target object. When there are multiple target objects on the road surface to be detected, the process of generating the final target object evaluation result based on the hazard situation of the multiple target objects can be: according to the target object hazard evaluation results corresponding to each road bulge obstacle and each pothole area contained in the road surface to be detected, the target object hazard average value of the road surface to be detected is calculated, and the hazard average value of each type of hazard is counted; according to the numerical relationship between the target object hazard average value and the second preset hazard threshold, the target object hazard degree assessment result is determined; the maximum value is selected from each hazard mean value, and the hazard type corresponding to the maximum value is used as the target object hazard category of the road surface to be detected; according to the target object hazard degree assessment result and the target object hazard category, the obstacle avoidance area of ​​the road surface to be detected is determined, and the vehicle driving route of the unmanned vehicle is generated based on the obstacle avoidance area.

[0105] In this embodiment, the road surface to be inspected includes multiple targets, which are cracks, shallow pits, deep pits, and low obstacles. When the target hazard assessment value of each target object is obtained, the following relationship (10) can be directly called to calculate the average target hazard value of the entire road surface to be inspected:

[0106] . (10)

[0107] In this embodiment, the second preset hazard threshold is a preset empirical value, and its numerical relationship with the target object hazard assessment value is used to determine the hazard level of all targets. The value range is , accordingly, all the targets The value range is Taking the second preset hazard threshold of 0.5 as an example, the following relationship (11) shows that when the target object hazard average value is greater than the second preset hazard threshold, the hazard level is high hazard; when the target object hazard average value is less than or equal to the first preset hazard threshold, the hazard level is low hazard:

[0108] . (11)

[0109] Of course, if you want to distinguish the degree of harm to the target object more carefully, you can also set multiple harm thresholds and define the degree of harm corresponding to each harm threshold interval. For example, if the second preset harm threshold is less than the fourth preset harm threshold, if it is greater than the fourth preset harm threshold, it is considered to be the maximum harm. If it is between the fourth preset harm threshold and the second preset harm threshold, it is considered to be a medium harm. If it is less than the second preset harm threshold, it is considered to be a low harm. After determining the degree of harm to the target object, the main harm types can also be evaluated. For this area, the average values ​​of scratch harm, entrapment harm, area ratio harm, and bump harm are calculated to obtain the mean value of scratch harm. , falling into the danger mean , area proportion hazard average , mean value of turbulence hazard , and then take the hazard category corresponding to the maximum value of the four hazard means as the main hazard category of the area, that is, the hazard category of the target object , as shown in the following formula (12):

[0110] . (12)

[0111] From the above, it can be seen that the present invention evaluates the hazards of the identified target objects, and can more accurately determine the obstacle avoidance area of ​​the road to be inspected according to the target object hazard level assessment result and the target object hazard category, and generate a safer and high-capacity unmanned vehicle driving route based on the obstacle avoidance area.

[0112] It should be noted that there is no strict order of execution between the steps in the present invention. As long as they comply with the logical order, these steps can be executed simultaneously or in a certain preset order. Figure 1 This is just a schematic and does not mean that this is the only execution order.

[0113] The present invention also provides a corresponding device for the road surface object detection method, which further makes the method more practical. Among them, the device can be described from the perspective of functional modules and hardware. The road surface object detection device provided by the present invention is introduced below. The device is used to implement the road surface object detection method provided by the present invention. In this embodiment, the road surface object detection device may include or be divided into one or more program modules, and the one or more program modules are stored in a storage medium and executed by one or more processors to complete the road surface object detection method disclosed in Example 1. The program module referred to in this embodiment refers to a series of computer program instruction segments that can complete specific functions, which is more suitable for describing the execution process of the road surface object detection device in the storage medium than the program itself. The following description will specifically introduce the functions of each program module of this embodiment. The road surface object detection device described below and the road surface object detection method described above can be referenced to each other.

[0114] From the perspective of functional modules, see Figure 5 , Figure 5 The structure diagram of the road surface object detection device provided in this embodiment under a specific implementation manner, the device may include:

[0115] The rasterization processing module 501 is used to perform rasterization processing on the road surface area to be detected in front of the driving direction, and generate a bird's-eye view grid area including multiple grids, each grid having spatial voxels in a horizontal direction perpendicular to the road surface.

[0116] The elevation difference determination module 502 is used to determine the elevation difference between the road surface where the spatial voxels of each grid in the bird's-eye view grid area are located and the horizontal plane of the autonomous driving vehicle coordinate system based on the inclination data of the horizontal plane of the road surface area to be detected and the autonomous driving vehicle coordinate system and the position information of each grid.

[0117] The target recognition module 503 is used to determine whether there is a target object in the road surface area to be detected based on the horizontal plane elevation difference corresponding to each grid and the spatial position correspondence between the bird's-eye view grid area and the road surface area to be detected.

[0118] Exemplarily, in some implementations of the present embodiment, the elevation difference determination module 502 may also be used to: divide the road surface area to be detected into multiple types of road surface areas according to the inclination of the road surface corresponding to each grid, and the inclination of the same type of road surface areas satisfies preset same or similar inclination conditions; determine the elevation difference between the road surface where the spatial voxels of each grid in the bird's-eye view grid area are located and the horizontal plane of the autonomous driving vehicle coordinate system according to the inclination data of the horizontal plane of each road surface area and the autonomous driving vehicle coordinate system and the position information of each grid.

[0119] As an exemplary implementation of the above embodiment, the elevation difference determination module 502 can also be further used to: fit the three-dimensional space plane equation of the corresponding grid area according to the three-dimensional position information of the point cloud data inside each grid; determine the average normal vector in each grid based on the three-dimensional space plane equation of each grid; and segment the road surface area to be detected according to the spatial position correspondence between the grid area of ​​the bird's-eye view and the road surface area to be detected according to the clustering result of the average normal vector of each grid, so as to obtain multiple types of road surface areas. As another exemplary implementation of the above embodiment, the elevation difference determination module 502 can also be further used to: determine the plane angle between the road surface area corresponding to each grid and the horizontal plane of the autonomous driving vehicle coordinate system according to the three-dimensional space plane equation of each grid; the autonomous driving vehicle coordinate system adopts the laser radar coordinate system; determine the coordinate system vertical distance between the laser radar coordinate system origin and the road surface according to the laser radar installation position and the vehicle result parameters, and determine the position information of the intersection of the horizontal coordinate axis of the autonomous driving vehicle coordinate system and the road surface corresponding to each grid according to the coordinate system vertical distance and the plane angle. As an exemplary implementation of the above embodiment, the above elevation difference determination module 502 can be further used to: for each grid, determine the position information of the intersection point of the horizontal axis of the autonomous driving vehicle coordinate system and the road surface corresponding to the current grid on the horizontal axis; based on the plane angle, according to the position information of each grid on the horizontal axis and the position information of the corresponding intersection on the horizontal axis, calculate the horizontal plane elevation difference between the road surface where the spatial voxels of each grid in the bird's-eye view grid area are located and the autonomous driving vehicle coordinate system.

[0120] As another exemplary implementation of the above embodiment, the above elevation difference determination module 502 can be further used to: perform plane fitting processing on the point cloud data in each grid to obtain the corresponding three-dimensional space plane equation; use the angle between the normal vector of the three-dimensional space plane equation of each grid and the normal vector of the horizontal plane of the autonomous driving vehicle coordinate system as the plane angle between the road surface area of ​​the corresponding grid and the horizontal plane of the autonomous driving vehicle coordinate system.

[0121] Exemplarily, in some other implementations of this embodiment, the above-mentioned target recognition module 503 can also be used for: if the horizontal plane elevation difference corresponding to the current grid is a positive value, then there is a raised obstacle at the corresponding position of the current grid in the road surface area to be detected, and the height of the raised obstacle is the horizontal plane elevation difference; if the horizontal plane elevation difference corresponding to the current grid is a negative value, then there is a pothole area at the corresponding position of the current grid in the road surface area to be detected, and the depth of the pothole area is the horizontal plane elevation difference.

[0122] Illustratively, in some other implementations of this embodiment, the above-mentioned rasterization processing module 501 can also be used to: obtain the number of laser radar lines installed on the autonomous driving vehicle; determine the number of grids and the grid size according to the number of laser radar lines, and determine the spatial size of the road surface area to be detected according to the number of grids and the grid size; and perform rasterization processing on the road surface area to be detected within the corresponding spatial size in front of the driving direction according to the number of grids and the grid size.

[0123] Illustratively, in some other implementations of this embodiment, the above-mentioned rasterization processing module 501 can also be used to: obtain the resolution of the laser radar installed on the autonomous driving vehicle; determine the number of grids and the grid size according to the resolution of the laser radar, and determine the spatial size of the road surface area to be detected according to the number of grids and the grid size; and perform rasterization processing on the road surface area to be detected within the corresponding spatial size in front of the driving direction according to the number of grids and the grid size.

[0124] Illustratively, in some other implementations of this embodiment, the above-mentioned rasterization processing module 501 can also be used to: obtain the maximum elevation of the target object in the road area to be detected; divide the maximum elevation of the target object according to a preset accuracy threshold to determine the total number of spatial cubes contained in the spatial voxels of each grid; determine the real physical space size corresponding to the spatial cube of the spatial voxel of each grid based on the maximum elevation of the target object, the total number of spatial cubes and the preset plane size of the spatial cube.

[0125] As an exemplary implementation of the above embodiment, the above rasterization processing module 501 can be further used for: taking the total number of spatial cubes according to the multiples of the number of divided segments of the maximum elevation of the target object, and taking the real physical space height corresponding to the spatial cubes according to the ratio of the maximum elevation of the target object and the total number of spatial cubes; and determining the real physical space size corresponding to the spatial cube of the spatial voxels of each grid according to the real physical space height and the grid size.

[0126] The functions of the functional modules of the road surface object detection device described in this embodiment can be specifically implemented according to the method in the above method embodiment. The specific implementation process can refer to the relevant description of the above method embodiment, which will not be repeated here.

[0127] The road surface object detection device mentioned above is described from the perspective of functional modules. Furthermore, the present invention also provides an electronic device, which is described from the perspective of hardware. Figure 6 The following is a schematic diagram of the structure of an electronic device provided by an embodiment of the present invention in one implementation manner. Figure 6 As shown, the electronic device includes a memory 60 for storing a computer program; and a processor 61 for implementing the steps of the road surface target detection method mentioned in any of the above embodiments when executing the computer program.

[0128] Among them, the processor 61 may include one or more processing cores, such as a 4-core processor, an 8-core processor, and the processor 61 may also be a controller, a microcontroller, a microprocessor or other data processing chip. The processor 61 may be implemented in at least one hardware form of DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). The processor 61 may also include a main processor and a coprocessor. The main processor is a processor for processing data in the awake state, also known as a CPU (Central Processing Unit); the coprocessor is a low-power processor for processing data in the standby state. In some embodiments, the processor 61 may be integrated with a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor 61 may also include an AI (Artificial Intelligence) processor, which is used to process computing operations related to machine learning.

[0129] The memory 60 may include one or more computer non-volatile storage media, which may be non-transitory. The memory 60 may also include high-speed random access memory and non-volatile memory, such as one or more disk storage devices and flash memory storage devices. In some embodiments, the memory 60 may be an internal storage unit of an electronic device, such as a hard disk of a server. In other embodiments, the memory 60 may also be an external storage device of an electronic device, such as a plug-in hard disk equipped on a server, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. Further, the memory 60 may also include both an internal storage unit of an electronic device and an external storage device. The memory 60 may not only be used to store application software and various types of data installed in the electronic device, such as: the code of the program in the process of executing the road target detection method, but also be used to temporarily store data that has been output or is to be output. In this embodiment, the memory 60 is at least used to store the following computer program 601, wherein, after the computer program is loaded and executed by the processor 61, the relevant steps of the road target detection method disclosed in any of the aforementioned embodiments can be implemented. In addition, the resources stored in the memory 60 may also include an operating system 602 and data 603, etc., and the storage method may be temporary storage or permanent storage. The operating system 602 may include Windows, Unix, Linux, etc. The data 603 may include but is not limited to data corresponding to the road target detection result, etc.

[0130] In some embodiments, the electronic device may further include a display screen 62, an input / output interface 63, a communication interface 64 or a network interface, a power supply 65 and a communication bus 66. Among them, the display screen 62 and the input / output interface 63, such as a keyboard, belong to the user interface, and exemplary user interfaces may also include standard wired interfaces, wireless interfaces, etc. Optionally, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, and an OLED (Organic Light-Emitting Diode) touch device, etc. The display may also be appropriately referred to as a display screen or a display unit, which is used to display information processed in the electronic device and to display a visual user interface. The communication interface 64 may exemplarily include a wired interface and / or a wireless interface, such as a WI-FI interface, a Bluetooth interface, etc., which are generally used to establish a communication connection between the electronic device and other electronic devices. The communication bus 66 may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 6 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0131] Those skilled in the art will understand that Figure 6 The structure shown in the figure does not constitute a limitation on the electronic device, and may include more or fewer components than those shown in the figure, for example, it may also include a sensor 67 for realizing various functions.

[0132] It is understandable that if the road target detection method in the above embodiment is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a non-volatile storage medium. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the relevant technology or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium to execute all or part of the steps of the methods of each embodiment of the present invention. The aforementioned storage medium includes but is not limited to: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), electrically erasable programmable ROM, register, hard disk, multimedia card, card-type memory (such as SD or DX memory, etc.), magnetic memory, removable disk, CD-ROM, magnetic disk or optical disk, etc. Various media that can store program codes. Based on this, the present invention also provides a non-volatile storage medium on which a computer program is stored, and when the computer program is executed by a processor, the steps of the road target detection method recorded in any of the above embodiments are executed.

[0133] It is understandable that if the road object detection method in the above embodiment is implemented in the form of a software functional unit and sold or used as an independent product, the computer software product does not need to be stored in a physical storage medium. For example, it can be directly transmitted to a computer or other device with information processing capabilities through a wired network or a wireless network to perform all or part of the steps of the methods of each embodiment of the present invention. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the relevant technology or all or part of the technical solution can be embodied in the form of a software product. Based on this, the present invention also provides a computer program product that stores a computer program. When the computer program is executed by a processor, the steps of the road object detection method recorded in any of the above embodiments are performed.

[0134] Finally, the present invention also provides an unmanned vehicle that fully reuses its own sensors, such as cameras and lidars, and includes an automatic driving perception system. The automatic driving perception system is also used to implement the steps of the road target detection method recorded in any of the previous embodiments when executing the computer program stored in the memory. By rasterizing the area to be detected and setting spatial voxels for each grid, considering that there may be one or more sections of the road surface in the near-field area with a certain inclination angle relative to the vehicle coordinate system, the corresponding elevation information of the spatial voxels corresponding to each grid is calculated respectively. In this way, the unmanned vehicle can accurately identify low raised obstacles such as speed bumps and cracks, deep pits, shallow pits or puddles as early as possible in the automatic driving scenario. Figure 7As shown in the figure, and making reasonable plans, the safety and comfort of the self-driving system can be effectively improved.

[0135] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the hardware disclosed in the embodiments, including electronic devices, non-volatile storage media, computer program products and unmanned vehicles, since they correspond to the methods disclosed in the embodiments, the description is relatively simple, and the relevant parts can be referred to the method part description.

[0136] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0137] The above is a detailed introduction to a road target detection method, electronic device, non-volatile storage medium, computer program product and unmanned vehicle provided by the present invention. Specific examples are used herein to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea. It should be pointed out that based on the embodiments of the present invention, for ordinary technicians in this technical field, all other embodiments obtained without creative work are within the scope of protection of the present invention. Without departing from the principles of the present invention, the present invention can also be improved and modified in a number of ways, and these improvements and modifications also fall within the scope of protection of the present invention.

Claims

1. A road target detection method, characterized in that: include: Rasterizing the road surface area to be detected in front of the driving direction to generate a bird's-eye view grid area including a plurality of grids, each grid having spatial voxels in a horizontal direction perpendicular to the road surface; Determine the elevation difference between the road surface where the spatial voxels of each grid in the bird's-eye view grid area are located and the horizontal plane of the autonomous driving vehicle coordinate system according to the inclination data of the road surface area to be detected and the horizontal plane of the autonomous driving vehicle coordinate system and the position information of each grid; Based on the horizontal plane elevation difference corresponding to each grid, determining whether there is a target object in the road surface area to be detected through the spatial position correspondence between the bird's-eye view grid area and the road surface area to be detected; The step of determining the elevation difference between the road surface where the spatial voxels of each grid in the bird's-eye view grid area are located and the horizontal plane of the autonomous driving vehicle coordinate system includes: According to the three-dimensional space plane equation of each grid, determine the plane angle between the road surface area corresponding to each grid and the horizontal plane of the autonomous driving vehicle coordinate system; the autonomous driving vehicle coordinate system adopts the laser radar coordinate system; Determine the coordinate system vertical distance between the origin of the laser radar coordinate system and the road surface according to the laser radar installation position and the vehicle result parameters, and determine the position information of the intersection of the horizontal coordinate axis of the autonomous driving vehicle coordinate system and the road surface corresponding to each grid according to the coordinate system vertical distance and the plane angle; For each grid, determining the position information on the horizontal axis of the intersection of the horizontal axis of the autonomous driving vehicle coordinate system and the road surface corresponding to the current grid; Based on the plane angle, according to the position information of each grid on the horizontal axis and the position information of the corresponding intersection on the horizontal axis, the elevation difference between the road surface where the spatial voxels of each grid in the bird's-eye view grid area are located and the horizontal plane elevation system of the autonomous driving vehicle coordinate system is calculated.

2. The road surface target detection method according to claim 1, characterized in that: Determining the elevation difference between the road surface where the spatial voxels of each grid in the bird's-eye view grid area are located and the horizontal plane of the autonomous driving vehicle coordinate system according to the inclination data of the road surface area to be detected and the horizontal plane of the autonomous driving vehicle coordinate system and the position information of each grid, including: According to the inclination degree of the road surface corresponding to each grid, the road surface area to be detected is divided into multiple types of road surface areas, and the inclination degree of the road surface area of ​​the same type meets the preset same similar inclination condition; Based on the inclination data of each road surface area and the horizontal plane of the autonomous driving vehicle coordinate system and the position information of each grid, the elevation difference between the road surface where the spatial voxels of each grid in the bird's-eye view grid area are located and the horizontal plane of the autonomous driving vehicle coordinate system is determined.

3. The road surface target detection method according to claim 2, characterized in that: According to the inclination of the road surface corresponding to each grid, the road surface area to be detected is divided into multiple types of road surface areas, including: According to the three-dimensional position information of the point cloud data inside each grid, the three-dimensional space plane equation of the corresponding grid area is fitted; Based on the three-dimensional space plane equation of each grid, determine the average normal vector in each grid; According to the clustering result of the average normal vector of each grid, the road surface area to be detected is segmented through the spatial position correspondence between the bird's-eye view grid area and the road surface area to be detected, so as to obtain multiple types of road surface areas.

4. The road surface object detection method according to claim 1, characterized in that: According to the three-dimensional space plane equation of each grid, the plane angle between the road area corresponding to each grid and the horizontal plane of the autonomous driving vehicle coordinate system is determined, including: Perform plane fitting processing on the point cloud data in each grid to obtain the corresponding three-dimensional space plane equation; The angle between the normal vector of the three-dimensional space plane equation of each grid and the normal vector of the horizontal plane of the autonomous driving vehicle coordinate system is used as the plane angle between the road surface area of ​​the corresponding grid and the horizontal plane of the autonomous driving vehicle coordinate system.

5. The road surface object detection method according to claim 1, characterized in that: Based on the horizontal plane elevation difference corresponding to each grid, determining whether there is a target object in the road surface area to be detected through the spatial position correspondence between the bird's-eye view grid area and the road surface area to be detected, including: If the horizontal plane elevation difference corresponding to the current grid is a positive value, then there is a raised obstacle at the corresponding position of the current grid in the road area to be detected, and the height of the raised obstacle is the horizontal plane elevation difference; If the horizontal plane elevation difference corresponding to the current grid is a negative value, then there is a pothole area at the corresponding position of the current grid in the road surface area to be detected, and the depth of the pothole area is the horizontal plane elevation difference.

6. The road surface object detection method according to claim 1, characterized in that: The road surface area to be detected in front of the driving direction is rasterized to generate a bird's-eye view grid area including multiple grids, each grid having spatial voxels in a horizontal direction perpendicular to the road surface, including: Get the number of laser radar lines installed on the autonomous vehicle; Determine the number of grids and the grid size according to the number of laser radar lines, and determine the spatial size of the road surface area to be detected according to the number of grids and the grid size; According to the number of grids and the grid size, a grid processing is performed on the road surface area to be detected within a corresponding space size ahead in the driving direction.

7. The road surface object detection method according to claim 1, characterized in that: The road surface area to be detected in front of the driving direction is rasterized to generate a bird's-eye view grid area including multiple grids, each grid having spatial voxels in a horizontal direction perpendicular to the road surface, including: Get the resolution of the lidar installed on the autonomous vehicle; Determine the number and size of grids according to the resolution of the laser radar, and determine the spatial size of the road surface area to be detected according to the number and size of grids; According to the number of grids and the grid size, a grid processing is performed on the road surface area to be detected within a corresponding space size ahead in the driving direction.

8. The road surface object detection method according to any one of claims 1 to 7, characterized in that: Before rasterizing the road surface area to be detected in front of the driving direction, the following steps are also included: Obtaining the maximum elevation of the target object in the road area to be detected; Dividing the maximum elevation of the target object according to a preset accuracy threshold to determine the total number of space cubes contained in the space voxels of each grid; The real physical space size corresponding to the space cube of the space voxel of each grid is determined according to the maximum elevation of the target object, the total number of space cubes and the preset plane size of the space cube.

9. The road surface object detection method according to claim 7, characterized in that: According to the maximum elevation of the target object, the total number of space cubes and the preset plane size of the space cube, the real physical space size corresponding to the space cube of the space voxel of each grid is determined, including: The total number of space cubes is taken as the multiple of the number of segments of the maximum elevation of the target object, and the real physical space height corresponding to the space cube is taken as the ratio of the maximum elevation of the target object to the total number of space cubes; The real physical space size corresponding to the space cube of the space voxels of each grid is determined according to the real physical space height and the grid size.

10. An electronic device, characterized in that: The method comprises a processor and a memory, wherein the processor is used to implement the steps of the road surface target detection method as claimed in any one of claims 1 to 9 when executing the computer program stored in the memory.

11. A non-volatile storage medium, characterized in that: The non-volatile storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the road surface target detection method according to any one of claims 1 to 9 are implemented.

12. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instructions are executed by a processor, the steps of the road surface object detection method as described in any one of claims 1 to 9 are implemented.

13. An unmanned vehicle, characterized in that: It includes an autonomous driving perception system, which is used to implement the steps of the road target detection method as described in any one of claims 1 to 9 when executing a computer program stored in a memory.

Images