Road environment state perception method, equipment, medium, program product and vehicle

Abstract

The present invention discloses a method, device, medium, program product and vehicle for perceiving a road environment state, which are applied in the field of autonomous driving technology. The method comprises dividing the dense point cloud data of the road surface in the target area ahead of the vehicle into near-field dense point cloud data and far-field dense point cloud data. The road surface flatness of the far-field dense point cloud data is analyzed to determine suspected obstacles. Whether the suspected obstacle is a target obstacle is verified according to the near-field dense point cloud data, and when the suspected obstacle is verified to be a target obstacle, the three-dimensional shape information of the target obstacle is determined according to the matching results of the point cloud data at adjacent moments; the road environment state perception result of the vehicle is generated according to the three-dimensional shape information of each target obstacle obtained during the vehicle's driving. The present invention can solve the problem that the related technology cannot accurately identify road obstacles, and can accurately identify road pothole areas and low raised obstacle areas, and assist the vehicle in more refined identification of the road environment state.

Description

Technical Field

[0001] The present invention relates to the field of autonomous driving technology, and in particular to a road environment state perception method, electronic equipment, non-volatile storage medium, computer program product and unmanned vehicle. Background Art

[0002] With the rapid development of artificial intelligence and Internet of Things technologies, autonomous driving technology has also developed accordingly. The smoothness of the road not only affects the normal operation of autonomous vehicles and the comfort of passengers, but also affects the life of the road and its infrastructure. The autonomous driving perception system not only needs to pay attention to typical obstacles such as pedestrians and vehicles, but also needs to monitor whether there are potholes and low raised obstacles on the road.

[0003] Related technologies use OCC (Occupancy Network) to predict the probability of each voxel grid being occupied and the target categories that may be included, thereby realizing the perception of the entire road scene. However, since the cost of labeling data for low-rise obstacles and potholes is too high, and the accuracy of the OCC algorithm cannot reach the recognition accuracy of potholes and low-rise obstacles, it is impossible to accurately determine whether there are potholes and low-rise obstacles on the road.

[0004] In view of this, accurately identifying road potholes and low raised obstacle areas on the road, and assisting vehicles to more finely identify road environment conditions, are technical problems that technical personnel in this field need to solve.

[0005] It should be noted that the information disclosed in the above background technology section is only used to enhance the understanding of the background of the present application, and therefore may include information that does not constitute the prior art known to ordinary technicians in the field. Summary of the invention

[0006] The present invention provides a road environment state perception method, electronic device, non-volatile storage medium, computer program product and unmanned vehicle, which can accurately identify road pothole areas and road low raised obstacle areas, and assist vehicles to more finely identify road environment states.

[0007] In order to solve the above technical problems, the present invention provides the following technical solutions:

[0008] The present invention provides a method for perceiving a road environment state, comprising:

[0009] Acquire dense point cloud data of a road surface in a target area ahead of a vehicle, and divide the dense point cloud data of the road surface into near-field dense point cloud data and far-field dense point cloud data according to the distance between the point cloud data and the vehicle; determine suspected obstacles in the target area by analyzing the road surface flatness of the far-field dense point cloud data; verify whether the suspected obstacle is a target obstacle based on the near-field dense point cloud data, and when the suspected obstacle is verified to be a target obstacle, determine the three-dimensional shape information of the target obstacle based on the point cloud data at adjacent moments and the matching results thereof; generate a road environment state perception result of the vehicle based on the three-dimensional shape information of each target obstacle acquired during the vehicle's driving; wherein the dense point cloud data of the road surface is obtained by removing the point cloud data of the road surface area that is not the target area in the splicing result of the point cloud data at the current moment and the splicing point cloud data at the previous moment.

[0010] In a first exemplary embodiment, the determining of suspected obstacles by analyzing the road surface flatness of the far-field dense point cloud data includes:

[0011] The first target sub-region corresponding to the far-field dense point cloud data is rasterized according to a first preset grid size, and the first target sub-region is segmented according to the clustering result of the average normal vector in each grid of the first target sub-region to obtain multiple types of first road surface regions; for each first road surface region, the current road surface region is rasterized according to a second preset grid size, and a road surface flatness region that meets a preset flatness is determined according to the direction of the average normal vector of each grid of the current first road surface region, and the remaining region of the current first road surface region after excluding the road surface flatness region is merged based on a spatial distance relationship to serve as the spatial position of the suspected obstacle; wherein the target region includes a first target sub-region and a second target sub-region, and the maximum distance between the second target sub-region and the vehicle along the vehicle driving direction is less than the minimum distance between the first target sub-region and the vehicle, the point cloud data corresponding to the first target sub-region is the far-field dense point cloud data, the point cloud data corresponding to the second target sub-region is the near-field dense point cloud data, and the second preset grid size is less than the first preset grid size.

[0012] In a second exemplary embodiment, the step of determining a road surface smoothing area that meets a preset smoothness according to the direction of the average normal vector of each grid in the current first road surface area includes:

[0013] According to the three-dimensional position information of the point cloud data inside each grid of the current first road surface area, the three-dimensional space plane equation of the corresponding grid is fitted; based on the three-dimensional space plane equation of each grid of the current first road surface area, the average normal vector in each grid of the current first road surface area is calculated; according to the direction of the average normal vector, at intervals of a preset angle threshold, a vote is performed on whether each average normal vector is a road surface normal vector, and the target average normal vector with the most votes is used as the road surface normal vector, and the grid corresponding to the target average normal vector is used as a road surface flattening area.

[0014] In a third exemplary embodiment, the step of acquiring dense point cloud data of a road surface in a target area ahead of the vehicle includes:

[0015] The vehicle motion data and vehicle motion posture data at the current moment are obtained as the posture point cloud data, and the initial posture change information is determined according to the posture point cloud data and the spliced ​​point cloud data at the previous moment of the current moment; the initial posture change information is used as the initial value of the iterative calculation, and the point cloud data at the current moment and the spliced ​​point cloud data at the previous moment are performed with point cloud matching calculation to determine the rotation change matrix and the translation vector; the spliced ​​point cloud data at the previous moment is transformed according to the rotation change matrix and the translation vector, and the transformed spliced ​​point cloud is removed. The point cloud data that is not at the vehicle's location at the current moment in the data is used as the dense point cloud data of the previous moment; the dense point cloud data of the previous moment is spliced ​​with the point cloud data of the current moment to obtain the dense point cloud data of the current moment; the image information of the driving area is obtained by performing semantic segmentation processing on the acquired driving scene image data; based on the driving area image information, the non-road data in the dense point cloud data of the current moment is eliminated according to the correspondence between the image pixels and the three-dimensional points of the point cloud, so as to obtain the road surface dense point cloud data corresponding to the road surface in the target area.

[0016] In a fourth exemplary implementation, before the dense point cloud data of the previous moment is spliced ​​with the point cloud data of the current moment, the method further includes:

[0017] The laser point cloud data, vehicle motion data and vehicle motion posture data with the timestamp at the current moment are obtained; the corresponding vehicle motion data and vehicle motion posture data are interpolated according to the timestamp of each three-dimensional point of the laser point cloud data; the interpolated point cloud data is compensated accordingly according to the vehicle movement information and the vehicle posture information to obtain the point cloud distortion calibration data as the point cloud data at the current moment.

[0018] In a fifth exemplary implementation, verifying whether the suspected obstacle is a target obstacle according to the near-field dense point cloud data includes:

[0019] The point cloud data of the smooth road surface area in the near-field dense point cloud data is removed to obtain a near-field non-smooth road surface dense point cloud; a rotation matrix and a translation vector are obtained according to the point cloud matching result of the point cloud data at a current moment and the dense point cloud data at a previous moment, and the spatial position information of the suspected obstacle in the vehicle coordinates is updated according to the rotation matrix and the translation vector; based on the spatial position information, the obstacle dense point cloud data of the suspected obstacle is obtained from the near-field non-smooth road surface dense point cloud; and target recognition is performed on the obstacle dense point cloud data to determine whether the suspected obstacle is a target obstacle.

[0020] In a sixth exemplary embodiment, removing the point cloud data of the road surface flat area in the near-field dense point cloud data includes:

[0021] According to a third preset grid size, the second target sub-region corresponding to the near-field dense point cloud data is rasterized; according to the clustering result of the average normal vector of each grid of the second target sub-region, the second target sub-region is divided into a plurality of second road surface regions; the corresponding point cloud data is corrected according to the inclination angle of each second road surface region, and the point cloud data of the smooth road surface area in each corrected second road surface area is eliminated according to the installation height of the laser radar; the remaining point cloud data of each second road surface area from which the point cloud data of the smooth road surface area is eliminated are spliced ​​according to the corresponding spatial position relationship to obtain a near-field dense point cloud without a smooth road surface.

[0022] In a seventh exemplary implementation, the performing target recognition on the obstacle dense point cloud data includes:

[0023] Pre-training a point cloud recognition model; inputting the dense point cloud data of the obstacle into the point cloud recognition model, and the point cloud recognition model outputting a recognition result that the dense point cloud data of the obstacle is a pothole area or a raised obstacle; if the dense point cloud data of the obstacle is a pothole area or a raised obstacle, determining that the suspected obstacle is a target obstacle.

[0024] In an eighth exemplary implementation, determining the three-dimensional shape information of the target obstacle according to the point cloud data at adjacent moments and the matching results thereof includes:

[0025] Performing three-dimensional shape fitting processing on the dense point cloud data of the target obstacle to obtain initial three-dimensional shape information of the target obstacle, and removing the dense point cloud data of the obstacle from the near-field uneven road surface dense point cloud to serve as the dense point cloud data of the previous moment; acquiring the point cloud data of the current moment when the vehicle continues to travel, splicing the point cloud data of the current moment with the dense point cloud data of the previous moment to obtain the dense point cloud data of the current moment; based on the new spatial position information of the target obstacle in the vehicle coordinate system at the current moment, determining the new obstacle dense point cloud data of the target obstacle from the dense point cloud data at the current moment, performing three-dimensional shape fitting on the new obstacle dense point cloud data to obtain the three-dimensional shape information of the target obstacle at the current moment, so as to update the initial three-dimensional shape information; wherein the near-field uneven road surface dense point cloud is obtained by removing the point cloud data of the flat road surface area in the near-field dense point cloud data.

[0026] In a ninth exemplary implementation, before the new spatial position information of the target obstacle in the vehicle coordinate system at the current moment is obtained, the method further includes:

[0027] Point cloud matching is performed on the point cloud data at the current moment and the dense point cloud data at the previous moment; a new rotation matrix and a new translation vector are determined according to the point cloud matching result, and the new spatial position information of the target obstacle in the vehicle coordinate system at the current moment is updated according to the new rotation matrix and the new translation vector.

[0028] In a tenth exemplary implementation, after determining the three-dimensional shape information of the target obstacle according to the point cloud data at adjacent moments and the matching results thereof, the method further includes:

[0029] The dense point cloud data remaining after removing the dense point cloud data of the obstacle at each moment is subjected to target recognition to determine whether there is a new target obstacle; if there is a new target obstacle, three-dimensional shape fitting is performed on the corresponding dense point cloud data to obtain the three-dimensional shape information of the new target obstacle.

[0030] The present invention also provides an electronic device, comprising a processor, wherein the processor is used to implement the steps of the road environment state perception method as described in any of the preceding items when executing a computer program stored in a memory.

[0031] Finally, the present invention also provides a non-volatile storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the road environment state perception method as described in any of the preceding items are implemented.

[0032] The present invention also provides a computer program product, comprising a computer program / instruction, which, when executed by a processor, implements the steps of the road environment state perception method as described in any of the preceding items.

[0033] Finally, the present invention also provides an unmanned vehicle, including an automatic driving perception system, wherein the automatic driving perception system is used to implement the steps of the road environment state perception method as described in any of the preceding items when executing a computer program stored in a memory.

[0034] The advantage of the technical solution provided by the present invention is that, based on the point cloud data collected by the laser radar, multi-level obstacle detection, identification and verification are set according to the data distribution characteristics, so as to accurately identify whether there are obstacles such as low raised obstacles and potholes in front of the vehicle, and when it is determined that such obstacles exist, as the vehicle continues to move forward, it is confirmed that the low raised obstacles and potholes will continue to approach the vehicle, and the granularity of the point cloud data inside them will be smaller. In this way, the three-dimensional morphology information of the target obstacle is determined according to the point cloud data at adjacent moments and their matching results, so that the accuracy of the three-dimensional morphology data of the obstacle can be gradually improved, thereby assisting the vehicle to more finely identify the environmental status and improve the safety and comfort of the vehicle.

[0035] In addition, the present invention also provides corresponding electronic devices, non-volatile storage media, computer program products and unmanned vehicles for implementing the road environment status perception method, further making the method more practical, and the electronic devices, non-volatile storage media, computer program products and unmanned vehicles have corresponding advantages.

[0036] The technical features mentioned above, the technical features to be mentioned below, and the technical features shown separately in the drawings can be arbitrarily combined with each other, as long as the combined technical features are not contradictory. All feasible feature combinations are technical contents clearly recorded in this article. Any of the multiple sub-features contained in the same sentence can be applied independently, and does not have to be applied together with other sub-features. It should be understood that the above general description and the detailed description below are only exemplary and cannot limit the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] In order to more clearly illustrate the technical solutions of the present invention or related technologies, the following briefly introduces the drawings required for use in the embodiments or related technical descriptions. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0038] Figure 1A schematic diagram of a road environment state perception method provided by the present invention;

[0039] Figure 2 A schematic diagram of a target area ahead of a vehicle in an exemplary example is provided for the present invention;

[0040] Figure 3 A schematic diagram of coordinate system definition in an exemplary example is provided for the present invention;

[0041] Figure 4 A schematic flow chart of a method for calculating dense point cloud data of a road surface in an exemplary example provided by the present invention;

[0042] Figure 5 A schematic diagram of a flow chart of another road environment state perception method provided by the present invention;

[0043] Figure 6 A structural diagram of a specific implementation of the road environment state perception device provided by the present invention;

[0044] Figure 7 A structural diagram of a specific implementation of an electronic device provided by the present invention;

[0045] Figure 8 A schematic diagram of obstacle recognition for an unmanned vehicle provided by the present invention. DETAILED DESCRIPTION

[0046] In order to enable those skilled in the art to better understand the technical solution of the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. Among them, the terms "first", "second", "third", "fourth", etc. in the specification and the above-mentioned drawings are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "including" and "having" and any variations of the two are intended to cover non-exclusive inclusions. The term "exemplary" means "used as an example, embodiment or illustrative". Any embodiment described here as "exemplary" is not necessarily interpreted as being superior or better than other embodiments.

[0047] It is understandable that potholes or low raised obstacles on the road can cause the autonomous vehicle to drive unsteadily and then bump, which will not only affect the comfort of the passengers, but also cause the vehicle to suddenly lose control and cause traffic accidents. If the vehicle is driven on potholes and low raised obstacles for a long time, it will damage the vehicle's suspension system, tires, and sensors, which will affect the vehicle's performance and safety. In addition, potholes on the road will damage the road material and cause the settlement of the roadbed. If they are not repaired in time, the potholes will easily expand and deepen, accelerating the damage to the road infrastructure and affecting the service life and carrying capacity of the road. In order to avoid the above problems, during the driving process of the vehicle, it will monitor in real time whether there are potholes or raised obstacle areas on the road ahead, remind the driver of the road detection situation, and issue a warning to slow down the speed of the car, or absorb and offset the impact and vibration caused by potholes / low raised obstacles on the road through the intelligent suspension system.

[0048] At present, the traditional way to monitor whether there are potholes or low raised obstacles on the road is manual visual inspection, where structural engineers and certified inspectors regularly check whether there are potholes or low raised obstacles on the road surface and report the location of such target areas. This method relies on the personal experience of inspectors and engineers, which not only cannot guarantee the accuracy of the detection results, but is also inefficient, costly, and dangerous. In order to solve the drawbacks of manual visual inspection, related technologies use computer vision technology to achieve more efficient and safer detection of road target areas. The target areas here include road obstacle areas that affect the normal operation of vehicles, including but not limited to potholes and low raised obstacle areas, such as raised cracks.

[0049] Related technologies use computer vision algorithms to achieve three-dimensional road imaging through 3D point cloud modeling, and then detect road obstacle areas based on segmentation methods to improve the accuracy of road target area detection. This method uses some algorithms, such as RANSAC (Random Sample Consensus) to improve the robustness of road point cloud modeling and DCNN (Deep Convolution Neural Network) to improve image classification accuracy. However, this type of method requires a small field of view and is based on a single-frame 3D road point cloud that is a plane or quadratic surface. The detection accuracy and scene adaptability are still difficult to achieve practicality, and it is difficult to obtain the width, depth, volume and other geometric characteristics of road pits with high precision. For example, a related technology realizes the detection of potholes on the road surface of unmanned vehicles through multi-sensor fusion. This method uses sound waves to scan the road surface ahead. When a pothole is detected, the camera is used to capture the road image, and then the pre-processed image is threshold segmented to obtain a binary image. The number of white pixels in the binary image is counted to determine whether there is water in the pothole. The image information captured by the camera is used to calculate the width of the pothole edge from the road edge on both sides, and the minimum torque for the unmanned vehicle to pass through the pothole is calculated. Finally, the relationship between the minimum torque and the maximum torque is used to determine whether the unmanned vehicle can pass safely. This method is greatly affected by the threshold value for image binarization, is difficult to adapt to various lighting conditions, and cannot provide information such as the depth of the pothole. Another related technology uses an RGB-D (Red Green Blue Depth, a color image with depth information) depth camera to detect low raised obstacles and potholes on the road. This method uses the original point cloud of the RGB-D depth camera for preprocessing to obtain the corresponding plane point cloud, extracts the raised low raised obstacle point cloud data from the plane point cloud, and projects the extracted raised low raised obstacle point cloud onto the plane model parameter plane, downsamples the raised low raised obstacle point cloud on the model parameter plane, performs convex hull segmentation, transforms the convex hull into a map coordinate system and publishes it, merges the projected raised low raised obstacle point cloud into the ground plane point cloud to form a new plane point cloud, transforms the new plane point cloud into a ground plane two-dimensional grid map under base_footprint (robot body coordinate system), performs pseudo pothole filtering on the ground plane two-dimensional grid map, and extracts potholes using an edge extraction method. Since the effective working distance of RGB-D cameras is very limited, usually within 10 meters, and they are basically ineffective under strong outdoor light, the accuracy of the point cloud generated by the RGB-D camera is poor due to the influence of the projection density of the generator. The effectiveness of the actual use process needs to be verified, the usage scenarios are limited, and the final detection accuracy cannot be guaranteed.Another related technology is to detect potholes on the road based on neural networks and binocular stereo vision. This method uses a binocular camera installed in front of the vehicle to obtain left and right views of the road scene; and uses a target detection neural network to detect the left view. When a pothole is detected, the left and right views are stereo matched to obtain a depth map. The pothole position information predicted by the neural network is used to cut out the depth map of the pothole and the road surface around the pothole. Finally, a combination of RANSAC and genetic algorithms is used to fit the road plane around the pothole, so as to calculate the distance from the pothole point cloud to the fitting plane, and the distance is summed and averaged to obtain the pothole depth. The minimum circumscribed rectangle of the projection point is determined by projecting the pothole point cloud onto the fitting road plane, and the area of ​​the pothole is obtained by determining the area of ​​the minimum circumscribed rectangle. However, due to the existence of void points in the parallax, this method has a large estimation error in the depth direction, and it is difficult to obtain high-precision pothole depth and bump height, resulting in low detection accuracy of road target areas. There is also a related technology that uses OCC to predict the probability of each voxel grid being occupied and the target categories that may be included, so as to achieve the perception of the entire road scene. However, since the cost of labeling data for low raised obstacles and potholes is too high, and the accuracy of the OCC algorithm cannot reach the recognition accuracy of potholes and low raised obstacles, it is impossible to accurately determine whether there are potholes and low raised obstacles on the road. It can be seen that the related technology cannot accurately identify potholes and low raised obstacle areas on the road, and cannot assist vehicles in more refined recognition of road environment conditions.

[0050] In view of this, in order to realize that the autonomous driving perception system meets the scene requirements of increasingly refined granularity, in addition to the identification of typical obstacles such as pedestrians and vehicles, the present invention is based on lidar data and, according to the data distribution characteristics, sets up multi-level obstacle detection, identification and verification, which can accurately identify low raised obstacles and potholes, and assist vehicles in more refined identification of road environment conditions. After introducing the technical scheme of the present invention, various non-limiting embodiments of the present invention are described in detail below. In order to better illustrate the present invention, numerous specific details are given in the specific embodiments below. Those skilled in the art should understand that the present invention can also be implemented without these specific details. In other examples, methods, means, components and circuits well known to those skilled in the art are not described in detail in order to highlight the subject matter of the present invention.

[0051] First see Figure 1 , Figure 1 A schematic diagram of a road environment state perception method provided in this embodiment, this embodiment may include the following contents:

[0052] S101: Acquire dense point cloud data of a road surface in a target area ahead of a vehicle, and divide the dense point cloud data of the road surface into near-field dense point cloud data and far-field dense point cloud data according to the distance between the point cloud data and the vehicle.

[0053] The target area is the area in front of the vehicle, which is an area extending from the front of the vehicle to the front in the direction of the vehicle. Figure 2 As shown, the vehicle target area can be an area with a length of 100m (meters) and a width of 20m in front of the vehicle. The dense point cloud data of the road surface is obtained by removing the point cloud data of the road surface area that is not the target area from the splicing result of the point cloud data at the current moment and the spliced ​​point cloud data at the previous moment. The point cloud data is collected by the vehicle's own point cloud data acquisition sensor. The vehicle of the present invention can be any vehicle with an automatic driving function, and is not limited to an automatic driving vehicle or an unmanned vehicle. It fully reuses the sensors that realize the automatic driving function, and does not require the installation of other sensors, such as the navigation system, inertial measurement unit, odometer, and lidar on the vehicle side. In order to obtain the accuracy of the point cloud data, the vehicle sensors can be calibrated before obtaining the point cloud data to obtain the coordinate system transformation relationship between the sensors. Furthermore, in the actual working process, the data time synchronization mechanism can also be used to ensure the time alignment of the data collected by the multi-source sensors. The dense point cloud data of the present invention refers to the point cloud data obtained by splicing the point cloud data of adjacent moments, that is, the point cloud data of the current moment is spliced ​​with the point cloud data of the previous moment. Considering that the laser radar has very limited lines or resolution, the point cloud data collected is very sparse, and the sparsity of the point cloud data becomes more prominent as the distance from the laser radar increases. In order to avoid the sparsity of the point cloud data causing missed detection of low raised obstacles or potholes with a small spatial scale, this step densifies the point cloud data by splicing the point cloud, reduces the sparsity of the point cloud, and thus reduces the missed detection rate. Since the probability of encountering obstacles is small when the vehicle is just started, the point cloud data of the previous moment is the splicing data of the previous point cloud data. In order to reduce the complexity of subsequent data processing and improve the accuracy and efficiency of obstacle recognition, the present invention removes the point cloud data that is not the road surface area in the spliced ​​point cloud data, so that the remaining point cloud data is the point cloud data of the road surface area in the target area, and this step is defined as the dense point cloud data of the road surface.

[0054] In the present invention, if there are obstacles in the target area in front of the vehicle, such as low raised obstacles and potholes, as the vehicle gradually approaches the vehicle during driving, the point cloud data information collected by the sensor will be richer. Since the near-field verification is more complicated and requires more computing resources, this step can first perform initial identification through the far-field area. If no suspected low raised obstacles / potholes are found in the far-field area, it is not necessary to start the near-field verification calculation process, thereby effectively reducing unnecessary computing burden. Based on this, this step divides the target area in front of the vehicle according to the distance between the vehicle and the pre-set distance threshold, and divides the target area into a first target sub-area and a second target sub-area, that is, the target area includes the first target sub-area and the second target sub-area. Among them, the distance threshold can be pre-set according to the current number of laser radar lines or resolution. The lower the number of lines or resolution, the smaller the distance threshold. The maximum distance between the second target sub-region and the vehicle along the vehicle's driving direction is less than the minimum distance between the first target sub-region and the vehicle, that is, the first target sub-region is a far-field region, and the second target sub-region is a near-field region. Taking the distance threshold of 20m as an example, the far-field region after 20m in front of the vehicle is the first target sub-region, and the size of the first target sub-region is: 80m×20m, and the near-field region within 20m in front of the vehicle is the second target sub-region, and the size of the second target sub-region is 20m×20m. For the convenience of distinction, the point cloud data collected for the first target sub-region is far-field point cloud data, and the point cloud data collected for the second target sub-region is near-field point cloud data. Correspondingly, the point cloud data corresponding to the first target sub-region is far-field dense point cloud data, and the point cloud data corresponding to the second target sub-region is near-field dense point cloud data.

[0055] In order to facilitate subsequent data processing, the present invention also defines the coordinate system involved accordingly, such as Figure 3 As shown in the figure, the vehicle includes a far-field camera, a near-field camera, a lidar, an odometer, and an inertial measurement unit. For the convenience of drawing, the far-field camera and the near-field camera are not distinguished, and the odometer and inertial measurement unit are not drawn. Figure 3 middle, is the camera coordinate system, Along the camera optical axis. is the standard camera coordinate system, which is Along Rotation, where Vertically downward. is the laser radar coordinate system, is the local world coordinate system, which is defined on the road surface, where Vertically upward, For horizontal forward, To the left horizontally, the vehicle coordinate system can usually adopt the lidar coordinate system, and the local world coordinate system is the coordinate system of the road surface on which the vehicle is traveling.

[0056] S102: Determine suspected obstacles in the target area by analyzing the road surface flatness of the far-field dense point cloud data.

[0057] In this embodiment, after the far-field dense point cloud data is obtained by the division in the previous step, the road surface flatness is analyzed by analyzing the point cloud data of the road surface area of ​​the dense point cloud data. This step can use any method that can realize the flatness calculation to calculate the distribution of road surface flatness in the far-field area. By analyzing the road surface flatness, the area where the road surface flatness exceeds the preset flatness threshold can be determined as an obstacle in the far-field area, that is, whether there are low raised obstacles / pothole areas in the far-field area. Since the obstacle detection accuracy of this step is relatively coarse, the accuracy of the detected obstacles is not high, and this step defines it as a suspected obstacle. In order to facilitate the subsequent processing of the verification of suspected obstacles, if this step determines that there are multiple suspected obstacles in the target area, the low raised obstacles / pothole areas identified inside can be generated through the spatial coordinate correspondence, which is convenient for association and tracking.

[0058] S103: Verify whether the suspected obstacle is the target obstacle based on the near-field dense point cloud data, and when the suspected obstacle is verified to be the target obstacle, determine the three-dimensional shape information of the target obstacle based on the point cloud data at adjacent moments and the matching results thereof.

[0059] Among them, the target obstacle is an obstacle with a relatively small or relatively small elevation on the road surface where the vehicle is traveling, including but not limited to potholes, low raised obstacles such as raised cracks, and speed bumps. After the obstacle is identified in the far field in the previous step, this step verifies the identified obstacle to determine whether the suspected obstacle is a real obstacle. In order to facilitate distinction, the suspected obstacle verified as a real obstacle is defined as the target obstacle. Considering that the error of the point cloud data collected by the laser radar increases with the distance, when the low raised obstacle / pothole area is far away from the vehicle, only primary recognition is performed through S102 to obtain the suspected low raised obstacle / pothole area. As the vehicle continues to move forward, low raised obstacles and potholes gradually approach the vehicle, and the collected point cloud data will become richer. The matching results of the point cloud data at adjacent moments can determine the position information of the target obstacle. The morphology information of each identified low raised obstacle / pothole can be obtained based on the point cloud data at adjacent moments. As the vehicle continues to approach the target obstacle, the point cloud data is richer and more accurate. By continuously iteratively updating the three-dimensional morphology information of the target obstacle, its morphological features can be gradually refined.

[0060] S104: Generate a road environment state perception result of the vehicle according to the three-dimensional shape information of each target obstacle obtained during the vehicle's driving process.

[0061] When the three-dimensional shape information of the target obstacle encountered by the vehicle during driving is determined in the previous step, the three-dimensional shape information includes but is not limited to the position information and shape information of the target obstacle, shape information such as the geometric shape and orientation of the target obstacle, and geometric shapes such as length, width, and height, then the corresponding road environment state perception result can be generated based on the three-dimensional shape information, that is, the three-dimensional shape information of the target obstacle is reflected in the road environment state perception result, so that it plays a corresponding role in roadside perception scenarios, smart traffic scenarios, V2X (vehicle to everything, that is, vehicle-to-outside information exchange) scenarios, smart security scenarios, and collaborative perception scenarios, such as being used to accurately assess the harmfulness of low raised obstacles / pothole areas, such as obstacle avoidance during automatic route planning.

[0062] In the technical solution provided in this embodiment, based on the point cloud data collected by the laser radar, multi-level obstacle detection, identification and verification are set according to the data distribution characteristics, so as to accurately identify whether there are obstacles such as low raised obstacles and potholes in front of the vehicle. When it is determined that such obstacles exist, as the vehicle continues to move forward, it is confirmed that the low raised obstacles and potholes will continue to approach the vehicle, and the granularity of the point cloud data inside them will be smaller. In this way, the three-dimensional shape information of the target obstacle is determined based on the point cloud data at adjacent moments and their matching results, so that the accuracy of the three-dimensional shape data of the obstacle can be gradually improved, thereby assisting the vehicle to more finely identify the environmental status and improve the safety and comfort of the vehicle.

[0063] In the above embodiment, there is no limitation on how to execute step S101. In this embodiment, an exemplary method for obtaining dense point cloud data of a road surface is provided, such as: Figure 4 As shown, it may include the following:

[0064] The vehicle motion data and vehicle motion posture data at the current moment are obtained as the pose point cloud data, and the initial pose change information is determined according to the pose point cloud data and the spliced ​​point cloud data at the previous moment before the current moment; the initial pose change information is used as the initial value of the iterative calculation, and the point cloud matching calculation is performed on the point cloud data at the current moment and the spliced ​​point cloud data at the previous moment to determine the rotation change matrix and the translation vector; the spliced ​​point cloud data at the previous moment is transformed according to the rotation change matrix and the translation vector, and the point cloud data that is not the position of the vehicle at the current moment in the transformed spliced ​​point cloud data is removed to serve as the dense point cloud data at the previous moment; the dense point cloud data at the previous moment is spliced ​​with the point cloud data at the current moment to obtain the dense point cloud data at the current moment; the image information of the driving area is obtained by performing semantic segmentation processing on the acquired driving scene image data; based on the image information of the driving area, the non-road data in the dense point cloud data at the current moment is eliminated according to the correspondence between the image pixels and the three-dimensional points of the point cloud, and the dense point cloud data of the road surface corresponding to the road surface in the target area is obtained.

[0065] In this step, since the single-frame point cloud data is relatively sparse, especially for low-beam or low-resolution laser radars, the sparsity is more obvious, and it is very easy to miss the low raised obstacles and potholes that the present invention is intended to identify. Therefore, this step splices the point cloud data in time sequence to increase the density of the point cloud data. Exemplarily, considering the spatiotemporal continuity of point cloud data, point cloud matching algorithms such as ICP (Iterative Closest Point) and NDT (Normal Distribution Transform) can be used to match point clouds. For the same scene, two frames of point cloud data are obtained from different distances or angles. Since the two frames of point cloud data have overlapping areas, that is, the spatial structures of the point clouds are similar, the ICP or NDT algorithm can be used to obtain the transformation relationship between the two frames of point cloud data based on the point cloud data in the overlapping area, that is, the rotation transformation matrix R and the translation vector T are obtained. In order to obtain more accurate matching results and reduce the amount of matching calculations, this step can obtain vehicle motion data through the odometer and vehicle motion posture data through the inertial strategy unit, and use it as the initial pose change parameter between the point cloud data at the current time t and the previously spliced ​​point cloud as the initial value of the iterative calculation, which can effectively reduce the number of iterations. After the point cloud matching calculation is completed, the dense point cloud at the previous time, that is, time t-1, is transformed by obtaining the rotation transformation matrix R and the translation vector T.

[0066] As the vehicle keeps moving forward, the dense point cloud data after transformation at time t-1 enters the area behind the front of the vehicle. Therefore, it is necessary to remove the point cloud data that is not the location of the vehicle at the current moment in the transformed spliced ​​point cloud data. For example, the laser radar coordinate system can be used as the vehicle coordinate system, and the part with an x ​​coordinate less than 0 in the dense point cloud after transformation at time t-1 is removed to remove the point cloud data that is not the location of the vehicle at the current moment in the transformed spliced ​​point cloud data, and then splice it with the point cloud data at the current moment t to obtain the dense point cloud data at the current moment.

[0067] In order to reduce the amount of subsequent calculations and improve the accuracy of subsequent road segmentation calculations, this embodiment can further pre-process the dense point cloud data, that is, remove the point cloud data of non-road areas. An exemplary implementation method for obtaining the road area is: obtain the image captured by the vehicle's forward camera, such as a far-field camera, a near-field camera, a wide-angle camera, etc., that is, the driving scene image data, and use a semantic segmentation algorithm to extract the road area from the driving scene image data. Then, use the calibration parameters of the camera and the lidar to obtain the correspondence between the image pixels and the point cloud points. Among them, an exemplary method for obtaining the correspondence between the two is: according to Figure 3 The three-dimensional coordinate systems of the camera imaging system and the lidar system are defined as shown. Through a finite number of known spatial points of the same name, the spatial transformation relationship between the two coordinate systems can be obtained. This process is called calibration. When the two sets of spatial correspondences are obtained, the correspondences between any spatial points of the same name in the two coordinate systems can be obtained. Based on this principle, combined with the imaging principle, the correspondence between image pixels and spatial points can be obtained first, and then based on the spatial point correspondence, the correspondence to the point cloud data can be obtained. According to the correspondence between image pixels and point clouds, it can be identified which point clouds belong to the road, so that the point cloud data can be judged point by point, thereby eliminating non-road point cloud data. The point cloud data in the non-road area is then removed, and finally, according to the pre-set area of ​​interest, such as Figure 2 The dense point cloud data of the road area in front of the vehicle is obtained by taking a 100m×20m area as an example, that is, obtaining the dense point cloud data of the road surface.

[0068] In order to further improve the processing flexibility of point cloud densification, this embodiment also provides another simpler point cloud densification method: considering the spatiotemporal continuity of point cloud data, the point cloud matching algorithm can also be used to match the point cloud data. Due to the movement of the vehicle, the position and posture of the vehicle at the current moment and the previous moment have changed, resulting in a certain position and posture change between the two frames of point cloud data. The inertial measurement unit can obtain the posture information of the vehicle at different times, and the odometer can obtain the moving distance of the vehicle between two moments. Through these two data, the posture change data can be estimated. In order to obtain more accurate matching results and reduce the amount of calculation, the initial data of the posture change between the point cloud data at the current moment and the previously spliced ​​point cloud can be obtained through the odometer and the inertial measurement unit. After the point cloud matching calculation is completed, the point cloud at the current moment is superimposed on the historical spliced ​​point cloud data to obtain dense point cloud data.

[0069] From the above, it can be seen that this implementation densifies the point cloud data based on the point cloud stitching method, effectively reduces the sparsity of the point cloud data, reduces the missed detection rate of road obstacles, and effectively improves the recognition accuracy of low raised obstacles or potholes.

[0070] In order to further improve the recognition accuracy of low raised obstacles or potholes, considering that the mechanical LiDAR (Light Laser Detection and Ranging, laser radar) will produce distortion along the direction of vehicle movement during vehicle driving due to its data receiving principle limitation, and the distortion increases with the increase of vehicle speed, which is very likely to cause a judgment error in the longitudinal direction consistent with the direction of the vehicle head. That is, if the distortion is not eliminated, it will cause large errors in the recognition of low raised obstacles / potholes, morphology judgment, etc. Based on the above embodiment, this embodiment also corrects the distortion of the laser radar point cloud data through the point cloud data collected by the odometer and the inertial measurement unit, such as Figure 4 As shown, the distortion correction process may include the following:

[0071] The laser point cloud data, vehicle motion data and vehicle motion posture data with the timestamp at the current moment are obtained; the corresponding vehicle motion data and vehicle motion posture data are interpolated according to the timestamp of each three-dimensional point of the laser point cloud data; the interpolated point cloud data is compensated accordingly according to the vehicle movement information and vehicle posture information to obtain the point cloud distortion calibration data as the point cloud data at the current moment.

[0072] Among them, the odometer is used to obtain the vehicle's movement distance information within a unit time period, and the inertial measurement unit is used to obtain the vehicle's movement posture data within the corresponding unit time period. The mechanical laser radar usually has a data frame rate of 10hz. Correspondingly, the unit time period can be set to 0.1 seconds, that is, the odometer is used to obtain the vehicle's movement distance information within 0.1 seconds, and the inertial measurement unit is used to obtain the vehicle's movement posture data within 0.1 seconds. Of course, it can also be flexibly selected according to the actual application scenario, which does not affect the implementation of the present invention. Each point in the laser point cloud data of this embodiment is accompanied by timestamp information. For each point in the point cloud, the acquired vehicle motion data and vehicle posture data are interpolated according to its timestamp information, and their coordinates are compensated. Taking the compensation of vehicle motion data as an example, the actual time of the current data frame acquisition is , for a point cloud data point , whose uncompensated coordinates are , whose timestamp is If The vehicle moves s meters along the x-axis during the time, so the coordinates after compensation are In actual application scenarios, when the vehicle's moving direction or posture changes, corresponding compensation can be performed to eliminate the point cloud data distortion caused by the vehicle's motion.

[0073] It can be seen from the above that this embodiment eliminates the distortion of the laser radar point cloud data through the vehicle posture data, thereby improving the accuracy of the point cloud data, which is beneficial to improving the recognition accuracy of low raised obstacles or potholes.

[0074] Considering that the road surface may have or continuously have multiple uphill and downhill sections or tilted sections, and since the elevation range of low raised obstacles / potholes is relatively small, if the road tilt factor is taken into account, it is easy to misjudge or miss low raised obstacles and potholes. In order to further improve the recognition accuracy of low raised obstacles / potholes, based on the above embodiment, the present invention also provides a high-precision suspected obstacle determination method, which may include the following contents:

[0075] The first target sub-region corresponding to the far-field dense point cloud data is rasterized according to a first preset grid size, and the first target sub-region is segmented according to the clustering result of the average normal vector in each grid of the first target sub-region to obtain multiple types of first road surface regions; for each first road surface region, the current road surface region is rasterized according to a second preset grid size, and a road surface flatness region that meets a preset flatness is determined according to the direction of the average normal vector of each grid of the current first road surface region, and the remaining area of ​​the current first road surface region after excluding the road surface flatness region is merged based on the spatial distance relationship to serve as the spatial position of the suspected obstacle.

[0076] In this embodiment, the rasterization process can be performed under the target perspective, such as generating a grid under a bird's-eye view perspective. The first preset grid size, which is a large spatial scale, is conducive to obtaining a flat road area, and then the second preset grid size, which is a small scale space, is used for calculation, which is conducive to obtaining more detailed information. For example, the first preset grid size can be 0.2 meters × 0.2 meters, and the second preset grid size can be 0.02 meters × 0.02 meters. Figure 2 and Figure 3 As shown, in A space of 0.2 m × 0.2 m in size is taken as a statistical area in the plane, and the far field area (80 × 20 m) 20 m in front of the vehicle is rasterized according to 0.2 m × 0.2 m. For each road surface area, in order to facilitate distinction, it is defined as a first road surface area, and the first road surface area is rasterized according to a grid size of 0.02 m × 0.02 m. Of course, the values ​​of the second preset grid size and the first preset grid size can be flexibly set according to the situation, as long as the second preset grid size is smaller than the first preset grid size, and the present invention does not impose any limitation on this. After the first target sub-region is rasterized, the three-dimensional position information of the point cloud data inside each grid of the first target sub-region is obtained, and the three-dimensional space plane equation of the corresponding grid is fitted; based on the three-dimensional space plane equation of each grid, the average normal vector in each grid is determined, and the average normal vector of each grid is clustered using any clustering method, such as mean shift (mean shift), k-mean (k-mean), and the first target sub-region is segmented according to the clustering results of the average normal vector of each grid to obtain multiple first road surface areas, such as different uphill and downhill sections. For each first road surface area, the grid is reset, and the local average normal vector of each grid is recalculated to analyze the road surface flatness.

[0077] Among them, satisfying the preset flatness is a pre-selected judgment standard for whether the road is flat or uneven, that is, there are obstacles. Exemplarily, the three-dimensional space plane equation of the corresponding grid can be fitted according to the three-dimensional position information of the point cloud data inside each grid of the current first road area; based on the three-dimensional space plane equation of each grid of the current first road area, the average normal vector in each grid of the current first road area is calculated; according to the direction of the average normal vector, at intervals of the preset angle threshold, vote on whether each average normal vector is a road surface normal vector, and take the target average normal vector with the most votes as the road surface normal vector, and take the grid corresponding to the target average normal vector as the road surface flat area. The preset angle threshold can be selected according to the actual situation. For example, 2° is used as an example. According to the local average normal vector direction, the vote is taken at intervals of 2°. The one with the most votes is considered to be the road surface normal, that is, the road surface flat area, and is eliminated. According to the spatial distance relationship, the remaining ones are merged to obtain one or more suspected low raised obstacles / pothole areas, including their spatial positions in the local world coordinate system.

[0078] Furthermore, as the vehicle continues to move forward, unique ID information can be assigned to suspected low raised obstacles and potholes that have newly entered the local world coordinate system. The ID information can be used for further tracking and continuous refinement calculation of each suspected low raised obstacle / pothole.

[0079] As can be seen from the above, this embodiment uses the consistency of local normal vectors as a criterion to identify suspected low raised obstacles and potholes, which is simple and easy to implement, with high accuracy, and is conducive to improving the recognition accuracy of low raised obstacles or potholes. Furthermore, the present invention obtains road surfaces in different areas through segmented road surface fitting, and then further identifies suspected obstacles in each road surface one by one, which is very easy to miss low raised obstacles and potholes, effectively improving the recognition accuracy of low raised obstacles or potholes.

[0080] Considering that the horizontal / vertical angular resolution of the laser radar configured on the vehicle is better than 0.06°, the laser radar's spatial perception capability increases with the closer the target distance is. Calculated at an angular resolution of 0.06°, the minimum spatial perception size is 5.2 centimeters at a distance of 50 meters from the radar, the minimum spatial perception capability is 2.0 centimeters at a distance of 20 meters from the radar, the minimum spatial perception capability is 1 centimeter at a distance of 10 meters from the radar, and the minimum spatial perception capability is 0.5 centimeters at a distance of 5 meters from the radar. The present invention improves the classification and recognition accuracy of low raised obstacles / pothole areas based on near-field dense point clouds, and the above embodiments do not impose any restrictions on how to verify suspected obstacles. The present invention also provides an exemplary implementation method for verifying whether a suspected obstacle is a target obstacle based on near-field dense point cloud data, which may include the following:

[0081] The point cloud data of the smooth road area in the near-field dense point cloud data is removed to obtain the near-field non-smooth road dense point cloud; the rotation matrix and the translation vector are obtained according to the point cloud matching results of the point cloud data at the current moment and the dense point cloud data at the previous moment, and the spatial position information of the suspected obstacle in the vehicle coordinates is updated according to the rotation matrix and the translation vector; based on the spatial position information, the obstacle dense point cloud data of the suspected obstacle is obtained from the near-field non-smooth road dense point cloud; and the target is identified by performing target recognition on the obstacle dense point cloud data to determine whether the suspected obstacle is a target obstacle.

[0082] like Figure 5 As shown, in this embodiment, as the distance between the obstacle and the vehicle becomes closer, the spatial resolution of the near-field dense point cloud data becomes higher. In order to further reduce the amount of subsequent calculations and improve the obstacle recognition accuracy, this embodiment extracts the dense point cloud of the near-field non-flat road surface from the acquired near-field dense point cloud. Exemplarily, the second target sub-area corresponding to the near-field dense point cloud data may be rasterized according to the third preset grid size; the second target sub-area is divided into a plurality of second road surface areas according to the clustering result of the average normal vector of each grid of the second target sub-area; the corresponding point cloud data is corrected according to the inclination angle of each second road surface area, and the point cloud data of the flat road surface area in each corrected second road surface area is eliminated according to the installation height of the laser radar; the remaining point cloud data of each second road surface area from which the point cloud data of the flat road surface area is eliminated is spliced ​​according to the corresponding spatial position relationship to obtain a near-field non-flat road surface dense point cloud. The third preset grid size can be flexibly selected according to actual needs, such as 0.2 meters × 0.2. For the near-field dense point cloud A, in A space of 0.2 m × 0.2 m in size is taken as the statistical area in the plane, and the near field area 20 m in front of the vehicle, i.e., the second target sub-area, is rasterized, and the average normal vector of the point cloud data fitting plane in each grid is calculated. Similarly, the local average normal vector of each grid can be clustered, and the second target sub-area can be further segmented according to the clustering results to obtain multiple road surface areas, which are defined as the second road surface area for distinction. ,in, , N is the total number of road surface areas, and the inclination angle of each second road surface area is obtained at the same time For each second road surface area , based on the point cloud coordinate transformation, that is, the three-dimensional coordinate transformation and its tilt angle Correction is performed by multiplying the three-dimensional coordinate transformation matrix with the point cloud coordinates, so that the inclination angle of the flat road surface area is 0. The corrected dense point cloud can be defined as For each rectified dense point cloud Each point in the point cloud data has xyz coordinates, where the z coordinate is the vertical direction. For vehicles with autonomous driving functions, the laser radar installation height is a basic parameter. Combined with the vehicle structure design parameters, the height H from the center of the laser radar to the ground can be known. Take this height H as the threshold and select ± , for example, 5 cm, the point cloud data can be filtered to retain only the points with coordinates in The points within the range are the point cloud data of the uneven road surface area. By removing the point cloud of the smooth road surface area, we can obtain the remaining dense point cloud in the area. .Will According to its corresponding The near-field area is stitched together to obtain a dense point cloud D of the near-field area without a smooth road surface.

[0083] As the vehicle moves forward, some suspected low raised obstacles / potholes identified in the far field gradually move to the near field. The granularity of the point cloud data inside is smaller, which provides convenient conditions for accurately judging its attributes. When the near field non-flat road dense point cloud is obtained, the suspected low raised obstacles / potholes entering the near field area are , start the verification: match the point cloud at the current time t with the dense point cloud at time t-1, obtain the rotation matrix R and translation vector T, use R and T to form a 4×4 matrix, and then use matrix multiplication to update The position and occupied space information in the vehicle coordinate system, that is, the updated spatial position information. Based on the near-field uneven road surface dense point cloud D and The spatial position information of the obstacle is obtained to obtain the dense point cloud inside it. Exemplarily, the point cloud recognition model can be trained in advance based on any point cloud recognition network model, such as pointnet (classification network name) and pointnet++ (classification network name). The dense point cloud data of the obstacle is input into the point cloud recognition model, and the point cloud recognition model outputs the recognition result that the dense point cloud data of the obstacle is a pothole area or a raised obstacle; if the dense point cloud data of the obstacle is a pothole area or a raised obstacle, the suspected obstacle is determined to be the target obstacle. If it is a low raised obstacle / pothole area, based on the dense point cloud inside it, any three-dimensional surface fitting method is used to perform three-dimensional shape fitting, and the three-dimensional shape (surface) information of the low obstacle / pothole area can be obtained, and key information such as its direction, length, width, and depth can be obtained.

[0084] Furthermore, for scenes with multiple target obstacles, since the vehicle is far away from the obstacle, missed detection may occur. After verifying that a suspected obstacle is a target obstacle, the dense point cloud data of the target obstacle can be removed from the near-field non-flat road dense point cloud D. After removing the identified obstacle point cloud, the remaining point cloud is further spliced ​​with the subsequent point cloud data frame to increase the density, which can improve the accuracy of obstacle recognition.

[0085] As can be seen from the above, this embodiment divides the near-field area into multiple segments and corrects the point cloud data based on the inclination of the segmented area, which is beneficial to further improve the classification and recognition accuracy of low raised obstacles / pothole areas.

[0086] The above embodiment does not limit how to gradually refine the three-dimensional shape information of the obstacle as the vehicle approaches the obstacle. Based on the above embodiment, the present invention also provides an implementation process for determining the three-dimensional shape information of the target obstacle based on the point cloud data at adjacent moments and the matching results thereof, which may include the following contents:

[0087] Perform three-dimensional shape fitting processing on the dense point cloud data of the target obstacle to obtain the initial three-dimensional shape information of the target obstacle, and remove the dense point cloud data of the obstacle from the near-field non-smooth road surface dense point cloud to serve as the dense point cloud data of the previous moment; obtain the point cloud data of the current moment when the vehicle continues to travel, and splice the point cloud data of the current moment with the dense point cloud data of the previous moment to obtain the dense point cloud data of the current moment; based on the new spatial position information of the target obstacle in the vehicle coordinate system at the current moment, determine the new obstacle dense point cloud data of the target obstacle from the dense point cloud data at the current moment, perform three-dimensional shape fitting on the new obstacle dense point cloud data, and obtain the three-dimensional shape information of the target obstacle at the current moment to update the initial three-dimensional shape information.

[0088] Among them, the near-field uneven road surface dense point cloud is obtained by removing the point cloud data of the flat road area from the near-field dense point cloud data. Since the spatial position of the obstacle does not move, but as the vehicle continues to move forward, the density of the obstacle point cloud will gradually increase. Based on this, Figure 5 As shown in the figure, for each identified obstacle, only the current frame point cloud is matched with the previous frame point cloud to obtain its position in the current dense point cloud, and it is updated to a denser (or finer) point cloud, further improving the accuracy of the point cloud shape data. Furthermore, in order to improve the accuracy and efficiency of determining the new spatial position information, the point cloud data at the current moment and the dense point cloud data at the previous moment can be matched, and then the new rotation matrix and new translation vector are determined according to the point cloud matching results, and the new spatial position information of the target obstacle in the vehicle coordinate system at the current moment is updated according to the new rotation matrix and new translation vector.

[0089] The refinement process of the 3D shape data of the low raised obstacle / pothole area identified in the previous frame in the near field may include: obtaining the rotation matrix R and the translation vector T according to the matching of the point cloud at the current time t and the dense point cloud obtained at the time t-1, and updating Position and occupied space information in the ego-vehicle coordinate system; based on the new near-field uneven road dense point cloud and The spatial position information of the target is obtained to obtain the dense point cloud inside it; based on the dense point cloud inside it, three-dimensional shape fitting is performed to obtain key information such as its orientation, length, width, depth, etc., and the dense point cloud data contained in the target is removed from the new near-field non-smooth road dense point cloud.

[0090] In order to further improve the recognition accuracy of target obstacles, target recognition is performed on the remaining dense point cloud data after removing the dense point cloud data of obstacles at each moment to determine whether there are new target obstacles; if there are new target obstacles, three-dimensional shape fitting is performed on the corresponding dense point cloud data to obtain the three-dimensional shape information of the new target obstacles. In other words, according to the above embodiment, after removing the point cloud data of the identified target obstacle from the latest near-field non-smooth road surface dense point cloud at the current moment, the remaining dense point cloud area can also be fitted. Identify: For each , we can obtain the dense point cloud inside it; using the point cloud recognition model The internal dense point cloud is classified and identified to determine whether it is a low raised obstacle / pothole area; if it is a low raised obstacle / pothole area, three-dimensional shape fitting is performed based on its internal dense point cloud to obtain key information such as its orientation, length, width, and depth.

[0091] As can be seen from the above, this embodiment is based on the distribution characteristics of point cloud data, that is, the closer to the laser radar, the denser the point cloud distribution, and the more accurate the spatial positioning accuracy. As the vehicle continues to move forward, the confirmed low raised obstacles / potholes will continue to approach the vehicle, and the granularity of the point cloud data inside them will become smaller, so the accuracy of its three-dimensional shape data can be gradually improved, thereby achieving high-precision calculation of the three-dimensional shape information of the target obstacle.

[0092] It should be noted that there is no strict order of execution between the steps in the present invention. As long as they comply with the logical order, these steps can be executed simultaneously or in a certain preset order. Figure 1 and Figure 5 This is just a schematic and does not mean that this is the only execution order.

[0093] The present invention also provides a corresponding device for the road environment state perception method, which further makes the method more practical. Among them, the device can be described from the perspective of functional modules and hardware. The road environment state perception device provided by the present invention is introduced below. The device is used to implement the road environment state perception method provided by the present invention. In this embodiment, the road environment state perception device may include or be divided into one or more program modules, and the one or more program modules are stored in a storage medium and executed by one or more processors to complete the road environment state perception method disclosed in Example 1. The program module referred to in this embodiment refers to a series of computer program instruction segments that can complete specific functions, which is more suitable for describing the execution process of the road environment state perception device in the storage medium than the program itself. The following description will specifically introduce the functions of each program module of this embodiment. The road environment state perception device described below and the road environment state perception method described above can be referenced to each other.

[0094] From the perspective of functional modules, see Figure 6 , Figure 6 The structure diagram of the road environment state perception device provided in this embodiment under a specific implementation manner, the device may include:

[0095] The point cloud data acquisition module 601 is used to acquire dense point cloud data of the road surface in the target area ahead of the vehicle, and divide the dense point cloud data of the road surface into near-field dense point cloud data and far-field dense point cloud data according to the distance between the point cloud data and the vehicle.

[0096] The far-field recognition module 602 is used to determine suspected obstacles in the target area by analyzing the road surface flatness of the far-field dense point cloud data.

[0097] The near-field verification module 603 is used to verify whether the suspected obstacle is the target obstacle based on the near-field dense point cloud data, and when the suspected obstacle is verified to be the target obstacle, determine the three-dimensional shape information of the target obstacle based on the point cloud data at adjacent moments and the matching results thereof.

[0098] The environment perception module 604 is used to generate the vehicle's road environment state perception results based on the three-dimensional shape information of each target obstacle obtained during the vehicle's driving process; wherein the road surface dense point cloud data is obtained by removing the point cloud data of the road surface area that is not the target area from the splicing result of the point cloud data at the current moment and the splicing point cloud data at the previous moment.

[0099] Exemplarily, in some implementations of this embodiment, the far-field recognition module 602 may also be used for:

[0100] The first target sub-region corresponding to the far-field dense point cloud data is rasterized according to the first preset grid size, and the first target sub-region is segmented according to the clustering result of the average normal vector in each grid of the first target sub-region to obtain multiple types of first road surface regions; for each first road surface region, the current road surface region is rasterized according to the second preset grid size, and a road surface flatness region that meets the preset flatness is determined according to the direction of the average normal vector of each grid of the current first road surface region, and the remaining area of ​​the current first road surface region after eliminating the road surface flatness region is merged based on the spatial distance relationship to serve as the spatial position of the suspected obstacle; wherein the target area includes the first target sub-region and the second target sub-region, the maximum distance between the second target sub-region and the vehicle along the vehicle driving direction is less than the minimum distance between the first target sub-region and the vehicle, the point cloud data corresponding to the first target sub-region is the far-field dense point cloud data, the point cloud data corresponding to the second target sub-region is the near-field dense point cloud data, and the second preset grid size is smaller than the first preset grid size.

[0101] As an exemplary implementation method of the above embodiment, the above far-field recognition module 602 can be further used to: fit the three-dimensional space plane equation of the corresponding grid according to the three-dimensional position information of the point cloud data inside each grid of the current first road surface area; calculate the average normal vector in each grid of the current first road surface area based on the three-dimensional space plane equation of each grid of the current first road surface area; vote on whether each average normal vector is a road surface normal vector according to the direction of the average normal vector, with a preset angle threshold as an interval, and use the target average normal vector with the most votes as the road surface normal vector, and use the grid corresponding to the target average normal vector as the road surface flattening area.

[0102] Exemplarily, in some implementations of the present embodiment, the point cloud data acquisition module 601 may also be used to: acquire the vehicle motion data and vehicle motion posture data at the current moment as the posture point cloud data, determine the initial posture change information according to the posture point cloud data and the spliced ​​point cloud data at the previous moment of the current moment; use the initial posture change information as the initial value of the iterative calculation, perform point cloud matching calculation on the point cloud data at the current moment and the spliced ​​point cloud data at the previous moment, and determine the rotation change matrix and translation vector; perform point cloud matching calculation on the spliced ​​point cloud data at the previous moment according to the rotation change matrix and translation vector. Transform, and remove the point cloud data that is not the location of the vehicle at the current moment in the transformed spliced ​​point cloud data to serve as the dense point cloud data of the previous moment; splice the dense point cloud data of the previous moment with the point cloud data of the current moment to obtain the dense point cloud data of the current moment; obtain the image information of the driving area by performing semantic segmentation processing on the acquired driving scene image data; based on the driving area image information, according to the correspondence between the image pixels and the three-dimensional points of the point cloud, eliminate the non-road data in the dense point cloud data of the current moment, and obtain the dense point cloud data of the road surface corresponding to the road surface in the target area.

[0103] As an exemplary implementation method of the above embodiment, the above point cloud data acquisition module 601 can also be used to: obtain laser point cloud data, vehicle motion data and vehicle motion posture data carrying a timestamp at the current moment; interpolate the corresponding vehicle motion data and vehicle motion posture data according to the timestamp of each three-dimensional point of the laser point cloud data; compensate the interpolated point cloud data accordingly according to the vehicle movement information and vehicle posture information to obtain point cloud distortion calibration data as the point cloud data at the current moment.

[0104] Exemplarily, in some other implementations of the present embodiment, the above-mentioned near-field verification module 603 can also be used to: remove the point cloud data of the flat road area in the near-field dense point cloud data to obtain a near-field dense point cloud with no flat road surface; obtain a rotation matrix and a translation vector based on the point cloud matching results of the point cloud data at the current moment and the dense point cloud data at the previous moment, and update the spatial position information of the suspected obstacle in the vehicle coordinates according to the rotation matrix and the translation vector; obtain the obstacle dense point cloud data of the suspected obstacle from the near-field dense point cloud with no flat road surface based on the spatial position information; and determine whether the suspected obstacle is a target obstacle by performing target recognition on the obstacle dense point cloud data.

[0105] As an exemplary implementation method of the above embodiment, the above-mentioned near-field verification module 603 can also be further used to: rasterize the second target sub-area corresponding to the near-field dense point cloud data according to a third preset grid size; divide the second target sub-area into multiple second road surface areas according to the clustering result of the average normal vector of each grid of the second target sub-area; correct the corresponding point cloud data according to the inclination angle of each second road surface area, and eliminate the point cloud data of the smooth road surface area in each second road surface area after correction according to the installation height of the laser radar; splice the remaining point cloud data of each second road surface area after eliminating the point cloud data of the smooth road surface area according to the corresponding spatial position relationship to obtain a near-field dense point cloud without a smooth road surface.

[0106] As another exemplary implementation method of the above embodiment, the above-mentioned near-field verification module 603 can be further used for: pre-training a point cloud recognition model; inputting dense point cloud data of an obstacle into the point cloud recognition model, and the point cloud recognition model outputs the recognition result that the dense point cloud data of the obstacle is a pothole area or a raised obstacle; if the dense point cloud data of the obstacle is a pothole area or a raised obstacle, determining that the suspected obstacle is a target obstacle.

[0107] Exemplarily, in some other implementations of the present embodiment, the near-field verification module 603 may also be used to: perform three-dimensional shape fitting processing on the dense point cloud data of the target obstacle to obtain the initial three-dimensional shape information of the target obstacle, and remove the dense point cloud data of the obstacle from the near-field non-flat road surface dense point cloud to serve as the dense point cloud data of the previous moment; obtain the point cloud data of the current moment when the vehicle continues to travel, and splice the point cloud data of the current moment with the dense point cloud data of the previous moment to obtain the dense point cloud data of the current moment; based on the new spatial position information of the target obstacle in the vehicle coordinate system at the current moment, determine the new obstacle dense point cloud data of the target obstacle from the dense point cloud data at the current moment, perform three-dimensional shape fitting on the new obstacle dense point cloud data, and obtain the three-dimensional shape information of the target obstacle at the current moment to update the initial three-dimensional shape information; wherein, the near-field non-flat road surface dense point cloud is obtained by removing the point cloud data of the flat road surface area in the near-field dense point cloud data.

[0108] As an exemplary implementation method of the above embodiment, the above-mentioned near-field verification module 603 can be further used to: perform point cloud matching on the point cloud data at the current moment and the dense point cloud data at the previous moment; determine a new rotation matrix and a new translation vector based on the point cloud matching results, and update the new spatial position information of the target obstacle in the vehicle coordinate system at the current moment based on the new rotation matrix and the new translation vector.

[0109] As an exemplary implementation method of the above embodiment, the above-mentioned near-field verification module 603 can be further used to: perform target recognition on the dense point cloud data remaining after removing the dense point cloud data of obstacles at each moment to determine whether there are new target obstacles; if there are new target obstacles, perform three-dimensional morphology fitting on the corresponding dense point cloud data to obtain the three-dimensional morphology information of the new target obstacle.

[0110] The road environment state perception device mentioned above is described from the perspective of functional modules. Furthermore, the present invention also provides an electronic device, which is described from the perspective of hardware. Figure 7 The following is a schematic diagram of the structure of an electronic device provided by an embodiment of the present invention in one implementation manner. Figure 7 As shown, the electronic device includes a memory 70 for storing computer programs; a processor 71 for implementing the steps of the road environment state perception method mentioned in any of the above embodiments when executing the computer program.

[0111] Among them, the processor 71 may include one or more processing cores, such as a 4-core processor, an 8-core processor, and the processor 71 may also be a controller, a microcontroller, a microprocessor or other data processing chip. The processor 71 may be implemented in at least one hardware form of DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). The processor 71 may also include a main processor and a coprocessor. The main processor is a processor for processing data in the awake state, also known as a CPU (Central Processing Unit); the coprocessor is a low-power processor for processing data in the standby state. In some embodiments, the processor 71 may be integrated with a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor 71 may also include an AI (Artificial Intelligence) processor, which is used to process computing operations related to machine learning.

[0112] The memory 70 may include one or more computer non-volatile storage media, which may be non-transitory. The memory 70 may also include high-speed random access memory and non-volatile memory, such as one or more disk storage devices and flash memory storage devices. In some embodiments, the memory 70 may be an internal storage unit of an electronic device, such as a hard disk of a server. In other embodiments, the memory 70 may also be an external storage device of an electronic device, such as a plug-in hard disk equipped on a server, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. Further, the memory 70 may also include both an internal storage unit of an electronic device and an external storage device. The memory 70 may not only be used to store application software and various types of data installed in the electronic device, such as: the code of the program in the process of executing the road environment state perception method, but also be used to temporarily store data that has been output or is to be output. In this embodiment, the memory 70 is at least used to store the following computer program 701, wherein, after the computer program is loaded and executed by the processor 71, the relevant steps of the road environment state perception method disclosed in any of the aforementioned embodiments can be implemented. In addition, the resources stored in the memory 70 may also include an operating system 702 and data 703, etc., and the storage method may be temporary storage or permanent storage. The operating system 702 may include Windows, Unix, Linux, etc. The data 703 may include but is not limited to data corresponding to the road environment state perception result, etc.

[0113] In some embodiments, the electronic device may further include a display screen 72, an input / output interface 73, a communication interface 74 or a network interface, a power supply 75 and a communication bus 76. Among them, the display screen 72 and the input / output interface 73, such as a keyboard, belong to the user interface, and exemplary user interfaces may also include standard wired interfaces, wireless interfaces, and the like. Optionally, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, and an OLED (Organic Light-Emitting Diode) touch device, and the like. The display may also be appropriately referred to as a display screen or a display unit, which is used to display information processed in the electronic device and to display a visual user interface. The communication interface 74 may exemplarily include a wired interface and / or a wireless interface, such as a WI-FI interface, a Bluetooth interface, and the like, which are generally used to establish a communication connection between the electronic device and other electronic devices. The communication bus 76 may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, and the like. The bus may be divided into an address bus, a data bus, a control bus, and the like. For ease of representation, Figure 7 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0114] Those skilled in the art will understand that Figure 7 The structure shown in the figure does not constitute a limitation on the electronic device, and may include more or fewer components than shown in the figure, for example, it may also include a sensor 77 to implement various functions.

[0115] It is understandable that if the road environment state perception method in the above embodiment is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a non-volatile storage medium. Based on such an understanding, the technical solution of the present invention is essentially or the part that contributes to the relevant technology or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium to execute all or part of the steps of the methods of each embodiment of the present invention. The aforementioned storage medium includes but is not limited to: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), electrically erasable programmable ROM, register, hard disk, multimedia card, card-type memory (such as SD or DX memory, etc.), magnetic memory, removable disk, CD-ROM, magnetic disk or optical disk, etc. Various media that can store program codes. Based on this, the present invention also provides a non-volatile storage medium on which a computer program is stored, and when the computer program is executed by a processor, the steps of the road environment state perception method recorded in any of the above embodiments are performed.

[0116] It is understandable that if the road environment state perception method in the above embodiment is implemented in the form of a software functional unit and sold or used as an independent product, the computer software product does not need to be stored in a physical storage medium. For example, it can be directly transmitted to a computer or other device with information processing capabilities through a wired network or a wireless network to execute all or part of the steps of the methods of each embodiment of the present invention. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the relevant technology or all or part of the technical solution can be embodied in the form of a software product. Based on this, the present invention also provides a computer program product that stores a computer program. When the computer program is executed by a processor, it performs the steps of the road environment state perception method recorded in any of the above embodiments.

[0117] Finally, the present invention also provides an unmanned vehicle that fully reuses its own sensors, such as cameras and lidars, and does not require the installation of other additional sensors. A computer program that implements the road environment state perception method as described in any of the previous embodiments is built into the existing automatic driving perception system. The automatic driving perception system can be used to implement the steps of the road environment state perception method as described in any of the previous embodiments when executing the computer program stored in the memory, and increase the accuracy of identifying low raised obstacles / potholes by densifying the point cloud. Then, considering the property that the spatial accuracy of the point cloud increases as the target is closer to the lidar, by first identifying in the far field and then verifying in the near field, not only can the computing consumption be effectively reduced, but also the position and shape information of each low raised obstacle / pothole can be corrected and refined step by step. In this way, the unmanned vehicle can accurately identify low raised obstacles such as speed bumps and cracks, deep pits, shallow pits or potholes as early as possible in the automatic driving scenario. Figure 8 As shown in the figure, and making reasonable plans, the safety and comfort of the self-driving system can be effectively improved.

[0118] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the hardware disclosed in the embodiments, including electronic devices, non-volatile storage media, computer program products and unmanned vehicles, since they correspond to the methods disclosed in the embodiments, the description is relatively simple, and the relevant parts can be referred to the method part description.

[0119] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0120] The above is a detailed introduction to a road environment state perception method, electronic device, non-volatile storage medium, computer program product and unmanned vehicle provided by the present invention. Specific examples are used herein to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea. It should be pointed out that based on the embodiments of the present invention, for ordinary technicians in this technical field, all other embodiments obtained without creative work are within the scope of protection of the present invention. Without departing from the principles of the present invention, the present invention can also be improved and modified in a number of ways, and these improvements and modifications also fall within the scope of protection of the present invention.

Claims

1. A method for perceiving a road environment state, characterized in that: include: Acquire dense point cloud data of a road surface in a target area ahead of the vehicle, and divide the dense point cloud data of the road surface into near-field dense point cloud data and far-field dense point cloud data according to the distance between the point cloud data and the vehicle; Determining suspected obstacles in the target area by analyzing the road surface flatness of the far-field dense point cloud data; Verifying whether the suspected obstacle is a target obstacle based on the near-field dense point cloud data, and when the suspected obstacle is verified to be a target obstacle, determining the three-dimensional shape information of the target obstacle based on the point cloud data at adjacent moments and the matching results thereof; Generate the vehicle's road environment state perception result based on the three-dimensional shape information of each target obstacle obtained during the vehicle's driving process; Wherein, determining the three-dimensional shape information of the target obstacle according to the point cloud data at adjacent moments and the matching results thereof includes: Performing three-dimensional shape fitting processing on the dense point cloud data of the target obstacle to obtain initial three-dimensional shape information of the target obstacle, and removing the dense point cloud data of the obstacle from the near-field non-smooth road surface dense point cloud to serve as the dense point cloud data of the previous moment; Acquire point cloud data at a current moment while the vehicle is continuing to travel, and splice the point cloud data at the current moment with dense point cloud data at a previous moment to obtain dense point cloud data at the current moment; Based on the new spatial position information of the target obstacle in the vehicle coordinate system at the current moment, new obstacle dense point cloud data of the target obstacle is determined from the dense point cloud data at the current moment, and three-dimensional shape fitting is performed on the new obstacle dense point cloud data to obtain the three-dimensional shape information of the target obstacle at the current moment, so as to update the initial three-dimensional shape information; Among them, the dense point cloud data of the road surface is obtained by removing the point cloud data of the road surface area that is not the target area in the splicing result of the point cloud data at the current moment and the spliced ​​point cloud data at the previous moment; the near-field uneven road surface dense point cloud is obtained by removing the point cloud data of the flat road surface area in the near-field dense point cloud data.

2. The road environment state perception method according to claim 1, characterized in that: By analyzing the road surface flatness of the far-field dense point cloud data, the suspected obstacles in the target area are determined, including: rasterizing the first target sub-region corresponding to the far-field dense point cloud data according to a first preset grid size, and segmenting the first target sub-region according to a clustering result of average normal vectors in each grid of the first target sub-region to obtain a plurality of first road surface regions; For each first road surface area, the current road surface area is rasterized according to the second preset grid size, and a road surface flatness area that meets the preset flatness is determined according to the direction of the average normal vector of each grid of the current first road surface area, and the remaining area of ​​the current first road surface area after excluding the road surface flatness area is merged based on the spatial distance relationship to serve as the spatial position of the suspected obstacle; Wherein, the target area includes a first target sub-area and a second target sub-area, along the driving direction of the vehicle, the maximum distance between the second target sub-area and the vehicle is smaller than the minimum distance between the first target sub-area and the vehicle, the point cloud data corresponding to the first target sub-area is the far-field dense point cloud data, the point cloud data corresponding to the second target sub-area is the near-field dense point cloud data, and the second preset grid size is smaller than the first preset grid size.

3. The road environment state perception method according to claim 2, characterized in that: Determining a road surface smoothing area that meets a preset smoothness according to the direction of the average normal vector of each grid in the current first road surface area includes: According to the three-dimensional position information of the point cloud data inside each grid of the current first road surface area, fitting to obtain the three-dimensional space plane equation of the corresponding grid; Calculating an average normal vector in each grid of the current first road surface area based on a three-dimensional space plane equation of each grid of the current first road surface area; According to the direction of the average normal vector, a vote is performed on whether each average normal vector is a road surface normal vector at intervals of a preset angle threshold, and the target average normal vector with the most votes is used as the road surface normal vector, and the grid corresponding to the target average normal vector is used as the road surface flattening area.

4. The road environment state perception method according to claim 1, characterized in that: Obtain dense point cloud data of the road surface in the target area ahead of the vehicle, including: Acquire vehicle motion data and vehicle motion posture data at the current moment as posture point cloud data, and determine initial posture change information based on the posture point cloud data and the spliced ​​point cloud data at the previous moment before the current moment; Using the initial position change information as the initial value for iterative calculation, performing point cloud matching calculation on the point cloud data at the current moment and the spliced ​​point cloud data at the previous moment, and determining the rotation change matrix and the translation vector; According to the rotation change matrix and the translation vector, the spliced ​​point cloud data of the previous moment is transformed, and the point cloud data that is not the position of the vehicle at the current moment in the transformed spliced ​​point cloud data is removed to serve as the dense point cloud data of the previous moment; The dense point cloud data of the previous moment is spliced ​​with the point cloud data of the current moment to obtain the dense point cloud data of the current moment; By performing semantic segmentation processing on the acquired driving scene image data, the driving area image information is obtained; Based on the driving area image information, non-road data in the dense point cloud data at the current moment is eliminated according to the correspondence between image pixels and three-dimensional points of the point cloud, so as to obtain road surface dense point cloud data corresponding to the road surface in the target area.

5. The road environment state perception method according to claim 4, characterized in that: Before splicing the dense point cloud data of the previous moment with the point cloud data of the current moment, it also includes: Obtain laser point cloud data, vehicle motion data, and vehicle motion posture data with timestamp at the current moment; According to the timestamp of each three-dimensional point of the laser point cloud data, interpolation processing is performed on the corresponding vehicle motion data and vehicle motion posture data; According to the vehicle movement information and vehicle posture information, the interpolated point cloud data is compensated accordingly to obtain the point cloud distortion calibration data as the point cloud data at the current moment.

6. The road environment state perception method according to claim 1, characterized in that: Verifying whether the suspected obstacle is a target obstacle according to the near-field dense point cloud data includes: Removing point cloud data of a smooth road surface area from the near-field dense point cloud data to obtain a near-field non-smooth road surface dense point cloud; Obtaining a rotation matrix and a translation vector according to a point cloud matching result between the point cloud data at the current moment and the dense point cloud data at the previous moment, and updating the spatial position information of the suspected obstacle in the vehicle coordinates according to the rotation matrix and the translation vector; Based on the spatial position information, obtaining obstacle dense point cloud data of the suspected obstacle from the near-field uneven road surface dense point cloud; By performing target recognition on the obstacle dense point cloud data, it is determined whether the suspected obstacle is a target obstacle.

7. The road environment state perception method according to claim 6, characterized in that: Removing point cloud data of a flat road surface area from the near-field dense point cloud data includes: Performing rasterization processing on the second target sub-region corresponding to the near-field dense point cloud data according to a third preset grid size; Dividing the second target sub-region into a plurality of second road surface regions according to a clustering result of average normal vectors of each grid of the second target sub-region; Correcting the corresponding point cloud data according to the inclination angle of each second road surface area, and removing the corrected point cloud data of the road surface flat area in each second road surface area according to the installation height of the laser radar; The remaining point cloud data of each second road surface area after removing the point cloud data of the smooth road surface area are spliced ​​according to the corresponding spatial position relationship to obtain a near-field non-smooth road surface dense point cloud.

8. The road environment state perception method according to claim 6, characterized in that: By performing target recognition on the obstacle dense point cloud data, including: Pre-train point cloud recognition model; Inputting the dense point cloud data of the obstacle into the point cloud recognition model, and the point cloud recognition model outputting the recognition result that the dense point cloud data of the obstacle is a pothole area or a raised obstacle; If the dense point cloud data of the obstacle is a pothole area or a raised obstacle, the suspected obstacle is determined to be a target obstacle.

9. The road environment state perception method according to claim 1, characterized in that: Before the new spatial position information of the target obstacle in the vehicle coordinate system at the current moment is obtained, the method further includes: Perform point cloud matching on the point cloud data at the current moment and the dense point cloud data at the previous moment; A new rotation matrix and a new translation vector are determined according to the point cloud matching result, and new spatial position information of the target obstacle in the vehicle coordinate system at the current moment is updated according to the new rotation matrix and the new translation vector.

10. The road environment state perception method according to claim 1, characterized in that: After determining the three-dimensional shape information of the target obstacle according to the point cloud data at adjacent moments and the matching results thereof, the method further includes: Performing target recognition on the remaining dense point cloud data after removing the dense point cloud data of the obstacle at each moment to determine whether there is a new target obstacle; If there is a new target obstacle, the corresponding dense point cloud data is fitted with a three-dimensional shape to obtain the three-dimensional shape information of the new target obstacle.

11. An electronic device, characterized in that: The method comprises a processor and a memory, wherein the processor is used to implement the steps of the road environment state perception method as claimed in any one of claims 1 to 10 when executing the computer program stored in the memory.

12. A non-volatile storage medium, characterized in that: The non-volatile storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the road environment state perception method as described in any one of claims 1 to 10 are implemented.

13. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instructions are executed by a processor, the steps of the road environment state perception method described in any one of claims 1 to 10 are implemented.

14. An unmanned vehicle, characterized in that: It includes an autonomous driving perception system, which is used to implement the steps of the road environment state perception method as described in any one of claims 1 to 10 when executing a computer program stored in a memory.

Images