Methods, devices, equipment, and storage media for detailed description of the vehicle's surrounding environment.

Abstract

This invention discloses a method, apparatus, device, and storage medium for refined description of the environment surrounding a vehicle, including: acquiring point cloud data, image data, and odometer data; performing target detection based on the acquired point cloud data and outputting dynamic target detection boxes; performing target detection based on the acquired image data to obtain a mask of the drivable area on the image; dividing the original point cloud data into unobstructed directional point cloud data, noisy point cloud data, dynamic obstacle point cloud data, static obstacle point cloud data, and refined ground point cloud data; and performing three-dimensional division of the space surrounding the vehicle, refreshing the five types of point cloud data with rays of different logics in a three-dimensional grid to refine the description of the three-dimensional environment information surrounding the vehicle. This invention, based on visual segmentation and classified point cloud ray refreshing, effectively combines visual and laser data to refine the description of the surrounding three-dimensional environment information and can effectively eliminate false detections caused by inherent defects in lasers such as noise and drag points.

Description

Technical Field

[0001] This invention belongs to the field of autonomous driving technology, specifically relating to a method, device, equipment, and storage medium for fine-grained description of the environment surrounding a vehicle. Background Technology

[0002] Autonomous navigation capability is crucial for the safe movement of autonomous vehicles in complex environments. A key aspect of autonomous vehicle navigation research is real-time environmental perception, including the perception of dynamic obstacles and the description of the static environment.

[0003] In urban traffic environments, dynamic obstacles are generally composed of pedestrians, motor vehicles, and non-motor vehicles, and the categories are relatively fixed. Therefore, in the field of autonomous driving, object detection models are usually used for detection, and they are described by detection boxes with categories and prediction lines.

[0004] Static obstacles in the traffic environment are quite complex, including not only various terrain obstacles and traffic equipment in the city, but also items that are not commonly seen in traffic scenarios, such as goods that have fallen off vehicles and temporary piles of construction materials. Due to their numerous semantics and varying sizes, they are usually described using a three-dimensional raster map.

[0005] Static environment perception technology involves vehicles using their own sensors (such as lidar and cameras) to perceive information about their surrounding environment and detect the passable areas.

[0006] Static 3D environment perception technology based on LiDAR time-series fusion has the characteristics of high accuracy, but it is difficult to solve the problems of water mist, dust noise, and the expansion and dragging phenomenon of LiDAR when facing highly reflective road signs; vision-based static environment perception technology usually has the problems of insufficient accuracy and weak ability to detect small obstacles.

[0007] How to better integrate visual and laser information to ensure an accurate description of the surrounding environment has become an urgent problem to be solved. Summary of the Invention

[0008] To address the aforementioned technical problems, this invention proposes a method, apparatus, device, and storage medium for fine-grained description of the environment surrounding a vehicle.

[0009] To achieve the above objectives, the technical solution of the present invention is as follows:

[0010] In a first aspect, the present invention discloses a method for refined description of the environment surrounding a vehicle, comprising:

[0011] Step S1: Collect point cloud data, image data, and odometry data;

[0012] Step S2: Target detection is performed based on the collected point cloud data to detect dynamic targets and output dynamic target detection boxes;

[0013] Step S3: Perform target detection based on the acquired image data to obtain a drivable area mask on the image;

[0014] Step S4: Based on the collected point cloud data, obtain unobstructed azimuth point cloud data and normal point cloud data through LiDAR polar coordinate sorting analysis;

[0015] Step S5: Based on the dynamic target detection box obtained in step S2, perform dynamic obstacle filtering on the normal point cloud data obtained in step S4 to obtain dynamic obstacle point cloud data and static environment point cloud data.

[0016] Step S6: Process the static environment point cloud data obtained in step S5 and divide it into fine ground point cloud data and fine obstacle point cloud data;

[0017] Step S7: Back-project the fine ground point cloud data and fine obstacle point cloud data obtained in step S6 onto the drivable area mask of the image obtained in step S3.

[0018] Step S8: For the orientation corresponding to the pixels in the drivable area, when both fine ground point cloud data and fine obstacle point cloud data exist, and the part of the fine obstacle point cloud data that is less than the distance from the camera center is less than the distance from the fine ground point cloud data, it is marked as noisy point cloud data, and the rest is marked as static obstacle point cloud data.

[0019] Step S9: Divide the space around the vehicle into three dimensions, and refresh the unobstructed directional point cloud data, noisy point cloud data, dynamic obstacle point cloud data, static obstacle point cloud data, and fine ground point cloud data with different logics in the three-dimensional grid to refine the description of the three-dimensional environment information around the vehicle.

[0020] Based on the above technical solution, the following improvements can be made:

[0021] As a preferred embodiment, step S4 includes:

[0022] Step S4.1: Based on the collected point cloud data, confirm the polar coordinate values ​​of the parts with no returned data through LiDAR polar coordinate sorting analysis;

[0023] Step S4.2: Based on the polar coordinate values ​​of the non-returning data portion obtained in step S4.1, and combined with the partial point cloud obtained from the nominal limit of laser ranging, mark it as unobstructed azimuth point cloud data;

[0024] The remaining portion is labeled as normal point cloud data.

[0025] As a preferred embodiment, step S6 includes:

[0026] Step S6.1: Process the static environment point cloud data obtained in step S5 using the ground segmentation algorithm, dividing it into coarse ground point cloud data and coarse obstacle point cloud data;

[0027] Step S6.2: The coarse ground point cloud data obtained in step S6.1 is overlaid with odometry data to obtain dense coarse ground point cloud data;

[0028] Step S6.3: Perform fine obstacle detection on the dense coarse ground point cloud data obtained in step S6.2 to detect small obstacle point cloud data;

[0029] Step S6.4: Remove the small obstacle point cloud data detected in step S6.3 from the coarse ground point cloud data to form fine ground point cloud data;

[0030] This data is then added to the coarse obstacle point cloud data to form the fine obstacle point cloud data.

[0031] As a preferred embodiment, step S9 includes:

[0032] For unobstructed point cloud data, noisy point cloud data, and dynamic obstacle point cloud data, the probability of the 3D grid occupancy by the rays of these three types of point clouds is updated by decreasing, while the 3D grid hit by these three types of point clouds is not processed.

[0033] For fine ground point cloud data, the probability of occupancy of the 3D grid through which such point cloud rays pass is decreased and updated, and the probability of occupancy of the 3D grid hit by such point cloud is increased and updated, and drivable attributes are assigned.

[0034] For static obstacle point cloud data, the probability of occupancy of the 3D grid through which such point cloud rays pass is decreased and updated, while the probability of occupancy of the 3D grid hit by such point cloud is increased and updated, and an inaccessible attribute is assigned.

[0035] Secondly, the present invention discloses a device for fine-grained description of the environment surrounding a vehicle, comprising:

[0036] The acquisition module is used to acquire point cloud data, image data, and odometer data;

[0037] The point cloud detection module is used to perform target detection based on the collected point cloud data, detect dynamic targets, and output dynamic target detection boxes;

[0038] The image detection module is used to perform target detection based on the acquired image data and obtain a mask of the drivable area on the image;

[0039] The polar coordinate sorting analysis module is used to obtain unobstructed azimuth point cloud data and normal point cloud data based on the collected point cloud data through LiDAR polar coordinate sorting analysis.

[0040] The dynamic obstacle filtering module is used to obtain dynamic target detection boxes based on the point cloud detection module, and to perform dynamic obstacle filtering processing on the normal point cloud data obtained by the polar coordinate sorting analysis module to obtain dynamic obstacle point cloud data and static environment point cloud data.

[0041] The ground segmentation module is used to process the static environment point cloud data obtained by the dynamic obstacle filtering module and divide it into fine ground point cloud data and fine obstacle point cloud data.

[0042] The back projection module is used to back project the fine ground point cloud data and fine obstacle point cloud data obtained by the ground segmentation module onto the drivable area mask of the image obtained by the image detection module.

[0043] The point cloud segmentation module is used to classify the orientation of pixels in the drivable area. When both fine ground point cloud data and fine obstacle point cloud data exist, and the part of the fine obstacle point cloud data that is less than the distance from the camera center is less than the distance from the fine ground point cloud data, it is marked as noisy point cloud data, and the rest is marked as static obstacle point cloud data.

[0044] The 3D grid refresh module is used to divide the space around the vehicle into three dimensions. It refreshes the unobstructed directional point cloud data, noisy point cloud data, dynamic obstacle point cloud data, static obstacle point cloud data, and fine ground point cloud data with different logics in the 3D grid, so as to describe the 3D environment information around the vehicle in a refined manner.

[0045] As a preferred solution, the polar coordinate sorting analysis module includes:

[0046] The sorting unit is used to identify the polar coordinate values ​​of parts with no returned data by analyzing the collected point cloud data using LiDAR polar coordinate sorting.

[0047] The partitioning unit is used to mark the partial point cloud obtained by combining the polar coordinate values ​​of the non-returning data part obtained by the sorting unit with the nominal limit of laser ranging as unobstructed azimuth point cloud data.

[0048] The remaining portion is labeled as normal point cloud data.

[0049] As a preferred embodiment, the ground segmentation module includes:

[0050] The coarse segmentation unit is used to process the static environment point cloud data obtained by the dynamic obstacle filtering module using the ground segmentation algorithm, and divides it into coarse ground point cloud data and coarse obstacle point cloud data.

[0051] The frame stacking unit is used to stack the coarse ground point cloud data obtained by the coarse segmentation unit with odometry data to obtain dense coarse ground point cloud data.

[0052] The detection unit is used to perform fine obstacle detection in the dense coarse ground point cloud data obtained by the frame stacking unit, and to detect small obstacle point cloud data.

[0053] The fine segmentation unit is used to remove the small obstacle point cloud data detected by the detection unit from the coarse ground point cloud data to form fine ground point cloud data;

[0054] This data is then added to the coarse obstacle point cloud data to form the fine obstacle point cloud data.

[0055] As a preferred embodiment, the 3D grid refresh module is used to perform the following methods:

[0056] For unobstructed point cloud data, noisy point cloud data, and dynamic obstacle point cloud data, the probability of the 3D grid occupancy by the rays of these three types of point clouds is updated by decreasing, while the 3D grid hit by these three types of point clouds is not processed.

[0057] For fine ground point cloud data, the probability of occupancy of the 3D grid through which such point cloud rays pass is decreased and updated, and the probability of occupancy of the 3D grid hit by such point cloud is increased and updated, and drivable attributes are assigned.

[0058] For static obstacle point cloud data, the probability of occupancy of the 3D grid through which such point cloud rays pass is decreased and updated, while the probability of occupancy of the 3D grid hit by such point cloud is increased and updated, and an inaccessible attribute is assigned.

[0059] Thirdly, the present invention discloses a computing device, comprising:

[0060] One or more processors;

[0061] Memory;

[0062] And one or more programs, wherein the one or more programs are stored in memory and configured to be executed by one or more processors, and the one or more programs include instructions for any of the above-described methods for fine-grained description of the vehicle's surrounding environment.

[0063] Fourthly, the present invention discloses a storage medium storing one or more computer-readable programs, the one or more programs including instructions adapted to be loaded by a memory and executed by any of the above-described methods for fine description of the vehicle's surrounding environment.

[0064] This invention discloses a method, apparatus, device, and storage medium for detailed description of the surrounding environment of a vehicle, which has the following beneficial effects:

[0065] First, this invention can effectively classify five types of point cloud data: unobstructed azimuth point cloud data, noisy point cloud data, dynamic obstacle point cloud data, static obstacle point cloud data, and fine ground point cloud data.

[0066] Secondly, this invention is based on visual segmentation and classification point cloud ray refresh, effectively combining visual and laser data to finely describe the surrounding three-dimensional environment information, and can effectively eliminate false detections caused by inherent laser defects such as noise and drag points. Attached Figure Description

[0067] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0068] Figure 1 A flowchart illustrating a method for refining the description of the vehicle's surrounding environment as provided in an embodiment of the present invention.

[0069] Figure 2 A block diagram of a computing device provided in an embodiment of the present invention. Detailed Implementation

[0070] The preferred embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0071] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0072] The expression “includes” is an “open-ended” expression, which means that there is a corresponding component or step, and should not be interpreted as excluding additional components or steps.

[0073] To achieve the objectives of this invention, some embodiments of the method, apparatus, device, and storage medium for detailed description of the vehicle's surrounding environment, such as... Figure 1 As shown, the methods for detailed description of the vehicle's surrounding environment include:

[0074] Step S101: Collect point cloud data, image data, and odometry data;

[0075] Step S102: Target detection is performed based on the collected point cloud data to detect dynamic targets and output dynamic target detection boxes;

[0076] Step S103: Perform target detection based on the acquired image data to obtain a drivable area mask on the image;

[0077] Step S104: Based on the collected point cloud data, obtain unobstructed azimuth point cloud data and normal point cloud data through LiDAR polar coordinate sorting analysis;

[0078] Step S105: Based on the dynamic target detection box obtained in step S102, perform dynamic obstacle filtering processing on the normal point cloud data obtained in step S104 to obtain dynamic obstacle point cloud data and static environment point cloud data.

[0079] Step S106: Process the static environment point cloud data obtained in step S105 and divide it into fine ground point cloud data and fine obstacle point cloud data;

[0080] Step S107: Back-project the fine ground point cloud data and fine obstacle point cloud data obtained in step S106 onto the drivable area mask of the image obtained in step S103.

[0081] Step S108: For the orientation corresponding to the pixels in the drivable area, when both fine ground point cloud data and fine obstacle point cloud data exist at the same time, and the part of the fine obstacle point cloud data that is less than the distance from the camera center is less than the distance from the fine ground point cloud data, it is marked as noisy point cloud data, and the rest is marked as static obstacle point cloud data.

[0082] Step S109: Divide the space around the vehicle into three dimensions, and refresh the unobstructed directional point cloud data, noisy point cloud data, dynamic obstacle point cloud data, static obstacle point cloud data, and fine ground point cloud data with different logics in the three-dimensional grid to refine the description of the three-dimensional environment information around the vehicle.

[0083] The steps described above are explained in detail below.

[0084] Step S101 can utilize sensors such as lidar and cameras carried by the vehicle to collect data.

[0085] The odometer data collected in step S101 specifically refers to the data accurately recording the vehicle's trajectory using sensors and SLAM algorithms. Common types of sensors include inertial odometers, laser odometers, and visual odometers. This invention can use a laser inertial odometer, which integrates an IMU inertial measurement sensor and a lidar sensor to accurately calculate the vehicle's trajectory data.

[0086] Step S102 involves target detection of the raw point cloud data. Through target detection model and target tracking technology, dynamic targets are detected in the real-time laser point cloud data, and dynamic target detection boxes are output.

[0087] Step S103 involves target detection in the camera image data to obtain the pixel description of obstacles and the drivable area mask in the image.

[0088] Step S104 further processes the point cloud data, including:

[0089] Step S4.1: Based on the collected point cloud data, the polar coordinate values ​​of the parts with no returned data are confirmed by the polar coordinate sorting analysis of the lidar (representing that the distance of objects in these directions exceeds the range measurement limit of the lidar).

[0090] Step S4.2: Based on the polar coordinate values ​​of the non-returning data portion obtained in step S4.1, and combined with the partial point cloud obtained from the nominal limit of laser ranging, mark it as unobstructed azimuth point cloud data;

[0091] The remaining portion is labeled as normal point cloud data.

[0092] Polar radius r:

[0093]

[0094] Polar angle θ:

[0095]

[0096] Polar angle ψ:

[0097]

[0098] Generally, point cloud data is parsed into xyz space rectangular coordinate system coordinates. The polar coordinate system parameters are obtained by using the above formulas from the coordinates (x, y, z) of each point cloud: polar radius r, polar angle θ, and polar angle ψ.

[0099] Under normal circumstances, the polar angles θ and ψ corresponding to the polar coordinates of each laser frame satisfy a definite list (e.g., the polar angle θ of a point cloud frame satisfies 0-360°, and the polar angle ψ satisfies -30°-25° according to the standard, etc.). When all points of a laser frame are transformed from the spatial rectangular coordinate system to the polar coordinate system, the missing parts in the list indicate that the distance measured by the laser point cloud emitted from that direction exceeds the nominal distance.

[0100] For example, according to the product manual of the lidar, if the range limit of the laser is 300m, then assume that the range in this direction is 300m. Therefore, the polar coordinates of the unobstructed point cloud can be obtained as follows: polar radius r' (e.g., 300m), polar angle θ', and polar angle ψ'.

[0101] Then, the xyz spatial rectangular coordinates of the unobstructed azimuth point cloud data are obtained by recalculating the formula.

[0102] Step S105 uses dynamic target detection boxes to process the normal point cloud data to obtain dynamic obstacle point cloud data and static environment point cloud data.

[0103] Step S106 further processes the static environment point cloud data, including:

[0104] Step S106.1: The static environment point cloud data obtained in step S105 is processed using a ground segmentation algorithm and divided into coarse ground point cloud data and coarse obstacle point cloud data.

[0105] Step S106.2: The coarse ground point cloud data obtained in step S106.1 is overlaid with odometry data to obtain dense coarse ground point cloud data;

[0106] Step S106.3: Perform fine obstacle detection (e.g., ground potholes and small obstacles) on the dense coarse ground point cloud data obtained in step S106.2, and detect the point cloud data of small obstacles;

[0107] Step S106.4: Remove the small obstacle point cloud data detected in step S106.3 from the coarse ground point cloud data to form fine ground point cloud data;

[0108] This data is then added to the coarse obstacle point cloud data to form the fine obstacle point cloud data.

[0109] The odometer data is the trajectory data of the vehicle's movement. Therefore, in step S106.2, the coarse ground point cloud data of each frame is stitched together and stacked according to the vehicle trajectory to become dense coarse ground point cloud data.

[0110] Statistical analysis of dense coarse ground point cloud data yields more accurate fine ground point cloud data and small obstacle point cloud data. The small obstacle point cloud data is then added to the coarse obstacle point cloud data to form even more accurate fine obstacle point cloud data.

[0111] After the above steps, single-frame point cloud data can be divided into four categories: unobstructed azimuth point cloud data, fine ground point cloud data, fine obstacle point cloud data, and dynamic obstacle point cloud data.

[0112] Camera images from all directions can be trained to identify obstacles and traversable areas through target recognition and traversable area training. The pixel set corresponding to the traversable area mask represents the absence of obstacles in a specified direction from the camera's optical center. This method can effectively eliminate false detections (noise) in the laser point cloud.

[0113] Steps S107 and S108 back-project the dense, precise ground point cloud data and the real-time, precise obstacle point cloud data onto the drivable area mask of the image.

[0114] For the orientation corresponding to the pixels in the drivable area, when both fine ground point cloud data and fine obstacle point cloud data exist simultaneously, and the part of the fine obstacle point cloud data that is less than the distance from the camera center than the distance from the fine ground point cloud data (which is the noise part of the laser), it is marked as noise point cloud data, and the rest is marked as static obstacle point cloud data.

[0115] Therefore, based on the single-frame point cloud, the fine obstacle point cloud data is further divided into noisy point cloud data and static obstacle point cloud data.

[0116] Finally, the space around the vehicle is divided into three dimensions, and the five types of point cloud data are refreshed with rays of different logics in the three-dimensional grid to obtain a three-dimensional grid data describing the environment around the vehicle.

[0117] Therefore, step S109 includes:

[0118] (1) For unobstructed directional point cloud data, noisy point cloud data and dynamic obstacle point cloud data, the probability of the three-dimensional grid occupancy through the rays of these three types of point clouds is updated by decreasing, and the three-dimensional grid hit by these three types of point clouds is not processed.

[0119] (2) For fine ground point cloud data, the probability of the 3D grid occupancy of the point cloud ray is decreased and the probability of the 3D grid occupancy of the point cloud hit is increased and drivable attributes are assigned.

[0120] (3) For static obstacle point cloud data, the probability of the three-dimensional grid occupancy when the point cloud ray passes through it is decreased and the probability of the three-dimensional grid occupancy when the point cloud hits it is increased and assigned the non-driving attribute.

[0121] This invention can accurately divide raw point cloud data into unobstructed directional point cloud data, noisy point cloud data, dynamic obstacle point cloud data, static obstacle point cloud data, and fine ground point cloud data.

[0122] In other embodiments, the present invention discloses a device for fine-grained description of the environment surrounding a vehicle, comprising:

[0123] The acquisition module is used to acquire point cloud data, image data, and odometer data;

[0124] The point cloud detection module is used to perform target detection based on the collected point cloud data, detect dynamic targets, and output dynamic target detection boxes;

[0125] The image detection module is used to perform target detection based on the acquired image data and obtain a mask of the drivable area on the image;

[0126] The polar coordinate sorting analysis module is used to obtain unobstructed azimuth point cloud data and normal point cloud data based on the collected point cloud data through LiDAR polar coordinate sorting analysis.

[0127] The dynamic obstacle filtering module is used to obtain dynamic target detection boxes based on the point cloud detection module, and to perform dynamic obstacle filtering processing on the normal point cloud data obtained by the polar coordinate sorting analysis module to obtain dynamic obstacle point cloud data and static environment point cloud data.

[0128] The ground segmentation module is used to process the static environment point cloud data obtained by the dynamic obstacle filtering module and divide it into fine ground point cloud data and fine obstacle point cloud data.

[0129] The back projection module is used to back project the fine ground point cloud data and fine obstacle point cloud data obtained by the ground segmentation module onto the drivable area mask of the image obtained by the image detection module.

[0130] The point cloud segmentation module is used to classify the orientation of pixels in the drivable area. When both fine ground point cloud data and fine obstacle point cloud data exist, and the part of the fine obstacle point cloud data that is less than the distance from the camera center is less than the distance from the fine ground point cloud data, it is marked as noisy point cloud data, and the rest is marked as static obstacle point cloud data.

[0131] The 3D grid refresh module is used to divide the space around the vehicle into three dimensions. It refreshes the unobstructed directional point cloud data, noisy point cloud data, dynamic obstacle point cloud data, static obstacle point cloud data, and fine ground point cloud data with different logics in the 3D grid, so as to describe the 3D environment information around the vehicle in a refined manner.

[0132] Furthermore, the polar coordinate sorting analysis module includes:

[0133] The sorting unit is used to identify the polar coordinate values ​​of parts with no returned data by analyzing the collected point cloud data using LiDAR polar coordinate sorting.

[0134] The partitioning unit is used to mark the partial point cloud obtained by combining the polar coordinate values ​​of the non-returning data part obtained by the sorting unit with the nominal limit of laser ranging as unobstructed azimuth point cloud data.

[0135] The remaining portion is labeled as normal point cloud data.

[0136] Furthermore, the ground segmentation module includes:

[0137] The coarse segmentation unit is used to process the static environment point cloud data obtained by the dynamic obstacle filtering module using the ground segmentation algorithm, and divides it into coarse ground point cloud data and coarse obstacle point cloud data.

[0138] The frame stacking unit is used to stack the coarse ground point cloud data obtained by the coarse segmentation unit with odometry data to obtain dense coarse ground point cloud data.

[0139] The detection unit is used to perform fine obstacle detection in the dense coarse ground point cloud data obtained by the frame stacking unit, and to detect small obstacle point cloud data.

[0140] The fine segmentation unit is used to remove the small obstacle point cloud data detected by the detection unit from the coarse ground point cloud data to form fine ground point cloud data;

[0141] This data is then added to the coarse obstacle point cloud data to form the fine obstacle point cloud data.

[0142] Furthermore, the 3D raster refresh module is used to perform the following methods:

[0143] For unobstructed point cloud data, noisy point cloud data, and dynamic obstacle point cloud data, the probability of the 3D grid occupancy by the rays of these three types of point clouds is updated by decreasing, while the 3D grid hit by these three types of point clouds is not processed.

[0144] For fine ground point cloud data, the probability of occupancy of the 3D grid through which such point cloud rays pass is decreased and updated, and the probability of occupancy of the 3D grid hit by such point cloud is increased and updated, and drivable attributes are assigned.

[0145] For static obstacle point cloud data, the probability of occupancy of the 3D grid through which such point cloud rays pass is decreased and updated, while the probability of occupancy of the 3D grid hit by such point cloud is increased and updated, and an inaccessible attribute is assigned.

[0146] Furthermore, it should be noted that: the vehicle surrounding environment fine description device provided in the above embodiments is only illustrated by the division of the above functional modules when performing fine description of the vehicle surrounding environment. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the vehicle surrounding environment fine description device can be divided into different functional modules to complete all or part of the functions described above.

[0147] Furthermore, the embodiments of the vehicle surrounding environment fine description device and the vehicle surrounding environment fine description method provided in the above embodiments belong to the same concept, and their specific implementation process can be found in the method embodiments, which will not be repeated here.

[0148] Furthermore, in some other embodiments, such as Figure 2 As shown, the present invention also discloses a computing device, comprising:

[0149] One or more processors 201;

[0150] Memory 202;

[0151] And one or more programs, wherein the one or more programs are stored in memory 202 and configured to be executed by one or more processors 201, the one or more programs including instructions for the vehicle surrounding environment fine description method disclosed in the above embodiments.

[0152] Processor 201 may include one or more processing cores, such as a quad-core processor, an octa-core processor, etc. Processor 201 may be implemented using at least one hardware form selected from DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). Processor 201 may also include a main processor and a coprocessor. The main processor, also known as a CPU (Central Processing Unit), is used to process data in the wake-up state; the coprocessor is a low-power processor used to process data in the standby state. In some embodiments, processor 201 may integrate a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the screen. In some embodiments, processor 201 may also include an AI (Artificial Intelligence) processor, which is used to handle computational operations related to machine learning.

[0153] The memory 202 may include one or more computer-readable storage media, which may be non-transitory. The memory 202 may also include high-speed random access memory and non-volatile memory, such as one or more disk storage devices or flash memory devices. In some embodiments, the non-transitory computer-readable storage media in the memory 202 are used to store at least one instruction, which is executed by the processor 201 to implement the refined description method of the vehicle's surrounding environment provided in the method embodiments of the present invention.

[0154] In addition, the computing device may optionally include: a peripheral device interface and at least one peripheral device. The processor 201, memory 202, and peripheral device interface can be connected via a bus or signal lines. Each peripheral device can be connected to the peripheral device interface via a bus, signal lines, or a circuit board. Illustratively, peripheral devices include, but are not limited to: radio frequency circuitry, a touchscreen display, audio circuitry, and a power supply.

[0155] Of course, the computing device may also include fewer or more components, and this embodiment does not limit this.

[0156] Furthermore, in other embodiments, the present invention also discloses a storage medium storing one or more computer-readable programs, the one or more programs including instructions adapted to be loaded by a memory and executed to perform the vehicle surrounding environment fine description method disclosed in the above embodiments.

[0157] This invention discloses a method, apparatus, device, and storage medium for detailed description of the surrounding environment of a vehicle, which has the following beneficial effects:

[0158] First, this invention can effectively classify five types of point cloud data: unobstructed azimuth point cloud data, noisy point cloud data, dynamic obstacle point cloud data, static obstacle point cloud data, and fine ground point cloud data.

[0159] Secondly, this invention is based on visual segmentation and classification point cloud ray refresh, effectively combining visual and laser data to finely describe the surrounding three-dimensional environment information, and can effectively eliminate false detections caused by inherent laser defects such as noise and drag points.

[0160] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely illustrative of the principles of the present invention. Various changes and modifications can be made to the present invention without departing from its spirit and scope. All such changes and modifications fall within the scope of the present invention as claimed, which is defined by the appended claims and their equivalents.

Claims

1. A method for fine description of a self-vehicle surrounding environment, characterized in that, The method comprises the following steps: Step S1: collecting point cloud data, image data and odometry data; Step S2: detecting a target based on the collected point cloud data, detecting a dynamic target, and outputting a dynamic target detection frame; Step S3: detecting a target based on the collected image data, obtaining a drivable area mask on the image; Step S4: based on the collected point cloud data, obtaining unobstructed azimuth point cloud data and normal point cloud data through laser radar polar coordinate sorting analysis; Step S5: based on the dynamic target detection frame obtained in step S2, performing dynamic obstacle filtering processing on the normal point cloud data obtained in step S4, to obtain dynamic obstacle point cloud data and static environment point cloud data; Step S6: processing the static environment point cloud data obtained in step S5, and dividing it into fine ground point cloud data and fine obstacle point cloud data; Step S7: projecting the fine ground point cloud data and the fine obstacle point cloud data obtained in step S6 onto the drivable area mask of the image obtained in step S3; Step S8: for the azimuth corresponding to the drivable area pixel, when both the fine ground point cloud data and the fine obstacle point cloud data exist, and the distance from the camera center in the fine obstacle point cloud data is less than the distance from the fine ground point cloud data, mark the part as noise point cloud data, and mark the remaining part as static obstacle point cloud data; Step S9: dividing the space around the ego vehicle in three dimensions, and performing ray refreshing of the unobstructed azimuth point cloud data, the noise point cloud data, the dynamic obstacle point cloud data, the static obstacle point cloud data and the fine ground point cloud data in the three-dimensional grid in different logics, to finely describe the three-dimensional environment information around the ego vehicle; The step S9 comprises: For the unobstructed azimuth point cloud data, the noise point cloud data and the dynamic obstacle point cloud data, the three types of point cloud rays are used to update the three-dimensional grid occupancy probability, and the three-dimensional grid hit by the three types of point cloud is not processed; For the fine ground point cloud data, the three-dimensional grid occupancy probability of the ray passing through the three-dimensional grid is updated, the three-dimensional grid occupancy probability of the ray hitting the three-dimensional grid is updated, and the drivable attribute is assigned; For the static obstacle point cloud data, the three-dimensional grid occupancy probability of the ray passing through the three-dimensional grid is updated, the three-dimensional grid occupancy probability of the ray hitting the three-dimensional grid is updated, and the non-drivable attribute is assigned.

2. The method of claim 1, wherein, The step S4 comprises: Step S4.1: based on the collected point cloud data, the polar coordinate values of the part without return data are confirmed through laser radar polar coordinate sorting analysis; Step S4.2: based on the polar coordinate values of the part without return data obtained in step S4.1, and combined with the part of the point cloud obtained from the laser ranging nominal limit, the unobstructed azimuth point cloud data is marked; The remaining part is marked as normal point cloud data.

3. The method of claim 1, wherein, The step S6 comprises: Step S6.1: processing the static environment point cloud data obtained in step S5 by using a ground segmentation algorithm, to divide it into coarse ground point cloud data and coarse obstacle point cloud data; Step S6.2: obtaining dense coarse ground point cloud data by stacking frames of the coarse ground point cloud data obtained in step S6.1 according to the odometry data; Step S6.3: Fine obstacle detection is performed in the dense coarse ground point cloud data obtained in step S6.2, and small obstacle point cloud data is detected; Step S6.4: The small obstacle point cloud data detected in step S6.3 is removed from the coarse ground point cloud data to form fine ground point cloud data; And add it to the coarse obstacle point cloud data to form fine obstacle point cloud data.

4. The device for refining the description of the environment surrounding the ego vehicle, characterized in that Comprise: The acquisition module is used for collecting point cloud data, image data and odometer data; The point cloud detection module is used for target detection based on the collected point cloud data, detecting dynamic targets, and outputting dynamic target detection frames; The image detection module is used for target detection based on the collected image data to obtain an image drivable area mask; The polar coordinate sorting analysis module is used for polar coordinate sorting analysis based on the collected point cloud data to obtain unobstructed azimuth point cloud data and normal point cloud data; The dynamic obstacle filtering module is used for dynamic obstacle filtering processing of the normal point cloud data obtained by the polar coordinate sorting analysis module based on the dynamic target detection frame obtained by the point cloud detection module to obtain dynamic obstacle point cloud data and static environment point cloud data; The ground segmentation module is used for processing the static environment point cloud data obtained by the dynamic obstacle filtering module to divide it into fine ground point cloud data and fine obstacle point cloud data; The back projection module is used for back projecting the fine ground point cloud data and the fine obstacle point cloud data obtained by the ground segmentation module to the drivable area mask of the image obtained by the image detection module; The point cloud division module is used for, for the azimuth corresponding to the drivable area pixel, when both the fine ground point cloud data and the fine obstacle point cloud data exist, and the distance from the camera center in the fine obstacle point cloud data is less than the distance from the fine ground point cloud data, marking the part as noise point cloud data, and marking the remaining part as static obstacle point cloud data; The three-dimensional grid refreshing module is used for three-dimensional division of the space around the ego vehicle, and different logic ray refreshing of the unobstructed azimuth point cloud data, noise point cloud data, dynamic obstacle point cloud data, static obstacle point cloud data and fine ground point cloud data in the three-dimensional grid to finely describe the three-dimensional environment information around the ego vehicle; The three-dimensional grid refreshing module is used to execute the following method: For the unobstructed azimuth point cloud data, noise point cloud data and dynamic obstacle point cloud data, the three-dimensional grid occupancy probability of the rays of the three types of point clouds is updated downward, and the three-dimensional grid hit by the rays is not processed; For the fine ground point cloud data, the three-dimensional grid occupancy probability of the rays of the point cloud is updated downward, the three-dimensional grid occupancy probability of the point cloud hit is updated upward, and the drivable attribute is assigned; For the static obstacle point cloud data, the three-dimensional grid occupancy probability of the rays of the point cloud is updated downward, the three-dimensional grid occupancy probability of the point cloud hit is updated upward, and the non-drivable attribute is assigned.

5. The apparatus according to claim 4, wherein The polar coordinate sorting analysis module comprises: The sorting unit is used for polar coordinate sorting analysis based on the collected point cloud data to confirm the polar coordinate values of the no-return data part; The dividing unit is configured to, based on the polar coordinate values of the no-return-data part obtained by the sorting unit, combine the part of the point cloud obtained based on the nominal limit of the laser ranging to mark the no-shielding-orientation point cloud data; The remaining part is marked as normal point cloud data.

6. The apparatus according to claim 4, wherein The ground segmentation module comprises: The coarse segmentation unit is configured to process the static environment point cloud data obtained by the dynamic obstacle filtering module by using a ground segmentation algorithm to divide the data into coarse ground point cloud data and coarse obstacle point cloud data; The stacking unit is configured to stack the coarse ground point cloud data obtained by the coarse segmentation unit based on the odometer data to obtain dense coarse ground point cloud data; The detection unit is configured to perform fine obstacle detection on the dense coarse ground point cloud data obtained by the stacking unit to detect small obstacle point cloud data; The fine segmentation unit is configured to remove the small obstacle point cloud data detected by the detection unit from the coarse ground point cloud data to form fine ground point cloud data; And add it to the coarse obstacle point cloud data to form fine obstacle point cloud data.

7. A computing device, characterized by Comprise: One or more processors; Memory; And one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by one or more processors, one or more programs comprising instructions of the self-vehicle surrounding environment refinement description method of any one of claims 1-3.

8. Storage medium, characterized in that The storage medium stores one or more computer-readable programs, and the one or more programs comprise instructions suitable for being loaded by the memory and executing the self-vehicle surrounding environment refinement description method of any one of claims 1-3.

Images