Vehicle surrounding environment refined description method, device and equipment and storage medium

Abstract

The invention discloses a fine description method, device and equipment for the surrounding environment of a vehicle and a storage medium. The method comprises the steps that point cloud data, image data and odometer data are collected; performing target detection based on the collected point cloud data, and outputting a dynamic target detection frame; performing target detection based on the acquired image data to obtain a drivable area mask on the image; dividing the original point cloud data into unshielded azimuth point cloud data, noisy point cloud data, dynamic obstacle point cloud data, static obstacle point cloud data and fine ground point cloud data; and performing three-dimensional division on the space around the vehicle, performing ray refreshing of different logics on five types of point cloud data in a three-dimensional grid, and finely describing three-dimensional environment information around the vehicle. Based on visual segmentation and classified point cloud ray refreshing, visual and laser data are effectively combined, surrounding three-dimensional environment information is finely described, and false detection caused by laser inherent defects such as noisy points and drag points can be effectively eliminated.

Description

Technical Field

[0001] The present invention belongs to the technical field of autonomous driving, and specifically relates to a method, device, equipment and storage medium for finely describing the environment surrounding a self-driving vehicle. Background Art

[0002] The autonomous navigation capability of self-driving vehicles is the key to their safe movement in complex environments. The key to vehicle autonomous navigation research is real-time environmental perception, including the perception of dynamic obstacles and the description of static environments.

[0003] Dynamic obstacles in urban traffic environments are generally composed of pedestrians, motor vehicles, and non-motor vehicles, and their categories are relatively fixed. Therefore, in the field of autonomous driving, target detection models are usually used for detection and described by detection boxes with categories and prediction lines.

[0004] The static obstacles in the traffic environment are more complex. They include not only various terrain barriers and traffic equipment in the city, but also items that are not common in traffic scenes. For example, goods dropped from vehicles and temporary stacking of construction materials. Due to their numerous semantics and different volumes, they are usually described using a three-dimensional grid occupancy map.

[0005] The static environment perception technology is that the vehicle perceives the surrounding environment information through its own sensors (such as lidar and camera) and detects the passable area of ​​the vehicle.

[0006] Static three-dimensional environment perception technology through LiDAR time-series fusion has the characteristics of high precision, but it is difficult to solve the problems of water mist, dust noise, and the expansion and dragging of LiDAR in the face of high-reflective road signs; static environment perception technology based on vision usually has the problems of insufficient precision and weak ability to detect small obstacles.

[0007] How to better integrate visual and laser information to ensure accurate description of the surrounding environment has become an urgent problem to be solved. Summary of the invention

[0008] In order to solve the above technical problems, the present invention proposes a method, device, equipment and storage medium for fine-grained description of the vehicle's surrounding environment.

[0009] In order to achieve the above object, the technical solution of the present invention is as follows:

[0010] In a first aspect, the present invention discloses a method for finely describing the surrounding environment of a vehicle, comprising:

[0011] Step S1: Collecting point cloud data, image data and odometer data;

[0012] Step S2: Target detection is performed based on the collected point cloud data, dynamic targets are detected, and a dynamic target detection frame is output;

[0013] Step S3: performing target detection based on the collected image data to obtain a drivable area mask on the image;

[0014] Step S4: Based on the collected point cloud data, the unobstructed azimuth point cloud data and normal point cloud data are obtained by laser radar polar coordinate sorting analysis;

[0015] Step S5: Based on the dynamic target detection frame obtained in step S2, the normal point cloud data obtained in step S4 is subjected to dynamic obstacle filtering processing to obtain dynamic obstacle point cloud data and static environment point cloud data;

[0016] Step S6: Processing the static environment point cloud data obtained in step S5, and dividing it into precise ground point cloud data and precise obstacle point cloud data;

[0017] Step S7: back-projecting the precise ground point cloud data and the precise obstacle point cloud data obtained in step S6 onto the drivable area mask of the image obtained in step S3;

[0018] Step S8: For the orientation corresponding to the pixel of the drivable area, when there are both precise ground point cloud data and precise obstacle point cloud data, and the part of the precise obstacle point cloud data whose distance from the camera center is smaller than that from the precise ground point cloud data is marked as noise point cloud data, and the rest is marked as static obstacle point cloud data;

[0019] Step S9: Divide the space around the vehicle into three dimensions, perform ray refresh of different logics on the unobstructed azimuth point cloud data, noise point cloud data, dynamic obstacle point cloud data, static obstacle point cloud data, and precise ground point cloud data in the three-dimensional grid, and describe the three-dimensional environment information around the vehicle in a refined manner.

[0020] Based on the above technical solution, the following improvements can be made:

[0021] As a preferred solution, step S4 includes:

[0022] Step S4.1: Based on the collected point cloud data, the polar coordinate values ​​of the part without return data are confirmed through the laser radar polar coordinate sorting analysis;

[0023] Step S4.2: Based on the polar coordinate values ​​of the part without return data obtained in step S4.1, the part of the point cloud obtained by combining the nominal limit of the laser ranging is marked as unobstructed azimuth point cloud data;

[0024] The rest are marked as normal point cloud data.

[0025] As a preferred solution, step S6 includes:

[0026] Step S6.1: using a ground segmentation algorithm to process the static environment point cloud data obtained in step S5, and divide it into coarse ground point cloud data and coarse obstacle point cloud data;

[0027] Step S6.2: The coarse surface point cloud data obtained in step S6.1 is stacked with the odometer data to obtain dense coarse surface point cloud data;

[0028] Step S6.3: performing fine obstacle detection in the dense coarse ground point cloud data obtained in step S6.2 to detect small obstacle point cloud data;

[0029] Step S6.4: removing the small obstacle point cloud data detected in step S6.3 from the coarse ground point cloud data to form fine ground point cloud data;

[0030] And add it to the coarse obstacle point cloud data to form the refined obstacle point cloud data.

[0031] As a preferred solution, step S9 includes:

[0032] For unobstructed azimuth point cloud data, noise point cloud data, and dynamic obstacle point cloud data, the occupancy probability of the three-dimensional grids through which the rays of these three types of point clouds pass is updated downward, and the three-dimensional grids hit by these three types of point clouds are not processed;

[0033] For the precise ground point cloud data, the occupancy probability of the three-dimensional grids that the point cloud rays pass through is updated downward, and the occupancy probability of the three-dimensional grids that the point cloud rays hit is updated upward, and the drivable attribute is given;

[0034] For static obstacle point cloud data, the occupancy probability of the three-dimensional grid passed by such point cloud rays is updated downward, and the occupancy probability of the three-dimensional grid hit by such point cloud rays is updated upward, and the non-drivable attribute is given.

[0035] In a second aspect, the present invention discloses a device for finely describing the surrounding environment of a vehicle, comprising:

[0036] Acquisition module, used to collect point cloud data, image data and odometer data;

[0037] Point cloud detection module, used to detect targets based on the collected point cloud data, detect dynamic targets, and output dynamic target detection frames;

[0038] An image detection module is used to detect targets based on the collected image data and obtain a drivable area mask on the image;

[0039] Polar coordinate sorting and analysis module, used to obtain unobstructed azimuth point cloud data and normal point cloud data through laser radar polar coordinate sorting and analysis based on the collected point cloud data;

[0040] The dynamic obstacle filtering module is used to obtain the dynamic target detection frame based on the point cloud detection module, and perform dynamic obstacle filtering processing on the normal point cloud data obtained by the polar coordinate sorting analysis module to obtain dynamic obstacle point cloud data and static environment point cloud data;

[0041] The ground segmentation module is used to process the static environment point cloud data obtained by the dynamic obstacle filtering module and divide it into precise ground point cloud data and precise obstacle point cloud data;

[0042] A back-projection module, used to back-project the refined ground point cloud data and the refined obstacle point cloud data obtained by the ground segmentation module onto the drivable area mask of the image obtained by the image detection module;

[0043] The point cloud segmentation module is used to mark the position corresponding to the pixel of the drivable area as noise point cloud data when there are both precise ground point cloud data and precise obstacle point cloud data, and the part of the precise obstacle point cloud data that is closer to the camera center than the precise ground point cloud data, and the rest of the part is marked as static obstacle point cloud data;

[0044] The three-dimensional grid refresh module is used to divide the space around the vehicle into three dimensions, and perform ray refresh of different logics on the unobstructed azimuth point cloud data, noise point cloud data, dynamic obstacle point cloud data, static obstacle point cloud data and precise ground point cloud data in the three-dimensional grid, so as to describe the three-dimensional environmental information around the vehicle in a refined manner.

[0045] As a preferred solution, the polar coordinate sorting analysis module includes:

[0046] A sorting unit is used to confirm the polar coordinate value of the part without return data through laser radar polar coordinate sorting analysis based on the collected point cloud data;

[0047] A division unit is used to mark the polar coordinate values ​​of the part without return data obtained by the sorting unit and the part of the point cloud obtained by the nominal limit of the laser ranging as the unobstructed azimuth point cloud data;

[0048] The rest are marked as normal point cloud data.

[0049] As a preferred solution, the ground segmentation module includes:

[0050] A coarse segmentation unit is used to process the static environment point cloud data obtained by the dynamic obstacle filtering module using a ground segmentation algorithm, and divide it into coarse ground point cloud data and coarse obstacle point cloud data;

[0051] A frame stacking unit is used to stack the coarse ground point cloud data obtained by the coarse segmentation unit based on the odometer data to obtain dense coarse ground point cloud data;

[0052] A detection unit, used for performing fine obstacle detection in the dense coarse ground point cloud data obtained by the frame stacking unit, and detecting small obstacle point cloud data;

[0053] A fine segmentation unit is used to remove the small obstacle point cloud data detected by the detection unit from the coarse ground point cloud data to form fine ground point cloud data;

[0054] And add it to the coarse obstacle point cloud data to form the refined obstacle point cloud data.

[0055] As a preferred solution, the three-dimensional grid refresh module is used to perform the following method:

[0056] For unobstructed azimuth point cloud data, noise point cloud data, and dynamic obstacle point cloud data, the occupancy probability of the three-dimensional grids through which the rays of these three types of point clouds pass is updated downward, and the three-dimensional grids hit by these three types of point clouds are not processed;

[0057] For the precise ground point cloud data, the occupancy probability of the three-dimensional grids that the point cloud rays pass through is updated downward, and the occupancy probability of the three-dimensional grids that the point cloud rays hit is updated upward, and the drivable attribute is given;

[0058] For static obstacle point cloud data, the occupancy probability of the three-dimensional grid passed by such point cloud rays is updated downward, and the occupancy probability of the three-dimensional grid hit by such point cloud rays is updated upward, and the non-drivable attribute is given.

[0059] In a third aspect, the present invention discloses a computing device, comprising:

[0060] one or more processors;

[0061] Memory;

[0062] And one or more programs, wherein the one or more programs are stored in a memory and configured to be executed by one or more processors, and the one or more programs include instructions of any of the above-mentioned methods for finely describing the surrounding environment of the vehicle.

[0063] In a fourth aspect, the present invention discloses a storage medium storing one or more computer-readable programs, wherein the one or more programs include instructions suitable for being loaded by a memory and executing any of the above-mentioned methods for finely describing the surrounding environment of the vehicle.

[0064] The present invention discloses a method, device, equipment and storage medium for finely describing the surrounding environment of a self-driving vehicle, which has the following beneficial effects:

[0065] First, the present invention can effectively and finely divide five types of point cloud data, namely, unobstructed azimuth point cloud data, noise point cloud data, dynamic obstacle point cloud data, static obstacle point cloud data and precise ground point cloud data.

[0066] Second, the present invention is based on visual segmentation and classified point cloud ray refresh, effectively combining visual and laser data to finely describe the surrounding three-dimensional environmental information, and can effectively eliminate false detections caused by inherent laser defects such as noise and dragging points. BRIEF DESCRIPTION OF THE DRAWINGS

[0067] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments are briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.

[0068] Figure 1 A flowchart of a method for finely describing the vehicle's surrounding environment provided in an embodiment of the present invention.

[0069] Figure 2 A block diagram of a computing device provided for an embodiment of the present invention. DETAILED DESCRIPTION

[0070] The preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0071] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0072] The expression of “comprising” an element is an “open” expression, which merely means that a corresponding component or step exists, and should not be interpreted as excluding additional components or steps.

[0073] In order to achieve the purpose of the present invention, some embodiments of the method, device, apparatus and storage medium for finely describing the surrounding environment of a self-driving vehicle are as follows: Figure 1 As shown in FIG. 1 , the method for refining the environment around the vehicle includes:

[0074] Step S101: collecting point cloud data, image data and odometer data;

[0075] Step S102: performing target detection based on the collected point cloud data, detecting dynamic targets, and outputting a dynamic target detection frame;

[0076] Step S103: performing target detection based on the collected image data to obtain a drivable area mask on the image;

[0077] Step S104: Based on the collected point cloud data, unobstructed azimuth point cloud data and normal point cloud data are obtained through laser radar polar coordinate sorting analysis;

[0078] Step S105: Based on the dynamic target detection frame obtained in step S102, the normal point cloud data obtained in step S104 is subjected to dynamic obstacle filtering processing to obtain dynamic obstacle point cloud data and static environment point cloud data;

[0079] Step S106: Processing the static environment point cloud data obtained in step S105 to divide it into precise ground point cloud data and precise obstacle point cloud data;

[0080] Step S107: back-projecting the precise ground point cloud data and the precise obstacle point cloud data obtained in step S106 onto the drivable area mask of the image obtained in step S103;

[0081] Step S108: For the orientation corresponding to the pixel of the drivable area, when there are both precise ground point cloud data and precise obstacle point cloud data, and the part of the precise obstacle point cloud data whose distance from the camera center is smaller than that from the precise ground point cloud data is marked as noise point cloud data, and the rest is marked as static obstacle point cloud data;

[0082] Step S109: Divide the space around the vehicle into three dimensions, perform ray refresh of different logics on the unobstructed azimuth point cloud data, noise point cloud data, dynamic obstacle point cloud data, static obstacle point cloud data and precise ground point cloud data in the three-dimensional grid, and describe the three-dimensional environment information around the vehicle in a refined manner.

[0083] The above steps are described in detail below.

[0084] In step S101, data can be collected using sensors such as a laser radar and a camera carried by the vehicle.

[0085] The odometer data collected in step S101 specifically refers to data that accurately records the moving trajectory of the vehicle through sensors and SLAM algorithms. According to the type of sensor, common ones include inertial navigation odometers, laser odometers, visual odometers, etc. The present invention can use a laser inertial navigation odometer, that is, a fusion of an imu inertial measurement sensor and a laser radar sensor to accurately calculate the moving trajectory data of the vehicle.

[0086] Step S102 performs target detection on the original point cloud data. Through the target detection model and target tracking technology, dynamic targets are detected in the real-time laser point cloud data and a dynamic target detection frame is output.

[0087] Step S103 performs target detection in the camera image data to obtain a pixel description of obstacles and a drivable area mask on the image.

[0088] Step S104 further processes the point cloud data, including:

[0089] Step S4.1: Based on the collected point cloud data, the polar coordinate values ​​of the parts without return data are confirmed through the laser radar polar coordinate sorting analysis (indicating that the distance of the objects in these directions exceeds the ranging limit of the laser);

[0090] Step S4.2: Based on the polar coordinate values ​​of the part without return data obtained in step S4.1, the part of the point cloud obtained by combining the nominal limit of the laser ranging is marked as unobstructed azimuth point cloud data;

[0091] The rest are marked as normal point cloud data.

[0092] Polar diameter r:

[0093]

[0094] Polar angle θ:

[0095]

[0096] Polar angle ψ:

[0097]

[0098] Generally, the point cloud data is parsed into the xyz space rectangular coordinate system coordinate position. The polar coordinate system parameters are obtained by using the above formula through the coordinates (x, y, z) of each point cloud: polar diameter r, polar angle θ, polar angle ψ.

[0099] Under normal circumstances, the polar angle θ and polar angle ψ of the polar coordinates corresponding to each frame of laser light satisfy a certain list (e.g., the polar angle θ of a frame of point cloud satisfies 0-360°, and the polar angle ψ satisfies -30°-25° according to the standard). When all points of a frame of laser light are converted from the spatial rectangular coordinate system to the polar coordinate system, the missing part in the list means that the distance measurement of the laser point cloud emitted from this orientation exceeds the nominal distance.

[0100] For example, according to the laser radar product manual, if the laser range limit is 300m, then assume that the range of this azimuth is 300m. Therefore, the polar coordinate system coordinates of the unobstructed azimuth point cloud can be obtained as follows: polar diameter r' (e.g.: 300m), polar angle θ', polar angle ψ'.

[0101] Then, the xyz spatial rectangular coordinates of the unobstructed azimuth point cloud data are recalculated by reversing the formula.

[0102] Step S105 uses the dynamic target detection frame to process the normal point cloud data to obtain dynamic obstacle point cloud data and static environment point cloud data.

[0103] Step S106 further processes the static environment point cloud data, including:

[0104] Step S106.1: using a ground segmentation algorithm to process the static environment point cloud data obtained in step S105, and divide it into coarse ground point cloud data and coarse obstacle point cloud data;

[0105] Step S106.2: The coarse surface point cloud data obtained in step S106.1 is stacked with the odometer data to obtain dense coarse surface point cloud data;

[0106] Step S106.3: Perform fine obstacle detection (such as ground potholes and small obstacles, etc.) in the dense coarse ground point cloud data obtained in step S106.2, and detect small obstacle point cloud data;

[0107] Step S106.4: removing the small obstacle point cloud data detected in step S106.3 from the coarse ground point cloud data to form fine ground point cloud data;

[0108] And add it to the coarse obstacle point cloud data to form the refined obstacle point cloud data.

[0109] The odometer data is the trajectory data of the vehicle, so step S106.2 splices and overlaps the coarse ground point cloud data of each frame according to the vehicle trajectory to form dense coarse ground point cloud data.

[0110] The dense coarse ground point cloud data is analyzed by statistical methods to obtain more accurate fine ground point cloud data and small obstacle point cloud data. The small obstacle point cloud data is added to the coarse obstacle point cloud data to form more accurate fine obstacle point cloud data.

[0111] After the above steps, single-frame point cloud data can be divided into four categories: unobstructed orientation point cloud data, precise ground point cloud data, precise obstacle point cloud data, and dynamic obstacle point cloud data.

[0112] Camera images in all directions can achieve the ability to identify obstacles and drivable areas through target recognition and drivable area training. The pixel point set of the drivable area mask represents that there are no obstacles in the specified direction starting from the camera optical center. This method can effectively eliminate the (noise) false detection part in the laser point cloud.

[0113] Step S107 and step S108 back-project the dense precise ground point cloud data and the real-time precise obstacle point cloud data onto the image drivable area mask.

[0114] For the orientation corresponding to the pixels in the drivable area, when there are both precise ground point cloud data and precise obstacle point cloud data, and the part of the precise obstacle point cloud data that is closer to the camera center than the precise ground point cloud data (the noise part of the laser), it is marked as noise point cloud data, and the rest is marked as static obstacle point cloud data.

[0115] Therefore, the precise obstacle point cloud data of the single-frame point cloud is divided into noise point cloud data and static obstacle point cloud data.

[0116] Finally, the space around the vehicle is divided into three dimensions, and the five types of point cloud data are refreshed with rays of different logics in the three-dimensional grid to obtain a three-dimensional grid data describing the environment around the vehicle.

[0117] Therefore, step S109 includes:

[0118] (1) For unobstructed azimuth point cloud data, noisy point cloud data, and dynamic obstacle point cloud data, the occupancy probability of the three-dimensional grids through which the rays of these three types of point clouds pass is updated downward, and the three-dimensional grids hit by these three types of point clouds are not processed;

[0119] (2) For the precise ground point cloud data, the occupancy probability of the three-dimensional grids that the point cloud rays pass through is updated downward, and the occupancy probability of the three-dimensional grids that the point cloud rays hit is updated upward, and the drivable attribute is given;

[0120] (3) For static obstacle point cloud data, the occupancy probability of the three-dimensional grid that the point cloud ray passes through is updated downward, and the occupancy probability of the three-dimensional grid that the point cloud hits is updated upward, and the non-drivable attribute is assigned.

[0121] The present invention can accurately divide the original point cloud data into unobstructed azimuth point cloud data, noise point cloud data, dynamic obstacle point cloud data, static obstacle point cloud data and precise ground point cloud data.

[0122] In some other embodiments, the present invention discloses a device for finely describing the surrounding environment of a vehicle, comprising:

[0123] Acquisition module, used to collect point cloud data, image data and odometer data;

[0124] Point cloud detection module, used to detect targets based on the collected point cloud data, detect dynamic targets, and output dynamic target detection frames;

[0125] An image detection module is used to detect targets based on the collected image data and obtain a drivable area mask on the image;

[0126] Polar coordinate sorting and analysis module, used to obtain unobstructed azimuth point cloud data and normal point cloud data through laser radar polar coordinate sorting and analysis based on the collected point cloud data;

[0127] The dynamic obstacle filtering module is used to obtain the dynamic target detection frame based on the point cloud detection module, and perform dynamic obstacle filtering processing on the normal point cloud data obtained by the polar coordinate sorting analysis module to obtain dynamic obstacle point cloud data and static environment point cloud data;

[0128] The ground segmentation module is used to process the static environment point cloud data obtained by the dynamic obstacle filtering module and divide it into precise ground point cloud data and precise obstacle point cloud data;

[0129] A back-projection module, used to back-project the refined ground point cloud data and the refined obstacle point cloud data obtained by the ground segmentation module onto the drivable area mask of the image obtained by the image detection module;

[0130] The point cloud segmentation module is used to mark the position corresponding to the pixel of the drivable area as noise point cloud data when there are both precise ground point cloud data and precise obstacle point cloud data, and the part of the precise obstacle point cloud data that is closer to the camera center than the precise ground point cloud data, and the rest of the part is marked as static obstacle point cloud data;

[0131] The three-dimensional grid refresh module is used to divide the space around the vehicle into three dimensions, and perform ray refresh of different logics on the unobstructed azimuth point cloud data, noise point cloud data, dynamic obstacle point cloud data, static obstacle point cloud data and precise ground point cloud data in the three-dimensional grid, so as to describe the three-dimensional environmental information around the vehicle in a refined manner.

[0132] Furthermore, the polar coordinate sorting analysis module includes:

[0133] A sorting unit is used to confirm the polar coordinate value of the part without return data through laser radar polar coordinate sorting analysis based on the collected point cloud data;

[0134] A division unit is used to mark the polar coordinate values ​​of the part without return data obtained by the sorting unit and the part of the point cloud obtained by the nominal limit of the laser ranging as the unobstructed azimuth point cloud data;

[0135] The rest are marked as normal point cloud data.

[0136] Furthermore, the ground segmentation module includes:

[0137] A coarse segmentation unit is used to process the static environment point cloud data obtained by the dynamic obstacle filtering module using a ground segmentation algorithm, and divide it into coarse ground point cloud data and coarse obstacle point cloud data;

[0138] A frame stacking unit is used to stack the coarse ground point cloud data obtained by the coarse segmentation unit based on the odometer data to obtain dense coarse ground point cloud data;

[0139] A detection unit, used for performing fine obstacle detection in the dense coarse ground point cloud data obtained by the frame stacking unit, and detecting small obstacle point cloud data;

[0140] A fine segmentation unit is used to remove the small obstacle point cloud data detected by the detection unit from the coarse ground point cloud data to form fine ground point cloud data;

[0141] And add it to the coarse obstacle point cloud data to form the refined obstacle point cloud data.

[0142] Furthermore, the three-dimensional grid refresh module is used to perform the following method:

[0143] For unobstructed azimuth point cloud data, noise point cloud data, and dynamic obstacle point cloud data, the occupancy probability of the three-dimensional grids through which the rays of these three types of point clouds pass is updated downward, and the three-dimensional grids hit by these three types of point clouds are not processed;

[0144] For the precise ground point cloud data, the occupancy probability of the three-dimensional grids that the point cloud rays pass through is updated downward, and the occupancy probability of the three-dimensional grids that the point cloud rays hit is updated upward, and the drivable attribute is given;

[0145] For static obstacle point cloud data, the occupancy probability of the three-dimensional grid passed by such point cloud rays is updated downward, and the occupancy probability of the three-dimensional grid hit by such point cloud rays is updated upward, and the non-drivable attribute is given.

[0146] Furthermore, it should be noted that: the device for finely describing the vehicle's surrounding environment provided in the above-mentioned embodiment only uses the division of the above-mentioned functional modules as an example when performing a fine description of the vehicle's surrounding environment. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the device for finely describing the vehicle's surrounding environment can be divided into different functional modules to complete all or part of the functions described above.

[0147] In addition, the device for finely describing the vehicle's surrounding environment provided in the above embodiments and the method for finely describing the vehicle's surrounding environment provided in the above embodiments belong to the same concept, and the specific implementation process is detailed in the method embodiment, which will not be repeated here.

[0148] In addition, in some other embodiments, Figure 2 As shown, the present invention also discloses a computing device, including:

[0149] One or more processors 201;

[0150] Memory 202;

[0151] And one or more programs, wherein the one or more programs are stored in the memory 202 and are configured to be executed by one or more processors 201, and the one or more programs include instructions of the method for finely describing the vehicle surrounding environment disclosed in the above embodiment.

[0152] The processor 201 may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor 201 may be implemented in at least one hardware form of DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). The processor 201 may also include a main processor and a coprocessor. The main processor is a processor for processing data in an awake state, also known as a CPU (Central Processing Unit); the coprocessor is a low-power processor for processing data in a standby state. In some embodiments, the processor 201 may be integrated with a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor 201 may also include an AI (Artificial Intelligence) processor, which is used to process computing operations related to machine learning.

[0153] The memory 202 may include one or more computer-readable storage media, which may be non-transitory. The memory 202 may also include a high-speed random access memory, and a non-volatile memory, such as one or more disk storage devices, flash memory storage devices. In some embodiments, the non-transitory computer-readable storage medium in the memory 202 is used to store at least one instruction, which is used to be executed by the processor 201 to implement the method for finely describing the surrounding environment of the vehicle provided in the method embodiment of the present invention.

[0154] In addition, the computing device may optionally include: a peripheral device interface and at least one peripheral device. The processor 201, the memory 202 and the peripheral device interface may be connected via a bus or a signal line. Each peripheral device may be connected to the peripheral device interface via a bus, a signal line or a circuit board. Schematically, the peripheral devices include but are not limited to: a radio frequency circuit, a touch display screen, an audio circuit, and a power supply.

[0155] Of course, the computing device may also include fewer or more components, which is not limited in this embodiment.

[0156] In addition, in some other embodiments, the present invention further discloses a storage medium, which stores one or more computer-readable programs, and the one or more programs include instructions, and the instructions are suitable for being loaded by a memory and executing the method for detailed description of the vehicle's surrounding environment disclosed in the above embodiment.

[0157] The present invention discloses a method, device, equipment and storage medium for finely describing the surrounding environment of a self-driving vehicle, which has the following beneficial effects:

[0158] First, the present invention can effectively and finely divide five types of point cloud data, namely, unobstructed azimuth point cloud data, noise point cloud data, dynamic obstacle point cloud data, static obstacle point cloud data and precise ground point cloud data.

[0159] Second, the present invention is based on visual segmentation and classified point cloud ray refresh, effectively combining visual and laser data to finely describe the surrounding three-dimensional environmental information, and can effectively eliminate false detections caused by inherent laser defects such as noise and dragging points.

[0160] The above shows and describes the basic principles and main features of the present invention and the advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The above embodiments and descriptions are only for illustrating the principles of the present invention. Without departing from the spirit and scope of the present invention, the present invention may have various changes and improvements, which shall fall within the scope of the present invention to be protected. The scope of protection of the present invention is defined by the attached claims and their equivalents.

Claims

1. A method for finely describing the environment surrounding the vehicle, characterized in that: include: Step S1: Collecting point cloud data, image data and odometer data; Step S2: Target detection is performed based on the collected point cloud data, dynamic targets are detected, and a dynamic target detection frame is output; Step S3: performing target detection based on the collected image data to obtain a drivable area mask on the image; Step S4: Based on the collected point cloud data, the unobstructed azimuth point cloud data and normal point cloud data are obtained by laser radar polar coordinate sorting analysis; Step S5: Based on the dynamic target detection frame obtained in step S2, the normal point cloud data obtained in step S4 is subjected to dynamic obstacle filtering processing to obtain dynamic obstacle point cloud data and static environment point cloud data; Step S6: Processing the static environment point cloud data obtained in step S5, and dividing it into precise ground point cloud data and precise obstacle point cloud data; Step S7: back-projecting the precise ground point cloud data and the precise obstacle point cloud data obtained in step S6 onto the drivable area mask of the image obtained in step S3; Step S8: For the orientation corresponding to the pixel of the drivable area, when there are both precise ground point cloud data and precise obstacle point cloud data, and the part of the precise obstacle point cloud data whose distance from the camera center is smaller than that from the precise ground point cloud data is marked as noise point cloud data, and the rest is marked as static obstacle point cloud data; Step S9: Divide the space around the vehicle into three dimensions, perform ray refresh of different logics on the unobstructed azimuth point cloud data, noise point cloud data, dynamic obstacle point cloud data, static obstacle point cloud data, and precise ground point cloud data in the three-dimensional grid, and describe the three-dimensional environment information around the vehicle in a refined manner.

2. The method for finely describing the vehicle's surrounding environment according to claim 1, characterized in that: The step S4 comprises: Step S4.1: Based on the collected point cloud data, the polar coordinate values ​​of the part without return data are confirmed through the laser radar polar coordinate sorting analysis; Step S4.2: Based on the polar coordinate values ​​of the part without return data obtained in step S4.1, the part of the point cloud obtained by combining the nominal limit of the laser ranging is marked as unobstructed azimuth point cloud data; The rest are marked as normal point cloud data.

3. The method for finely describing the vehicle's surrounding environment according to claim 1, characterized in that: The step S6 comprises: Step S6.1: using a ground segmentation algorithm to process the static environment point cloud data obtained in step S5, and divide it into coarse ground point cloud data and coarse obstacle point cloud data; Step S6.2: The coarse surface point cloud data obtained in step S6.1 is stacked with the odometer data to obtain dense coarse surface point cloud data; Step S6.3: performing fine obstacle detection in the dense coarse ground point cloud data obtained in step S6.2 to detect small obstacle point cloud data; Step S6.4: removing the small obstacle point cloud data detected in step S6.3 from the coarse ground point cloud data to form fine ground point cloud data; And add it to the coarse obstacle point cloud data to form the refined obstacle point cloud data.

4. The method for finely describing the vehicle's surrounding environment according to claim 1, characterized in that: The step S9 comprises: For unobstructed azimuth point cloud data, noise point cloud data, and dynamic obstacle point cloud data, the occupancy probability of the three-dimensional grids through which the rays of these three types of point clouds pass is updated downward, and the three-dimensional grids hit by these three types of point clouds are not processed; For the precise ground point cloud data, the occupancy probability of the three-dimensional grids that the point cloud rays pass through is updated downward, and the occupancy probability of the three-dimensional grids that the point cloud rays hit is updated upward, and the drivable attribute is given; For static obstacle point cloud data, the occupancy probability of the three-dimensional grid passed by such point cloud rays is updated downward, and the occupancy probability of the three-dimensional grid hit by such point cloud rays is updated upward, and the non-drivable attribute is given.

5. A device for describing the surrounding environment of the vehicle in detail, characterized in that: include: Acquisition module, used to collect point cloud data, image data and odometer data; Point cloud detection module, used to detect targets based on the collected point cloud data, detect dynamic targets, and output dynamic target detection frames; An image detection module is used to detect targets based on the collected image data and obtain a drivable area mask on the image; Polar coordinate sorting and analysis module, used to obtain unobstructed azimuth point cloud data and normal point cloud data through laser radar polar coordinate sorting and analysis based on the collected point cloud data; The dynamic obstacle filtering module is used to obtain the dynamic target detection frame based on the point cloud detection module, and perform dynamic obstacle filtering processing on the normal point cloud data obtained by the polar coordinate sorting analysis module to obtain dynamic obstacle point cloud data and static environment point cloud data; The ground segmentation module is used to process the static environment point cloud data obtained by the dynamic obstacle filtering module and divide it into precise ground point cloud data and precise obstacle point cloud data; A back-projection module, used to back-project the refined ground point cloud data and the refined obstacle point cloud data obtained by the ground segmentation module onto the drivable area mask of the image obtained by the image detection module; The point cloud segmentation module is used to mark the position corresponding to the pixel of the drivable area as noise point cloud data when there are both precise ground point cloud data and precise obstacle point cloud data, and the part of the precise obstacle point cloud data that is closer to the camera center than the precise ground point cloud data, and the rest of the part is marked as static obstacle point cloud data; The three-dimensional grid refresh module is used to divide the space around the vehicle into three dimensions, and perform ray refresh of different logics on the unobstructed azimuth point cloud data, noise point cloud data, dynamic obstacle point cloud data, static obstacle point cloud data and precise ground point cloud data in the three-dimensional grid, so as to describe the three-dimensional environmental information around the vehicle in a refined manner.

6. The device for finely describing the surrounding environment of the vehicle according to claim 5, characterized in that: The polar coordinate sorting analysis module comprises: A sorting unit is used to confirm the polar coordinate value of the part without return data through laser radar polar coordinate sorting analysis based on the collected point cloud data; A division unit is used to mark the polar coordinate values ​​of the part without return data obtained by the sorting unit and the part of the point cloud obtained by the nominal limit of the laser ranging as the unobstructed azimuth point cloud data; The rest are marked as normal point cloud data.

7. The device for finely describing the surrounding environment of the vehicle according to claim 5, characterized in that: The ground segmentation module comprises: A coarse segmentation unit is used to process the static environment point cloud data obtained by the dynamic obstacle filtering module using a ground segmentation algorithm, and divide it into coarse ground point cloud data and coarse obstacle point cloud data; A frame stacking unit is used to stack the coarse ground point cloud data obtained by the coarse segmentation unit based on the odometer data to obtain dense coarse ground point cloud data; A detection unit, used for performing fine obstacle detection in the dense coarse ground point cloud data obtained by the frame stacking unit, and detecting small obstacle point cloud data; A fine segmentation unit is used to remove the small obstacle point cloud data detected by the detection unit from the coarse ground point cloud data to form fine ground point cloud data; And add it to the coarse obstacle point cloud data to form the refined obstacle point cloud data.

8. The device for finely describing the surrounding environment of the vehicle according to claim 5, characterized in that: The three-dimensional grid refresh module is used to execute the following method: For unobstructed azimuth point cloud data, noise point cloud data, and dynamic obstacle point cloud data, the occupancy probability of the three-dimensional grids through which the rays of these three types of point clouds pass is updated downward, and the three-dimensional grids hit by these three types of point clouds are not processed; For the precise ground point cloud data, the occupancy probability of the three-dimensional grids that the point cloud rays pass through is updated downward, and the occupancy probability of the three-dimensional grids that the point cloud rays hit is updated upward, and the drivable attribute is given; For static obstacle point cloud data, the occupancy probability of the three-dimensional grid passed by such point cloud rays is updated downward, and the occupancy probability of the three-dimensional grid hit by such point cloud rays is updated upward, and the non-drivable attribute is given.

9. A computing device, characterized in that include: one or more processors; Memory; And one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by one or more processors, and one or more of the programs include instructions for the method for fine-grained description of the vehicle's surrounding environment as described in any one of claims 1-4 above.

10. A storage medium, characterized in that The storage medium stores one or more computer-readable programs, and the one or more programs include instructions, and the instructions are suitable for being loaded by the memory and executing the method for fine-grained description of the vehicle's surrounding environment as described in any one of claims 1-4.

Images