Obstacle segmentation recognition method and system

Abstract

The application relates to an obstacle segmentation and identification method and system, which comprises the following steps: projecting point cloud data to a first two-dimensional grid map, traversing all point clouds of the point cloud data, if a certain point cloud x1 belongs to a certain dense point cloud grid y1, and meanwhile the difference between the maximum height and the minimum height of the point clouds in the grid y1 is greater than a preset first threshold value, and the difference between the point cloud height of the point cloud x1 and the minimum height of the point cloud in the grid y1 is greater than a preset second threshold value, obtaining other point clouds in the grid y1 with a point cloud height higher than that of the point cloud x1, and segmenting the point cloud x1 and the other point clouds into candidate obstacle point clouds; traversing all point clouds of the point cloud data, if a certain point cloud x g is a candidate obstacle point cloud, projecting the point cloud x g onto an obstacle segmentation image, if the projection position of the point cloud x g in the obstacle segmentation image is located in an obstacle region, determining that the point cloud x g is a final obstacle point cloud. The application performs obstacle segmentation and identification based on a solid-state laser radar, and avoids generating a large number of laser noise points.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of vehicle perception technology, in particular to an obstacle segmentation and identification method and system. BACKGROUND

[0002] With the development of current automatic driving perception systems, various types of sensors suitable for vehicle-mounted have emerged accordingly, and the performance of laser radar is particularly outstanding. From the original bulky mechanical rotating laser radar, it has gradually evolved into a small embedded solid-state laser radar. Due to the replacement of laser radar hardware, there is an urgent need to migrate the original mechanical rotating laser radar obstacle detection segmentation algorithm to solid-state laser radar. However, the current laser radar segmentation algorithm is still mainly based on rotating mechanical laser radar. By utilizing the geometric relationship between the rotating laser beam, the undirected graph structure of the laser point cloud can be constructed, and based on this, the obstacles can be extracted. However, there are great differences between the point cloud data scanning of solid-state laser radar and mechanical rotating laser radar. The original obstacle segmentation method based on rotating laser radar is no longer applicable. In addition, when laser radar extracts obstacles alone, a large number of laser noise points will be randomly generated, which will be segmented as obstacles, easily leading to over-segmentation, causing automatic driving to brake and change lanes, and affecting the comfort of passengers.

[0003] In summary, how to perform obstacle segmentation and identification based on solid-state laser radar and avoid generating a large number of laser noise points is a technical problem that needs to be solved at present. SUMMARY

[0004] The purpose of the present application is to provide an obstacle segmentation and identification method and system to realize obstacle segmentation and identification based on solid-state laser radar and avoid generating a large number of laser noise points.

[0005] To achieve the above-mentioned purpose, an embodiment of the present application provides an obstacle segmentation and identification method, comprising the following steps:

[0006] Step S10, acquiring point cloud data collected by a vehicle-mounted laser radar, projecting the point cloud data to a first two-dimensional grid map, and constructing a data structure based on grid storage point cloud; wherein the data structure includes the number of point clouds projected to the grid, the height of the point cloud, and the identification of dense point cloud grids or sparse point cloud grids;

[0007] Step S20, traversing all point clouds of the point cloud data, if a point cloud x1 belongs to a certain dense point cloud grid y1, and simultaneously satisfies that the difference between the maximum height and the minimum height of the point cloud in the grid y1 is greater than a preset first threshold value, and the difference between the point cloud height of the point cloud x1 and the minimum height of the point cloud of the grid y1 is greater than a preset second threshold value, then acquiring other point clouds in the grid y1 whose point cloud height is higher than the point cloud x1, and segmenting the point cloud x1 and the other point clouds into candidate obstacle point clouds;

[0008] Step S30, obtaining an obstacle segmentation image; wherein the obstacle segmentation image is obtained by image segmentation on a visual image collected by a vehicle-mounted camera;

[0009] Step S40, traversing all point clouds of the point cloud data again, if a point cloud x g is a candidate obstacle point cloud, projecting the point cloud x g into the obstacle segmentation image, if the projection position of the point cloud x g in the obstacle segmentation image is located in an obstacle region, determining the point cloud x g as a final obstacle point cloud.

[0010] Preferably, the step S20 further comprises:

[0011] If a point cloud x2 belongs to a sparse point cloud grid y2, obtaining all point clouds in a first preset size region centered on the grid y2, respectively calculating the slope between the all point clouds and the point cloud x2, and counting the number of slopes greater than or equal to a third preset threshold in the calculated slopes, if the number of slopes is greater than a fourth preset threshold, obtaining other point clouds in the first preset size region with a height greater than the point cloud x2, and segmenting the point cloud x2 and the other point clouds into candidate obstacle point clouds.

[0012] Preferably, the step S20 further comprises:

[0013] If the number of slopes is less than or equal to the fourth preset threshold, respectively calculating the height difference between the all point clouds and the point cloud x2, and counting the number of differences greater than a fifth preset threshold in the calculated differences, if the number of differences is greater than a sixth preset threshold, obtaining all point clouds in a second preset size region centered on the grid y2, obtaining other point clouds in the second preset size region with a height greater than the point cloud x2, and segmenting the point cloud x2 and the other point clouds into candidate obstacle point clouds.

[0014] Preferably, the first preset size region is a circular region centered on the grid y2 with R as the radius, wherein R is a preset value; the first preset size region is a circular region centered on the grid y2 with r as the radius, wherein r is a preset value; R is less than r.

[0015] Preferably, the step S30 further comprises:

[0016] Projecting the obstacle segmentation image into a second two-dimensional grid map to obtain an impassable region of the obstacle in the second two-dimensional grid map.

[0017] The step S40 further comprises:

[0018] If the projected position of the point cloud x g in the second two-dimensional grid map is not located in the obstacle region, the point cloud x g is further projected into the second two-dimensional grid map, and it is judged whether the projected position of the point cloud x g in the second two-dimensional grid map is in the impassable region, and if yes, the point cloud x g is determined as the final obstacle point cloud, and if no, the point cloud x g is determined as a non-obstacle point cloud.

[0019] Preferably, the judging whether the projected position of the point cloud x g in the second two-dimensional grid map is in the impassable region comprises:

[0020] judging whether at least a part of the impassable region exists in a third preset size region centered on the projected position of the point cloud x g in the second two-dimensional grid map, and if yes, the point cloud x g is determined as the final obstacle point cloud, and if no, the point cloud x g is determined as a non-obstacle point cloud.

[0021] Preferably, when the number of point clouds in a grid exceeds a preset seventh threshold value, the grid belongs to a dense point cloud grid; otherwise, it belongs to a sparse point cloud grid.

[0022] As the same inventive concept, embodiments of the present application also propose an obstacle segmentation and recognition system, comprising:

[0023] a point cloud preprocessing unit, configured to acquire point cloud data collected by a vehicle-mounted laser radar, project the point cloud data into a first two-dimensional grid map, and construct a data structure for storing point clouds based on grids; wherein the data structure comprises the number of point clouds projected into a grid, the height of a point cloud, and the identification of a dense point cloud grid or a sparse point cloud grid;

[0024] a point cloud preliminary screening unit, configured to traverse all point clouds of the point cloud data, if a point cloud x1 belongs to a certain dense point cloud grid y1, and simultaneously satisfies that the difference between the maximum height and the minimum height of the point clouds in the grid y1 is greater than a preset first threshold value, and the difference between the height of the point cloud x1 and the minimum height of the point clouds in the grid y1 is greater than a preset second threshold value, the point clouds in the grid y1 whose height is higher than that of the point cloud x1 are acquired, and the point cloud x1 and the other point clouds are segmented as candidate obstacle point clouds;

[0025] a visual processing unit, configured to acquire an obstacle segmentation image; wherein the obstacle segmentation image is obtained by image segmentation on a visual image collected by a vehicle-mounted camera;

[0026] a point cloud re-screening unit, configured to traverse all point clouds of the point cloud data again, and when a certain point cloud x g is a candidate obstacle point cloud, project the point cloud x g into the obstacle segmentation image, and when the projection position of the point cloud x g in the obstacle segmentation image is located in an obstacle region, determine the point cloud x g as a final obstacle point cloud.

[0027] Preferably, the point cloud preliminary screening unit is further configured to:

[0028] when a certain point cloud x2 belongs to a certain sparse point cloud grid y2, obtain all point clouds in a first preset size region centered on the grid y2, respectively calculate the slope between the all point clouds and the point cloud x2, and count the number of slopes greater than or equal to a third preset threshold value in the calculated slopes;

[0029] when the number of slopes is greater than a fourth preset threshold value, obtain other point clouds in the first preset size region whose height is greater than the point cloud x2, and divide the point cloud x2 and the other point clouds into candidate obstacle point clouds;

[0030] when the number of slopes is less than or equal to the fourth preset threshold value, respectively calculate the height difference between the all point clouds and the point cloud x2, count the number of differences greater than a fifth preset threshold value in the calculated differences, and if the number of differences is greater than a sixth preset threshold value, obtain all point clouds in a second preset size region centered on the grid y2, obtain other point clouds in the second preset size region whose height is greater than the point cloud x2, and divide the point cloud x2 and the other point clouds into candidate obstacle point clouds.

[0031] Preferably, the visual processing unit is further configured to project the obstacle segmentation image into a second two-dimensional grid map to obtain an impassable region of the obstacle in the second two-dimensional grid map.

[0032] The point cloud re-screening unit is further configured to:

[0033] when the projection position of the point cloud x g in the obstacle segmentation image is not located in the obstacle region, further project the point cloud x g into the second two-dimensional grid map, and determine whether the projection position of the point cloud x g in the second two-dimensional grid map is in the impassable region, if yes, determine the point cloud x g as a final obstacle point cloud, and if no, determine the point cloud x g is not an obstacle point cloud.

[0034] The embodiments of the present invention have at least the following beneficial effects:

[0035] By projecting point cloud data collected by vehicle-mounted solid-state LiDAR onto a two-dimensional grid map, the point cloud data is rasterized. Based on the point cloud density and height in each grid after rasterization, obstacle segmentation is performed to determine candidate obstacle point clouds. This method can simultaneously adapt to both solid-state LiDAR and mechanical LiDAR for obstacle segmentation and recognition. Furthermore, by combining the obstacle segmentation images from the vehicle vision system, the candidate obstacle point clouds are further filtered to determine the final obstacle point clouds, ensuring the accuracy of obstacle segmentation and avoiding the generation of a large amount of laser noise.

[0036] Other features and advantages of embodiments of the present invention will be set forth in the following description. Attached Figure Description

[0037] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0038] Figure 1 This is a flowchart illustrating an obstacle segmentation and recognition method according to an embodiment of the present invention.

[0039] Figure 2 This is a schematic diagram of the dimensions of a two-dimensional raster map in an embodiment of the present invention.

[0040] Figure 3 This is a schematic diagram illustrating the specific process of point cloud segmentation in the dense point cloud grid in step S20 of this embodiment of the invention.

[0041] Figure 4 This is a schematic diagram illustrating the specific process of point cloud segmentation in the sparse point cloud raster in step S20 of this embodiment of the invention.

[0042] Figure 5 This is a schematic diagram of the specific process of step S40 in an embodiment of the present invention.

[0043] Figure 6 This is a schematic diagram of an obstacle segmentation and recognition system according to an embodiment of the present invention. Detailed Implementation

[0044] The various exemplary embodiments, features, and aspects of this disclosure will now be described in detail with reference to the accompanying drawings. Furthermore, numerous specific details are set forth in the following detailed embodiments to better illustrate the invention. Those skilled in the art will understand that the invention can be practiced without certain specific details. In some instances, means well known to those skilled in the art have not been described in detail in order to highlight the spirit of the invention.

[0045] See Figure 1 One embodiment of the present invention proposes an obstacle segmentation and recognition method applicable to autonomous vehicles, which includes the segmentation and recognition of obstacles on the road during the autonomous driving process of the vehicle, comprising the following steps S10 to S40:

[0046] Step S10: Acquire point cloud data collected by vehicle-mounted LiDAR, project the point cloud data onto a first two-dimensional grid map, and construct a data structure based on grid storage of point clouds; wherein the data structure includes the number of point clouds projected onto the grid, the height of the point cloud, and the identifier of whether it belongs to a dense point cloud grid or a sparse point cloud grid.

[0047] Specifically, when applying the method of this embodiment, it is necessary to install a LiDAR and a vision system in the vehicle. The LiDAR is not limited to one type; both new solid-state LiDAR and traditional mechanical LiDAR can be used, such as the RS-LIDAR-M1 solid-state LiDAR. During the autonomous driving process, the point cloud data collected by the solid-state LiDAR is received in real time. This point cloud data is based on the vehicle coordinate system. Projecting this point cloud data onto a first two-dimensional grid map can construct a grid-based data structure for storing point clouds, including multiple grids. Each grid stores point cloud data for the corresponding coordinate region, including the number of point clouds projected onto the grid, the point cloud height, and an identifier indicating whether it belongs to a dense point cloud grid or a sparse point cloud grid. The dense point cloud grid and the sparse point cloud grid are divided according to the number of point clouds in the grid.

[0048] For example, in one application scenario, the size of a two-dimensional raster map is as follows: Figure 2 As shown, the size is 401×101, the origin is the top left corner with coordinates (0,0), and the pixel (50,400) in the image is the center point of the front of the vehicle. Each pixel represents an actual geometric distance of 20cm. Therefore, the range of obstacles in front of the vehicle is about 80 meters, and the range of obstacles on the vehicle body is 20 meters to the left and right.

[0049] Step S20: Traverse all point clouds of the point cloud data. If a point cloud x1 belongs to a uniform dense point cloud grid y1, and simultaneously satisfies that the difference between the maximum height and minimum height of the point cloud in grid y1 is greater than a preset first threshold, and the difference between the point cloud height of point cloud x1 and the minimum height of the point cloud in grid y1 is greater than a preset second threshold, then obtain other point clouds in grid y1 whose point cloud height is higher than that of point cloud x1, and segment point cloud x1 and the other point clouds into candidate obstacle point clouds.

[0050] For a detailed process of point cloud segmentation in a dense point cloud raster, please refer to [link / reference]. Figure 3 In the process, when a certain point cloud is determined to be a candidate obstacle point cloud, all point clouds in the grid where the point cloud is located that are higher than the point cloud are expanded and segmented into candidate obstacle point clouds.

[0051] Step S30: Obtain obstacle segmentation image; wherein the obstacle segmentation image is obtained by segmenting the visual image captured by the vehicle-mounted camera;

[0052] Specifically, the vehicle vision system receives visual images of the vehicle's surrounding environment from the vehicle's camera in real time, and performs obstacle segmentation on the visual images using a pre-set image segmentation algorithm. For example, it segments obstacles such as pedestrians, two-wheeled vehicles, vehicles, roadblocks, and non-road areas. It should be noted that since the acquired segmented images are distorted, the method in this embodiment also obtains a distortion-free obstacle segmentation image based on the distortion parameters for use in subsequent calculations.

[0053] It should be noted that steps S20 and S30 can be executed simultaneously, and there is no order in their execution.

[0054] Step S40: Traverse all point clouds of the point cloud data again. If a certain point cloud x g For the point cloud of candidate obstacles, then the point cloud x g Projected onto the obstacle segmentation image, if the point cloud x g If the projection position in the obstacle segmentation image is located within the obstacle region, then the point cloud x is determined. g To form the final obstacle point cloud.

[0055] Specifically, in step S40, the candidate obstacle point cloud obtained in step S20 is verified using the distortion-free obstacle segmentation image to filter out missegmentation caused by lidar noise and determine the final obstacle point cloud.

[0056] Further, see Figure 4 Step S20 may further include:

[0057] If a point cloud x2 belongs to a sparse point cloud grid y2, then all point clouds in the first preset size area centered on grid y2 are obtained, and the slope between each point cloud and point cloud x2 in the first preset size area is calculated. The number of slopes with a slope greater than or equal to a preset third threshold is counted among the multiple slopes calculated.

[0058] If the number of slopes is greater than a preset fourth threshold, then other point clouds in the first preset size region with a point cloud height greater than point cloud x2 are obtained, and point cloud x2 and the other point clouds are segmented into candidate obstacle point clouds;

[0059] If the number of slopes is less than or equal to a preset fourth threshold, then the height difference between all point clouds and point cloud x2 is calculated, and the number of differences among the calculated height differences that are greater than a preset fifth threshold is counted. If the number of differences is greater than a preset sixth threshold, then all point clouds in the second preset size area centered on grid y2 are obtained, and other point clouds in the second preset size area whose height is greater than that of point cloud x2 are divided into point cloud x2 and other point clouds are divided into candidate obstacle point clouds.

[0060] Specifically, the slope gradient is calculated as follows in this embodiment:

[0061] K = abs(ΔZ / D)

[0062] Where K is the slope between point cloud x2 and any other point cloud, ΔZ is the height difference between point cloud x2 and any other point cloud, D is the distance between point cloud x2 and any other point cloud, and abs() is the absolute value operation expression.

[0063] Specifically, when the slope is greater than or equal to the third threshold, a count is performed. When the cumulative count exceeds the fourth threshold, the point cloud x2 is segmented into candidate obstacle point clouds, and all other point clouds in the first preset size region that are higher than the point cloud x2 are further segmented into candidate obstacle point clouds. The third threshold is preferably, but not limited to, 1.

[0064] Specifically, when the height difference is greater than or equal to the fifth threshold, a count is made. When the cumulative count exceeds the sixth threshold, the point cloud x2 is segmented into candidate obstacle point clouds, and all other point clouds in the second preset size region that are higher than the height of point cloud x2 are further segmented and all of them are segmented into candidate obstacle point clouds. The fifth threshold is preferably, but not limited to, 20.

[0065] Further, the first preset size area is a circular area centered on grid y2 with radius R, where R is a preset value; the second preset size area is a circular area centered on grid y2 with radius r, where r is a preset value; R is less than r. Here, R and r are preferably, but not limited to, the number of pixels, that is, the length represented by several pixels on the grid map, preferably, but not limited to, R = 2, r = 5.

[0066] Furthermore, step S30 also includes:

[0067] The obstacle segmentation image is projected onto a second two-dimensional grid map to obtain the impassable area of ​​the obstacle in the second two-dimensional grid map;

[0068] Specifically, the distortion-corrected obstacle segmentation image can be transformed into vehicle coordinates (X, V) using the following linear camera model, based on the camera's calibrated intrinsic and extrinsic parameters. H ,Y H Z H The coordinates under the image are u and v, which are the X and Y coordinates of a pixel in the image. H ,Y H Z H Let M be the X, Y, and Z coordinates in the vehicle coordinate system, and M be the projection transformation function, where Z is assumed to be... H =0. After obtaining the coordinates of the image-segmented obstacle in the vehicle coordinate system, it can be projected onto a two-dimensional raster map, thereby obtaining the impassable area of ​​the image-segmented obstacle in the raster, as shown in the following expression:

[0069]

[0070] Further, see Figure 5 Step S40 further includes:

[0071] If point cloud x g If the projection position in the obstacle segmentation image is not located in the obstacle region, then the point cloud x g Projected onto the second two-dimensional raster map, and the point cloud x is determined. g If the projection position in the second two-dimensional raster map is located in the impassable area, then determine the point cloud x. g For the final obstacle point cloud, if not, determine point cloud x. g It is not an obstacle point cloud.

[0072] Furthermore, the determination of the point cloud x g Whether the projected position in the second two-dimensional grid map is located in the impassable area includes:

[0073] Determine the point cloud x g If at least a portion of the impassable area exists within a third preset-size region centered on the projection position in the second two-dimensional raster map, then determine the point cloud x. g For the final obstacle point cloud, if not, determine point cloud x. g It is not an obstacle point cloud.

[0074] Specifically, since there is an error between the grid position of the candidate obstacle point cloud and the grid position of the obstacle in the image segmentation, this embodiment introduces a certain error range. The third preset size region is the error range. The corresponding point cloud x is determined by judging whether there is at least a part of the impassable region in the third preset size region, that is, whether the third preset size region overlaps with the impassable region. g Is it an obstacle point cloud?

[0075] Furthermore, the third preset size area is a rectangular area with a size of m*n, where m is the horizontal dimension and n is the vertical dimension. Preferably, m = 4 pixels in length and n = 25 pixels in length.

[0076] Furthermore, when the point cloud data is projected onto the first two-dimensional raster map, if the number of point clouds in the raster exceeds a preset seventh threshold, then the raster belongs to a dense point cloud raster; otherwise, it belongs to a sparse point cloud raster.

[0077] See Figure 6 Another embodiment of the present invention also proposes an obstacle segmentation and recognition system. The system of this embodiment corresponds to the method of the above embodiments and can be used to perform the steps of the method of the above embodiments. The system of this embodiment includes:

[0078] Point cloud preprocessing unit 1 is used to acquire point cloud data collected by vehicle-mounted lidar, project the point cloud data onto a first two-dimensional grid map, and construct a data structure based on the grid to store the point cloud; wherein the data structure includes the number of point clouds projected onto the grid, the point cloud height, and the identifier of whether it belongs to a dense point cloud grid or a sparse point cloud grid.

[0079] The point cloud initial screening unit 2 is used to traverse all point clouds of the point cloud data. If a point cloud x1 belongs to a certain dense point cloud grid y1, and simultaneously satisfies that the difference between the maximum height and the minimum height of the point cloud in grid y1 is greater than a preset first threshold, and the difference between the point cloud height of point cloud x1 and the minimum height of the point cloud in grid y1 is greater than a preset second threshold, then other point clouds in grid y1 with point cloud heights higher than point cloud x1 are obtained, and point cloud x1 and the other point clouds are divided into candidate obstacle point clouds.

[0080] The visual processing unit 3 is used to acquire obstacle segmentation images; wherein the obstacle segmentation images are obtained by image segmentation of visual images captured by the vehicle-mounted camera;

[0081] Point cloud re-screening unit 4 is used to re-traverse all point clouds of the point cloud data, when a certain point cloud x g When creating a point cloud for candidate obstacles, x g Projected onto the obstacle segmentation image, and when the point cloud x g The projection position in the obstacle segmentation image is located within the obstacle region, determining the point cloud x. g To form the final obstacle point cloud.

[0082] Preferably, the point cloud initial screening unit 2 is further used for:

[0083] When a point cloud x2 belongs to a sparse point cloud grid y2, all point clouds in the first preset size area centered on grid y2 are obtained, the slope between each of these point clouds and point cloud x2 is calculated, and the number of slopes with a slope greater than or equal to a preset third threshold is counted among the multiple slopes obtained.

[0084] When the number of slopes is greater than a preset fourth threshold, other point clouds with a height greater than that point cloud x2 in the first preset size region are obtained, and that point cloud x2 and the other point clouds are segmented into candidate obstacle point clouds;

[0085] When the number of slopes is less than or equal to a preset fourth threshold, the height difference between all point clouds and point cloud x2 is calculated, and the number of differences among the calculated height differences that are greater than a preset fifth threshold is counted. If the number of differences is greater than a preset sixth threshold, all point clouds in the second preset size area centered on grid y2 are obtained, and other point clouds in the second preset size area whose height is greater than that of point cloud x2 are divided into point cloud x2 and other point clouds are divided into candidate obstacle point clouds.

[0086] Preferably, the visual processing unit is further configured to: project the obstacle segmentation image onto a second two-dimensional grid map to obtain the impassable area of ​​the obstacle in the second two-dimensional grid map;

[0087] The point cloud rescreening unit 4 is also used for:

[0088] When point cloud x g When the projection position in the obstacle segmentation image is not located in the obstacle region, the point cloud x is further... g Projected onto the second two-dimensional raster map, and the point cloud x is determined. g If the projection position in the second two-dimensional raster map is located in the impassable area, then determine the point cloud x.g For the final obstacle point cloud, if not, determine point cloud x. g It is not an obstacle point cloud.

[0089] The systems described in the embodiments above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of the system solution in the embodiments, depending on actual needs.

[0090] It should be noted that the system in the above embodiments corresponds to the method in the above embodiments. Therefore, the parts of the system in the above embodiments that are not described in detail can be obtained by referring to the content of the method in the above embodiments. That is, the specific steps recorded in the method in the above embodiments can be understood as the functions that the system in the above embodiments can achieve, and will not be described again here.

[0091] Furthermore, if the obstacle segmentation and recognition system described in the above embodiments is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium.

[0092] In various embodiments of the present invention, point cloud data collected by vehicle-mounted solid-state LiDAR is projected onto a two-dimensional grid map to rasterize the point cloud data. Based on the point cloud density and height in each grid after rasterization, obstacle segmentation is performed to determine candidate obstacle point clouds. This method can simultaneously adapt to both solid-state LiDAR and mechanical LiDAR for obstacle segmentation and recognition. Furthermore, by combining the obstacle segmentation image from the vehicle vision system, the candidate obstacle point clouds are further filtered to determine the final obstacle point cloud, ensuring the accuracy of obstacle segmentation and avoiding the generation of a large amount of laser noise.

[0093] The various embodiments of the present invention have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical application, or technical improvements to the embodiments in the market, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. An obstacle segmentation and recognition method, characterized in that, Includes the following steps: Step S10: Acquire point cloud data collected by vehicle-mounted LiDAR, project the point cloud data onto a first two-dimensional grid map, and construct a data structure based on grid storage of point clouds; wherein the data structure includes the number of point clouds projected onto the grid, the height of the point cloud, and the identifier of whether it belongs to a dense point cloud grid or a sparse point cloud grid. Step S20: Traverse all point clouds of the point cloud data. If a point cloud x1 belongs to a dense point cloud grid y1, and simultaneously satisfies that the difference between the maximum and minimum heights of the point clouds in grid y1 is greater than a preset first threshold, and the difference between the point cloud height of point cloud x1 and the minimum height of the point clouds in grid y1 is greater than a preset second threshold, then obtain other point clouds in grid y1 whose point cloud height is higher than point cloud x1, and segment point cloud x1 and the other point clouds into candidate obstacle point clouds. If a point cloud x2 belongs to a sparse point cloud grid y2, then obtain all point clouds in a first preset size area centered on grid y2, calculate the slope between all point clouds and point cloud x2, and count the number of slopes among the calculated slopes that are greater than or equal to a preset third threshold. If the number of slopes is greater than a preset fourth threshold, then obtain other point clouds in the first preset size area whose point cloud height is greater than point cloud x2, and segment point cloud x2 and the other point clouds into candidate obstacle point clouds. Step S30: Obtain obstacle segmentation image; wherein the obstacle segmentation image is obtained by segmenting the visual image captured by the vehicle-mounted camera; Step S40: Traverse all point clouds of the point cloud data again. If a certain point cloud x g For the point cloud of candidate obstacles, then the point cloud x g Projected onto the obstacle segmentation image, if the point cloud x g If the projection position in the obstacle segmentation image is located within the obstacle region, then the point cloud x is determined. g To form the final obstacle point cloud.

2. The method according to claim 1, characterized in that, Step S20 further includes: If the number of slopes is less than or equal to a preset fourth threshold, then the height difference between all point clouds and point cloud x2 is calculated, and the number of differences among the calculated height differences that are greater than a preset fifth threshold is counted. If the number of differences is greater than a preset sixth threshold, then all point clouds in the second preset size area centered on grid y2 are obtained, and other point clouds in the second preset size area whose height is greater than that of point cloud x2 are divided into point cloud x2 and other point clouds are divided into candidate obstacle point clouds.

3. The method according to claim 2, characterized in that, The first preset size area is a circular area centered on grid y2 with radius R, where R is a preset value; the second preset size area is a circular area centered on grid y2 with radius r, where r is a preset value; R is less than r.

4. The method according to claim 1, characterized in that, Step S30 further includes: The obstacle segmentation image is projected onto a second two-dimensional grid map to obtain the impassable area of ​​the obstacle in the second two-dimensional grid map; Step S40 further includes: If point cloud x g If the projection position in the obstacle segmentation image is not located in the obstacle region, then the point cloud x g Projected onto the second two-dimensional raster map, and the point cloud x is determined. g If the projection position in the second two-dimensional raster map is located in the impassable area, then determine the point cloud x. g For the final obstacle point cloud, if not, determine point cloud x. g It is not an obstacle point cloud.

5. The method according to claim 4, characterized in that, The determination of the point cloud x g Whether the projected position in the second two-dimensional grid map is located in the impassable area includes: Determine the point cloud x g If at least a portion of the impassable area exists within a third preset-size region centered on the projection position in the second two-dimensional raster map, then determine the point cloud x. g For the final obstacle point cloud, if not, determine point cloud x. g It is not an obstacle point cloud.

6. The method according to claim 1, characterized in that, If the number of point clouds in a raster exceeds the preset seventh threshold, the raster is classified as a dense point cloud raster; otherwise, it is classified as a sparse point cloud raster.

7. An obstacle segmentation and recognition system, characterized in that, include: A point cloud preprocessing unit is used to acquire point cloud data collected by vehicle-mounted lidar, project the point cloud data onto a first two-dimensional grid map, and construct a data structure based on the grid to store the point cloud; wherein the data structure includes the number of point clouds projected onto the grid, the point cloud height, and the identifier of whether it belongs to a dense point cloud grid or a sparse point cloud grid. A point cloud initial screening unit is used to traverse all point clouds of the point cloud data. If a point cloud x1 belongs to a dense point cloud grid y1, and simultaneously satisfies that the difference between the maximum and minimum heights of the point clouds in grid y1 is greater than a preset first threshold, and the difference between the point cloud height of point cloud x1 and the minimum height of the point clouds in grid y1 is greater than a preset second threshold, then other point clouds in grid y1 with higher point cloud heights than point cloud x1 are obtained, and point cloud x1 and the other point clouds are segmented into candidate obstacle point clouds. If a point cloud x2 belongs to a sparse point cloud grid y2, then all point clouds in a first preset size area centered on grid y2 are obtained, the slope between all point clouds and point cloud x2 is calculated, and the number of slopes with slopes greater than or equal to a preset third threshold is counted. If the number of slopes is greater than a preset fourth threshold, then other point clouds in the first preset size area with higher point cloud heights than point cloud x2 are obtained, and point cloud x2 and the other point clouds are segmented into candidate obstacle point clouds. A vision processing unit is used to acquire obstacle segmentation images; wherein the obstacle segmentation images are obtained by image segmentation of visual images captured by an onboard camera; The point cloud re-screening unit is used to re-traverse all point clouds of the point cloud data, when a certain point cloud x g When creating a point cloud for candidate obstacles, x g Projected onto the obstacle segmentation image, and when the point cloud x g The projection position in the obstacle segmentation image is located within the obstacle region, determining the point cloud x. g To form the final obstacle point cloud.

8. The system according to claim 7, characterized in that, The point cloud initial screening unit is also used for: When the number of slopes is less than or equal to a preset fourth threshold, the height difference between all point clouds and point cloud x2 is calculated, and the number of differences among the calculated height differences that are greater than a preset fifth threshold is counted. If the number of differences is greater than a preset sixth threshold, all point clouds in the second preset size area centered on grid y2 are obtained, and other point clouds in the second preset size area whose height is greater than that of point cloud x2 are divided into point cloud x2 and other point clouds are divided into candidate obstacle point clouds.

9. The system according to claim 8, characterized in that, The visual processing unit is further configured to: project the obstacle segmentation image onto a second two-dimensional grid map to obtain the impassable area of ​​the obstacle in the second two-dimensional grid map; The point cloud rescreening unit is also used for: When point cloud x g When the projection position in the obstacle segmentation image is not located in the obstacle region, the point cloud x is further... g Projected onto the second two-dimensional raster map, and the point cloud x is determined. g If the projection position in the second two-dimensional raster map is located in the impassable area, then determine the point cloud x. g For the final obstacle point cloud, if not, determine point cloud x. g It is not an obstacle point cloud.

Images