Ground point cloud segmentation method and device, vehicle and storage medium

Abstract

This application discloses a method, apparatus, vehicle, and storage medium for segmenting ground point clouds. The method involves acquiring a first set of point clouds located within a road-accessible area from a laser point cloud corresponding to the current environment; projecting the first set of point clouds onto an environmental image corresponding to the current environment; and filtering the first set of point clouds based on the segmented ground regions in the environmental image to obtain a second set of point clouds; performing plane fitting on the second set of point clouds; and determining the ground point clouds in the first set of point clouds based on the fitted plane obtained from the plane fitting. By filtering the first set of point clouds through the segmented ground regions in the environmental image, all laser point clouds in the obtained second set are confirmed to be located within the ground region. Therefore, plane fitting based on the second set of point clouds yields a more accurate fitted plane, thereby precisely determining the ground point clouds in the first set of point clouds and making the segmentation of ground point clouds more accurate.

Description

Technical Field

[0001] This application relates to the field of point cloud data processing technology, and more specifically, to a method, apparatus, vehicle, and storage medium for segmenting ground point clouds. Background Technology

[0002] With the development of intelligent vehicles, vehicle-mounted LiDAR is widely used during vehicle operation, for example, in obstacle detection and LiDAR point cloud localization. However, before realizing these functions, the vehicle first needs to perform ground point cloud segmentation on the LiDAR-acquired LiDAR point cloud to remove ground point clouds. Typically, vehicles can use features such as height differences and gradients between LiDAR point clouds, or determine the ground point cloud using plane fitting methods. However, this method is easily affected by point cloud noise, leading to missegmentation and low accuracy in ground point cloud segmentation. Summary of the Invention

[0003] In view of the above problems, this application proposes a method, apparatus, vehicle and storage medium for segmenting ground point clouds, so as to reduce the influence of point cloud noise and make the segmentation of ground point clouds more accurate.

[0004] In a first aspect, embodiments of this application provide a method for segmenting ground point clouds. The method includes: obtaining a first set of point clouds located in a road-accessible area from a laser point cloud corresponding to a current environment; projecting the first set of point clouds onto an environmental image corresponding to the current environment, and filtering the first set of point clouds based on the ground area segmented from the environmental image to obtain a second set of point clouds, the second set of point clouds including laser point clouds located in the ground area; performing plane fitting on the second set of point clouds, and determining the ground point clouds in the first set of point clouds based on the fitted plane obtained by the plane fitting.

[0005] Secondly, embodiments of this application provide a ground point cloud segmentation device, the device comprising: a point cloud acquisition module, a point cloud processing module, and a point cloud determination module, wherein the point cloud acquisition module is used to acquire a first point cloud set located in a road-accessible area from a laser point cloud corresponding to the current environment; the point cloud processing module is used to project the first point cloud set onto an environmental image corresponding to the current environment, and filter the first point cloud set based on the ground area segmented from the environmental image to obtain a second point cloud set, the second point cloud set including laser point clouds located in the ground area; the point cloud determination module is used to perform plane fitting on the second point cloud set, and determine the ground point cloud in the first point cloud set based on the fitted plane obtained by the plane fitting.

[0006] Thirdly, embodiments of this application provide a vehicle, including: one or more processors; a memory; and one or more application programs, wherein the one or more application programs are stored in the memory and configured to be executed by the one or more processors, and the one or more application programs are configured to perform the ground point cloud segmentation method provided in the first aspect above.

[0007] Fourthly, embodiments of this application provide a computer-readable storage medium storing program code, which can be called by a processor to execute the ground point cloud segmentation method provided in the first aspect above.

[0008] The solution provided in this application involves obtaining a first set of laser point clouds located within the passable road area from the laser point cloud corresponding to the current environment; projecting the first set of point clouds onto an environmental image corresponding to the current environment; and filtering the first set of point clouds based on the segmented ground areas in the environmental image to obtain a second set of point clouds; performing plane fitting on the second set of point clouds; and determining the ground point clouds in the first set of point clouds based on the fitted plane obtained from the plane fitting. By filtering the first set of point clouds through the segmented ground areas in the environmental image, the laser point clouds in the obtained second set are all laser point clouds located within the ground area. Therefore, plane fitting based on the second set of point clouds can obtain a more accurate fitted plane, thereby accurately determining the ground point clouds in the first set of point clouds and making the segmentation of the ground point clouds more accurate. Attached Figure Description

[0009] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0010] Figure 1 A flowchart illustrating a ground point cloud segmentation method provided in one embodiment of this application is shown.

[0011] Figure 2 A schematic diagram of an environmental image provided in one embodiment of this application is shown.

[0012] Figure 3 A flowchart illustrating a ground point cloud segmentation method provided in another embodiment of this application is shown.

[0013] Figure 4 A schematic diagram illustrating the determination of the direction of expansion of the passable area in an embodiment of this application is shown.

[0014] Figure 5The diagram illustrates several road passable areas provided in the embodiments of this application.

[0015] Figure 6 A schematic diagram of the specific process of step S240 in another embodiment of this application is shown.

[0016] Figure 7 A schematic diagram showing the position of the vehicle in the grid diagram according to an embodiment of this application is shown.

[0017] Figure 8 A schematic diagram of the specific process of step S260 in another embodiment of this application is shown.

[0018] Figure 9 A schematic diagram of the specific process of step S280 in another embodiment of this application is shown.

[0019] Figure 10 A structural block diagram of the ground point cloud segmentation device provided in an embodiment of this application is shown.

[0020] Figure 11 A structural block diagram of a vehicle provided in an embodiment of this application is shown.

[0021] Figure 12 A structural block diagram of a computer-readable storage medium provided in an embodiment of this application is shown. Detailed Implementation

[0022] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings.

[0023] As the application of LiDAR in intelligent vehicles continues to develop, during intelligent driving, vehicles often need to acquire laser point cloud data of the surrounding environment through laser types, and then perform ground segmentation processing on the current environment's laser point cloud to achieve functions such as obstacle detection and laser point cloud localization. In implementing these functions, the vehicle needs to screen the laser point cloud data of the current environment directly acquired by the LiDAR to remove ground point clouds.

[0024] Specifically, current laser point cloud ground segmentation algorithms are typically implemented based on point cloud spatial features or model fitting methods. Algorithms based on point cloud spatial features utilize methods such as sampling maximum and minimum height differences and neighborhood features between laser scanning beams. For example, by calculating the difference between the maximum and minimum heights of laser point clouds within a grid, if the difference is less than a set threshold, it is determined to be a ground point cloud; or by calculating the gradient between each laser point cloud and its neighboring laser point clouds, if the gradient value is small, it is determined to be a ground point cloud. However, these methods are easily limited by feature thresholds and the influence of noisy point clouds, resulting in under-segmentation or over-segmentation. Furthermore, some features are only applicable to certain types of LiDAR, limiting their wide applicability. Additionally, model fitting-based ground segmentation methods typically utilize an assumed mathematical model. Since certain assumptions are difficult to meet the complex real-world road conditions, their applicability is also limited. While planar fitting methods can solve the problem of segmenting curved surfaces under certain conditions, fixed planar regions are difficult to adapt to complex real-world road environments.

[0025] Therefore, this application provides a method, apparatus, vehicle, and storage medium for segmenting ground point clouds. By segmenting ground regions from environmental images, a first set of point clouds corresponding to passable areas is filtered, ensuring that all laser point clouds in the resulting second set are definitively located within the ground region. Based on this second set, plane fitting is performed to obtain a more accurate fitting plane, thereby precisely determining the ground point clouds in the first set and making the ground point cloud segmentation more accurate. The specific ground point cloud segmentation method will be described in detail in subsequent embodiments.

[0026] Please see Figure 1 , Figure 1 This paper illustrates a flowchart of a ground point cloud segmentation method according to an embodiment of this application. The following will focus on... Figure 1 The process shown is described in detail. The ground point cloud segmentation method may specifically include the following steps:

[0027] Step S110: Obtain the first set of point clouds in the passable area of ​​the road from the laser point cloud corresponding to the current environment.

[0028] In this embodiment, the vehicle can acquire the laser point cloud corresponding to its current environment using LiDAR. Since the acquired laser point cloud is unprocessed, it includes both ground and non-ground point clouds. However, it is understandable that if the vehicle directly segments the ground point cloud using spatial features such as height and gradient values, the segmentation result will be inaccurate and easily affected by noisy point clouds. Therefore, the vehicle can first segment a first set of point clouds within the road-accessible area from the laser point cloud corresponding to the current environment. This allows for further screening based on the first set of point clouds to obtain an accurate ground point cloud.

[0029] In some implementations, the vehicle can determine the passable road area in the current environment using a pre-acquired high-precision map corresponding to the current environment. It can then project the laser point cloud corresponding to the current environment onto the high-precision map to obtain the first set of point clouds within the passable road area. Specifically, the high-precision map stores the coordinate data of discrete points on the left and right road boundaries in the current environment using the Universal Transverse Mercator Grid System (UTM). After acquiring the high-precision map, the vehicle can transform the UTM coordinates of the road boundary points stored in the high-precision map to the vehicle's corresponding vehicle coordinate system and store them separately for left and right sides. Then, by connecting the left and right road boundary points, the passable road area can be obtained.

[0030] Step S120: Project the first point cloud set onto the environmental image corresponding to the current environment, and filter the first point cloud set based on the ground area segmented in the environmental image to obtain a second point cloud set, wherein the second point cloud set includes laser point clouds located in the ground area.

[0031] In this embodiment, after the vehicle acquires the first point cloud set that is within the passable area of ​​the current environment, it can further extract the environmental image of the ground area corresponding to the current environment, remove the laser point clouds in the non-ground area of ​​the first point cloud set, and obtain the second point cloud set that is determined to be completely ground point clouds. This allows the vehicle to perform plane fitting based on the laser point clouds in the second point cloud set that are determined to be ground areas to obtain the accurate fitting plane corresponding to the current environment.

[0032] Understandably, the first point cloud set obtained by the vehicle through the above steps is merely a set of point clouds determined by using discrete points on the road boundaries in a high-precision map to determine whether the laser point clouds are within the passable road area. However, even if some laser point clouds are within the passable road area, they may not necessarily be identified as ground point clouds. For example, the first point cloud set will also include the laser point clouds corresponding to pedestrians and vehicles within the passable road area. Therefore, after obtaining the first point cloud set corresponding to the passable road area, the vehicle can further remove laser point clouds that do not belong to the ground area through image segmentation. This ensures that every point cloud in the final second point cloud set is an accurate ground-based laser point cloud, guaranteeing the accuracy of all laser point clouds in the second point cloud set being ground point clouds.

[0033] Specifically, when a vehicle acquires the laser point cloud corresponding to the current environment using LiDAR, it can simultaneously acquire the corresponding environmental image using devices such as surround-view cameras, and identify the ground region within that image to obtain the ground and non-ground regions. Figure 2 As shown, it illustrates the results of the vehicle's ground region recognition of the environmental image. Figure 2 The shaded area filled with diagonal lines represents the ground region identified by the vehicle. The vehicle can then project the laser point clouds corresponding to its surroundings onto this environmental image, which includes both ground and non-ground regions. This allows for the determination of a second point cloud set, composed of all laser point clouds within the ground region from the first point cloud set. In other words, by removing laser point clouds from the first set that do not belong to the ground region, a second point cloud set consisting entirely of laser point clouds belonging to the ground region is obtained.

[0034] It should be understood that while the second point cloud set obtained by the vehicle at this point can confirm that all laser point clouds belong to the ground area, some laser point clouds in the current environment, excluding those in the second point cloud set, may still belong to the ground area. These laser point clouds were not included in the second point cloud set during the aforementioned elimination steps. However, this error is acceptable because the vehicle does not directly use the second point cloud set as the final set of confirmed ground point clouds. Instead, it performs a planar fitting of the ground based on the laser point clouds in the second point cloud set that are definitively identified as ground point clouds. After obtaining the fitted plane, the initial first point cloud set is then filtered based on the fitted plane. In other words, even if some laser point clouds in the first point cloud set belong to the ground but were not included in the second point cloud set, the vehicle can still identify these ground point clouds through subsequent steps. Therefore, to improve the accuracy of the fitted plane obtained by the vehicle through the planar fitting of the second laser point clouds, the vehicle only needs to ensure the accuracy of the laser point clouds in the second point cloud set belonging to the ground point clouds. In other words, when determining the second point cloud set, the vehicle only needs to determine that the laser point cloud belongs to the ground point cloud, and does not need to ensure that all ground point clouds are included in the second point cloud set.

[0035] Step S130: Perform plane fitting on the second point cloud set, and determine the ground point cloud in the first point cloud set based on the fitted plane obtained by the plane fitting.

[0036] In this embodiment, after obtaining a second point cloud set from an environmental image, the vehicle can perform planar fitting on the second point cloud set. Since the vehicle filters the first point cloud set using the environmental image, all laser point clouds included in the second point cloud set are ground point clouds. Therefore, the fitted plane obtained by the vehicle based on these laser point clouds identified as ground point clouds is closer to the actual ground. Based on this, the vehicle can filter out all laser point clouds in the first point cloud set that belong to the ground point cloud based on this fitted plane with higher similarity to the actual ground. Specifically, after obtaining the fitted plane, the vehicle can determine whether a laser point cloud belongs to the ground point cloud based on the distance between each laser point cloud in the first point cloud set and the fitted plane.

[0037] In some implementations, considering that the actual ground may be a complex curved surface, the vehicle can perform regional planar fitting on the second point cloud set, and filter the laser point cloud in the first point cloud set within each region according to the different fitting planes obtained from the fitting of different regions, thereby obtaining the ground point cloud in different regions, and thus obtaining all the ground point clouds in the entire first point cloud set. The ground point cloud selected in this way will be more accurate and better adapted to complex actual road environments.

[0038] The ground point cloud segmentation method provided in this application involves obtaining a first set of point clouds located in the passable road area from the laser point cloud corresponding to the current environment; projecting the first set of point clouds onto an environmental image corresponding to the current environment; and filtering the first set of point clouds based on the segmented ground area in the environmental image to obtain a second set of point clouds; performing plane fitting on the second set of point clouds; and determining the ground point clouds in the first set of point clouds based on the fitted plane obtained by plane fitting. By filtering the first set of point clouds through the segmented ground area in the environmental image, all laser point clouds in the obtained second set of point clouds are laser point clouds located in the ground area. Therefore, plane fitting based on the second set of point clouds can obtain a more accurate fitted plane, thereby accurately determining the ground point clouds in the first set of point clouds and making the segmentation of ground point clouds more accurate.

[0039] Please see Figure 3 , Figure 3 This paper illustrates a flowchart of a ground point cloud segmentation method according to another embodiment of this application. The following will focus on... Figure 3 The process shown is described in detail. The ground point cloud segmentation method may specifically include the following steps:

[0040] Step S210: Based on the road boundary points in the high-precision map and the boundary type corresponding to each road boundary point, determine the passable area of ​​the road. The boundary type is used to characterize whether the road boundary point is a left boundary point or a right boundary point.

[0041] In this embodiment, the high-precision map pre-acquired by the vehicle only includes the coordinate data of road boundary points, and whether the boundary type of each road boundary point is a left or right road boundary point. The vehicle then needs to process these road boundary points, connecting them sequentially to form a passable road area. Specifically, the vehicle can first convert the UTM coordinates of the road boundary points in the high-precision map into coordinate data in the vehicle coordinate system and store them according to different boundary types, that is, storing all left road boundary points together and right road boundary points together. Then, these points can be projected onto a first grid map, and the left and right road boundary points can be sequentially connected by straight lines, thereby forming the outlines of the left and right boundaries in the grid map. Finally, based on whether each boundary line outline belongs to the left or right boundary, the passable road area between two road boundary lines can be determined.

[0042] Specifically, after obtaining the left and right boundary lines in the grid map, the vehicle can traverse the boundary points from farthest to nearest to determine the direction of expansion of the passable area corresponding to each boundary line. For example, for boundary point 1 on the left boundary of the road and boundary point 2 adjacent to it, the vehicle can determine the direction of expansion of the passable area corresponding to the line segment formed by boundary points 1 and 2 based on the coordinate relationship between these two boundary points in the grid map—that is, the angle between the lines connecting the two points and the boundary types to which the two points belong. The various correspondences are as follows: Figure 4 As shown, Figure 4 As shown in the first row and first column of the diagram, if both boundary points 1 and 2 are left boundary points, and the vehicle's direction of travel is from boundary point 1 to boundary point 2, then the corresponding traversable area expansion direction is perpendicular to the line segment between boundary points 1 and 2 and points to the right. Similarly, as... Figure 4 As shown in the figure in the third row and first column, both boundary points 1 and 2 are right boundary points. The vehicle travels from boundary point 1 to boundary point 2. Therefore, the direction of expansion of the corresponding passable area is perpendicular to the line segment and points to the left.

[0043] Subsequently, the vehicle can translate each line segment according to the direction of the traversable area expansion to obtain the traversable area corresponding to each line segment. The traversable area of ​​the road is then obtained by the intersection of these traversable areas. For example, translating two adjacent left boundary points according to their corresponding traversable area expansion directions yields the left boundary traversable area L_FS, and translating two adjacent right boundary points according to their corresponding traversable area expansion directions yields the right boundary traversable area R_FS. The final traversable road area is W_FS = L_FS ∩ R_FS. Figure 5 As shown, the shaded areas filled with diagonal lines indicate several different shapes of road passable areas obtained by vehicles in the grid map.

[0044] Step S220: Obtain the first set of point clouds in the passable area of ​​the road from the laser point cloud corresponding to the current environment.

[0045] In this embodiment, step S220 can be found in other embodiments and will not be repeated here.

[0046] Step S230: Obtain the image coordinates of each laser point cloud in the first point cloud set in the environmental image.

[0047] In this embodiment, after acquiring a first point cloud set, the vehicle can project each laser point cloud in the first point cloud set onto the environmental image corresponding to the current environment, so as to identify the laser point clouds in the first point cloud set that are located in the ground area, thus obtaining a second point cloud set. Specifically, the vehicle can convert the coordinate data of each laser point cloud in the first point cloud set in the three-dimensional coordinate system into image coordinate data in the image coordinate system corresponding to the environmental image, so as to determine whether the laser point cloud belongs to the ground point cloud based on the image coordinates corresponding to each laser point cloud. Before projecting the first point cloud set onto the environmental image, the vehicle can also identify the ground area of ​​the environmental image, which can be done using a pre-trained ground area recognition model. After obtaining the image coordinates corresponding to each laser point cloud in the environmental image, the vehicle can directly determine whether the laser point cloud corresponding to the image coordinates is a ground point cloud by judging whether the projection point of the image coordinates is located in the ground area based on the position of the image coordinates in the environmental image.

[0048] Specifically, the vehicle can determine the image coordinates of each laser point cloud in the first point cloud set in the environmental image using the following formula:

[0049]

[0050] Among them, (X) H Y H Z H Let (u, v) be the coordinates of the laser point cloud to be projected, (u, v) be the image coordinates of the laser point cloud projected onto the environmental image, and M be the projection matrix, which can be obtained by calibrating the camera that acquires the environmental image. c It is a constant and can be obtained during the solution of (u, v).

[0051] Step S240: Based on the image coordinates corresponding to each laser point cloud in the first point cloud set, delete the laser point cloud whose image coordinates are located outside the ground area from the first point cloud set to obtain the second point cloud set.

[0052] In this embodiment, after projecting each laser point cloud in the first point cloud set onto the environmental image to obtain the corresponding image coordinates, the vehicle can directly determine whether each laser point cloud is a ground point cloud by checking whether its corresponding image coordinates belong to the ground area in the environmental image. Specifically, if the vehicle determines that the image coordinates of a certain laser point cloud in the first point cloud set are outside the ground area of ​​the environmental image, it can directly delete that laser point cloud from the first point cloud set. Thus, after deleting all laser point clouds outside the ground area from the first point cloud set, the vehicle can obtain a second point cloud set that only includes laser point clouds from the ground area.

[0053] In some implementations, such as Figure 6 As shown, after obtaining the second point cloud set, the computer device can further filter out some noisy point clouds in the second point cloud set to more accurately include laser point clouds located in the ground area. Specifically, this includes the following steps:

[0054] Step S241: Based on the maximum and minimum laser point cloud height values ​​corresponding to the second point cloud set in each grid of the first grid map, determine the median maximum height value and the median minimum height value corresponding to the second target point cloud.

[0055] Step S242: If the height value corresponding to the second target point cloud is greater than the maximum median height value or less than the minimum median height value, then delete the second target point cloud from the second point cloud set.

[0056] In this embodiment, the median maximum height is the median value of the maximum laser point cloud height corresponding to each grid cell within the target area where the second target point cloud is located. The median minimum height is the median value of the minimum laser point cloud height corresponding to each grid cell within the target area. The target area is a region within a preset range centered on the location of the second target point cloud. The second target point cloud is any laser point cloud in the second point cloud set. During the process of determining the first point cloud set, the vehicle can project the laser point cloud corresponding to the current environment onto the first grid image to facilitate the determination of the first point cloud set located in the road-accessible area. Therefore, the second point cloud set obtained by the vehicle based on the first point cloud set is also distributed in the first grid image. Thus, the vehicle can obtain a portion of the laser point clouds from the second point cloud set included in each grid cell of the first grid image, and then determine the maximum and minimum laser point cloud height values ​​corresponding to each grid cell based on the coordinate data of each laser point cloud. Subsequently, the vehicle can traverse each laser point cloud in the second point cloud set based on the maximum and minimum laser point cloud height values ​​corresponding to each grid, that is, determine whether the height value corresponding to the laser point cloud is too high or too low. If it is too high or too low, the vehicle can directly delete the laser point cloud from the second point cloud set.

[0057] Specifically, the vehicle can use any laser point cloud in the second point cloud set as the second target point cloud, and with the second target point cloud as the origin, obtain the median value (ME) of the maximum laser point cloud height value corresponding to each of the multiple grids in the surrounding target area. max And the median value ME of the minimum laser point cloud height corresponding to each of the multiple grids in the target area. min Furthermore, based on the median value ME of the maximum laser point cloud height. maxand the median value ME of the minimum laser point cloud height. min This involves determining whether the height value ZH of the second target point cloud is too high or too low. In other words, if ME... min ≤ZH≤ME max If the target laser point cloud is not found in the second point cloud set, it can be retained; otherwise, it can be removed. This allows for further filtering of abnormal laser point clouds in the second point cloud set, ensuring that all laser point clouds included in the second point cloud set are from the ground region, thus improving the accuracy of subsequent plane fitting.

[0058] Step S250: Project the laser point cloud in the second point cloud set onto the first grid image to obtain laser point clouds in different grids.

[0059] Step S260: Based on the laser point cloud in each grid of the first grid image, merge the grids in the first grid image that meet the preset merging conditions to obtain the second grid image.

[0060] In this embodiment, after obtaining the second point cloud set, to make the subsequent planar fitting more accurate, the vehicle can perform regional planar fitting on each laser point cloud in the second point cloud set on the first grid image. This involves adjusting the resolution of the first grid image, merging certain grids that meet the conditions, and performing planar fitting according to the distribution of each merged grid. Understandably, the vehicle can determine the laser point clouds in the second point cloud set included in different grids of the first grid image. Therefore, the vehicle can determine whether each grid meets the preset merging conditions based on the laser point clouds in the second point cloud set included in each grid. If the preset merging conditions are met, the grids are merged, and the corresponding size of the grid is updated. After merging all grids that meet the conditions, the vehicle obtains the second grid image.

[0061] In some implementations, such as Figure 7 As shown, the first grid image can be a grid of fixed size, which can be 600×600, with the upper left corner as the origin. The pixel point (300, 300) in Figure N is the center point of the vehicle. The resolution can be 20cm, which means that the actual geometric distance represented by each pixel is 20cm.

[0062] In some implementations, such as Figure 8 As shown, a computer device can merge graticules in a first raster image that meet preset merging conditions through the following steps:

[0063] Step S261: Based on the laser point cloud in each grid of the first grid image, determine the first quantity, first height difference, and first slope corresponding to each grid of the first grid image.

[0064] Step S262: If the first quantity corresponding to the first target grid of the first grid image is less than or equal to a preset quantity, the first height difference is less than or equal to a preset height difference, and the first slope is less than or equal to a preset slope, then the first target grid is merged, and the first target grid is any grid in the first grid image.

[0065] In this embodiment of the application, the first quantity is the number of laser point clouds in the grid, the first height difference is the height difference between the first point cloud corresponding to the maximum laser point cloud height value and the second point cloud corresponding to the minimum laser point cloud height value in the grid, the first slope is the ratio between the first height difference and the first horizontal distance, and the first horizontal distance is the horizontal distance between the first point cloud and the second point cloud.

[0066] Understandably, the reason vehicles need to merge some grids in the first grid image is to ensure that each grid contains a sufficient number of laser point clouds for accurate planar fitting during subsequent planar fitting. Therefore, if some grids contain too few laser point clouds, they need to be merged. Simultaneously, when merging grids, factors such as the height difference and slope between the laser point clouds within a grid should also be considered. If the height difference or slope between the laser point clouds in a grid is large, merging this grid with other grids may reduce the accuracy of the fitted plane in those grids. Therefore, grids with excessively large height differences or slopes exceeding a certain value should not be merged. Thus, the vehicle can determine whether to merge grids in the first grid image based on the first number of laser point clouds corresponding to each grid, the first height difference, and the first slope.

[0067] Specifically, the vehicle can determine the first quantity NP, the first height difference DH, and the first slope K corresponding to each grid cell based on the laser point cloud in the second point cloud set included in each grid cell. Wherein, the first height difference DH = (Z... max -Z min ), where Z max Z represents the maximum laser point cloud height value among all laser point clouds included in the grid. min The minimum laser point cloud height value among all laser point clouds included in the grid; the first slope slope K = (Z max -Z min ) / D, where D is Zmax Corresponding laser point cloud and Z min The horizontal distance between the corresponding laser point clouds. If the first number NP corresponding to the first target grid is ≤ NP max First height difference DH≤DH max Or, the first slope K ≤ K max Then the first target grid will be merged.

[0068] In some implementations, NP max It can take the value 50, DH max The value can be 20cm, K max The value can be 0.3. In fact, different thresholds can be set for vehicles based on different application scenarios or accuracy requirements, and no restrictions are imposed here.

[0069] Step S270: Perform plane fitting on the laser point cloud in each grid of the second grid image to obtain the fitting plane corresponding to each grid of the second grid image.

[0070] Step S280: Based on the fitting plane corresponding to each grid cell of the second grid map, determine the ground point cloud in the first point cloud set.

[0071] In this embodiment, after obtaining the laser point cloud in the second point cloud set included in each grid of the second grid image through the above steps, the vehicle can perform plane fitting based on the laser point cloud included in each grid to obtain the fitting plane corresponding to each grid in the second grid image. Then, based on the fitting plane corresponding to each grid, it can determine which laser point cloud in each grid of the first point cloud set belongs to the ground point cloud. Specifically, the vehicle can determine the distance between the laser point cloud and the fitting plane corresponding to the grid based on the height value of the laser point cloud included in each grid of the second grid image of the first point cloud set, and then determine whether the laser point cloud belongs to the ground point cloud based on the distance.

[0072] In some implementations, such as Figure 9 As shown, a computer device can determine the ground point cloud in the first point cloud set based on the fitting plane corresponding to each grid cell of the second grid image through the following steps:

[0073] Step S281: Project the laser point cloud in the first point cloud set onto the second grid image to obtain the laser point cloud in different grids of the second grid image.

[0074] In this embodiment of the application, after the vehicle merges some grids in the first grid image to obtain the second grid image, it can project the laser point cloud in the first point cloud set onto the second grid image to determine the distribution of the laser point cloud in the first point cloud set in each grid of the second grid image. This allows the ground point cloud in the first point cloud set to be determined based on the fitting plane corresponding to the distribution of each grid in the second grid image and the laser point cloud in the first point cloud set included in each grid distribution.

[0075] Step S282: If the distance between the first target point cloud in the second target grid of the second grid image and the fitting plane corresponding to the second target grid is less than the first preset distance, then the first target point cloud is determined to be the ground point cloud, the second target grid is any grid in the second grid image, and the first target point cloud is any laser point cloud in the second target grid.

[0076] In this embodiment, after determining the laser point clouds in the first point cloud set included in each grid of the second grid image, the vehicle can determine the distance between the laser point cloud and the fitting plane corresponding to the grid by using the height value corresponding to each laser point cloud. Then, based on the relationship between this distance and a first preset distance, it can determine whether the laser point cloud belongs to the ground point cloud. It is understood that if the distance between the first target point cloud in the second target grid and the fitting plane corresponding to the second target grid is less than the first preset distance, the vehicle can determine the first target point cloud as a ground point cloud; conversely, if the distance between the first target point cloud and the fitting plane is too large, the first target point cloud is obviously not a ground point cloud. In some embodiments, the first preset distance can be 5 cm. Depending on different application scenarios and accuracy requirements, the vehicle can set different sizes of the first preset distance, which is not limited here.

[0077] The ground point cloud segmentation method provided in this application embodiment determines the passable area of ​​a road by using road boundary points in a high-precision map and the boundary type corresponding to each road boundary point. It then obtains a first set of point clouds located within the passable area of ​​the laser point cloud in the current environment. Next, it obtains the image coordinates of each laser point cloud in the first set in the environmental image. Based on the image coordinates of each laser point cloud in the first set, it deletes laser point clouds whose image coordinates are located outside the ground area from the first set, obtaining a second set of point clouds. Finally, based on the laser point clouds in each grid of the first grid image, it merges the grids in the first grid image that meet preset merging conditions, obtaining a second grid image. Finally, it performs plane fitting on the laser point clouds in each grid of the second grid image to obtain a fitting plane corresponding to each grid of the second grid image. Based on the fitting plane corresponding to each grid of the second grid image, it determines the ground point cloud in the first set of point clouds. The ground regions obtained by segmenting the environmental image are used to filter the first point cloud set, so that the laser point clouds in the second point cloud set are more certain to be located in the ground region. Furthermore, by merging the grids in the first grid image that meet the preset merging conditions, the laser point clouds in each grid are fitted to a plane, so that the fitted plane is more consistent with the actual complex road environment. Thus, the ground point clouds in the first point cloud set can be more accurately determined based on the fitted plane.

[0078] Please see Figure 10 This document illustrates a structural block diagram of a ground point cloud segmentation device 200 provided in an embodiment of this application. The ground point cloud segmentation device 200 includes: a point cloud acquisition module 210, a point cloud processing module 220, and a point cloud determination module 230. Specifically, the point cloud acquisition module 210 acquires a first set of point clouds located in the passable area of ​​a road within the laser point cloud corresponding to the current environment; the point cloud processing module 220 projects the first set of point clouds onto an environmental image corresponding to the current environment, and filters the first set of point clouds based on the segmented ground area in the environmental image to obtain a second set of point clouds, which includes laser point clouds located in the ground area; the point cloud determination module 230 performs plane fitting on the second set of point clouds and determines the ground point clouds in the first set of point clouds based on the fitted plane obtained from the plane fitting.

[0079] As one possible implementation, the point cloud processing module 220 is further configured to obtain the image coordinates of each laser point cloud in the first point cloud set in the environmental image; based on the image coordinates of each laser point cloud in the first point cloud set, the laser point cloud whose image coordinates are located outside the ground area is deleted from the first point cloud set to obtain the second point cloud set.

[0080] As one possible implementation, the point cloud determination module 230 includes a point cloud grid unit, a grid merging unit, a plane fitting unit, and a point cloud determination unit. Specifically, the point cloud grid unit projects the laser point cloud from the second point cloud set onto the first grid image to obtain laser point clouds in different grids; the grid merging unit merges grids in the first grid image that meet preset merging conditions based on the laser point cloud in each grid of the first grid image to obtain the second grid image; the plane fitting unit performs plane fitting on the laser point cloud in each grid of the second grid image to obtain a fitting plane corresponding to each grid of the second grid image; and the point cloud determination unit determines the ground point cloud in the first point cloud set based on the fitting plane corresponding to each grid of the second grid image.

[0081] As one possible implementation, the grid merging unit is further configured to determine, based on the laser point clouds in each grid of the first grid image, a first quantity, a first height difference, and a first slope corresponding to each grid of the first grid image. The first quantity is the number of laser point clouds in the grid, the first height difference is the height difference between the first point cloud corresponding to the maximum laser point cloud height value and the second point cloud corresponding to the minimum laser point cloud height value in the grid, the first slope is the ratio between the first height difference and the first horizontal distance, and the first horizontal distance is the horizontal distance between the first point cloud and the second point cloud. If the first quantity corresponding to the first target grid of the first grid image is less than or equal to a preset quantity, the first height difference is less than or equal to a preset height difference, and the first slope is less than or equal to a preset slope, then the first target grid is merged, and the first target grid is any grid in the first grid image.

[0082] As one possible implementation, the point cloud determination unit is further configured to project the laser point cloud in the first point cloud set onto the second grid image to obtain the laser point cloud in different grids of the second grid image; if the distance between the first target point cloud in the second target grid of the second grid image and the fitting plane corresponding to the second target grid is less than the first preset distance, then the first target point cloud is determined to be a ground point cloud, the second target grid is any grid in the second grid image, and the first target point cloud is any laser point cloud in the second target grid.

[0083] As one possible implementation, the ground point cloud segmentation device 200 also includes a region determination module, which is used to determine the passable area of ​​the road based on the road boundary points in the high-precision map and the boundary type corresponding to each road boundary point. The boundary type is used to characterize whether the road boundary point is a left boundary point or a right boundary point.

[0084] As one possible implementation, the ground point cloud segmentation device 200 further includes a threshold determination module and a point cloud deletion module. The threshold determination module is used to determine the median maximum height and median minimum height of the second target point cloud based on the maximum and minimum laser point cloud height values ​​corresponding to each grid cell in the first grid map of the second point cloud set. The median maximum height is the median of the maximum laser point cloud height values ​​corresponding to each grid cell in the target area where the second target point cloud is located, and the median minimum height is the median of the minimum laser point cloud height values ​​corresponding to each grid cell in the target area. The second target point cloud is any laser point cloud in the second point cloud set. The point cloud deletion module is used to delete the second target point cloud from the second point cloud set if the height value corresponding to the second target point cloud is greater than the median maximum height or less than the median minimum height.

[0085] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the above-described device and module can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0086] In the several embodiments provided in this application, the coupling between modules can be electrical, mechanical, or other forms of coupling.

[0087] Furthermore, the functional modules in the various embodiments of this application can be integrated into one processing module, or each module can exist physically separately, or two or more modules can be integrated into one module. The integrated modules described above can be implemented in hardware or as software functional modules.

[0088] In summary, the solution provided in this application obtains a first set of laser point clouds located within the passable road area from the laser point cloud corresponding to the current environment; projects the first set of point clouds onto an environmental image corresponding to the current environment; and filters the first set of point clouds based on the segmented ground region in the environmental image to obtain a second set of point clouds; performs plane fitting on the second set of point clouds, and determines the ground point clouds in the first set of point clouds based on the fitted plane obtained from the plane fitting. By filtering the first set of point clouds through the segmented ground region in the environmental image, the laser point clouds in the obtained second set are all laser point clouds located within the ground region. Therefore, plane fitting based on the second set of point clouds can obtain a more accurate fitted plane, thereby accurately determining the ground point clouds in the first set of point clouds and making the segmentation of the ground point clouds more accurate.

[0089] Please refer to Figure 11The diagram illustrates a structural block diagram of a vehicle 300 provided in an embodiment of this application. The vehicle 300 in this application may include one or more of the following components: a processor 310, a memory 320, and one or more application programs. The one or more application programs may be stored in the memory 320 and configured to be executed by one or more processors 310. The one or more programs are configured to perform the methods described in the foregoing method embodiments.

[0090] Processor 310 may include one or more processing cores. Processor 310 connects to various parts of the computer device using various interfaces and lines, and performs various functions and processes data by running or executing instructions, programs, code sets, or instruction sets stored in memory 320, and by calling data stored in memory 320. Optionally, processor 310 may be implemented using at least one hardware form of Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), or Programmable Logic Array (PLA). Processor 310 may integrate one or a combination of several of the following: Central Processing Unit (CPU), Graphics Processing Unit (GPU), and modem. The CPU primarily handles the operating system, user interface, and applications; the GPU is responsible for rendering and drawing the displayed content; and the modem handles wireless communication. It is understood that the modem may also not be integrated into processor 310 and may be implemented separately using a communication chip.

[0091] The memory 320 may include random access memory (RAM) or read-only memory (ROM). The memory 320 can be used to store instructions, programs, code, code sets, or instruction sets. The memory 320 may include a program storage area and a data storage area. The program storage area may store instructions for implementing an operating system, instructions for implementing at least one function (such as touch functionality, sound playback functionality, image playback functionality, etc.), and instructions for implementing the various method embodiments described below. The data storage area may also store data created during the use of the computer device (such as phonebook data, audio and video data, chat log data, etc.).

[0092] Please refer to Figure 12This diagram illustrates a structural block diagram of a computer-readable storage medium provided in an embodiment of this application. The computer-readable medium 800 stores program code that can be called by a processor to execute the methods described in the above method embodiments.

[0093] The computer-readable storage medium 800 may be an electronic memory such as flash memory, EEPROM (Electrically Erasable Programmable Read-Only Memory), EPROM, hard disk, or ROM. Optionally, the computer-readable storage medium 800 includes a non-transitory computer-readable storage medium. The computer-readable storage medium 800 has storage space for program code 810 that performs any of the method steps described above. This program code can be read from or written to one or more computer program products. The program code 810 may be compressed, for example, in a suitable form.

[0094] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A method of segmenting a ground point cloud, characterized in that, The method includes: Obtain the first set of point clouds in the road-accessible area from the laser point cloud corresponding to the current environment; The first point cloud set is projected onto the environmental image corresponding to the current environment, and the first point cloud set is filtered based on the ground area segmented in the environmental image to obtain a second point cloud set, the second point cloud set including laser point clouds located in the ground area; Projecting the laser point cloud in the second point cloud set onto the first grid image yields laser point clouds in different grids; Based on the laser point cloud in each grid of the first grid image, the grids in the first grid image that meet the preset merging conditions are merged to obtain the second grid image. Plane fitting is performed on the laser point cloud in each grid of the second grid image to obtain the fitting plane corresponding to each grid of the second grid image; Based on the fitting plane corresponding to each grid cell of the second grid map, the ground point cloud in the first point cloud set is determined.

2. The method according to claim 1, characterized in that, The step of projecting the first point cloud set onto the environmental image corresponding to the current environment, and filtering the first point cloud set based on the ground region segmented from the environmental image to obtain a second point cloud set, includes: Obtain the image coordinates of each laser point cloud in the first point cloud set in the environmental image; Based on the image coordinates corresponding to each laser point cloud in the first point cloud set, the laser point cloud whose image coordinates are located outside the ground area is deleted from the first point cloud set to obtain the second point cloud set.

3. The method according to claim 1, characterized in that, The process of merging the laser point clouds in each grid of the first grid image, and merging the grids in the first grid image that meet preset merging conditions to obtain the second grid image includes: Based on the laser point cloud in each grid of the first grid image, a first quantity, a first height difference, and a first slope are determined for each grid of the first grid image. The first quantity is the number of laser point clouds in the grid. The first height difference is the height difference between the first point cloud corresponding to the maximum laser point cloud height value and the second point cloud corresponding to the minimum laser point cloud height value in the grid. The first slope is the ratio between the first height difference and the first horizontal distance. The first horizontal distance is the horizontal distance between the first point cloud and the second point cloud. If the first number corresponding to the first target grid cell in the first grid image is less than or equal to a preset number, the first height difference is less than or equal to a preset height difference, and the first slope is less than or equal to a preset slope, then the first target grid cells are merged, and the first target grid cell is any grid cell in the first grid image.

4. The method according to claim 1, characterized in that, The step of determining the ground point cloud in the first point cloud set based on the fitting plane corresponding to each grid cell of the second grid image includes: Projecting the laser point cloud in the first point cloud set onto the second grid image yields laser point clouds in different grids of the second grid image. If the distance between the first target point cloud in the second target grid of the second grid image and the fitting plane corresponding to the second target grid is less than the first preset distance, then the first target point cloud is determined to be the ground point cloud, the second target grid is any grid in the second grid image, and the first target point cloud is any laser point cloud in the second target grid.

5. The method according to claim 1, characterized in that, Before obtaining the first set of point clouds located in the road-accessible area from the laser point cloud corresponding to the current environment, the method further includes: Based on the road boundary points in the high-precision map and the boundary type corresponding to each road boundary point, the passable area of ​​the road is determined, and the boundary type is used to characterize whether the road boundary point is a left boundary point or a right boundary point.

6. The method according to any one of claims 1-5, characterized in that, After obtaining the second point cloud set, the method further includes: Based on the maximum and minimum laser point cloud height values ​​corresponding to each grid cell in the first grid map of the second point cloud set, the median maximum height value and the median minimum height value corresponding to the second target point cloud are determined. The median maximum height value is the median value of the maximum laser point cloud height value corresponding to each grid cell in the target area where the second target point cloud is located. The median minimum height value is the median value of the minimum laser point cloud height value corresponding to each grid cell in the target area. The target area is a region within a preset range centered on the location of the second target point cloud. The second target point cloud is any laser point cloud in the second point cloud set. If the height value corresponding to the second target point cloud is greater than the maximum median height value or less than the minimum median height value, then the second target point cloud is deleted from the second point cloud set.

7. A ground point cloud segmentation device, characterized in that, The device includes a point cloud acquisition module, a point cloud processing module, and a point cloud determination module, wherein, The point cloud acquisition module is used to acquire the first set of point clouds in the passable area of ​​the road from the laser point cloud corresponding to the current environment. The point cloud processing module is used to project the first point cloud set onto the environmental image corresponding to the current environment, and filter the first point cloud set based on the ground area segmented in the environmental image to obtain a second point cloud set, wherein the second point cloud set includes laser point clouds located in the ground area; The point cloud determination module is used to project the laser point cloud in the second point cloud set onto the first grid image to obtain laser point clouds in different grids; based on the laser point cloud in each grid of the first grid image, merge the grids in the first grid image that meet the preset merging conditions to obtain the second grid image; perform plane fitting on the laser point cloud in each grid of the second grid image to obtain the fitting plane corresponding to each grid of the second grid image; and determine the ground point cloud in the first point cloud set based on the fitting plane corresponding to each grid of the second grid image.

8. A vehicle, characterized in that, The vehicles include: One or more processors; Memory; One or more applications, wherein the one or more applications are stored in the memory and configured to be executed by the one or more processors, the one or more applications being configured to perform the method as described in any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores program code that can be invoked by a processor to execute the method as described in any one of claims 1-6.

Images