Ground point cloud segmentation method, system, ground modeling method and medium

Abstract

The present application discloses a ground point cloud segmentation method, system, ground modeling method and medium, belonging to the field of autonomous driving technology, the method comprising: pre-processing ground point cloud data and dividing it into ground two-dimensional grids; determining grid features corresponding to the ground two-dimensional grids through a pre-trained ground network model; processing the grid features through the ground network model to obtain a grid height prediction value; comparing the grid height prediction value with the corresponding grid height threshold, if the grid height prediction value is greater than the grid height threshold, the corresponding ground point cloud data is determined as an obstacle point; if the grid height prediction value is not greater than the grid height threshold, the corresponding ground point cloud data is determined as a ground point. The present application calculates the height of each point cloud grid through a pre-trained ground network model, and compares it with the corresponding threshold of each grid to determine whether it is a ground point, thereby improving the accuracy of point cloud segmentation.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of autonomous driving technology, and in particular to a ground point cloud segmentation method, system, ground modeling method and medium. Background Art

[0002] LiDAR can obtain environmental information in real time, accurately and reliably, and is widely used in the environmental perception system of driverless cars. In road traffic scenes, obstacles such as vehicles, pedestrians, and buildings are distributed on the ground and connected to the ground. To perceive these obstacles, they must first be distinguished from ground points. Therefore, point cloud ground segmentation is a basic and important link in the laser perception system.

[0003] There are currently some methods for segmenting point cloud ground, such as clustering algorithms, grid methods, and model fitting methods. However, these methods have problems such as reliance on fixed thresholds, simple road models, and reliance on sensor data characteristics, making it difficult to cope with complex scenes in actual road environments. The current point cloud segmentation method relies on fixed thresholds for judgment, which makes it impossible to accurately divide ground points and obstacle points in the point cloud; or it uses special laser radars to obtain the inherent properties of point clouds for point cloud segmentation. This method cannot be applied to other models of laser radars and is not universal. There are also some methods that directly use deep learning to segment point clouds, but problems such as over-segmentation and under-segmentation are inevitable. Summary of the invention

[0004] The present application proposes a ground point cloud segmentation method, a ground modeling method, a system, a medium and a device, which are used to solve the problem that when using a laser radar to obtain a ground point cloud for ground modeling and to segment the ground point cloud and non-ground ground, the ground point cloud segmentation is inaccurate when segmentation is performed using a fixed threshold, and there is an over-segmentation or under-segmentation problem.

[0005] In a first aspect, the present application provides a ground point cloud segmentation method, comprising: acquiring ground point cloud data; preprocessing the ground point cloud data and dividing it into a pre-constructed ground two-dimensional grid; determining the grid features corresponding to the ground two-dimensional grid according to the height information of the ground point cloud data through a pre-trained ground network model; processing the grid features through the ground network model to obtain a grid height prediction value corresponding to the ground two-dimensional grid; comparing the grid height prediction value with a grid height threshold corresponding to the ground two-dimensional grid, wherein under the condition that the grid height prediction value is greater than the grid height threshold, the ground point cloud data corresponding to the ground two-dimensional grid is determined as an obstacle point; under the condition that the grid height prediction value is not greater than the grid height threshold, the ground point cloud data corresponding to the ground two-dimensional grid is determined as a ground point.

[0006] Optionally, the ground point cloud data is preprocessed and divided into a pre-constructed ground two-dimensional grid, including: converting the ground point cloud data in the lidar coordinate system into the vehicle coordinate system, and determining the three-dimensional coordinates corresponding to the ground point cloud data in the vehicle coordinate system; according to the three-dimensional coordinates, dividing the ground point cloud data into corresponding ground grids.

[0007] Optionally, the ground point cloud data is divided into corresponding ground grids according to the three-dimensional coordinates, including: determining the area range corresponding to the ground two-dimensional grid according to the coordinates of each vertex in the ground two-dimensional grid; and dividing the ground point cloud data within the area range into ground point cloud data corresponding to the ground two-dimensional grid according to the position of the three-dimensional coordinates in the vehicle coordinate system.

[0008] Optionally, the pre-training process of the ground network model includes: acquiring ground point cloud training data through a lidar; projecting the ground point cloud training data into a ground two-dimensional grid, and predicting the grid height of the ground two-dimensional grid according to the ground point cloud training data in the ground two-dimensional grid through a prediction network model to obtain a preliminary prediction value of the grid height; the prediction network model performs loss learning according to the true value of the grid height, the height loss between adjacent grids, and the preliminary prediction value of the grid height to obtain a ground network model.

[0009] Optionally, the pre-training process of the face network model also includes: manually labeling the ground point cloud training data to obtain real ground points; projecting the real ground points into a ground two-dimensional grid, and taking the average height of the real ground points in the ground two-dimensional grid as the true value of the grid height of the ground two-dimensional grid.

[0010] Optionally, the prediction network model performs loss learning based on the true value of the grid height, the height loss between adjacent grids, and the preliminary predicted value of the grid height to obtain a ground network model, including: the prediction network model adjusts the preliminary predicted value of the grid height of the current grid based on the height loss between adjacent grids to obtain a grid height prediction value; based on the difference between the true value of the grid height corresponding to the current grid and the predicted value of the grid height, the prediction network model performs loss learning to obtain a ground network model.

[0011] Optionally, the grid height is compared with a grid height threshold corresponding to the ground two-dimensional grid, including: determining the corresponding grid height threshold based on a preset grid height value and a height change threshold between adjacent grids, wherein the greater the ground slope between adjacent grids, the greater the height change threshold.

[0012] In a second aspect, the present application provides a ground point cloud segmentation system, comprising: a preprocessing module, which preprocesses the acquired ground point cloud data and divides it into a pre-constructed ground two-dimensional grid; a ground point segmentation module, which determines the grid features corresponding to the ground two-dimensional grid according to the height information of the ground point cloud data through a pre-trained ground network model; processes the grid features through the ground network model to obtain a grid height prediction value corresponding to the ground two-dimensional grid; compares the grid height prediction value with a grid height threshold corresponding to the ground two-dimensional grid, wherein under the condition that the grid height prediction value is greater than the grid height threshold, the ground point cloud data corresponding to the ground two-dimensional grid is determined as an obstacle point; under the condition that the grid height prediction value is not greater than the grid height threshold, the ground point cloud data corresponding to the ground two-dimensional grid is determined as a ground point.

[0013] In a third aspect, the present application provides a ground modeling method, including: obtaining ground point cloud data; preprocessing the ground point cloud data and dividing it into a pre-constructed ground two-dimensional grid; determining the grid features corresponding to the ground two-dimensional grid according to the height information of the ground point cloud data through a pre-trained ground network model; processing the grid features through the ground network model to obtain a grid height prediction value corresponding to the ground two-dimensional grid; comparing the grid height prediction value with a grid height threshold corresponding to the ground two-dimensional grid, wherein under the condition that the grid height prediction value is greater than the grid height threshold, the ground point cloud data corresponding to the ground two-dimensional grid is determined as an obstacle point; under the condition that the grid height prediction value is not greater than the grid height threshold, the ground point cloud data corresponding to the ground two-dimensional grid is determined as a ground point; and ground modeling is performed based on the ground points to obtain a road ground.

[0014] In a fourth aspect, the present application provides a computer-readable storage medium storing computer instructions, wherein the computer instructions are operated to execute the ground point cloud segmentation method in scheme one or the ground modeling method in scheme three.

[0015] The beneficial effect of the present application is as follows: when segmenting the acquired ground point cloud data into ground points and non-ground points, the present application segments the ground point cloud data through a two-dimensional grid, and calculates the height of each point cloud grid through a pre-trained ground network model, and compares it with the corresponding threshold of each grid to determine whether it is a ground point. By using a dynamic threshold judgment method, the accuracy of point cloud segmentation is improved when segmenting ground points, over-segmentation and under-segmentation are avoided, and the accuracy of point cloud segmentation is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 It is a schematic diagram of an implementation method of the ground point cloud segmentation method of the present application;

[0017] Figure 2is a schematic diagram of an implementation of the ground point cloud segmentation system of the present application;

[0018] Figure 3 is a schematic diagram of an implementation of the ground modeling method of the present application;

[0019] Figure 4 is a schematic diagram of an embodiment of a ground modeling device of the present application;

[0020] Figure 5 It is a schematic diagram of an implementation of a processing chip of the present application. DETAILED DESCRIPTION

[0021] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0022] The terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a product or device that includes a series of steps or units is not necessarily limited to those units that are clearly listed, but may include other units that are not clearly listed or inherent to these products or devices.

[0023] LiDAR can obtain environmental information in real time, accurately and reliably, and is widely used in the environmental perception system of driverless cars. In road traffic scenes, obstacles such as vehicles, pedestrians, and buildings are distributed on the ground and connected to the ground. To perceive these obstacles, they must first be distinguished from ground points. Therefore, point cloud ground segmentation is a basic and important link in the laser perception system.

[0024] Laser radar can obtain environmental information in real time, accurately and reliably, and is widely used in the environmental perception system of unmanned vehicles. In road traffic scenes, obstacles such as vehicles, pedestrians, and buildings are distributed on the ground and connected to the ground. To perceive these obstacles, they must first be distinguished from ground points. Therefore, point cloud ground segmentation is a basic and important link in the laser perception system. At present, there are some methods for segmenting point cloud ground, such as clustering algorithm, grid method, and model fitting method, but these methods have problems such as relying on fixed thresholds, simple road models, and relying on sensor data characteristics, and it is difficult to cope with complex scenes in actual road environments. The current point cloud segmentation method relies on fixed thresholds for judgment, resulting in the inability to accurately divide ground points and obstacle points in the point cloud; or using special laser radars to obtain the inherent properties of point clouds for point cloud segmentation. This method cannot be applied to other models of laser radars and is not universal. There are also some methods that directly use deep learning to segment point clouds, but there are inevitably problems such as over-segmentation and under-segmentation.

[0025] Therefore, in order to improve the accuracy of ground point cloud segmentation and avoid the problems of over-segmentation and under-segmentation, in the face of the actual scenario where the road surface is subject to slope bullying, different thresholds need to be adopted for different road sections to judge ground points and non-ground points, rather than only using fixed parameter thresholds for segmentation.

[0026] In response to the above problems, the present application proposes a ground point cloud segmentation method, system, ground modeling method and medium. The ground point cloud segmentation method includes: acquiring ground point cloud data; preprocessing the ground point cloud data and dividing it into a pre-constructed ground two-dimensional grid; determining the grid features corresponding to the ground two-dimensional grid according to the height information of the ground point cloud data through a pre-trained ground network model; processing the grid features through the ground network model to obtain a grid height prediction value corresponding to the ground two-dimensional grid; comparing the grid height prediction value with the grid height threshold corresponding to the ground two-dimensional grid, wherein under the condition that the grid height prediction value is greater than the grid height threshold, the ground point cloud data corresponding to the ground two-dimensional grid is determined as an obstacle point; under the condition that the grid height prediction value is not greater than the grid height threshold, the ground point cloud data corresponding to the ground two-dimensional grid is determined as a ground point.

[0027] When segmenting the acquired ground point cloud data into ground points and non-ground points, the present application segments the ground point cloud data through a two-dimensional grid, and calculates the height of each point cloud grid through a pre-trained ground network model, and compares it with the corresponding threshold of each grid to determine whether it is a ground point. In the judgment process, the slope change between adjacent grids is taken into account, and the slope change is included in the consideration of whether the grid is the ground or an obstacle, so that one two-dimensional grid corresponds to one grid height threshold, the accuracy of the judgment is achieved, the actual situation of the actual road surface is fully considered, and the accuracy of the ground point cloud data segmentation between ground points and obstacle points is improved.

[0028] The technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems are described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0029] Figure 1 An implementation of the ground point cloud segmentation method of the present application is shown.

[0030] exist Figure 1 In the illustrated embodiment, the ground point cloud segmentation method of the present application includes process S101 of acquiring ground point cloud data.

[0031] In this implementation, the ground point cloud is first collected by a laser radar device installed on a collection vehicle or other vehicle to obtain the original ground power data. The ground point cloud data includes ground points, obstacle points, and some noise points. In addition, the ground point cloud segmentation method of the present application does not rely on the characteristics of the laser radar data for subsequent analysis, or the ground point cloud segmentation method of the present application can process point cloud data obtained by various forms of laser radars such as mechanical radar, semi-solid radar, solid-state radar, etc., and is not limited to the model of the laser radar.

[0032] exist Figure 1 In the illustrated embodiment, the ground point cloud segmentation method of the present application includes a process S102 of preprocessing the ground point cloud data and dividing it into a pre-constructed ground two-dimensional grid.

[0033] In this embodiment, the ground point cloud data collected by the laser radar are respectively in the coordinate system of the corresponding laser radar. In particular, in the case of multiple laser radars, the coordinate systems of the point cloud data are different. Therefore, when preprocessing the ground point cloud data, on the one hand, noise is filtered out, and on the other hand, the ground point cloud data is coordinate-converted and converted to the vehicle coordinate system to facilitate subsequent unified processing. For example, after the coordinate conversion of the ground point cloud data through the external parameter conversion process of the laser radar equipment, the ground point cloud data in the vehicle coordinate system is projected and divided into the pre-constructed ground two-dimensional grid.

[0034] Optionally, the ground point cloud data is preprocessed and divided into a pre-constructed ground two-dimensional grid, including: converting the ground point cloud data in the lidar coordinate system into the vehicle coordinate system, and determining the three-dimensional coordinates corresponding to the ground point cloud data in the vehicle coordinate system; and dividing the ground point cloud data into corresponding ground grids according to the three-dimensional coordinates.

[0035] In this optional embodiment, the ground point cloud data in the laser radar coordinate system is converted to the vehicle coordinate system to obtain the three-dimensional coordinates corresponding to each point cloud point. The corresponding ground point cloud can be divided into corresponding ground grids according to the three-dimensional coordinates and the divided ground grids. By converting the point cloud data obtained by multiple radars to the same coordinate system, the overall processing of the point cloud data is facilitated. In the vehicle coordinate system, the three-dimensional coordinates of each point cloud point in the point cloud data are determined, which facilitates the use of coordinates to divide the point cloud and improves the efficiency of point cloud data processing.

[0036] Optionally, the ground point cloud data is divided into corresponding ground grids according to the three-dimensional coordinates, including: determining the area range corresponding to the ground two-dimensional grid according to the coordinates of each vertex in the ground two-dimensional grid; and dividing the ground point cloud data within the area range into ground point cloud data corresponding to the ground two-dimensional grid according to the position of the three-dimensional coordinates in the vehicle coordinate system.

[0037] In this optional embodiment, the shape of the ground two-dimensional grid can be reasonably divided according to the actual road conditions, and a rectangle can generally be selected. When determining which ground point clouds belong to which ground two-dimensional grid, the coordinates of each vertex of the ground two-dimensional grid can be used to determine the area range corresponding to the ground two-dimensional grid, and then the coordinates of the ground point cloud data and the area range of the ground two-dimensional grid are compared to determine the ground point cloud data in the ground two-dimensional grid. When the point cloud data is divided using the divided two-dimensional grid, it can be divided according to the relationship between the three-dimensional coordinates of the point cloud and the two-dimensional grid range to improve the efficiency of point cloud segmentation.

[0038] Specifically, taking the ground two-dimensional grid as a rectangle as an example, the four vertices of the ground two-dimensional grid correspond to corresponding two-dimensional coordinates respectively, and the grid area range can be expressed as range = [x_min, x_max, y_min, y_max]. By comparing the relationship between the x-coordinate and y-coordinate of the ground point cloud data in the vehicle coordinate system and the grid area range, the ground point cloud data within the range is determined as the ground point cloud data corresponding to the ground two-dimensional grid.

[0039] exist Figure 1 In the illustrated embodiment, the ground point cloud segmentation method of the present application includes process S103, which determines the grid features corresponding to the ground two-dimensional grid according to the height information of the ground point cloud data through a pre-trained ground network model.

[0040] In this embodiment, the point cloud is projected into a two-dimensional grid, and a computing network such as PillarNet or PointNet is applied to the points in the grid to calculate the grid features containing the height information of the point cloud in the grid.

[0041] exist Figure 1 In the illustrated embodiment, the ground point cloud segmentation method of the present application includes a process S104 of processing grid features through a ground network model to obtain a grid height prediction value corresponding to the ground two-dimensional grid.

[0042] In this implementation, the grid features are processed by a ground network model, and grid height prediction values ​​corresponding to each ground two-dimensional grid are obtained based on the grid features corresponding to each ground two-dimensional grid.

[0043] Specifically, the backbone network in the ground network model, such as commonly used networks such as ResNet and UNet, can be used to process the grid features. Since the grid features contain the height information of the corresponding two-dimensional grid on the ground, the grid height prediction value corresponding to the two-dimensional grid on the ground can be obtained.

[0044] Optionally, the pre-training process of the ground network model includes: acquiring ground point cloud training data through a lidar; projecting the ground point cloud training data into a ground two-dimensional grid, and predicting the grid height of the ground two-dimensional grid according to the ground point cloud training data in the ground two-dimensional grid through a prediction network model to obtain a preliminary prediction value of the grid height; the prediction network model performs loss learning according to the true value of the grid height, the height loss between adjacent grids, and the preliminary prediction value of the grid height to obtain a ground network model.

[0045] In this optional embodiment, the training of the ground network model can be roughly divided into two steps. First, the prediction network model predicts the grid height of the ground two-dimensional grid based on the ground point cloud training data projected onto the ground two-dimensional grid to obtain a preliminary prediction value of the grid height. Then, using the true value of the grid height and the height loss of the height continuity between adjacent grids, the prediction network model is subjected to loss learning to obtain the ground network model. When the obtained ground network model processes the ground point cloud data, the difference between the predicted value and the true value of the ground two-dimensional grid coverage is within the preset range, thereby ensuring the effectiveness of the model training.

[0046] Optionally, the pre-training process of the ground network model also includes: manually labeling the ground point cloud training data to obtain real ground points; projecting the real ground points into a ground two-dimensional grid, and taking the average height of the real ground points in the ground two-dimensional grid as the true value of the grid height of the ground two-dimensional grid.

[0047] In this optional embodiment, when training the ground network model, in order to ensure the simulation accuracy of the trained model, the authenticity of the true value needs to be ensured when the loss learning is performed on the true value. Therefore, in the training process of the ground network model, the ground point cloud training data is obtained by laser radar, and the ground points in the ground point cloud training data are manually labeled to obtain the real ground points. Through manual labeling, it can be ensured that the determined real ground points are real enough, thereby ensuring the accuracy of model training. By projecting the real ground points into the divided two-dimensional ground grid, the height of the corresponding ground grid is determined according to the height of the real ground points. That is, the average height of all the real ground points in a grid is calculated as the grid height true value of the grid. Then the prediction network model is based on the obtained grid true height of each grid, the coverage loss between adjacent grids and a large amount of ground point cloud data for loss learning, and finally the ground network model is obtained, so that the trained ground network model can determine the height of the ground grid according to the input ground point cloud data, and then make a subsequent judgment on whether it is a ground point. By determining the true value of the grid height and ensuring its accuracy, you can ensure the accuracy of the final model training results and improve the effectiveness of your model training.

[0048] Optionally, the prediction network model performs loss learning based on the true value of the grid height, the height loss between adjacent grids, and the preliminary predicted value of the grid height to obtain a ground network model, including: the prediction network model adjusts the preliminary predicted value of the grid height of the current grid based on the height loss between adjacent grids to obtain a grid height prediction value; based on the difference between the true value of the grid height corresponding to the current grid and the predicted value of the grid height, the prediction network model performs loss learning to obtain a ground network model.

[0049] In this optional embodiment, the prediction network model performs loss learning according to the true value of the grid height, the height loss between adjacent grids, and the preliminary predicted value of the grid height. First, the prediction network model adjusts the preliminary predicted value of the grid height of the current grid according to the height loss between adjacent grids to obtain the predicted value of the grid height; then, based on the difference between the true value of the height of the grid and the predicted value of the grid height, loss learning is performed to finally train the ground network model. In the actual operation process, the height loss between adjacent grids and the loss with the true value of the grid height can be simultaneously learned. Using the model training method of loss learning, it is ensured that there is a small difference between the predicted result obtained by the model and the true value, and the accuracy and rationality of the model processing results are guaranteed. .

[0050] Specifically, the training process during model training can be roughly described by the following formula:

[0051]

[0052]

[0053] Where G is the true value of the ground height, is the ground height predicted by the network, is the total loss of the network, is the regression loss, α and β are the weight coefficients of the two losses, which can generally be taken as 0.9 and 0.1;

[0054] is the regression loss, which is the absolute value of the interpolation between the ground height predicted by the network and the ground truth at the corresponding grid position;

[0055] To predict the ground height smoothness loss, are the gradients of the ground height map in the x and y directions respectively.

[0056] When training the ground network model, the present application fully considers the height loss between adjacent grids and predicts the height of the ground grid, thereby achieving accurate segmentation of ground points in the ground point cloud data.

[0057] exist Figure 1 In the illustrated embodiment, the ground point cloud segmentation method of the present application includes process S105, comparing the grid height with the grid height threshold corresponding to the ground two-dimensional grid, wherein under the condition that the grid height prediction value is greater than the grid height threshold, the ground point cloud data corresponding to the ground two-dimensional grid is determined as an obstacle point; under the condition that the grid height prediction value is not greater than the grid height threshold, the ground point cloud data corresponding to the ground two-dimensional grid is determined as a ground point.

[0058] In this embodiment, after predicting the grid height prediction value of each ground two-dimensional grid through the ground network model, the grid height prediction value is compared with the corresponding grid height threshold. When the grid height prediction value is greater than the grid height threshold, the ground two-dimensional grid corresponds to an obstacle, and the point cloud points in the grid are obstacle points; when the grid height prediction value is not greater than the grid height threshold, the ground two-dimensional grid corresponds to the ground, and the point cloud points in the grid are ground points.

[0059] Optionally, the grid height prediction value is compared with the grid height threshold corresponding to the ground two-dimensional grid, including: determining the corresponding grid height threshold based on a preset grid height value and a height change threshold between adjacent grids, wherein the greater the ground slope between adjacent grids, the greater the height change threshold.

[0060] In this optional embodiment, since the road is not always flat and has ups and downs in slope, when judging the grid height of the ground two-dimensional grid, if the judgment is made through a fixed height threshold, wrong judgments will occur for some ground with ups and downs in slope, resulting in wrong judgments of ground points, which in turn affects the subsequent division of ground points and obstacle points, and causes problems of over-segmentation or under-segmentation of ground points. Therefore, when determining the height threshold, the present application takes into account the reasonable relationship between the height changes of adjacent ground two-dimensional grids, and by setting the height change threshold between adjacent grids, it can also accurately judge when the ground has a slope change. Among them, the height change threshold will be carefully and reasonably set in the trained ground network model, wherein the greater the ground slope between adjacent grids, the greater the height change threshold.

[0061] When segmenting the acquired ground point cloud data into ground points and non-ground points, the present application segments the ground point cloud data through a two-dimensional grid, and calculates the height of each point cloud grid through a pre-trained ground network model, and compares it with the corresponding threshold of each grid to determine whether it is a ground point. In the judgment process, the slope change between adjacent grids is taken into account, and the slope change is included in the consideration of whether the grid is the ground or an obstacle, so that one two-dimensional grid corresponds to one grid height threshold, the accuracy of the judgment is achieved, the actual situation of the road surface is fully considered, and the accuracy of the ground point cloud data segmentation between ground points and obstacle points is improved.

[0062] Figure 2 An embodiment of the ground point cloud segmentation system of the present application is shown.

[0063] exist Figure 2In the illustrated embodiment, the ground point cloud segmentation system of the present application includes: a preprocessing module 201, which preprocesses the acquired ground point cloud data and divides it into a pre-constructed ground two-dimensional grid; a ground point segmentation module 202, which determines the grid features corresponding to the ground two-dimensional grid according to the height information of the ground point cloud data through a pre-trained ground network model; processes the grid features through the ground network model to obtain a grid height prediction value corresponding to the ground two-dimensional grid; compares the grid height prediction value with a grid height threshold corresponding to the ground two-dimensional grid, wherein under the condition that the grid height prediction value is greater than the grid height threshold, the ground point cloud data corresponding to the ground two-dimensional grid is determined as an obstacle point; under the condition that the grid height prediction value is not greater than the grid height threshold, the ground point cloud data corresponding to the ground two-dimensional grid is determined as a ground point.

[0064] Optionally, in the preprocessing module 201, the ground point cloud data in the lidar coordinate system is converted to the vehicle coordinate system, and the three-dimensional coordinates corresponding to the ground point cloud data are determined in the vehicle coordinate system; according to the three-dimensional coordinates, the ground point cloud data is divided into corresponding ground grids.

[0065] Optionally, in the preprocessing module 201, the area range corresponding to the ground two-dimensional grid is determined according to the coordinates of each vertex in the ground two-dimensional grid; and the ground point cloud data within the area range is divided into ground point cloud data corresponding to the ground two-dimensional grid according to the position of the three-dimensional coordinates in the vehicle coordinate system.

[0066] Optionally, the pre-training process of the ground network model includes: acquiring ground point cloud training data through a lidar; projecting the ground point cloud training data into a ground two-dimensional grid, and predicting the grid height of the ground two-dimensional grid according to the ground point cloud training data in the ground two-dimensional grid through a prediction network model to obtain a preliminary prediction value of the grid height; the prediction network model performs loss learning according to the true value of the grid height, the height loss between adjacent grids, and the preliminary prediction value of the grid height to obtain a ground network model.

[0067] Optionally, the pre-training process of the face network model also includes: manually labeling the ground point cloud training data to obtain real ground points; projecting the real ground points into a ground two-dimensional grid, and taking the average height of the real ground points in the ground two-dimensional grid as the true value of the grid height of the ground two-dimensional grid.

[0068] Optionally, the prediction network model performs loss learning based on the true value of the grid height, the height loss between adjacent grids, and the preliminary predicted value of the grid height to obtain a ground network model, including: the prediction network model adjusts the preliminary predicted value of the grid height of the current grid based on the height loss between adjacent grids to obtain a grid height prediction value; based on the difference between the true value of the grid height corresponding to the current grid and the predicted value of the grid height, the prediction network model performs loss learning to obtain a ground network model.

[0069] Optionally, the grid height is compared with a grid height threshold corresponding to the ground two-dimensional grid, including: determining the corresponding grid height threshold based on a preset grid height value and a height change threshold between adjacent grids, wherein the greater the ground slope between adjacent grids, the greater the height change threshold. In another technical solution of the present application, a ground modeling method is provided, including: acquiring ground point cloud data through a lidar; preprocessing the ground point cloud data, and dividing it into a pre-constructed ground two-dimensional grid; processing the ground two-dimensional grid and its corresponding ground point cloud data through a pre-trained ground network model to obtain a grid height of the ground two-dimensional grid; comparing the grid height with a grid height threshold corresponding to the ground two-dimensional grid, wherein under the condition that the grid height is not greater than the grid height threshold, the ground point cloud data corresponding to the ground two-dimensional grid is determined as a ground point; and ground modeling is performed based on the ground point to obtain a road ground.

[0070] When the ground point cloud segmentation system of the present application segments the acquired ground point cloud data into ground points and non-ground points, the ground point cloud data is segmented by a two-dimensional grid, and the height of each point cloud grid is calculated by a pre-trained ground network model, and compared with the corresponding threshold of each grid to determine whether it is a ground point. In the judgment process, the slope change between adjacent grids is taken into account, and the slope change is included in the consideration of whether the grid is the ground or an obstacle, so that one two-dimensional grid corresponds to one grid height threshold, the accuracy of the judgment is achieved, the actual situation of the road surface is fully considered, and the accuracy of the ground point cloud data segmentation between ground points and obstacle points is improved.

[0071] Figure 3 An implementation of the ground modeling method of the present application is shown.

[0072] exist Figure 3In the illustrated embodiment, the ground modeling method of the present application includes: process S301, acquiring ground point cloud data; process S302, preprocessing the ground point cloud data and dividing it into a pre-constructed ground two-dimensional grid; process S303, determining the grid features corresponding to the ground two-dimensional grid according to the height information of the ground point cloud data through a pre-trained ground network model; process S304, processing the grid features through the ground network model to obtain a grid height prediction value corresponding to the ground two-dimensional grid; process S305, comparing the grid height prediction value with the grid height threshold corresponding to the ground two-dimensional grid, wherein under the condition that the grid height prediction value is greater than the grid height threshold, the ground point cloud data corresponding to the ground two-dimensional grid is determined as an obstacle point; under the condition that the grid height prediction value is not greater than the grid height threshold, the ground point cloud data corresponding to the ground two-dimensional grid is determined as a ground point; and process S306, performing ground modeling according to the ground point to obtain a road ground.

[0073] In this embodiment, the description of each process can refer to the description of the ground point cloud segmentation method in the above specification, which will not be repeated here. When the ground modeling method of the present application segments the acquired ground point cloud data into ground points and non-ground points, the ground point cloud data is segmented by a two-dimensional grid, and the height of each point cloud grid is calculated by a pre-trained ground network model, and compared with the corresponding threshold of each grid to determine whether it is a ground point. In the judgment process, the slope change between adjacent grids is taken into account, and the slope change is included in the consideration of whether the grid is the ground or an obstacle, so that one two-dimensional grid corresponds to one grid height threshold, the accuracy of the judgment is achieved, the actual situation of the road surface is fully considered, and the accuracy of the ground point cloud data segmentation of ground points and obstacle points is improved. When the acquired ground points are used for ground modeling, a ground map is obtained to achieve the accuracy of ground modeling. The specific process of each of the above steps refers to the ground point cloud segmentation method described in any example in the specification, which will not be repeated here.

[0074] Figure 4 An embodiment of the ground modeling device of the present application is shown.

[0075] exist Figure 4In the illustrated embodiment, the ground modeling device of the present application includes: a preprocessing module 401, which preprocesses the acquired ground point cloud data and divides it into a pre-constructed ground two-dimensional grid; a ground point segmentation module 402, which determines the grid features corresponding to the ground two-dimensional grid according to the height information of the ground point cloud data through a pre-trained ground network model; processes the grid features through the ground network model to obtain a grid height prediction value corresponding to the ground two-dimensional grid; compares the grid height prediction value with the grid height threshold corresponding to the ground two-dimensional grid, wherein under the condition that the grid height prediction value is greater than the grid height threshold, the ground point cloud data corresponding to the ground two-dimensional grid is determined as an obstacle point; under the condition that the grid height prediction value is not greater than the grid height threshold, the ground point cloud data corresponding to the ground two-dimensional grid is determined as a ground point; and a ground modeling module 403, which performs ground modeling according to the ground points to obtain a road ground.

[0076] In this embodiment, the principles of the processing process of each module in the ground modeling device are detailed in the description of the ground point cloud segmentation method in the above specification, and will not be repeated here. When the ground modeling device of the present application segments the acquired ground point cloud data into ground points and non-ground points, the ground point cloud data is segmented by a two-dimensional grid, and the height of each point cloud grid is calculated by a pre-trained ground network model, and compared with the corresponding threshold of each grid to determine whether it is a ground point. In the judgment process, the slope change between adjacent grids is taken into account, and the slope change is included in the consideration of whether the grid is the ground or an obstacle, so that one two-dimensional grid corresponds to one grid height threshold, the accuracy of the judgment is achieved, the actual situation of the road surface is fully considered, and the accuracy of the ground point cloud data segmentation of ground points and obstacle points is improved. When the acquired ground points are used for ground modeling, a ground map is obtained to achieve the accuracy of ground modeling.

[0077] In a specific embodiment of the present application, a computer-readable storage medium stores computer instructions, wherein the computer instructions are operated to execute the ground point cloud segmentation method or ground modeling method described in any embodiment. The storage medium may be directly in hardware, in a software module executed by a processor, or in a combination of the two.

[0078] The software modules may reside in RAM memory, flash memory, ROM memory, EPROM memory, EEPROM memory, registers, hard disk, removable disk, CD-ROM or any other form of storage medium known in the art. An exemplary storage medium is coupled to the processor so that the processor can read information from and write information to the storage medium.

[0079] The processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field programmable gate arrays (FPGA), or other programmable logic devices, discrete gates or transistor logic, discrete hardware components, or any combination thereof. A general-purpose processor may be a microprocessor, but in an alternative, the processor may be any conventional processor, controller, microcontroller, or state machine. The processor may also be implemented as a combination of computing devices, such as a combination of a DSP and a microprocessor, a plurality of microprocessors, one or more microprocessors combined with a DSP core, or any other such configuration. In an alternative, the storage medium may be integrated with the processor. The processor and the storage medium may reside in an ASIC. The ASIC may reside in a user terminal. In an alternative, the processor and the storage medium may reside in a user terminal as discrete components.

[0080] In a specific embodiment of the present application, a computer device includes a processor and a memory, wherein the memory stores computer instructions, wherein: the processor operates the computer instructions to execute the ground point cloud segmentation method or the ground modeling method described in any embodiment.

[0081] Figure 5 An implementation of a processing chip of the present application is shown.

[0082] exist Figure 5 In the illustrated embodiment, the processing chip of the present application includes a memory and a processor, wherein the processor is provided with a preprocessing module and a ground point segmentation module, for executing the ground point cloud segmentation process of the ground point cloud segmentation method described in the above specification. The specific processing process is described in the above specification, and will not be repeated here. The memory is used to store the intermediate data or final results of the processing of each module, and to export the processed data, etc.

[0083] In the embodiments provided in the present application, it should be understood that the disclosed device can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0084] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0085] The above are merely embodiments of the present application, and are not intended to limit the patent scope of the present application. Any equivalent structural transformations made using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A ground point cloud segmentation method, characterized in that: include: Obtain ground point cloud data; Preprocessing the ground point cloud data and dividing it into a pre-constructed ground two-dimensional grid; Determine, by a pre-trained ground network model, the grid features corresponding to the ground two-dimensional grid according to the height information of the ground point cloud data; Processing the grid features through the ground network model to obtain a grid height prediction value corresponding to the ground two-dimensional grid; The grid height prediction value is compared with the grid height threshold corresponding to the ground two-dimensional grid, wherein Under the condition that the grid height prediction value is greater than the grid height threshold, determining the ground point cloud data corresponding to the ground two-dimensional grid as an obstacle point; Under the condition that the grid height prediction value is not greater than the grid height threshold, determining the ground point cloud data corresponding to the ground two-dimensional grid as a ground point; The pre-training process of the ground network model includes: Obtain ground point cloud training data through lidar, The ground point cloud training data is projected into the ground two-dimensional grid, and the grid height of the ground two-dimensional grid is predicted according to the ground point cloud training data in the ground two-dimensional grid by a prediction network model to obtain a preliminary prediction value of the grid height, The prediction network model performs loss learning according to the true value of grid height, the height loss between adjacent grids and the preliminary prediction value of the grid height to obtain the ground network model, including: The prediction network model adjusts the initial prediction value of the grid height of the current grid according to the height loss between adjacent grids to obtain the grid height prediction value. According to the difference between the true value of the grid height corresponding to the current grid and the predicted value of the grid height, loss learning is performed on the prediction network model to obtain the ground network model.

2. The ground point cloud segmentation method according to claim 1, characterized in that: The preprocessing of the ground point cloud data and dividing it into a pre-constructed ground two-dimensional grid includes: Convert the ground point cloud data in the laser radar coordinate system to the vehicle coordinate system, and determine the three-dimensional coordinates corresponding to the ground point cloud data in the vehicle coordinate system; According to the three-dimensional coordinates, the ground point cloud data is divided into corresponding ground grids.

3. The ground point cloud segmentation method according to claim 2, characterized in that: The dividing the ground point cloud data into corresponding ground grids according to the three-dimensional coordinates includes: Determine the area range corresponding to the two-dimensional ground grid according to the coordinates of each vertex in the two-dimensional ground grid; According to the position of the three-dimensional coordinates in the vehicle coordinate system, the ground point cloud data within the area is divided into the ground point cloud data corresponding to the ground two-dimensional grid.

4. The ground point cloud segmentation method according to claim 1, characterized in that: Also includes: Manually labeling the ground point cloud training data to obtain real ground points; The real ground points are projected into the ground two-dimensional grid, and the average height of the real ground points in the ground two-dimensional grid is used as the true value of the grid height of the ground two-dimensional grid.

5. The ground point cloud segmentation method according to claim 1, characterized in that: The comparing the grid height with a grid height threshold corresponding to the ground two-dimensional grid includes: The corresponding grid height threshold is determined according to a preset grid height value and a height change threshold between adjacent grids, wherein the greater the ground slope between adjacent grids, the greater the height change threshold.

6. A ground point cloud segmentation system, characterized in that: include: A preprocessing module, which preprocesses the acquired ground point cloud data and divides it into a pre-constructed ground two-dimensional grid; A ground point segmentation module, which determines the grid features corresponding to the ground two-dimensional grid according to the height information of the ground point cloud data through a pre-trained ground network model; Processing the grid features through the ground network model to obtain a grid height prediction value corresponding to the ground two-dimensional grid; The grid height prediction value is compared with the grid height threshold corresponding to the ground two-dimensional grid, wherein Under the condition that the grid height prediction value is greater than the grid height threshold, determining the ground point cloud data corresponding to the ground two-dimensional grid as an obstacle point; Under the condition that the grid height prediction value is not greater than the grid height threshold, determining the ground point cloud data corresponding to the ground two-dimensional grid as a ground point; The pre-training process of the ground network model includes: Obtain ground point cloud training data through lidar, The ground point cloud training data is projected into the ground two-dimensional grid, and the grid height of the ground two-dimensional grid is predicted according to the ground point cloud training data in the ground two-dimensional grid by a prediction network model to obtain a preliminary prediction value of the grid height, The prediction network model performs loss learning according to the true value of grid height, the height loss between adjacent grids and the preliminary prediction value of the grid height to obtain the ground network model, including: The prediction network model adjusts the initial prediction value of the grid height of the current grid according to the height loss between adjacent grids to obtain the grid height prediction value. According to the difference between the true value of the grid height corresponding to the current grid and the predicted value of the grid height, loss learning is performed on the prediction network model to obtain the ground network model.

7. A ground modeling method, characterized in that: include: Obtain ground point cloud data; Preprocessing the ground point cloud data and dividing it into a pre-constructed ground two-dimensional grid; Determine, by a pre-trained ground network model, the grid features corresponding to the ground two-dimensional grid according to the height information of the ground point cloud data; Processing the grid features through the ground network model to obtain a grid height prediction value corresponding to the ground two-dimensional grid; The grid height prediction value is compared with the grid height threshold corresponding to the ground two-dimensional grid, wherein Under the condition that the grid height prediction value is greater than the grid height threshold, determining the ground point cloud data corresponding to the ground two-dimensional grid as an obstacle point; Under the condition that the grid height prediction value is not greater than the grid height threshold, determining the ground point cloud data corresponding to the ground two-dimensional grid as a ground point; Perform ground modeling according to the ground points to obtain a road ground; The pre-training process of the ground network model includes: Obtain ground point cloud training data through lidar, The ground point cloud training data is projected into the ground two-dimensional grid, and the grid height of the ground two-dimensional grid is predicted according to the ground point cloud training data in the ground two-dimensional grid by a prediction network model to obtain a preliminary prediction value of the grid height, The prediction network model performs loss learning according to the true value of grid height, the height loss between adjacent grids and the preliminary prediction value of the grid height to obtain the ground network model, including: The prediction network model adjusts the initial prediction value of the grid height of the current grid according to the height loss between adjacent grids to obtain the grid height prediction value. According to the difference between the true value of the grid height corresponding to the current grid and the predicted value of the grid height, loss learning is performed on the prediction network model to obtain the ground network model.

8. A computer-readable storage medium storing computer instructions, wherein the computer instructions are operated to execute the ground point cloud segmentation method described in any one of claims 1 to 5 or the ground modeling method described in claim 7.

Images