Ground segmentation method, device and computer equipment based on point cloud data

Abstract

A ground segmentation method based on point cloud data, comprising: obtaining point cloud data; performing ground segmentation on the point cloud data to obtain a ground segmentation result; performing object detection on the point cloud data to obtain a three-dimensional bounding box of the target object corresponding to the point cloud data; extracting the point cloud data in the three-dimensional bounding box, selecting target point cloud data from the extracted point cloud data, performing plane fitting on the target point cloud data corresponding to the three-dimensional bounding box to obtain a fitting plane; extracting the position regions corresponding to each target object according to the three-dimensional bounding box; and updating the ground segmentation result according to the position regions and the fitting plane to obtain the target ground segmentation result corresponding to the point cloud data.

Description

Technical Field

[0001] The present application relates to a ground segmentation method, device, computer device and storage medium based on point cloud data. Background Art

[0002] LiDAR sensors can provide real-time and accurate three-dimensional scene information, with a large ranging scope, high precision, and measurement values unaffected by environmental lighting. They are applied in fields such as driverless, security monitoring, surveying and mapping exploration, etc. The information collected by LiDAR sensors is usually presented in the form of point clouds. By correspondingly processing the point cloud data, tasks such as high-precision map reconstruction, detection and tracking of obstacles can be completed. Since ground point cloud data usually occupies a large amount of data in the entire observation, and the amount of useful information contained in the point cloud data for completing the target task is small, it is necessary to perform ground segmentation on the point cloud data, so as to complete the target task based on the remaining point cloud data after removing the ground point cloud data.

[0003] In traditional methods, ground segmentation of point cloud data is mainly performed through ground fitting or methods based on depth images and geometric relationships. However, these methods are only applicable to ground segmentation of point cloud data in specific scenarios. For example, the ground fitting method is only applicable to the case where the road is relatively flat, and the method based on depth images and geometric relationships requires the LiDAR sensor to be installed in a horizontal state. When the scene changes, if the traditional method is used to perform ground segmentation on the point cloud data, the accuracy of ground segmentation of the point cloud data will be relatively low. Summary of the Invention

[0004] According to various embodiments disclosed in the present application, a ground segmentation method, device, computer device and storage medium based on point cloud data are provided.

[0005] A ground segmentation method based on point cloud data includes:

[0006] Obtain point cloud data;

[0007] Perform ground segmentation on the point cloud data to obtain a ground segmentation result;

[0008] Perform target detection on the point cloud data to obtain a three-dimensional bounding box of the target object corresponding to the point cloud data;

[0009] Extract the point cloud data in the three-dimensional bounding box, select target point cloud data from the extracted point cloud data, and perform plane fitting on the target point cloud data corresponding to the three-dimensional bounding box to obtain a fitting plane;

[0010] Extract the position area corresponding to each target object according to the three-dimensional bounding box; and

[0011] Update the ground segmentation result according to the position area and the fitted plane to obtain the target ground segmentation result corresponding to the point cloud data.

[0012] A ground segmentation device based on point cloud data, comprising:

[0013] A communication module for acquiring point cloud data;

[0014] A ground segmentation module for performing ground segmentation on the point cloud data to obtain a ground segmentation result;

[0015] A target detection module for performing target detection on the point cloud data to obtain a three-dimensional bounding box of the target object corresponding to the point cloud data;

[0016] A plane fitting module for extracting the point cloud data in the three-dimensional bounding box, selecting target point cloud data from the extracted point cloud data, and performing plane fitting on the target point cloud data corresponding to the three-dimensional bounding box to obtain a fitted plane;

[0017] An extraction module for extracting the position area corresponding to each target object according to the three-dimensional bounding box; and

[0018] An update module for updating the ground segmentation result according to the position area and the fitted plane to obtain the target ground segmentation result corresponding to the point cloud data.

[0019] A computer device comprising a memory and one or more processors, wherein the memory stores computer-readable instructions, and when the computer-readable instructions are executed by the processors, the one or more processors perform the following steps:

[0020] Acquire point cloud data;

[0021] Perform ground segmentation on the point cloud data to obtain a ground segmentation result;

[0022] Perform target detection on the point cloud data to obtain a three-dimensional bounding box of the target object corresponding to the point cloud data;

[0023] Extract the point cloud data in the three-dimensional bounding box, select target point cloud data from the extracted point cloud data, and perform plane fitting on the target point cloud data corresponding to the three-dimensional bounding box to obtain a fitted plane;

[0024] Extract the position area corresponding to each target object according to the three-dimensional bounding box; and

[0025] Update the ground segmentation result according to the position area and the fitted plane to obtain the target ground segmentation result corresponding to the point cloud data.

[0026] One or more non - volatile computer - readable storage media storing computer - readable instructions, which, when executed by one or more processors, cause the one or more processors to perform the following steps:

[0027] Obtain point cloud data;

[0028] Perform ground segmentation on the point cloud data to obtain a ground segmentation result;

[0029] Perform object detection on the point cloud data to obtain a three - dimensional bounding box of the target object corresponding to the point cloud data;

[0030] Extract the point cloud data in the three - dimensional bounding box, select target point cloud data from the extracted point cloud data, and perform plane fitting on the target point cloud data corresponding to the three - dimensional bounding box to obtain a fitting plane;

[0031] Extract the position area corresponding to each target object according to the three - dimensional bounding box; and

[0032] Update the ground segmentation result according to the position area and the fitting plane to obtain a target ground segmentation result corresponding to the point cloud data.

[0033] Details of one or more embodiments of the present application are set forth in the following drawings and description. Other features and advantages of the present application will become apparent from the specification, drawings, and claims. Brief Description of the Drawings

[0034] To more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings required for use in the embodiments. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0035] Figure 1 It is an application scenario diagram of the ground segmentation method based on point cloud data in one or more embodiments.

[0036] Figure 2 It is a flow schematic diagram of the ground segmentation method based on point cloud data in one or more embodiments.

[0037] Figure 3 It is a flow schematic diagram of the step of selecting target point cloud data from the extracted point cloud data in one or more embodiments.

[0038] Figure 4 It is a flow schematic diagram of the step of updating the ground segmentation result according to the position area and the fitting plane to obtain the target ground segmentation result in one or more embodiments.

[0039] Figure 5Block diagram of a ground segmentation device based on point cloud data in one or more embodiments.

[0040] Figure 6 Block diagram of a computer device in one or more embodiments. Detailed implementation manners

[0041] In order to make the technical solutions and advantages of this application clearer, the following further details this application in combination with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not used to limit this application.

[0042] It should be noted that the terms "first", "second", etc. in the description and claims of this application are used to distinguish similar objects and do not necessarily need to describe a specific order or sequence.

[0043] The ground segmentation method based on point cloud data provided by this application can be applied to, for example, Figure 1 the application scenario of driverless as shown. In a driverless vehicle, a vehicle-mounted sensor 102 and a vehicle-mounted computer device 104 are pre-installed. The vehicle-mounted computer device can be simply referred to as a computer device. The vehicle-mounted sensor 102 communicates with the computer device 104. During the driverless process, the vehicle-mounted sensor 102 transmits the collected point cloud data to the computer device 104. The computer device 104 performs ground segmentation on the point cloud data to obtain a ground segmentation result. The computer device 104 can also perform target detection on the point cloud data to obtain a three-dimensional bounding box of the target object corresponding to the point cloud data. The computer device 104 extracts the point cloud data in the three-dimensional bounding box, selects target point cloud data from the extracted point cloud data, and performs plane fitting on the target point cloud data corresponding to the three-dimensional bounding box to obtain a fitting plane. The computer device 104 extracts the position area corresponding to each target object according to the three-dimensional bounding box, and thus updates the ground segmentation result according to the position area and the fitting plane to obtain the target ground segmentation result corresponding to the point cloud data. The vehicle-mounted sensor 102 can be a lidar.

[0044] In one of the embodiments, as Figure 2 shown, a ground segmentation method based on point cloud data is provided. Taking the method applied to Figure 1 the computer device therein as an example for illustration, it includes the following steps:

[0045] Step 202, obtain point cloud data.

[0046] The point cloud data can be the data recorded in the form of point cloud by on-vehicle sensors scanning the surrounding environment information. The point cloud data can specifically include the three-dimensional coordinates (x, y, z) of each point, the laser reflection intensity (Intensity), the color information (RGB), etc. The three-dimensional coordinates are used to represent the position information of the surface of the target object in the surrounding environment. For example, the three-dimensional coordinates can be the coordinates of a point in the Cartesian coordinate system, specifically including the horizontal axis coordinate, the vertical axis coordinate, and the vertical axis coordinate of the point in the Cartesian coordinate system. The Cartesian coordinate system is a three-dimensional space coordinate system established with the on-vehicle sensor as the origin. The three-dimensional space coordinate system includes the horizontal axis (x-axis), the vertical axis (y-axis), and the vertical axis (z-axis). The three-dimensional space coordinate system established with the on-vehicle sensor as the origin satisfies the right-hand rule. The x-axis coordinate in the three-dimensional coordinates represents the longitudinal distance of the scanned target object surface relative to the on-vehicle sensor, the y-axis coordinate represents the lateral offset of the scanned target object surface relative to the on-vehicle sensor, and the z-axis coordinate represents the height of the scanned target object surface relative to the on-vehicle sensor.

[0047] During the process of driverless driving, the vehicle can scan the current environment through on-vehicle sensors installed on the vehicle to obtain corresponding point cloud data, and transmit the collected point cloud data to a computer device. For example, the on-vehicle sensor can be a lidar.

[0048] Step 204: Perform ground segmentation on the point cloud data to obtain a ground segmentation result.

[0049] Ground segmentation refers to identifying ground points and non-ground points in the point cloud data. Since the point cloud data includes a large amount of ground point cloud data, and the amount of useful information contained in the point cloud data for tasks such as high-precision map reconstruction, obstacle detection and tracking is small, it is necessary to perform ground segmentation on the point cloud data, so as to complete the target task based on the remaining point cloud data after removing the ground points.

[0050] A computer device can perform ground segmentation on point cloud data to identify the categories of each point in the point cloud data and obtain the corresponding ground segmentation result. The categories specifically include ground points and non-ground points. The ground segmentation method can be to first divide the point cloud area where the point cloud data is located into multiple sub-areas. The point cloud area refers to the three-dimensional data space where the point cloud data is located. The division method can be to perform grid division on the point cloud area, that is, to divide the horizontal plane formed by the x-axis and y-axis directions of the point cloud area. The grid division method can be equal division or random division. For example, for a vehicle-mounted sensor with a visible range of 100m, the size of the horizontal plane within the scanning area of the vehicle-mounted sensor is 100m * 100m, then the point cloud area can be equally divided into 10 * 10 horizontal grids. For each sub-area obtained by the division, the computer device can estimate the corresponding ground using the least squares method according to the preset plane equation, so as to obtain the ground corresponding to each sub-area. For example, the preset plane equation can be a ternary linear equation. The ground corresponding to each sub-area is represented in the form of a ternary linear equation. The computer device traverses and inputs the point coordinates in the corresponding sub-area into the equation of the ground, calculates the distance between each point and the corresponding ground, and when the distance is less than the threshold, the point is determined as a ground point. When the distance is greater than or equal to the threshold, the point is determined as a non-ground point. The threshold refers to the distance threshold used to determine whether the point is a ground point.

[0051] In one embodiment, the computer device can also perform ground segmentation on the point cloud data using any one of the point cloud ground segmentation methods such as the method based on depth image and geometric relationship, the normal vector method, the absolute height method, the average height method, etc., which do not require standard ground data and training models.

[0052] Step 206, perform object detection on the point cloud data to obtain the three-dimensional bounding box of the target object corresponding to the point cloud data.

[0053] Object detection refers to identifying the category of the target object corresponding to the point cloud data and the three-dimensional bounding box corresponding to each target object. The target object refers to an object whose bottom is the ground.

[0054] The computer device stores an object detection unit, which can directly use the three-dimensional object detection unit to perform object detection on the point cloud data, identify the category of the target object corresponding to the point cloud data and the point cloud data corresponding to each target object. For example, the target object can be a vehicle, a pedestrian, etc. The bottom of the target object is the ground. The computer device can calculate the three-dimensional bounding box of the corresponding target object according to the point cloud data corresponding to each target object. The three-dimensional bounding box has corresponding center point coordinates, size, orientation, etc. The object detection method is any one of the object detection methods such as three-dimensional object detection based on shape matching, three-dimensional object detection based on target local features, and object detection method based on deep learning.

[0055] Step 208: Extract the point cloud data in the three-dimensional bounding box, select the target point cloud data from the extracted point cloud data, and perform plane fitting on the target point cloud data corresponding to the three-dimensional bounding box to obtain a fitted plane.

[0056] The target point cloud data refers to the plane fitting points used to fit the ground plane at the bottom of the target.

[0057] After the computer device obtains the three-dimensional bounding boxes corresponding to each target object, it extracts the point cloud data in each three-dimensional bounding box, and selects the target point cloud data for the point cloud data extracted from each three-dimensional bounding box. Specifically, the computer device can perform grid division on the point cloud data extracted from each three-dimensional bounding box to obtain multiple grids. There can be multiple point cloud data corresponding to each grid. The computer device selects the target point cloud data from the point cloud data corresponding to each grid, and performs plane fitting on the target point cloud data corresponding to each three-dimensional bounding box to obtain the fitted plane corresponding to each three-dimensional bounding box. The fitted plane is the ground plane at the bottom of the target object corresponding to the three-dimensional bounding box. The fitted plane is represented in the form of an equation.

[0058] In one embodiment, the method of performing plane fitting on the target point cloud data corresponding to the three-dimensional bounding box can be any one of plane fitting algorithms such as the least squares method, the eigenvalue method, and the total least squares method.

[0059] Step 210: Extract the position regions corresponding to each target object according to the three-dimensional bounding box.

[0060] Step 212: Update the ground segmentation result according to the position region and the fitted plane to obtain the target ground segmentation result corresponding to the point cloud data.

[0061] Since the ground segmentation result is affected by various factors such as noise, road slope, and weather, the accuracy of the ground segmentation result will be reduced. Therefore, the computer device can update the ground segmentation result through the fitted plane obtained by target detection. Specifically, the computer device can determine the position region corresponding to each target object according to the three-dimensional bounding box obtained by target detection. The position region can be the horizontal plane region where the target object is located, that is, the region composed of the x-axis direction and the y-axis direction. The computer device can then update the ground segmentation result according to the position region and the fitted plane, so as to update the data with lower reliability in the ground segmentation result and obtain a more accurate target ground segmentation result.

[0062] In one embodiment, the ground segmentation result corresponding to the point cloud data is updated according to the position area and the fitting plane, and the obtained target ground segmentation result includes: identifying the position relationship between each point in the point cloud data and the position area; when the position relationship is that the point is not near the position area, retaining the ground segmentation result corresponding to the points not near the position area in the ground segmentation result.

[0063] The computer device identifies the position relationship between each point in the point cloud data and the position area. Specifically, the computer device only needs to identify the position relationship between the point and the position area according to the abscissa and ordinate of each point. When any one of the abscissa and ordinate of the point is outside the position area, it indicates that the point is not near the position area. At this time, the computer device can retain the ground segmentation result corresponding to the point in the ground segmentation result and use the category corresponding to the point in the ground segmentation result as the final category of the point. When both the abscissa and ordinate of the point are within the position area, it indicates that the point is near the position area. At this time, the computer device can calculate the distance between the point and the fitting plane, compare the distance with a threshold, and update the ground segmentation result according to the comparison result, so as to obtain the target ground segmentation result corresponding to the point cloud data.

[0064] In this embodiment, by acquiring point cloud data, ground segmentation is performed on the point cloud data to obtain a ground segmentation result. Target detection can also be performed on the point cloud data to obtain a three-dimensional bounding box of the target object corresponding to the point cloud data. By extracting the point cloud data in the three-dimensional bounding box and selecting target point cloud data from the extracted point cloud data, the target point cloud data corresponding to the three-dimensional bounding box is then subjected to plane fitting to obtain a fitting plane. Furthermore, the position regions corresponding to each target object can be extracted according to the three-dimensional bounding box, and the ground segmentation result can be updated based on the position regions and the fitting plane. Since the ground segmentation result is obtained by processing the point cloud data through traditional ground segmentation methods, such as methods based on depth images and geometric relationships, normal vector methods, absolute height methods, average height methods, etc., it is greatly affected by various factors such as noise, road slope, and weather, and is only applicable to specific scenarios, which may lead to a decrease in the accuracy of the ground segmentation result. However, the target detection unit based on deep learning can adapt to various scenarios and is less affected by various factors such as noise, road slope, and weather. By using the target detection unit based on deep learning stored in the computer device to perform target detection on the point cloud data and obtain the fitting plane at the bottom of the target object, the ground segmentation result can be updated based on the fitting plane, which can improve the accuracy of the ground segmentation result. In addition, the traditional ground segmentation method used for ground segmentation of point cloud data does not require manual annotation of ground data and training of the model. Compared with the method of using a large number of sample data annotated with ground data and a deep neural network model for model training, and performing ground segmentation of point cloud data through the trained model, it avoids the problems of large difficulty and time consumption in ground data annotation, and effectively improves the efficiency of ground segmentation of point cloud data.

[0065] In one of the embodiments, as Figure 3 shown, the step of selecting target point cloud data from the extracted point cloud data specifically includes:

[0066] Step 302, dividing the extracted point cloud data into multiple grids according to preset parameters.

[0067] Step 304, selecting the point with the minimum height value from the point cloud data corresponding to each grid.

[0068] Step 306, calculating the height difference between each point in the point cloud data corresponding to each grid and the point with the minimum height value.

[0069] Step 308, selecting the points with a height difference less than the first threshold, and obtaining the target point cloud data according to the selected points.

[0070] The preset parameters can be parameters for rasterizing the data area where the extracted point cloud data is located. For example, the preset parameters can be length * width, indicating the length and width of each raster obtained after rasterization. The length and width can be the same or different. The preset parameters can also be equal division, the number of target rasters, etc. The extracted point cloud data refers to the point cloud data in the three-dimensional bounding box corresponding to each target object. The target point cloud data refers to the set of plane fitting points for fitting the ground plane at the bottom of the target.

[0071] Specifically, the computer device can divide the data area corresponding to the extracted point cloud data in the x-axis direction and the y-axis direction according to the preset parameters, so as to obtain multiple rasters. The heights of the multiple rasters are the same. When rasterizing the data area corresponding to the extracted point cloud data, the order of division in the x-axis direction and the y-axis direction is not limited. For example, the computer device can first divide the data area corresponding to the extracted point cloud data in the x-axis direction according to the preset parameters, and then divide the data area corresponding to the extracted point cloud data in the y-axis direction according to the preset parameters. It can also first divide the data area corresponding to the extracted point cloud data in the y-axis direction according to the preset parameters, and then divide the data area corresponding to the extracted point cloud data in the x-axis direction according to the preset parameters.

[0072] For each raster, the computer device selects the point with the minimum height value in the point cloud data corresponding to each raster, and calculates the height difference between each point in the point cloud data corresponding to the corresponding raster and the point with the minimum height value. The computer device pre-stores a first threshold for determining whether it is a plane fitting point. When the height difference is less than the first threshold, it indicates that the point corresponding to the height difference is a plane fitting point. The computer device compares the height difference with the first threshold, selects the points with height differences less than the first threshold, and determines the selected points as plane fitting points, thereby obtaining the target point cloud data.

[0073] In this embodiment, the extracted point cloud data is divided into multiple rasters according to the preset parameters, and the point with the minimum height value is selected from the point cloud data corresponding to each raster, and the height difference between each point in the point cloud data corresponding to each raster and the point with the minimum height value is calculated, so as to select the points with height differences less than the first threshold, and the target point cloud data is obtained according to the selected points. Since rasterization only needs to divide the data area corresponding to the point cloud data in the x-axis direction and the y-axis direction, and only needs to calculate the height difference between each point in the point cloud data corresponding to each raster and the point with the minimum height value, the target point cloud data can be quickly selected. Therefore, when the computing resources of the computer device are limited and the real-time requirement is high in the driverless mode, the extraction efficiency of the target point cloud data can be improved.

[0074] In one embodiment, performing plane fitting on the target point cloud data corresponding to the three-dimensional bounding box to obtain the fitting plane includes: combining the target point cloud data of multiple grids corresponding to the three-dimensional bounding box to obtain the combined point cloud data corresponding to the three-dimensional bounding box; performing plane fitting on the combined point cloud data of the three-dimensional bounding box to obtain the fitting plane corresponding to the three-dimensional bounding box.

[0075] After extracting the point cloud data in the three-dimensional bounding box and dividing the extracted point cloud data into multiple grids, each three-dimensional bounding box can correspond to multiple grids. After selecting the target point cloud data in each grid, the target point cloud data of multiple grids corresponding to each three-dimensional bounding box can be combined together to obtain the combined point cloud data corresponding to multiple three-dimensional bounding boxes. Then, the computer device performs plane fitting on the combined point cloud data corresponding to each three-dimensional bounding box respectively to obtain the fitting plane corresponding to each three-dimensional bounding box. The plane fitting method can be any one of plane fitting algorithms such as the least squares method, the eigenvalue method, and the total least squares method. The fitting plane refers to the plane at the bottom of the target object corresponding to the three-dimensional bounding box. The fitting plane can be represented in the form of a ternary linear equation.

[0076] In this embodiment, the computer device combines the target point cloud data of multiple grids corresponding to each three-dimensional bounding box, which can combine the target point cloud data belonging to the same target object together, and is beneficial to improving the efficiency of calculating the fitting plane at the bottom of each target object. The computer device performs plane fitting on the combined point cloud data corresponding to each three-dimensional bounding box to obtain the fitting plane corresponding to the three-dimensional bounding box, and can quickly calculate the bottom plane corresponding to each target object.

[0077] In one embodiment, as Figure 4 shown, the steps of updating the ground segmentation result according to the position area and the fitting plane to obtain the target ground segmentation result specifically include:

[0078] Step 402, identifying the position relationship between each point in the point cloud data and the position area.

[0079] Step 404, when the position relationship is that the point is not near the position area, retaining the ground segmentation result corresponding to the point not near the position area in the ground segmentation result.

[0080] Step 406, when the position relationship is that the point is located near the position area, calculating the first distance value between the point located near the position area and the fitting plane.

[0081] Step 408, comparing the first distance value with the second threshold, and determining the first category corresponding to the point located near the position area according to the comparison result.

[0082] Step 410: Update the ground segmentation result according to the first category to obtain the target ground segmentation result corresponding to the point cloud data.

[0083] The position area can be the horizontal plane area where the target object is located, that is, the area formed by the x-axis direction and the y-axis direction.

[0084] The computer device can identify the position relationship between each point in the point cloud data and the position area according to the abscissa and ordinate of each point in the point cloud data. When any one of the abscissa and ordinate of the point is outside the position area, it indicates that the point is not near the position area, and the point does not belong to the points on the surface of the target object corresponding to the corresponding three-dimensional bounding box. At this time, the computer device can retain the ground segmentation result corresponding to the point in the ground segmentation result and use the category corresponding to the point in the ground segmentation result as the final category of the point. When both the abscissa and ordinate of the point are within the position area, it indicates that the point is near the position area, and the point belongs to the points on the surface of the target object corresponding to the corresponding three-dimensional bounding box. Since the accuracy of the segmentation result of the points near the position area is easily affected by various factors. At this time, the computer device can calculate the distance between the point and the fitting plane to obtain the first distance value. A second threshold for judging the category of the point is pre-stored in the computer device. Therefore, the computer device compares the first distance with the second threshold and determines the first category corresponding to the point according to the comparison result. The first category refers to the category of the point identified by the fitting plane.

[0085] In one embodiment, the first category includes ground points and non-ground points. Comparing the first distance value with the second threshold, the first category corresponding to the points located near the position area determined according to the comparison result includes: determining the points with the first distance value less than the second threshold as ground points; determining the points with the first distance value greater than or equal to the second threshold as non-ground points.

[0086] When the comparison result is that the first distance is less than the second threshold, the computer device determines that the first category corresponding to the point is a ground point. Thus, the ground segmentation result is updated according to the first category of the point. When the comparison result is that the first distance is greater than or equal to the second threshold, the computer device can retain the ground segmentation result corresponding to the point and use the ground segmentation result corresponding to the point as the final ground segmentation result.

[0087] In this embodiment, by identifying whether each point in the point cloud data is near the position area, when the point is not near the position area, the ground segmentation result corresponding to the point not near the position area in the ground segmentation result is retained. When the point is near the position area, the distance between the point and the fitted plane is calculated, the distance is compared with the corresponding threshold, and the ground segmentation result is updated according to the comparison result. Since the fitted plane is obtained by detection processing based on the object detection unit of deep learning and has high accuracy, the ground segmentation result corresponding to the point near the position area can be updated according to the fitted plane, thereby improving the accuracy of the ground segmentation result. At the same time, it can also avoid the problem of dividing the points belonging to the surface of the same target object into multiple parts.

[0088] In one embodiment, updating the ground segmentation result according to the first category includes: finding the ground segmentation result corresponding to the point near the position area in the ground segmentation result; comparing the found ground segmentation result with the first category; when the comparison is consistent, retaining the found ground segmentation result; when the comparison is inconsistent, replacing the found ground segmentation result according to the first category.

[0089] When there are points near the position area in the point cloud data, the first distance value between the point and the fitted plane can be calculated, and by comparing the first distance value with the second threshold, the first category corresponding to the point can be obtained. Then the computer device finds the ground segmentation result corresponding to the point in the ground segmentation result and identifies whether the ground segmentation result found is consistent with the first category corresponding to the calculated point. When the comparison is consistent, the computer device retains the ground segmentation result of the point. When the comparison is inconsistent, the ground segmentation result corresponding to the point needs to be replaced with the first category, thereby updating the ground segmentation result, using the first category with higher accuracy as the ground segmentation result of the point, and further improving the accuracy of the ground segmentation result.

[0090] In one embodiment, performing ground segmentation on the point cloud data to obtain the ground segmentation result includes: dividing the point cloud area corresponding to the point cloud data into multiple sub-regions; calculating the ground corresponding to the point cloud data in each sub-region according to the preset plane equation; calculating the second distance value between each point in the point cloud data of each sub-region and the corresponding ground; comparing the second distance value with the third threshold, determining the second category corresponding to each point according to the comparison result, and using the second category corresponding to each point as the ground segmentation result.

[0091] The point cloud region refers to the three-dimensional data space where the point cloud data is located. The second distance value refers to the distance between each point in the point cloud data and the corresponding ground calculated during the process of ground segmentation of the point cloud data. The third threshold value refers to the threshold value used to determine the category of each point in the point cloud data during the process of ground segmentation of the point cloud data. The second category refers to the category corresponding to each point in the ground segmentation result.

[0092] After obtaining the point cloud data, the computer device performs ground segmentation on the point cloud data. Specifically, the computer device divides the horizontal plane formed by the point cloud region corresponding to the point cloud data in the x-axis direction and the y-axis direction to obtain multiple sub-regions. Each sub-region may include multiple points. The division method can be equal division or random division. For each sub-region obtained by the division, the computer device can estimate the corresponding ground by using the least squares method according to the preset plane equation, so as to obtain the ground corresponding to each sub-region. The ground corresponding to each sub-region is represented in the form of a ternary linear equation. The computer device traverses and inputs the point coordinates of each point in the corresponding sub-region in the equation corresponding to the ground, calculates the second distance between each point and the corresponding ground, and compares the second distance with the third threshold value to obtain a comparison result. When the comparison result is that the second distance is less than the third threshold value, the second category of the point corresponding to the comparison result is a ground point. When the comparison result is that the second distance is greater than or equal to the third threshold value, the second category of the point corresponding to the comparison result is a non-ground point. Furthermore, the second categories corresponding to multiple points are used as the ground segmentation result.

[0093] In this embodiment, during the process of ground segmentation of the point cloud data by the computer device, it is not necessary to manually annotate the ground data, nor to train a model according to the sample data annotated with ground data and perform ground segmentation on the point cloud data according to the trained model, thus avoiding the problems of high annotation difficulty and time-consuming and laborious work, and improving the efficiency of ground segmentation of the point cloud data.

[0094] In one of the embodiments, as Figure 5 shown, a ground segmentation device based on point cloud data is provided, including: a communication module 502, a ground segmentation module 504, a target detection module 506, a plane fitting module 508, an extraction module 510, and an update module 512, where:

[0095] The communication module 502 is used to obtain point cloud data.

[0096] The ground segmentation module 504 is used to perform ground segmentation on the point cloud data to obtain a ground segmentation result.

[0097] The target detection module 506 is used to perform target detection on the point cloud data to obtain a three-dimensional bounding box of the target object corresponding to the point cloud data.

[0098] A plane fitting module 508 is configured to extract point cloud data in a three-dimensional bounding box, select target point cloud data from the extracted point cloud data, and perform plane fitting on the target point cloud data corresponding to the three-dimensional bounding box to obtain a fitted plane.

[0099] An extraction module 510 is configured to extract the position regions corresponding to each target object according to the three-dimensional bounding box.

[0100] An update module 512 is configured to update the ground segmentation result according to the position region and the fitted plane to obtain the target ground segmentation result corresponding to the point cloud data.

[0101] In one embodiment, the plane fitting module 508 is further configured to divide the extracted point cloud data into multiple grids according to preset parameters; select the points with the minimum height value in the point cloud data corresponding to each grid; calculate the height differences between each point in the point cloud data corresponding to each grid and the point with the minimum height value; select the points with height differences less than a first threshold, and obtain the target point cloud data according to the selected points.

[0102] In one embodiment, the plane fitting module 508 is further configured to combine the target point cloud data of multiple grids corresponding to the three-dimensional bounding box to obtain the combined point cloud data corresponding to the three-dimensional bounding box; perform plane fitting on the combined point cloud data of the three-dimensional bounding box to obtain the fitted plane corresponding to the three-dimensional bounding box.

[0103] In one embodiment, the update module 512 is further configured to identify the position relationship between each point in the point cloud data and the position region; when the position relationship is that the point is not near the position region, retain the ground segmentation result corresponding to the points not near the position region in the ground segmentation result.

[0104] In one embodiment, the above device further includes:

[0105] A distance calculation module is configured to calculate a first distance value between the points near the position region and the fitted plane when the position relationship is that the point is near the position region.

[0106] A category determination module is configured to compare the first distance value with a second threshold, and determine a first category corresponding to the points near the position region according to the comparison result.

[0107] The update module 512 is further configured to update the ground segmentation result according to the first category to obtain the target ground segmentation result corresponding to the point cloud data.

[0108] In one embodiment, the update module 512 is further configured to find the ground segmentation results corresponding to the points located near the position area in the ground segmentation results; compare the found ground segmentation results with the first category; when the comparison is consistent, retain the found ground segmentation results; when the comparison is inconsistent, replace the found ground segmentation results according to the first category.

[0109] In one embodiment, the first category includes ground points and non-ground points, and the category determination module is further configured to determine the points with the first distance value less than the second threshold as ground points; determine the points with the first distance value greater than or equal to the second threshold as non-ground points.

[0110] In one embodiment, the ground segmentation module 504 is further configured to divide the point cloud region corresponding to the point cloud data into multiple sub-regions; calculate the ground corresponding to the point cloud data of each sub-region according to the preset plane equation; calculate the second distance value between each point in the point cloud data of each sub-region and the corresponding ground; compare the second distance value with the third threshold, determine the second category corresponding to each point according to the comparison result, and use the second category corresponding to each point as the ground segmentation result.

[0111] For the specific limitations on the ground segmentation device based on point cloud data, reference can be made to the limitations on the ground segmentation method based on point cloud data in the foregoing text, which will not be elaborated here. Each module in the above-mentioned ground segmentation device based on point cloud data can be implemented in whole or in part by software, hardware, and their combination. The above-mentioned modules can be embedded in the processor of the computer device in hardware form or be independent of it, or be stored in the memory of the computer device in software form, so that the processor can call and execute the operations corresponding to the above-mentioned modules.

[0112] In one embodiment, a computer device is provided, and its internal structure diagram can be as Figure 6 shown. The computer device includes a processor, a memory, a communication interface, and a database connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store point cloud data, ground segmentation results, target segmentation results, etc. The communication interface of the computer device is used to connect and communicate with vehicle-mounted sensors. When the computer program is executed by the processor, it implements a ground segmentation method based on point cloud data.

[0113] Those skilled in the art can understand, Figure 6The structure shown is only a block diagram of some structures related to the solution of this application, and does not constitute a limitation on the computer device to which the solution of this application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine some components, or have different component arrangements.

[0114] A computer device includes a memory and one or more processors. Computer-readable instructions are stored in the memory. When the computer-readable instructions are executed by the one or more processors, the one or more processors are caused to execute the steps in each of the above method embodiments.

[0115] One or more non-volatile computer-readable storage media storing computer-readable instructions, which when executed by one or more processors, cause the one or more processors to execute the steps in each of the above method embodiments.

[0116] Those of ordinary skill in the art can understand that all or part of the processes of implementing the methods in the above embodiments can be completed by instructing relevant hardware through computer-readable instructions. The computer-readable instructions can be stored in a non-volatile computer-readable storage medium. When the computer-readable instructions are executed, they may include the processes of the embodiments of the above methods. Among them, any reference to memory, storage, database, or other media used in the embodiments provided in this application may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and Rambus dynamic RAM (RDRAM), etc.

[0117] The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity of description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered to be within the scope described in this specification.

[0118] The above-described embodiments merely represent several implementation manners of the present application. The description thereof is relatively specific and detailed, but it should not be construed as a limitation on the scope of the invention patent. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present application, several modifications and improvements can still be made, and these all fall within the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the appended claims.

Claims

1. A ground segmentation method based on point cloud data, comprising: obtaining point cloud data; performing ground segmentation on the point cloud data to obtain a ground segmentation result; performing object detection on the point cloud data to obtain a three-dimensional bounding box of the target object corresponding to the point cloud data; extracting the point cloud data in the three-dimensional bounding box, selecting target point cloud data from the extracted point cloud data, and performing plane fitting on the target point cloud data corresponding to the three-dimensional bounding box to obtain a fitted plane; extracting the position regions corresponding to the respective target objects according to the three-dimensional bounding box; identifying the positional relationship between each point in the point cloud data and the position regions; when the positional relationship is that the point is not near the position region, retaining the ground segmentation result corresponding to the points not near the position region in the ground segmentation result; when the positional relationship is that the point is located near the position region, calculating a first distance value between the points located near the position region and the fitted plane; comparing the first distance value with a second threshold, and determining a first category corresponding to the points located near the position region according to the comparison result; and updating the ground segmentation result according to the first category to obtain a target ground segmentation result corresponding to the point cloud data.

2. The method according to claim 1, wherein the selecting target point cloud data from the extracted point cloud data includes: dividing the extracted point cloud data into a plurality of grids according to preset parameters; selecting the point with the minimum height value from the point cloud data corresponding to each grid; calculating the height difference between each point in the point cloud data corresponding to each grid and the point with the minimum height value; and selecting the points with a height difference less than a first threshold, and obtaining target point cloud data according to the selected points.

3. The method according to claim 2, wherein the performing plane fitting on the target point cloud data corresponding to the three-dimensional bounding box to obtain a fitted plane includes: combining the target point cloud data of a plurality of grids corresponding to the three-dimensional bounding box to obtain combined point cloud data corresponding to the three-dimensional bounding box; and performing plane fitting on the combined point cloud data of the three-dimensional bounding box to obtain a fitted plane corresponding to the three-dimensional bounding box.

4. The method according to claim 1, wherein the updating the ground segmentation result according to the first category includes: searching for the ground segmentation result corresponding to the points located near the position region in the ground segmentation result; comparing the found ground segmentation result with the first category; when the comparison is consistent, retaining the found ground segmentation result; and when the comparison is inconsistent, replacing the found ground segmentation result according to the first category.

5. The method according to claim 1, wherein the first category includes ground points and non-ground points, and the comparing the first distance value with a second threshold and determining a first category corresponding to the points located near the position region according to the comparison result includes: determining the points with a first distance value less than the second threshold as ground points; and determining the points with a first distance value greater than or equal to the second threshold as non-ground points.

6. The method according to any one of claims 1 to 5, characterized in that, the ground segmentation of the point cloud data to obtain a ground segmentation result includes: dividing the point cloud region corresponding to the point cloud data into a plurality of sub-regions; calculating the ground corresponding to the point cloud data of each sub-region according to a preset plane equation; calculating a second distance value between each point in the point cloud data of each sub-region and the corresponding ground; and comparing the second distance value with a third threshold, determining a second category corresponding to each point according to the comparison result, and taking the second category corresponding to each point as the ground segmentation result.

7. A ground segmentation device based on point cloud data, comprising: a communication module for acquiring point cloud data; a ground segmentation module for performing ground segmentation on the point cloud data to obtain a ground segmentation result; a target detection module for performing target detection on the point cloud data to obtain a three-dimensional bounding box of the target object corresponding to the point cloud data; a plane fitting module for extracting the point cloud data in the three-dimensional bounding box, selecting target point cloud data from the extracted point cloud data, and performing plane fitting on the target point cloud data corresponding to the three-dimensional bounding box to obtain a fitted plane; an extraction module for extracting a position region corresponding to each target object according to the three-dimensional bounding box; and an update module for identifying the position relationship between each point in the point cloud data and the position region; when the position relationship is that the point is not near the position region, retaining the ground segmentation result corresponding to the points not near the position region in the ground segmentation result; when the position relationship is that the point is located near the position region, calculating a first distance value between the points located near the position region and the fitted plane; comparing the first distance value with a second threshold, determining a first category corresponding to the points located near the position region according to the comparison result; and updating the ground segmentation result according to the first category to obtain a target ground segmentation result corresponding to the point cloud data.

8. The device according to claim 7, characterized in that, the plane fitting module is further configured to divide the extracted point cloud data into a plurality of grids according to preset parameters; select the point with the smallest height value from the point cloud data corresponding to each grid; calculate the height difference between each point in the point cloud data corresponding to each grid and the point with the smallest height value; and select the points with a height difference less than a first threshold, and obtain target point cloud data according to the selected points.

9. A computer device includes a memory and one or more processors, and computer-readable instructions are stored in the memory. When the computer-readable instructions are executed by the one or more processors, the one or more processors are caused to perform the following steps: acquire point cloud data; perform ground segmentation on the point cloud data to obtain a ground segmentation result; perform target detection on the point cloud data to obtain a three-dimensional bounding box of the target object corresponding to the point cloud data; extract the point cloud data in the three-dimensional bounding box, select target point cloud data from the extracted point cloud data, and perform plane fitting on the target point cloud data corresponding to the three-dimensional bounding box to obtain a fitted plane; Extract the position regions corresponding to each target object according to the three-dimensional bounding box; Identify the positional relationship between each point in the point cloud data and the position region; When the positional relationship is that the point is not near the position region, retain the ground segmentation result corresponding to the points in the ground segmentation result that are not near the position region; When the positional relationship is that the point is located near the position region, calculate the first distance value between the points located near the position region and the fitted plane; Compare the first distance value with a second threshold, and determine the first category corresponding to the points located near the position region according to the comparison result; and Update the ground segmentation result according to the first category to obtain the target ground segmentation result corresponding to the point cloud data.

10. The computer device according to claim 9, wherein, when the processor executes the computer-readable instructions, the following steps are further executed: divide the extracted point cloud data into multiple grids according to preset parameters; select the point with the minimum height value in the point cloud data corresponding to each grid; calculate the height difference between each point in the point cloud data corresponding to each grid and the point with the minimum height value; and select the points with the height difference less than a first threshold, and obtain the target point cloud data according to the selected points.

11. The computer device according to claim 10, wherein, when the processor executes the computer-readable instructions, the following steps are further executed: combine the target point cloud data of the multiple grids corresponding to the three-dimensional bounding box to obtain the combined point cloud data corresponding to the three-dimensional bounding box; and perform plane fitting on the combined point cloud data of the three-dimensional bounding box to obtain the fitted plane corresponding to the three-dimensional bounding box.

12. One or more non-transitory computer-readable storage media storing computer-readable instructions, which when executed by one or more processors, cause the one or more processors to perform the following steps: Obtain point cloud data; Perform ground segmentation on the point cloud data to obtain a ground segmentation result; Perform target detection on the point cloud data to obtain the three-dimensional bounding box of the target object corresponding to the point cloud data; Extract the point cloud data in the three-dimensional bounding box, select the target point cloud data from the extracted point cloud data, perform plane fitting on the target point cloud data corresponding to the three-dimensional bounding box to obtain a fitted plane; Extract the position regions corresponding to each target object according to the three-dimensional bounding box; Identify the positional relationship between each point in the point cloud data and the position region; When the positional relationship is that the point is not near the position region, retain the ground segmentation result corresponding to the points in the ground segmentation result that are not near the position region; When the positional relationship is that the point is located near the position region, calculate the first distance value between the points located near the position region and the fitted plane; Compare the first distance value with a second threshold, and determine the first category corresponding to the points located near the position region according to the comparison result; and Update the ground segmentation result according to the first category to obtain the target ground segmentation result corresponding to the point cloud data.

13. The storage medium according to claim 12, wherein, when the computer-readable instructions are executed by the processor, the following steps are further performed: dividing the extracted point cloud data into a plurality of grids according to preset parameters; selecting a point with the minimum height value in the point cloud data corresponding to each grid; calculating the height difference between each point in the point cloud data corresponding to each grid and the point with the minimum height value; and selecting points with a height difference less than a first threshold, and obtaining target point cloud data according to the selected points.

14. The storage medium according to claim 13, wherein, when the computer-readable instructions are executed by the processor, the following steps are further performed: combining the target point cloud data of the plurality of grids corresponding to the three-dimensional bounding box to obtain combined point cloud data corresponding to the three-dimensional bounding box; and performing plane fitting on the combined point cloud data of the three-dimensional bounding box to obtain a fitting plane corresponding to the three-dimensional bounding box.

Images