Method and device for extracting road point cloud based on cloth simulation

Abstract

The present application discloses a method and device for extracting road point cloud based on cloth simulation. The method includes: projecting the first point cloud data of the target road area into a two-dimensional image; determining the boundary of the target road based on fitting the two-dimensional image; determining the target rotation angle based on the boundary; rotating the first point cloud data around the Z-axis based on the target rotation angle to obtain second point cloud data; the direction of the target road in the second point cloud data is parallel to the X-axis; extracting the second point cloud data based on cloth simulation filtering. In this way, based on the road boundary determined by the two-dimensional image, the point cloud data can be rotated to make the road parallel to the X-axis, which can optimize the filtering effect.

Description

Technical Field

[0001] This application relates to the technical field of road monitoring, and particularly to a method and device for extracting road point clouds based on cloth simulation. Background Art

[0002] Currently, airborne lidar (Light Detection and Ranging, LiDAR) has been widely used in fields such as basic surveying and mapping, 3D modeling, and disaster assessment due to its advantages of large measurement range, all-weather operation, and being unaffected by terrain and environmental limitations. Generally, airborne LiDAR is used to collect point cloud data, and the terrain features of roads and their slopes are interpreted from three-dimensional space to achieve monitoring.

[0003] As a novel filtering method, cloth simulation filtering (CSF) separates ground points from non-ground points by simulating the process of cloth covering the ground surface, and has the characteristics of few parameters and strong adaptability. However, in steep slope areas, it cannot fit the terrain well due to the internal force constraints of cloth particles. When there is a deviation between the slope direction of the point cloud and the distribution direction of the cloth grid, the filtering effect is unstable, which is particularly obvious when processing point clouds in road and slope areas, resulting in low accuracy of point cloud extraction. Summary of the Invention

[0004] Embodiments of this application provide a method and device for extracting road point clouds based on cloth simulation to solve the technical problem of low accuracy of point cloud extraction.

[0005] According to the first aspect of the embodiments of this application, a method for extracting road point clouds based on cloth simulation is provided. The method includes: projecting the first point cloud data of the target road area into a two-dimensional image; determining the boundary of the target road based on the two-dimensional image by fitting; determining the target rotation angle based on the boundary; rotating the first point cloud data around the Z axis based on the target rotation angle to obtain second point cloud data; the direction of the target road in the second point cloud data is parallel to the X axis; and extracting the second point cloud data based on cloth simulation filtering.

[0006] Further, the step of rotating the first point cloud data around the Z axis based on the target rotation angle to obtain second point cloud data includes: determining the center coordinate data of the bounding box of the first point cloud data; processing the coordinate data corresponding to the X axis and Y axis based on the target rotation angle and the center coordinate data of the bounding box to determine the rotation transformation matrix; and processing the first point cloud data based on the rotation transformation matrix to obtain second point cloud data.

[0007] Further, determining the boundary of the target road based on the two-dimensional image includes: rearranging all pixels in the two-dimensional image into a two-dimensional array; each row of the two-dimensional array corresponding to the color data of one pixel; clustering all pixels based on the two-dimensional array; extracting the boundary points of the target road based on all the clustered pixels; and determining the boundary of the target road by fitting based on the boundary points.

[0008] Further, clustering all pixels based on the two-dimensional array includes: dividing all color data included in the two-dimensional array into a predetermined number of clusters; dividing each pixel into the cluster with the closest color to the pixel based on the Euclidean distance between the color data corresponding to the pixel and the centroid of the cluster; recalculating the centroid of the cluster after all pixels are divided, and re-dividing the pixels based on the recalculated centroid of the cluster until the change amount of the centroid of the cluster after recalculation is less than a predetermined threshold or the number of divisions reaches a predetermined number.

[0009] Further, determining the boundary of the target road by fitting based on the boundary points includes: arbitrarily selecting two target points from all the boundary points to form a target line; calculating the residuals between the remaining boundary points and the target line, and selecting the boundary points with residuals less than or equal to a preset value to form a point set; determining the point set with the most boundary points among all the boundary points as the target point set; and fitting the boundary points in the target point set based on the least squares method to obtain the boundary of the target road.

[0010] Further, determining the target rotation angle based on the boundary includes: determining the angle between the boundary and the X-axis as the target rotation angle based on the slope of the boundary.

[0011] Further, projecting the first point cloud data of the target road area into a two-dimensional image includes: filtering out noise points in the initial point cloud data of the target road area based on a predetermined elevation interval; determining the slope data of the initial point cloud data after filtering based on the predetermined elevation interval; rendering and coloring the initial point cloud data based on the predetermined elevation interval and / or the slope data to obtain the first point cloud data; and projecting the first point cloud data onto the plane formed by the X-axis and the Y-axis to obtain a two-dimensional image.

[0012] According to the second aspect of the embodiments of the present application, there is also provided a device for extracting road point clouds based on cloth simulation. The device includes: a processing unit configured to project first point cloud data of a target road area into a two-dimensional image; a determination unit configured to determine the boundary of the target road by fitting based on the two-dimensional image; determine a target rotation angle based on the boundary; rotate the first point cloud data around the Z-axis based on the target rotation angle to obtain second point cloud data; the direction of the target road in the second point cloud data is parallel to the X-axis; an extraction unit configured to extract the second point cloud data based on cloth simulation filtering.

[0013] According to the third aspect of the embodiments of the present application, there is also provided a storage medium including a stored program that, when running, executes the steps in the above method.

[0014] According to the fourth aspect of the embodiments of the present application, there is also provided an electronic device including a processor, a communication interface, a memory, and a communication bus. Among them, the processor, the communication interface, and the memory communicate with each other through the communication bus. Among them: the memory is configured to store a computer program; the processor is configured to execute the steps in the above method by running the program stored in the memory.

[0015] The embodiments of the present application also provide a computer program product containing instructions that, when running on a computer, cause the computer to execute the steps in the above method.

[0016] A method for extracting road point clouds based on cloth simulation proposed by the embodiments of the present application projects first point cloud data of a target road area into a two-dimensional image; determines the boundary of the target road by fitting based on the two-dimensional image; determines a target rotation angle based on the boundary; rotates the first point cloud data around the Z-axis based on the target rotation angle to obtain second point cloud data; the direction of the target road in the second point cloud data is parallel to the X-axis; extracts the second point cloud data based on cloth simulation filtering. In this way, by projecting the point cloud data into a two-dimensional image, the road boundary in the image can be more accurately identified. Then, after rotating the point cloud data based on the boundary until the road direction is parallel to the X-axis and then performing cloth simulation filtering, the poor filtering effect caused by the deviation between the road slope direction and the cloth grid direction can be avoided, the filtering effect can be improved, and thus the extraction accuracy of the filtered point cloud can be enhanced. Description of the Drawings

[0017] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation to the present application. In the drawings:

[0018] Figure 1It is a flowchart of a method for extracting road point clouds based on cloth simulation according to an embodiment of the present application;

[0019] Figure 2 It is a schematic diagram of the effect of performing clustering segmentation and road fitting according to an embodiment of the present application;

[0020] Figure 3 It is a schematic flowchart of a method for extracting road slope point clouds optimized based on cloth simulation according to an embodiment of the present application;

[0021] Figure 4 It is a schematic diagram of the effect of two-way cloth simulation filtering according to an embodiment of the present application;

[0022] Figure 5 It is a structural diagram of a device for extracting road point clouds based on cloth simulation according to an embodiment of the present application;

[0023] Figure 6 It is a structural diagram of an electronic device according to an embodiment of the present application. Detailed implementation manners

[0024] In order to enable those skilled in the art to better understand the solution of the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present application. It should be noted that, without conflict, the embodiments in the present application and the features in the embodiments may be combined with each other.

[0025] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above drawings are used to distinguish similar objects, and do not necessarily need to be used to describe a specific order or sequence. It should be understood that such data can be interchanged under appropriate circumstances so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "comprising" and "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, product or device including a series of steps or units does not necessarily need to be limited to those steps or units clearly listed, but may include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0026] In this embodiment, a method for extracting road point clouds based on cloth simulation is provided. Figure 1 It is a flowchart of a method for extracting road point clouds based on cloth simulation according to an embodiment of the present application, as Figure 1As shown, the process includes the following steps:

[0027] S10: Project the first point cloud data of the target road area into a two-dimensional image;

[0028] S20: Determine the boundary of the target road based on fitting the two-dimensional image;

[0029] S30: Determine the target rotation angle based on the boundary;

[0030] S40: Rotate the first point cloud data around the Z-axis based on the target rotation angle to obtain the second point cloud data; the direction of the target road in the second point cloud data is parallel to the X-axis;

[0031] S50: Extract the second point cloud data based on cloth simulation filtering.

[0032] In this embodiment, the target road area may be an area containing the target road, such as an area containing the target road and the slopes around the target road. The first point cloud data may be the point cloud data collected from the target road area by LiDAR.

[0033] In one embodiment, determining the boundary of the target road may refer to determining the boundary line equation of the target road. For example, the boundary line equation may be expressed as y = k * x + b * , where k * is the slope and b * is the intercept, which are constant terms.

[0034] In one embodiment, step S10 may include: filtering out the noise points outside the elevation range of the target scene from the initial point cloud data, and rendering and coloring the initial point cloud data after filtering out the noise points to obtain the first point cloud data; projecting the first point cloud data into a two-dimensional image.

[0035] Among them, filtering out the noise points outside the elevation range of the target scene from the initial point cloud data may include: filtering out the noise points in the initial point cloud data of the target road area based on a predetermined elevation interval.

[0036] Among them, rendering and coloring the initial point cloud data to obtain the first point cloud data may include: determining the slope data of the initial point cloud data after filtering based on the predetermined elevation interval; rendering and coloring the initial point cloud data based on the predetermined elevation interval and / or the slope data to obtain the first point cloud data.

[0037] In one embodiment, after determining the slope data of the initial point cloud data after filtering based on the predetermined elevation range, and before rendering and coloring the initial point cloud data based on the predetermined elevation range and / or the slope data, the method may further include: filtering out noise points outside the predetermined slope range from the initial point cloud data based on the slope data and the predetermined slope range. Here, the predetermined slope range may be a slope threshold, for example, including a slope lower limit value s min and a slope upper limit value s max .

[0038] In one embodiment, rendering and coloring the initial point cloud data based on the predetermined elevation range and / or the slope data may include: rendering and coloring the initial point cloud data based on at least one of the predetermined elevation range, the slope data, and the predetermined slope range.

[0039] Among them, projecting the first point cloud data into a two-dimensional image may include: projecting the first point cloud data onto the plane formed by the X-axis and the Y-axis to obtain a two-dimensional image.

[0040] In one embodiment, rotating around the Z-axis may refer to rotating around the Z-axis with the center of the bounding box of the first point cloud data as the origin while keeping the elevation coordinate unchanged. For example, rotating around the Z-axis may refer to translating the center of the bounding box of the first point cloud data to the origin of the original coordinate system and rotating around this origin while keeping the Z coordinate unchanged. The orientation of the target road is parallel to the X-axis, which may indicate that the slope direction (i.e., the aspect) of the target road remains unchanged along the X-axis direction, or the change amplitude or change ratio of the slope direction is lower than the preset lower limit value.

[0041] In this way, by projecting the point cloud data into a two-dimensional image, the road boundary in the image can be recognized more accurately. Then, after rotating the point cloud data to make the road orientation parallel to the X-axis based on the boundary and performing cloth simulation filtering, the poor filtering effect caused by the deviation between the road slope direction and the cloth grid direction can be avoided, the filtering effect can be improved, and further the extraction accuracy of the filtered point cloud can be enhanced.

[0042] In some embodiments, the step S40 may include:

[0043] Determining the coordinate data of the center of the bounding box of the first point cloud data;

[0044] Based on the target rotation angle and the coordinate data of the center of the bounding box, processing the coordinate data corresponding to the X-axis and the Y-axis to determine the rotation transformation matrix;

[0045] Processing the first point cloud data based on the rotation transformation matrix to obtain the second point cloud data.

[0046] In one embodiment, the bounding box center may refer to the geometric center of the bounding box of the first point cloud data. For example, the coordinate data of the bounding box center may be (c x , c y , c z ).

[0047] In one embodiment, determining the coordinate data of the bounding box center of the first point cloud data may include: determining the coordinate data of the bounding box center of the first point cloud data based on the maximum and minimum coordinate values on the X-axis, Y-axis, and Z-axis in the first point cloud data. For example, min(x) is the minimum coordinate value on the X-axis, max(x) is the maximum coordinate value on the X-axis, min(y) is the minimum coordinate value on the Y-axis, max(y) is the maximum coordinate value on the Y-axis, min(z) is the minimum coordinate value on the Z-axis, and max(z) is the maximum coordinate value on the Z-axis. Then, the coordinate data of the bounding box center can be expressed as:

[0048]

[0049] In one embodiment, the coordinate data corresponding to the first point cloud data before rotation transformation can be expressed as (x, y, z), and the coordinate data corresponding to the second point cloud data can be expressed as (x', y', z'). Then, the homogeneous coordinate data corresponding to the first point cloud data is (x, y, z, 1), and the homogeneous coordinate data corresponding to the second point cloud data is (x', y', z', 1).

[0050] Therefore, the rotation transformation matrix can be expressed as:

[0051]

[0052] where α d is the target rotation angle. Based on the target rotation angle and the coordinate data of the bounding box center may refer to based on the target rotation angle and the coordinate data of the bounding box center on the X-axis and Y-axis.

[0053] In one embodiment, processing the first point cloud data based on the rotation transformation matrix to obtain the second point cloud data may include: multiplying the first point cloud data by the rotation transformation matrix to obtain the second point cloud data.

[0054] In this way, processing the coordinate data of the X-axis and Y-axis based on the bounding box center can achieve rotation around the Z-axis, and processing based on the center of the point cloud data can maintain the stability of the rotation and improve the accuracy of the rotation processing.

[0055] In some embodiments, the determining the boundary of the target road by fitting based on the two-dimensional image includes:

[0056] Rearrange all the pixels in the two-dimensional image into a two-dimensional array; each row of the two-dimensional array corresponds to the color data of one of the pixels;

[0057] Cluster all the pixels based on the two-dimensional array;

[0058] Extract the boundary points of the target road based on all the pixels after clustering;

[0059] Determine the boundary of the target road by fitting based on the boundary points.

[0060] In one embodiment, each row in the two-dimensional array represents the color data of one pixel. For example, the color data can be the LAB value, where L represents the luminance channel, and A and B represent the color channels. For example, before step S20, the method may further include: converting the RGB channel data of the two-dimensional image into color data, i.e., the LAB value. In this way, it is converted from the RGB color space to the LAB color space with higher robustness. Its characteristic of separating luminance and color makes it perform better in scenarios that require precise color analysis and is more suitable for image segmentation tasks that need to simulate human vision.

[0061] In one embodiment, due to the sparsity of the point cloud, there will inevitably be pores in the projected two-dimensional image, and not all regions in the input top view may have point cloud distribution. Therefore, it is necessary to filter the white or transparent non-point cloud pixel regions and perform morphological optimization on the filtered image to fill the image holes and smooth the boundaries.

[0062] Therefore, after converting the RGB channel data of the two-dimensional image into color data, i.e., the LAB value, and before step S20, the method may further include: performing a morphological inverse operation including a dilation operation and an erosion operation on the converted two-dimensional image based on a morphological operation kernel with a predetermined size and a predetermined shape. The predetermined shape can be an ellipse, etc., and the predetermined size can be 7x7.

[0063] In one embodiment, the morphological operation kernel can select rectangular kernels, cross-shaped kernels or elliptical kernels with different sizes to adapt to different image processing scenarios.

[0064] Among them, the dilation operation can make the target area "become larger", merge the background points in contact with the target area into the target object, and expand the target boundary outward. It can be used to fill some holes in the target area and eliminate the fine-grained noise contained in the target area. The erosion operation, on the other hand, can make the target area "become smaller". Its essence is to cause the boundary of the image to shrink and can be used to eliminate small and meaningless target objects.

[0065] In one embodiment, the expression of the dilation operation is as follows:

[0066]

[0067] The expression of the erosion operation is as follows:

[0068]

[0069] In the formula, ⊕ represents the dilation operation, ⊙ represents the erosion operation, D is the two-dimensional image to be processed, and C is the structuring element, which is the core for performing these operations.

[0070] In one embodiment, clustering all pixels may refer to performing clustering segmentation on the two-dimensional image based on all pixels in the two-dimensional image. Extracting the boundary points of the target road based on all pixels after clustering may refer to extracting the boundary points of the target road based on the label results corresponding to all pixels after clustering, where the label results may characterize the cluster where they are located, etc. For example, according to the pixel label results after clustering segmentation, traverse row by row to extract the boundary points. Fitting based on the boundary points may refer to using the RANSAC algorithm to fit the road boundary based on the boundary points.

[0071] In one embodiment, the clustering all pixels based on the two-dimensional array includes:

[0072] Dividing all color data included in the two-dimensional array into a predetermined number of clusters;

[0073] Based on the Euclidean distance between the color data corresponding to each pixel and the centroid of the cluster, dividing the pixel into the cluster with the closest color to the pixel;

[0074] After all pixels are divided, recalculate the centroid of the cluster, and re-divide the pixels based on the recalculated centroid of the cluster until the change amount of the centroid of the cluster after recalculation is less than a predetermined threshold or the number of divisions reaches a predetermined number.

[0075] In one embodiment, the predetermined number K may be 2 to distinguish the road and other areas. The centroid can be expressed as the mean of the LAB three channels within the cluster.

[0076] Here, the color data may be LAB values. The Euclidean distance between the LAB values and the centroid of the cluster may refer to the Euclidean distance between the LAB values and the centroid of the cluster in the LAB color space. The cluster with the closest color to the pixel may refer to the cluster corresponding to the centroid with the smallest Euclidean distance between the LAB values of the pixel, and the pixels within a cluster are pixels with the closest color.

[0077] In one embodiment, clustering all pixels based on the two-dimensional array may refer to clustering all pixels based on the two-dimensional array using the K-means algorithm. The core idea of the K-means algorithm is to partition the data set into K clusters by iteratively optimizing the positions of the cluster centroids, maximizing the similarity of data points within the same cluster and the difference between different clusters.

[0078] In the LAB color space, the objective function SSE is to minimize the difference in LAB values of all pixels px within the clusters, i.e., the SSE value:

[0079]

[0080] In the formula, u i =[u iL , u iA , u iB represents the centroid (the average value of the three LAB channels) of the i-th cluster C i , where i is a positive integer less than or equal to K, L px is the L-channel value of the pixel px, A px is the A-channel value of the pixel px, and B px is the B-channel value of the pixel px. px i =[p iL , p iA , p iB represents the LAB value of a single pixel.

[0081] In one embodiment, the average values of the three LAB channels of each cluster centroid are calculated independently, and the three channel values corresponding to the centroid are calculated as follows:

[0082]

[0083] In one embodiment, the Euclidean distance between the color data corresponding to each pixel and the centroid of the cluster may refer to the Euclidean distance based on the three channels of the color data between each pixel and the centroid of the cluster. The basis for assigning a pixel to the nearest cluster is the Euclidean distance in the LAB color space:

[0084]

[0085] Here, p refers to a pixel. The Euclidean distance d from each pixel to each centroid is calculated, and after it is assigned to the nearest cluster, i.e., the cluster with the minimum Euclidean distance, the average LAB value within the cluster, i.e., the centroid, is recalculated. In this way, the cluster centroid is continuously updated until the change in the centroid position is less than the threshold or the maximum number of iterations is reached, at which point the iteration terminates.

[0086] In one embodiment, re - partitioning the pixels based on the centroids of the recalculated clusters may refer to partitioning the pixels into the cluster with the minimum Euclidean distance based on the Euclidean distance between the color data corresponding to each pixel and the centroids of the recalculated clusters.

[0087] In one embodiment, the change amount of the centroid after recalculation may refer to the change amount of the color data of the centroid after recalculation compared with the color data of the centroid before the current recalculation. Here, the change amount may refer to the changed value or the changed ratio.

[0088] In one embodiment, the number of partitioning times may refer to the number of times of completing the partitioning of all pixels, or may also refer to the number of times of re - partitioning all pixels based on the centroids of the recalculated clusters.

[0089] In this way, partitioning the pixels based on the centroids of the clusters calculated from the three channels of LAB values can better achieve the segmentation of the two - dimensional image, which is conducive to accurately determining the target road therein.

[0090] In some embodiments, determining the boundary of the target road based on the boundary points includes:

[0091] Selecting any two target points from all the boundary points to form a target straight line;

[0092] Calculating the residuals between the remaining boundary points and the target straight line, and selecting the boundary points with residuals less than or equal to a preset value to form a point set;

[0093] Determining the point set with the most boundary points among all the boundary points as the target point set;

[0094] Fitting the boundary points in the target point set based on the least - squares method to obtain the boundary of the target road.

[0095] In one embodiment, as Figure 2 shown is the result comparison after clustering segmentation and road boundary fitting of the original image, i.e., the two - dimensional image. For the road scene, the RANSAC algorithm is used to perform the fitting task of the target straight line of the form y = kx + b.

[0096] Randomly selecting 2 points (x i , y i ) from all the boundary points, i.e., the boundary point set bp={(x i , y i )} as the sampling point set (two points determine a straight line), and calculating the parameters of the target straight line using the sampling point set S={(x1, y1), (x2, y2)}, where k is the slope and b is the intercept:

[0097]

[0098] Calculate the residuals r between the remaining boundary points and the target line i , and set a preset value ε = 3 to filter out the point set I:

[0099]

[0100]

[0101] Determine that the point set with the most boundary points among all the boundary points is the target point set, that is, record the one with the largest number of points in the point set as the target point set.

[0102] In one embodiment, if the number of points |I*| in the current point set > |I|, update the optimal parameters θ*=(k*, b*):

[0103]

[0104] When the number of iterations T reaches the theoretically calculated value or the preset maximum value T max = 100, stop:

[0105]

[0106] Among them, p is the expected success rate, usually set to 0.99; ω is the inlier ratio, that is, the ratio of the number of points |I| in the point set to the total number of boundary points N, which can be set by prior estimation or updated dynamically.

[0107] In one embodiment, based on the least squares method, fit the boundary points in the target point set to obtain the boundary of the target road, which may include: select the model with the largest number of points in the point set as the optimal candidate, and refit the line using all the boundary points in the point set by the least squares method:

[0108]

[0109]

[0110] Among them, ,

[0111] , from which the boundary line, that is, the equation of the boundary, can be fitted as y = k*x + b*.

[0112] In one embodiment, determining the target rotation angle based on the boundary includes: determining the angle between the boundary and the X-axis as the target rotation angle based on the slope of the boundary.

[0113] Among them, determining the angle between the boundary and the X-axis as the target rotation angle based on the slope of the boundary, that is, further obtaining the angle α between the boundary line and the x-axis according to the slope k* d , for subsequent point cloud rotation:

[0114]

[0115]

[0116] Among them, α d ranges from (−90°, 90°). When the slope k* → ∞, the line is perpendicular to the X-axis, and at this time α d = ±90°; when k* = 0, the line is parallel to the X-axis, and α d = 0°.

[0117] Here, a negative target rotation angle represents clockwise rotation, and in the image coordinate system, the origin is located in the upper left corner, and the Y-axis direction is downward.

[0118] In this way, the angle between the boundary determined based on the boundary line fitting and the X-axis is used as the angle to be rotated, and the point cloud data can be accurately rotated, so as to facilitate the parallelism between the road boundary in the point cloud data and the X-axis, so as to accurately filter and improve the extraction accuracy.

[0119] In some embodiments, the projecting the first point cloud data of the target road area into a two-dimensional image includes:

[0120] Filtering out noise points in the initial point cloud data of the target road area based on a predetermined elevation range;

[0121] Determining slope data of the filtered initial point cloud data based on the predetermined elevation range;

[0122] Rendering and coloring the initial point cloud data based on the predetermined elevation range and / or the slope data to obtain the first point cloud data;

[0123] Projecting the first point cloud data onto the plane formed by the X-axis and the Y-axis to obtain a two-dimensional image.

[0124] In one embodiment, for the initial point cloud data collected by airborne LiDAR, first filter out noise points outside the target scene elevation range, that is, the predetermined elevation range, to complete the first step of rough denoising:

[0125]

[0126] In the formula, P represents the input original point set, that is, the initial point cloud data, P1 represents the point set after the first step of denoising, that is, the filtered initial point cloud data, z max is the elevation value of the highest road surface, and z min is the elevation value of the lowest ground. These two values can be roughly set by visual discrimination, and most of the noise points far from the terrain surface can be filtered to avoid interfering with the subsequent cloth simulation filtering.

[0127] In one embodiment, filtering out the noise points in the initial point cloud data of the target road area based on a predetermined elevation range is the first step of denoising. After determining the slope data of the initial point cloud data after filtering based on the predetermined elevation range, and before rendering and coloring the initial point cloud data based on the predetermined elevation range and / or the slope data, the method may further include a second step of denoising, that is, including: filtering out the noise points outside the predetermined slope range from the initial point cloud data based on the slope data and the predetermined slope range. Here, the predetermined slope range may be a slope threshold, for example, including a slope lower limit value s min and a slope upper limit value s max .

[0128] In one embodiment, based on the predetermined elevation range and / or the slope data, it may include: based on at least one of the predetermined elevation range, the slope data, and the predetermined slope range.

[0129] In one embodiment, in the second step of denoising, determining the slope data of the initial point cloud data after filtering based on the predetermined elevation range may refer to further calculating the point cloud slope s according to the above-mentioned predetermined elevation range, and then generating a slope statistical histogram of the initial point cloud data after filtering. Optionally, after determining the slope data of the initial point cloud data after filtering based on the predetermined elevation range, a slope threshold may also be set to eliminate the slope abnormal points, further suppressing the influence of the noise points on the subsequent cloth simulation filtering.

[0130]

[0131]

[0132] In the formula, (x1, y1, z1), (x2, y2, z2) represent the coordinates of two adjacent points, P2 represents the point set after the second-step slope filtering denoising, s min and s max are the slope lower limit value and the slope upper limit value included in the slope threshold.

[0133] In some embodiments, rendering and coloring the three-dimensional point cloud according to the predetermined elevation range and / or the slope data, that is, the above-mentioned elevation value or slope value, so that the road area can be roughly distinguished. Rendering and coloring the initial point cloud data may include rendering the point cloud according to the scalar field in point cloud data processing software such as CloudCompare software, and the color scale selection is [blue > green > yellow > red].

[0134] As a possible implementation manner, a method for extracting the point cloud of the road slope based on cloth simulation optimization is provided, and the specific process is as Figure 3 shown.

[0135] Among them, for the extraction of the second point cloud data based on cloth simulation filtering, the second point cloud data can be inverted based on the CSF algorithm, simulating a piece of cloth falling from above under gravity, and the final shape of the cloth is the digital surface model. Based on this digital surface model, the second point cloud data can be extracted.

[0136] In one embodiment, the target road area may include the target road and the slopes around the target road. For the case where the roadside of the target road in the target road area is higher than the slope, the extraction of the second point cloud data based on cloth simulation filtering may include: after filtering out the noise points below the target road in the second point cloud data based on reverse cloth simulation filtering, then filtering out the noise points above the target road in the second point cloud data based on forward cloth simulation filtering; performing extraction based on the second point cloud data after filtering out the noise points.

[0137] In one embodiment, for the case where the roadside of the target road in the target road area is lower than the slope, the extraction of the second point cloud data based on cloth simulation filtering may include: after filtering out the noise points above the target road in the second point cloud data based on forward cloth simulation filtering, then filtering out the noise points below the target road in the second point cloud data based on reverse cloth simulation filtering; performing extraction based on the second point cloud data after filtering out the noise points. In this way, more accurate extraction can be performed for road areas with different concave and convex terrains.

[0138] As Figure 4 shown, perform bidirectional cloth filtering on the original section point cloud, that is, the second point cloud data. First perform reverse CSF and then perform forward CSF. Finally, obtain the filtered point cloud, and extraction can be performed based on the filtered point cloud. For the point cloud of the convex terrain where the road surface is higher than the slope, considering that the road slope of the convex terrain will become a concave terrain during the forward filtering of CSF (i.e., inverting the point cloud), first perform reverse filtering to filter out the noise points below the road surface, and then perform forward filtering to filter out the noise above the road surface. The road slope of the concave terrain is the opposite.

[0139] Exemplarily, the specific steps for extracting the second point cloud data based on cloth simulation filtering are as follows:

[0140] 1. Flip the point cloud.

[0141] 2. Determine the number of cloth particles according to the defined grid resolution, set the cloth stiffness, and initialize the cloth above the highest point.

[0142] 3. Project the point cloud and the cloth particles onto the x - y plane, find the nearest point to each cloth particle in the plane, and record the elevation data IHV of this point.

[0143] 4. For each fabric particle, if the particle is movable, calculate and update its position under the influence of gravity (the time step for the gravity effect is defaulted to 0.65 seconds), and compare the height of the fabric particle with the IHV. If the particle height is less than or equal to the IHV, place the particle at the height of the IHV and set it to "immovable".

[0144] 5. For each fabric particle, calculate its displacement under the influence of internal forces.

[0145] 6. Repeat steps 4 - 5. When the maximum height change of all particles is small enough or exceeds the specified maximum number of iterations, the simulation process terminates.

[0146] 7. Calculate the distance between the fabric particles and the LiDAR point cloud.

[0147] 8. If the distance from the point to the fabric particle is less than the classification threshold, the point is classified as a ground point; otherwise, it is classified as a non - ground point.

[0148] Due to situations such as secondary echo reflections during the LiDAR data acquisition process, abnormal points below the road surface will inevitably appear in the measured terrain, resulting in noise above the road surface and noise below the road surface in the acquired point cloud. Especially for road slopes with concave - convex terrains, there may be a large number of abnormal points in the concave parts, while relatively fewer abnormal points above the convex parts.

[0149] In one embodiment, when processing the target road and its slope area as described above, slope smoothing can be enabled, the fabric grid resolution can be set to about 0.5, a relatively soft value can be selected for the fabric stiffness, the classification threshold can be set to about 0.5 according to the thickness of the second point cloud data, and default values can be selected for the time step and the number of iterations, etc.

[0150] Through the description of the above - mentioned implementation manners, those skilled in the art can clearly understand that the method according to the above - mentioned embodiments can be implemented by means of software plus a necessary general - purpose hardware platform. Of course, it can also be implemented by hardware, but in many cases, the former is a better implementation manner. Based on such an understanding, the technical solution of the present invention, in essence, or the part that makes a contribution to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disc) and includes several instructions for causing a terminal device (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in various embodiments of the present invention.

[0151] In this embodiment, a device is also provided to implement the above embodiments and preferred implementation manners, and those that have been described will not be repeated. As used hereinafter, the term "unit" can be a combination of software and / or hardware that can achieve a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, implementation in hardware, or a combination of software and hardware is also possible and contemplated.

[0152] Figure 5 is a structural block diagram of a road point cloud extraction device based on cloth simulation according to an embodiment of the present application. As Figure 5 shown, the device includes:

[0153] A processing unit 100, configured to project the first point cloud data of a target road area into a two-dimensional image;

[0154] A determination unit 200, configured to determine the boundary of the target road based on fitting of the two-dimensional image; determine a target rotation angle based on the boundary; rotate the first point cloud data around the Z-axis based on the target rotation angle to obtain second point cloud data; the direction of the target road in the second point cloud data is parallel to the X-axis;

[0155] An extraction unit 300, configured to extract the second point cloud data based on cloth simulation filtering.

[0156] In some embodiments, the determination unit 200 is configured to: determine the center coordinate data of the bounding box of the first point cloud data; process the coordinate data corresponding to the X-axis and Y-axis based on the target rotation angle and the center coordinate data of the bounding box to determine a rotation transformation matrix; process the first point cloud data based on the rotation transformation matrix to obtain second point cloud data.

[0157] In some embodiments, the determination unit 200 is configured to: rearrange all pixels in the two-dimensional image into a two-dimensional array; each row of the two-dimensional array corresponds to the color data of one pixel; cluster all pixels based on the two-dimensional array; extract the boundary points of the target road based on the clustered all pixels; determine the boundary of the target road based on fitting of the boundary points.

[0158] In some embodiments, the determination unit 200 is configured to: divide all color data included in the two-dimensional array into a predetermined number of clusters; based on the Euclidean distance between the color data corresponding to each pixel and the centroid of the cluster, divide the pixel into the cluster with the closest color to the pixel; after all pixels are divided, recalculate the centroid of the cluster, and re-divide the pixels based on the recalculated centroid of the cluster until the change amount of the centroid of the cluster after recalculation is less than a predetermined threshold or the number of divisions reaches a predetermined number.

[0159] In some embodiments, the determining unit 200 is configured to: randomly select two target points from all the boundary points to form a target straight line; calculate the residuals between the remaining boundary points and the target straight line, and select the boundary points with residuals less than or equal to a preset value to form a point set; determine the point set with the largest number of boundary points among all the boundary points as the target point set; and fit the boundary points in the target point set based on the least squares method to obtain the boundary of the target road.

[0160] In some embodiments, the determining unit 200 is configured to: determine, based on the slope of the boundary, the angle between the boundary and the X-axis as the target rotation angle.

[0161] In some embodiments, the processing unit 100 is configured to: filter out noise points in the initial point cloud data of the target road area based on a predetermined elevation range; determine the slope data of the initial point cloud data after filtering based on the predetermined elevation range; render and color the initial point cloud data based on the predetermined elevation range and / or the slope data to obtain first point cloud data; and project the first point cloud data onto the plane formed by the X-axis and the Y-axis to obtain a two-dimensional image.

[0162] It should be noted that the above-mentioned various modules can be implemented by software or hardware. For the latter, it can be implemented in the following ways, but not limited to this: all the above-mentioned modules are located in the same processor; or, the above-mentioned various modules are separately located in different processors in any combination form.

[0163] The embodiment of the present application further provides an electronic device, which can be a robot or applied in a robot. Figure 6 is a structural diagram of an electronic device according to an embodiment of the present application, as Figure 6 shown, including a processor 41, a communication interface 42, a memory 43, and a communication bus 44. Among them, the processor 41, the communication interface 42, and the memory 43 complete mutual communication through the communication bus 44. The memory 43 is used to store a computer program; when the processor 41 executes the program stored on the memory 43, it implements the steps of the method described in any one or more of the foregoing method embodiments.

[0164] The communication bus mentioned in the above terminal may be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into an address bus, a data bus, a control bus, etc. For the sake of convenience of representation, only a thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.

[0165] The communication interface is used for communication between the above-mentioned terminal and other devices.

[0166] The memory may include a Random Access Memory (RAM), or may also include a non-volatile memory, such as at least one disk memory. Optionally, the memory may also be at least one storage device located away from the aforementioned processor.

[0167] The above-mentioned processor may be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc.; it may also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0168] In another embodiment provided by the present application, there is also provided a computer-readable storage medium, in which instructions are stored. When it runs on a computer, it causes the computer to execute any one of the methods described in the above embodiments.

[0169] In another embodiment provided by the present application, there is also provided a computer program product containing instructions. When it runs on a computer, it causes the computer to execute any one of the methods described in the above embodiments.

[0170] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the processes or functions described in the embodiments of the present application are generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (such as infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or a data center that includes one or more integrated available media. The available medium can be a magnetic medium (for example, a floppy disk, a hard disk, a magnetic tape), an optical medium (for example, a DVD), or a semiconductor medium (for example, a solid state disk (SSD)), etc.

[0171] The foregoing is only a preferred embodiment of the present application and is not intended to limit the protection scope of the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present application are included in the protection scope of the present application.

[0172] The foregoing is only the specific implementation manner of the present application, enabling those skilled in the art to understand or implement the present application. Various modifications to these embodiments will be obvious to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will be accorded the widest scope consistent with the principles and novel features claimed herein.

Claims

1. A method for extracting road point clouds based on cloth simulation, characterized in that, The method includes: Projecting the first point cloud data of the target road area into a two-dimensional image, where projecting the first point cloud data of the target road area into a two-dimensional image includes: filtering out noise points in the initial point cloud data of the target road area based on a predetermined elevation range; determining slope data of the filtered initial point cloud data based on the predetermined elevation range; rendering and coloring the initial point cloud data based on the predetermined elevation range and / or the slope data to obtain the first point cloud data; projecting the first point cloud data onto a plane formed by the X-axis and the Y-axis to obtain a two-dimensional image; Determining the boundary of the target road by fitting based on the two-dimensional image, where determining the boundary of the target road by fitting based on the two-dimensional image includes: rearranging all pixels in the two-dimensional image into a two-dimensional array; each row of the two-dimensional array corresponding to the color data of one pixel; clustering all pixels based on the two-dimensional array; extracting boundary points of the target road based on the clustered pixels; determining the boundary of the target road by fitting based on the boundary points; Determining a target rotation angle based on the boundary, where determining a target rotation angle based on the boundary includes: determining an angle between the boundary and the X-axis as the target rotation angle based on the slope of the boundary; Rotating the first point cloud data around the Z-axis based on the target rotation angle to obtain a second point cloud data; the direction of the target road in the second point cloud data is parallel to the X-axis; Extracting the second point cloud data based on cloth simulation filtering.

2. The method according to claim 1, wherein The step of rotating the first point cloud data around the Z-axis based on the target rotation angle to obtain a second point cloud data includes: Determining the center coordinate data of the bounding box of the first point cloud data; Processing the coordinate data corresponding to the X-axis and the Y-axis based on the target rotation angle and the center coordinate data of the bounding box to determine a rotation transformation matrix; Processing the first point cloud data based on the rotation transformation matrix to obtain a second point cloud data.

3. The method according to claim 1, wherein The clustering of all pixels based on the two-dimensional array includes: Dividing all color data included in the two-dimensional array into a predetermined number of clusters; Dividing each pixel into the cluster with the closest color to the pixel based on the Euclidean distance between the color data corresponding to the pixel and the centroid of the cluster; After all pixels are divided, recalculating the centroid of the cluster and re-dividing the pixels based on the recalculated centroid of the cluster until the change amount of the centroid of the cluster after recalculation is less than a predetermined threshold or the number of divisions reaches a predetermined number.

4. The method according to claim 1, wherein The determining the boundary of the target road by fitting based on the boundary points includes: Selecting any two target points from all the boundary points to form a target line; Calculating the residuals between the remaining boundary points and the target line, and selecting the boundary points with residuals less than or equal to a preset value to form a point set; Determining the point set with the most boundary points among all the boundary points as the target point set; Fitting the boundary points in the target point set based on the least squares method to obtain the boundary of the target road.

5. A road point cloud extraction device based on cloth simulation, characterized in that The device includes: A processing unit for projecting the first point cloud data of a target road area into a two-dimensional image, wherein the processing unit is configured to: filter out noise points in the initial point cloud data of the target road area based on a predetermined elevation range; determine slope data of the filtered initial point cloud data based on the predetermined elevation range; perform rendering and coloring on the initial point cloud data based on the predetermined elevation range and / or the slope data to obtain first point cloud data; project the first point cloud data onto a plane formed by the X-axis and the Y-axis to obtain a two-dimensional image; A determining unit for determining the boundary of the target road by fitting based on the two-dimensional image; determining a target rotation angle based on the boundary; rotating the first point cloud data around the Z-axis based on the target rotation angle to obtain second point cloud data; the orientation of the target road in the second point cloud data is parallel to the X-axis, and the determining unit is configured to: rearrange all pixels in the two-dimensional image into a two-dimensional array; each row of the two-dimensional array corresponds to the color data of one of the pixels; perform clustering on all pixels based on the two-dimensional array; extract boundary points of the target road based on all the clustered pixels; determine the boundary of the target road by fitting based on the boundary points; the determining unit is further configured to: determine the angle between the boundary and the X-axis as the target rotation angle based on the slope of the boundary; An extraction unit for extracting the second point cloud data by cloth simulation filtering.

6. An electronic device, characterized in that, The electronic device includes a memory, a processor, and a computer program stored on the memory and executable on the processor. When the computer program is executed by the processor, the steps of the method for extracting road point clouds based on cloth simulation according to any one of claims 1 to 5 are implemented.

7. A storage medium, characterized in that, A computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the method for extracting road point clouds based on cloth simulation according to any one of claims 1 to 5 are implemented.

Images