Grid-based multi-source data ground extraction method

Abstract

The invention discloses a multi-source data ground extraction method based on grids, and the method comprises the steps: calibrating multi-source data from a laser radar and a depth camera in a same coordinate system, and converting the multi-source data into point cloud data; dividing regional grids according to a point cloud data range, calculating point cloud features, and carrying out plane fitting by taking points meeting a set threshold condition as seed points; and setting points which do not meet a seed point threshold value in the region as to-be-recovered points until plane fitting of all regions is completed. And finally, according to the obtained plane models of all the areas, counting all points meeting ground conditions, and completing ground extraction. According to the method, a larger sensing range and effective data can be obtained, meanwhile, a plurality of plane models can be obtained according to different areas, the robustness of an algorithm under the condition that the ground is uneven in the transverse direction and the longitudinal direction is improved, and a more accurate extraction result is obtained.

Description

Technical Field

[0001] The invention relates to the field of geographic information technology, and in particular to a multi-source data ground extraction method based on a grid. Background Art

[0002] In the process of using multiple sensor data to perceive the surrounding environment, accurate segmentation of ground areas and non-ground areas is the basis. However, existing algorithms have many shortcomings: on the one hand, most of them use direct plane fitting in the data range to extract the ground; on the other hand, they only obtain data through a single sensor. These methods are time-consuming due to their reliance on manual features and the number of iterations. In addition, when the ground area is uneven in the horizontal and vertical directions, the ground extraction effect is poor and it is difficult to meet actual needs.

[0003] Therefore, in order to improve the completeness of ground extraction and the robustness to the ground area, it is necessary to propose a new ground extraction method. Summary of the invention

[0004] In view of the deficiencies of the prior art, the present invention discloses a raster-based multi-source data ground extraction method to solve the problems raised in the above background technology.

[0005] To achieve the above object, the present invention provides the following technical solution: a grid-based multi-source data ground extraction method, comprising the following steps:

[0006] S1, calibrate the multi-source data from the lidar and depth camera in the same coordinate system and convert them into point cloud data;

[0007] S2, divide the regional grid according to the point cloud data range, calculate the point cloud features, take the points that meet the set threshold conditions as seed points, and perform plane fitting;

[0008] S3, setting the points in the region that do not meet the seed point threshold as points to be restored until all regions complete the plane fitting;

[0009] S4. Based on the plane models obtained from all areas, count all points that meet the ground conditions to complete ground extraction.

[0010] Preferably, step S1 specifically comprises the following steps:

[0011] 1) First, convert the data obtained by the depth camera into a point cloud format. The mapping relationship between the world coordinates and the camera pixels includes:

[0012]

[0013] Where u and v are arbitrary coordinate points in the image coordinate system, u 0 , v0 are the center coordinates of the image, x w ,y w , z w Represents the three-dimensional coordinate point in the world coordinate system, R and T are the 3x3 rotation matrix and 3x1 translation matrix of the external parameter matrix, f represents the focal length of the camera, dx and dy represent the actual physical value represented by a pixel in the x direction and y direction respectively;

[0014] 2) Since the world coordinate origin coincides with the camera origin, the transformation formula from the image point to the world coordinate point is calculated:

[0015]

[0016] Among them, z c Indicates the z-axis value of the camera coordinates, that is, the distance from the target to the camera.

[0017] Preferably, step S2 specifically comprises the following steps:

[0018] 1) According to the actual point cloud data range, the point cloud data is divided into regions and grids. The width and height of the data range are cloud_width and cloud_height respectively, the number of regions is zone_number, the number of vertical grids in each region is ring_number, and the number of horizontal grids is sector_number. The region boundary is:

[0019]

[0020] Among them, the grid height Grid width

[0021] While dividing the area, filter out the points with a depth value of zero and outside the perception range within the data range to reduce the amount of calculation. Then traverse each point and calculate the indexes zone_idx, ring_idx, sector_idx according to the coordinates in the image, where:

[0022]

[0023] sector_idx=min((u-zone_bound[zone_idx]) / sector_size,(sector_number-1)))

[0024] ring_idx=min(v / ring_size,(ring_number-1))

[0025] 2) After the grid division is completed, the average height, standard deviation, and normal vector features of the point cloud are calculated respectively, and divided into seed points according to the set feature threshold to prepare for plane fitting.

[0026] Preferably, the average height, standard deviation, and normal vector characteristics of the point cloud are calculated, specifically including:

[0027] Note the average height:

[0028]

[0029] Standard Deviation:

[0030]

[0031] Covariance:

[0032]

[0033] Mean distance:

[0034]

[0035] Line variables:

[0036]

[0037] angle:

[0038] θ=cos -1 (n 1 ·n 2 )

[0039] Where N represents the total number of points in the current grid, z i Represents the z-axis coordinate of the i-th point in the current grid, represents the average value of all point coordinates, v i Represents the vector from the point to the center point; perform SVD decomposition on the covariance to obtain eigenvalues ​​and eigenvectors, define the line variable as the ratio between the two main eigenvalues, and use it to describe the linearity and planarity of the point; define the angle as the angle between the eigenvector and the unit normal vector, and use it to describe the deviation; finally, complete the screening of seed points under the constraints of multiple features.

[0040] Preferably, step S3 specifically comprises the following steps:

[0041] Before performing plane fitting on the seed points, a plane model is given first:

[0042] Ax+By+Cz+D=0

[0043] Then, the Householder QR decomposition is used to solve the linear equation system Ax=b. For each column of the matrix A, the Householder matrix is ​​constructed where w j is a vector extracted from the jth column of matrix A and the processed column, and I is the identity matrix;

[0044] The Householder transformation is applied to the matrix A to convert the matrix A into an upper triangular matrix R, and at the same time, an orthogonal matrix Q is constructed; the decomposition is expressed as A = QR, and the transpose of the orthogonal matrix Q is used to calculate y = Q T b, where y is the transformed constant term vector;

[0045] Since R is an upper triangular matrix, use the back-substitution method to solve x from back to front:

[0046]

[0047] Among them, r ij is the element of matrix R, y i is the i-th element of the vector y, and the solved x value is the parameter in the plane equation.

[0048] Preferably, step S4 specifically comprises the following steps:

[0049] After obtaining the regional plane model, first traverse the points in the region and calculate the distance between the points and the plane:

[0050]

[0051] If it is within the set threshold, it is counted as a plane point, otherwise it is recorded as a point to be restored;

[0052] If there are not enough seed points in the area, resulting in plane fitting failure, all points in the area will be recorded as points to be recovered;

[0053] Finally, the plane models of all areas are traversed, and the distance values ​​of all points to be restored between the plane models are counted. If the distance is less than the set threshold, the point is recorded through the index and counted into the plane points in the final result. When the ground models are merged, the segmentation between the ground area and the non-ground area is finally completed, and the ground extraction is completed.

[0054] Compared with the prior art, the present invention has the following beneficial effects:

[0055] 1. In the present invention, multi-source data is used to replace a single data source, so as to reduce the influence of the external environment on a single sensor during data acquisition and obtain a larger perception range and effective data.

[0056] 2. In the present invention, the grid is subdivided on the basis of the divided area, and then the plane model is fitted according to the seed points, and the invalid points in the depth camera are filtered out, which effectively reduces the amount of data and the number of iterations during fitting and improves real-time performance.

[0057] 3. In the present invention, multiple plane models are obtained according to different regions, which improves the robustness of the algorithm when the ground is uneven in the horizontal and vertical directions, and obtains more accurate extraction results. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] The accompanying drawings are used to provide further understanding of the present invention and constitute a part of the specification. They are used to explain the present invention together with the embodiments of the present invention and do not constitute a limitation of the present invention.

[0059] In the attached picture:

[0060] Figure 1 The present invention is a flowchart of a multi-source data ground extraction method based on a grid. DETAILED DESCRIPTION

[0061] The preferred embodiments of the present invention are described below in conjunction with the accompanying drawings. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present invention, and are not used to limit the present invention.

[0062] Example: Figure 1 As shown in the figure, a grid-based multi-source data ground extraction method is implemented as follows: the multi-source data from the lidar and depth camera are calibrated in the same coordinate system and converted into point cloud data. The regional grid is divided according to the point cloud data range, the point cloud features are calculated, and the points that meet the set threshold conditions are used as seed points for plane fitting. The points in the area that do not meet the seed point threshold are set as points to be restored until all areas complete the plane fitting. Finally, based on the plane model obtained in all areas, all points that meet the ground conditions are counted to complete the ground extraction.

[0063] The specific steps include:

[0064] (1) First, the data obtained by the depth camera is converted into a point cloud format. The mapping relationship from the world coordinates to the camera pixels includes:

[0065]

[0066] Where u and v are arbitrary coordinate points in the image coordinate system, u 0 , v 0 are the center coordinates of the image, x w ,y w , z w Represents the three-dimensional coordinate point in the world coordinate system, z cRepresents the z-axis value of the camera coordinates, that is, the distance from the target to the camera. R, T are the 3x3 rotation matrix and 3x1 translation matrix of the external parameter matrix, f represents the focal length of the camera, dx, dy represent the actual physical value represented by a pixel in the x direction and y direction, respectively. Since the world coordinate origin coincides with the camera origin, the transformation formula between the image point and the world coordinate point can be calculated:

[0067]

[0068] (2) According to the actual data range, the point cloud data is divided into regions and grids. The width and height of the data range are cloud_width and cloud_height respectively, the number of regions is zone_number, the number of vertical grids in each region is ring_number, and the number of horizontal grids is sector_number. Then there are regional boundaries:

[0069]

[0070] Grid height Grid width While dividing the area, filter out the points with a depth value of zero and outside the perception range within the data range to reduce the amount of calculation, then traverse each point and calculate the indexes zone_idx, ring_idx, and sector_idx based on the coordinates in the image. Among them:

[0071]

[0072] sector_idx=min((u-zone_bound[zone_idx]) / sector_size,(sector_number-1)))

[0073] ring_idx=min(v / ring_size,(ring_number-1))

[0074] (3) After the grid division is completed, the average height, standard deviation, normal vector and other features of the point cloud are calculated respectively, and divided into seed points according to the set feature threshold to prepare for plane fitting.

[0075] Note the average height:

[0076]

[0077] Standard Deviation:

[0078]

[0079] Covariance:

[0080]

[0081] Mean distance:

[0082]

[0083] Line variables:

[0084]

[0085] angle:

[0086] θ=cos -1 (n 1 ·n 2 0

[0087] Where N represents the total number of points in the current grid, z i Represents the z-axis coordinate of the i-th point in the current grid, represents the average value of all point coordinates, v i Represents the vector from the point to the center point. Perform SVD decomposition on the covariance to obtain eigenvalues ​​and eigenvectors. Define the line variable as the ratio between the two main eigenvalues ​​to describe the linearity and planarity of the point. Define the angle as the angle between the eigenvector and the unit normal vector to describe the deviation. Finally, complete the selection of seed points under the constraints of multiple features.

[0088] (4) Before plane fitting of the seed points, a plane model is given:

[0089] Ax+By+Cz+D=0

[0090] Then, the Householder QR decomposition is used to solve the linear equation system Ax=b. For each column of the matrix A, the Householder matrix is ​​constructed where w j is the vector extracted from the jth column of matrix A and the processed columns, and I is the identity matrix. Apply Householder transformation to matrix A, transform matrix A into upper triangular matrix R, and construct orthogonal matrix Q. The decomposition can be expressed as A = QR, and using the transpose of orthogonal matrix Q, calculate y = Q T b, where y is the transformed constant term vector. Since R is an upper triangular matrix, back substitution is applied to solve x from back to front:

[0091]

[0092] Among them, r ij is the element of matrix R, y i is the i-th element of the vector y, and the solved x value is the parameter in the plane equation.

[0093] (5) After obtaining the regional plane model, first traverse the points in the region and calculate the distance between the points and the plane:

[0094]

[0095] If it is within the set threshold range, the plane point is counted, otherwise it is recorded as a point to be restored. If there are not enough seed points in the area, resulting in plane fitting failure, all points in the area are recorded as points to be restored. Finally, the plane models of all areas are traversed, and the distance values ​​of all points to be restored between the plane models are counted. If the distance is less than the set threshold, the point is recorded by index and counted as a plane point in the final result. When the ground model is merged, the segmentation between the ground area and the non-ground area is finally completed, and the ground extraction is completed.

[0096] Finally, it should be noted that the above description is only a preferred example of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art can still modify the technical solutions described in the aforementioned embodiments or replace some of the technical features therein by equivalents. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A grid-based multi-source data ground extraction method, characterized in that: The following steps are involved: S1, calibrate the multi-source data from the lidar and depth camera in the same coordinate system and convert them into point cloud data; S2, divide the regional grid according to the point cloud data range, calculate the point cloud features, take the points that meet the set threshold conditions as seed points, and perform plane fitting; S3, setting the points in the region that do not meet the seed point threshold as points to be restored until all regions complete the plane fitting; S4. Based on the plane models obtained from all areas, count all points that meet the ground conditions to complete ground extraction.

2. The method for ground extraction of multi-source data based on a grid according to claim 1, characterized in that: Step S1, specifically includes: 1) First, convert the data obtained by the depth camera into a point cloud format. The mapping relationship between the world coordinates and the camera pixels includes: Where u and v are arbitrary coordinate points in the image coordinate system, u0 and v0 are the center coordinates of the image, and x w ,y w , z w Represents the three-dimensional coordinate point in the world coordinate system, R and T are the 3x3 rotation matrix and 3x1 translation matrix of the external parameter matrix, f represents the focal length of the camera, dx and dy represent the actual physical value represented by a pixel in the x direction and y direction respectively; 2) Since the world coordinate origin coincides with the camera origin, the transformation formula from the image point to the world coordinate point is calculated: Among them, z c Indicates the z-axis value of the camera coordinates, that is, the distance from the target to the camera.

3. The method for ground extraction of multi-source data based on a grid according to claim 2, characterized in that: Step S2, specifically comprising: 1) According to the actual point cloud data range, the point cloud data is divided into regions and grids. The width and height of the data range are cloud_width and cloud_height respectively, the number of regions is zone_number, the number of vertical grids in each region is ring_number, and the number of horizontal grids is sector_number. The region boundary is: Among them, the grid height Grid width While dividing the area, filter out the points with a depth value of zero and outside the perception range within the data range to reduce the amount of calculation. Then traverse each point and calculate the indexes zone_idx, ring_idx, sector_idx according to the coordinates in the image, where: sector_idx=min((u-zone_bound[zone_idx]) / sector_size,(sector_number-1))) ring_idx=min(v / ring_size,(ring_number-1)) 2) After the grid division is completed, the average height, standard deviation, and normal vector features of the point cloud are calculated respectively, and divided into seed points according to the set feature threshold to prepare for plane fitting.

4. The method for ground extraction of multi-source data based on a grid according to claim 3, characterized in that: Calculate the average height, standard deviation, and normal vector characteristics of the point cloud, including: Note the average height: Standard Deviation: Covariance: Mean distance: Line variables: angle: θ=cos -1 (n1·n2) Where N represents the total number of points in the current grid, z i Represents the z-axis coordinate of the i-th point in the current grid, represents the average value of all point coordinates, v i Represents the vector from the point to the center point; perform SVD decomposition on the covariance to obtain eigenvalues ​​and eigenvectors, define the line variable as the ratio between the two main eigenvalues, and use it to describe the linearity and planarity of the point; define the angle as the angle between the eigenvector and the unit normal vector, and use it to describe the deviation; finally, complete the screening of seed points under the constraints of multiple features.

5. The method for ground extraction of multi-source data based on a grid according to claim 4, characterized in that: Step S3, specifically includes: Before performing plane fitting on the seed points, a plane model is given first: Ax+By+Cz+D=0 Then, the Householder QR decomposition is used to solve the linear equation system Ax=b. For each column of the matrix A, the Householder matrix is ​​constructed where w j is a vector extracted from the jth column of matrix A and the processed column, and I is the identity matrix; The Householder transformation is applied to the matrix A to convert the matrix A into an upper triangular matrix R, and at the same time, an orthogonal matrix Q is constructed; the decomposition is expressed as A = QR, and the transpose of the orthogonal matrix Q is used to calculate y = Q T b, where y is the transformed constant term vector; Since R is an upper triangular matrix, use the back-substitution method to solve x from back to front: Among them, r ij is the element of matrix R, y i is the i-th element of the vector y, and the solved x value is the parameter in the plane equation.

6. The method for ground extraction of multi-source data based on grid according to claim 5, characterized in that: Step S4, specifically includes: After obtaining the regional plane model, first traverse the points in the region and calculate the distance between the points and the plane: If it is within the set threshold, it is counted as a plane point, otherwise it is recorded as a point to be restored; If there are not enough seed points in the area, resulting in plane fitting failure, all points in the area will be recorded as points to be recovered; Finally, the plane models of all areas are traversed, and the distance values ​​of all points to be restored between the plane models are counted. If the distance is less than the set threshold, the point is recorded through the index and counted into the plane points in the final result. When the ground models are merged, the segmentation between the ground area and the non-ground area is finally completed, and the ground extraction is completed.

Images