Ground feature extraction method based on multi-source data

Abstract

The invention discloses a ground feature extraction method based on multi-source data, and the method comprises the following specific steps: S1, calibrating the multi-source data from a laser radar and a depth camera in the same coordinate system, and converting the multi-source data into point cloud data; s2, dividing regional grids according to a point cloud data range, calculating point cloud features, and performing plane fitting by taking points meeting a set threshold condition as seed points; s3, setting points which do not meet a seed point threshold value in the region as to-be-recovered points until plane fitting of all regions is completed; and S4, according to the obtained plane models of all the areas, counting all points meeting ground conditions, and completing ground extraction. Depth camera and laser radar data are fused, and the adaptability to the external environment is higher; grids are subdivided on the basis of area division, a plurality of plane models are obtained, and robustness to a ground area is higher; in the data processing process, invalid data are removed at the same time, and higher real-time performance is achieved.

Description

Technical Field

[0001] The present invention relates to the technical field of data acquisition, and in particular to a ground feature extraction method based on multi-source data. Background Art

[0002] The ground feature extraction method based on multi-source data is a method that combines multiple data sources (such as satellite images, terrain data, lidar data, etc.) to extract ground feature information from these data through comprehensive analysis. This method is commonly used in geographic information systems (GIS), remote sensing technology, and earth science. In this method, information from different data sources can complement each other, thereby improving the accuracy and reliability of ground feature extraction.

[0003] The basis for realizing the perception of the surrounding environment by using data from multiple sensors is to realize the accurate segmentation of the ground area and the non-ground area. Most existing algorithms adopt plane fitting directly in the data range to extract the ground. Due to the reliance on manual features and the number of iterations, it is not only time-consuming, but also the ground extraction effect is poor when the ground area is uneven in the horizontal and vertical directions. In view of the above problems, the present invention proposes a ground feature extraction method based on multi-source data. Summary of the invention

[0004] 1. Technical issues to be resolved

[0005] In view of the shortcomings of the existing technology, the present invention provides a ground feature extraction method based on multi-source data, which solves the problem that the existing algorithms mostly use plane fitting directly in the data range to extract the ground. Due to the reliance on manual features and the number of iterations, it is not only time-consuming, but also the ground extraction effect is poor when the ground area is uneven in the horizontal and vertical directions.

[0006] (II) Technical solution

[0007] To achieve the above objectives, the present invention is implemented through the following technical solutions:

[0008] The technical solution of the present invention is: a ground feature extraction method based on multi-source data, comprising the following steps:

[0009] S1, calibrate the multi-source data from the lidar and depth camera in the same coordinate system and convert them into point cloud data;

[0010] S2, divide the regional grid according to the point cloud data range, calculate the point cloud features, take the points that meet the set threshold conditions as seed points, and perform plane fitting;

[0011] S3, setting the points in the region that do not meet the seed point threshold as points to be restored until all regions complete the plane fitting;

[0012] S4. Based on the plane models obtained from all areas, count all points that meet the ground conditions to complete ground extraction.

[0013] As a preferred technical solution of the present invention, in step S1, the data acquired by the depth camera is converted into a point cloud format, according to the mapping relationship between the world coordinates and the camera pixels:

[0014]

[0015] Where u and v are arbitrary coordinate points in the image coordinate system, u0 and v0 are the center coordinates of the image, and x w ,y w , z w Represents the three-dimensional coordinate point in the world coordinate system, z c Represents the z-axis value of the camera coordinates, that is, the distance from the target to the camera. R and T are the 3x3 rotation matrix and 3x1 translation matrix of the external parameter matrix, respectively. f represents the focal length of the camera. dx and dy represent the actual physical value represented by a pixel in the x and y directions, respectively. The world coordinate origin coincides with the camera origin. The transformation formula between the image point and the world coordinate point is calculated as follows:

[0016]

[0017] As a preferred technical solution of the present invention, step S1 also includes dividing the point cloud data into regions and grids according to the actual data range, recording the width and height of the data range as cloud_width and cloud_height respectively, the number of regions as zone_number, the number of vertical grids in each region as ring_number, and the number of horizontal grids as sector_number, then there is a regional boundary:

[0018]

[0019] Grid height Grid width While dividing the area, filter out the points with a depth value of zero and outside the perception range within the data range to reduce the amount of calculation. Then traverse each point and calculate the indexes zone_idx, ring_idx, sector_idx according to the coordinates in the image, where:

[0020]

[0021] sector_idx=min((u-zone_bound[zone_idx]) / sector_size,(sector_number-1)))

[0022] ring_idx=min(v / ring_size,(ring_number-1)).

[0023] As a preferred technical solution of the present invention, in step S2, after the grid division is completed, the average height, standard deviation, normal vector and other features of the point cloud are calculated respectively, and divided into seed points according to the set feature threshold, in preparation for plane fitting;

[0024] Note the average height:

[0025]

[0026] Standard Deviation:

[0027]

[0028] Covariance:

[0029]

[0030] Mean distance:

[0031]

[0032] Line variables:

[0033]

[0034] angle:

[0035] θ=cos -1 (n1·n2)

[0036] Where N represents the total number of points in the current grid, z i Represents the z-axis coordinate of the i-th point in the current grid, represents the average value of all point coordinates, v i Represent the vector from the point to the center point, perform SVD decomposition on the covariance to obtain the eigenvalues ​​and eigenvectors, define the line variable as the ratio between the two main eigenvalues, and use it to describe the linearity and planarity of the point; define the angle as the angle between the eigenvector and the unit normal vector, and use it to describe the deviation. Finally, complete the screening of seed points under the constraints of multiple features.

[0037] As a preferred technical solution of the present invention, in step S3, before performing plane fitting on the seed points, a plane model is first given:

[0038] Ax+By+Cz+D=0

[0039] Then, the Householder QR decomposition is used to solve the linear equation system Ax=b. For each column of the matrix A, the Householder matrix is ​​constructed where w j is the vector extracted from the jth column of matrix A and the processed columns. I is the identity matrix. Apply Householder transformation to matrix A to transform matrix A into an upper triangular matrix R and construct an orthogonal matrix Q. The decomposition can be expressed as A = QR. Using the transpose of the orthogonal matrix Q, calculate y = Q T b, where y is the transformed constant term vector; since R is an upper triangular matrix, back substitution is used to solve x from back to front;

[0040]

[0041] Among them, r ij is the element of matrix R, y i is the i-th element of the vector y, and the solved x value is the parameter in the plane equation.

[0042] As a preferred technical solution of the present invention, in step S4, after obtaining the regional plane model, first traverse the points in the region to calculate the distance value between the point and the plane:

[0043]

[0044] If it is within the set threshold range, the plane point is counted, otherwise it is recorded as a point to be restored; if there are not enough seed points in the area, resulting in plane fitting failure, all points in the area are recorded as points to be restored. Finally, the plane models of all areas are traversed, and the distance values ​​of all points to be restored between the plane models are counted. If the distance is less than the set threshold, the point is recorded by index and counted as a plane point in the final result. When the ground model is merged, the segmentation between the ground area and the non-ground area is finally completed, and the ground extraction is completed.

[0045] (III) Beneficial effects

[0046] The present invention provides a method for extracting ground features based on multi-source data. It has the following beneficial effects:

[0047] (1) The present invention integrates the depth camera and lidar data, and has stronger adaptability to the external environment; the grid is subdivided on the basis of the divided area to obtain multiple plane models, which are more robust to the ground area; in the process of data processing, invalid data is eliminated at the same time, and it has stronger real-time performance.

[0048] (2) Data fusion and processing efficiency of the present invention: By calibrating the multi-source data from the lidar and depth camera in the same coordinate system and converting them into point cloud data, the efficiency and accuracy of data fusion and processing are achieved. Region division and feature extraction: The method of dividing the region by a grid, calculating the point cloud features and performing plane fitting helps to extract ground features more finely and improve the accuracy of ground extraction. Data processing optimization: Before performing plane fitting on the seed points, a detailed plane model and a method of solving the linear equations using Householder QR decomposition are given, thereby optimizing the data processing process. Automation and accuracy: By setting feature thresholds and screening seed points, automatic identification and extraction of ground conditions are achieved, improving the accuracy and reliability of ground extraction. Result analysis and extraction effect: By calculating the distance values ​​of points in the region and processing the points to be restored, the segmentation of the ground area and the non-ground area is finally completed, achieving effective result analysis and extraction effect of ground extraction. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 It is a schematic diagram of the process of the present invention. DETAILED DESCRIPTION

[0050] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0051] See also Figure 1 , an embodiment of the present invention provides a technical solution: a ground feature extraction method based on multi-source data, and the specific implementation process of the algorithm is as follows: the multi-source data from the lidar and the depth camera are calibrated in the same coordinate system and converted into point cloud data. The regional grid is divided according to the range of the point cloud data, the point cloud features are calculated, and the points that meet the set threshold conditions are used as seed points for plane fitting. The points in the area that do not meet the seed point threshold are set as points to be restored until all areas complete the plane fitting. Finally, based on the plane model obtained in all areas, all points that meet the ground conditions are counted to complete the ground extraction.

[0052] The data obtained by the depth camera is converted into a point cloud format. According to the mapping relationship between the world coordinates and the camera pixels, there are:

[0053]

[0054] Where u and v are arbitrary coordinate points in the image coordinate system, u0 and v0 are the center coordinates of the image, and x w,y w , z w Represents the three-dimensional coordinate point in the world coordinate system, z c Represents the z-axis value of the camera coordinates, that is, the distance from the target to the camera. R and T are the 3x3 rotation matrix and 3x1 translation matrix of the external parameter matrix, respectively. f represents the focal length of the camera. dx and dy represent the actual physical value represented by a pixel in the x and y directions, respectively. The world coordinate origin coincides with the camera origin. The transformation formula between the image point and the world coordinate point is calculated as follows:

[0055]

[0056] It also includes dividing the point cloud data into regions and grids according to the actual data range. The width and height of the data range are cloud_width and cloud_height respectively, the number of regions is zone_number, the number of vertical grids in each region is ring_number, and the number of horizontal grids is sector_number, then there are regional boundaries:

[0057]

[0058] Grid height Grid width While dividing the area, filter out the points with a depth value of zero and outside the perception range within the data range to reduce the amount of calculation. Then traverse each point and calculate the indexes zone_idx, ring_idx, sector_idx according to the coordinates in the image, where:

[0059]

[0060] sector_idx=min((u-zone_bound[zone_idx]) / sector_size,(sector_number-1)))

[0061] ring_idx=min(v / ring_size,(ring_number-1))

[0062] After the grid division is completed, the average height, standard deviation, normal vector and other features of the point cloud are calculated respectively, and divided into seed points according to the set feature threshold to prepare for plane fitting;

[0063] Note the average height:

[0064]

[0065] Standard Deviation:

[0066]

[0067] Covariance:

[0068]

[0069] Mean distance:

[0070]

[0071] Line variables:

[0072]

[0073] angle:

[0074] θ=cos -1 (n1·n2)

[0075] Where N represents the total number of points in the current grid, z i Represents the z-axis coordinate of the i-th point in the current grid, represents the average value of all point coordinates, v i Represent the vector from the point to the center point, perform SVD decomposition on the covariance to obtain the eigenvalues ​​and eigenvectors, define the line variable as the ratio between the two main eigenvalues, and use it to describe the linearity and planarity of the point; define the angle as the angle between the eigenvector and the unit normal vector, and use it to describe the deviation. Finally, complete the screening of seed points under the constraints of multiple features.

[0076] Before performing plane fitting on the seed points, a plane model is given first:

[0077] Ax+By+Cz+D=0

[0078] Then, the Householder QR decomposition is used to solve the linear equation system Ax=b. For each column of the matrix A, the Householder matrix is ​​constructed where w j is the vector extracted from the jth column of matrix A and the processed columns. I is the identity matrix. Apply Householder transformation to matrix A to transform matrix A into an upper triangular matrix R and construct an orthogonal matrix Q. The decomposition can be expressed as A = QR. Using the transpose of the orthogonal matrix Q, calculate y = Q T b, where y is the transformed constant term vector; since R is an upper triangular matrix, back substitution is used to solve x from back to front;

[0079]

[0080] Among them, r ij is the element of matrix R, y i is the i-th element of the vector y, and the solved x value is the parameter in the plane equation.

[0081] After obtaining the regional plane model, first traverse the points in the region and calculate the distance between the points and the plane:

[0082]

[0083] If it is within the set threshold range, the plane point is counted, otherwise it is recorded as a point to be restored; if there are not enough seed points in the area, resulting in plane fitting failure, all points in the area are recorded as points to be restored. Finally, the plane models of all areas are traversed, and the distance values ​​of all points to be restored between the plane models are counted. If the distance is less than the set threshold, the point is recorded by index and counted as a plane point in the final result. When the ground model is merged, the segmentation between the ground area and the non-ground area is finally completed, and the ground extraction is completed.

[0084] The above shows and describes the basic principles and main features of the present invention and the advantages of the present invention. It is obvious to those skilled in the art that the present invention is not limited to the details of the above exemplary embodiments, and the present invention can be implemented in other specific forms without departing from the spirit or basic features of the present invention. Therefore, no matter from which point of view, the embodiments should be regarded as exemplary and non-restrictive. The scope of the present invention is defined by the attached claims rather than the above description, and it is intended that all changes falling within the meaning and scope of the equivalent elements of the claims are included in the present invention. Any figure mark in the claims should not be regarded as limiting the claims involved.

[0085] In addition, it should be understood that although the present specification is described according to implementation modes, not every implementation mode contains only one independent technical solution. This description of the specification is only for the sake of clarity. Those skilled in the art should regard the specification as a whole. The technical solutions in each embodiment may also be appropriately combined to form other implementation modes that can be understood by those skilled in the art.

Claims

1. A ground feature extraction method based on multi-source data, characterized in that: The specific steps include the following: S1, calibrate the multi-source data from the lidar and depth camera in the same coordinate system and convert them into point cloud data; S2, divide the regional grid according to the point cloud data range, calculate the point cloud features, take the points that meet the set threshold conditions as seed points, and perform plane fitting; S3, setting the points in the region that do not meet the seed point threshold as points to be restored until all regions complete the plane fitting; S4. Based on the plane models obtained from all areas, count all points that meet the ground conditions to complete ground extraction.

2. The method for extracting ground features based on multi-source data according to claim 1, characterized in that: In step S1, the data acquired by the depth camera is converted into a point cloud format. According to the mapping relationship between the world coordinates and the camera pixels, there is: Where u and v are arbitrary coordinate points in the image coordinate system, u0 and v0 are the center coordinates of the image, and x w ,y w , z w Represents the three-dimensional coordinate point in the world coordinate system, z c Represents the z-axis value of the camera coordinates, that is, the distance from the target to the camera. R and T are the 3x3 rotation matrix and 3x1 translation matrix of the external parameter matrix, respectively. f represents the focal length of the camera. dx and dy represent the actual physical value represented by a pixel in the x and y directions, respectively. The world coordinate origin coincides with the camera origin. The transformation formula between the image point and the world coordinate point is calculated as follows:

3. The method for extracting ground features based on multi-source data according to claim 2, characterized in that: Step S1 also includes dividing the point cloud data into regions and grids according to the actual data range. The width and height of the data range are respectively cloud_width and cloud_height, the number of regions is zone_number, the number of vertical grids in each region is ring_number, and the number of horizontal grids is sector_number. Then there are regional boundaries: Grid height Grid width While dividing the area, filter out the points with a depth value of zero and outside the perception range within the data range to reduce the amount of calculation. Then traverse each point and calculate the indexes zone_idx, ring_idx, sectpr_idx according to the coordinates in the image, where: sector_idx=min((u-zone_bound[zone_idx]) / sector_size,(sector_number-1))) ring_idx=min(v / ring_size,(ring_number-1)).

4. The method for extracting ground features based on multi-source data according to claim 3 is characterized in that: In step S2, after the grid division is completed, the average height, standard deviation, normal vector and other features of the point cloud are calculated respectively, and divided into seed points according to the set feature threshold to prepare for plane fitting; the average height is recorded as: Standard Deviation: Covariance: Mean distance: Line variables: angle: θ=cos -1 (n1·n2) Where N represents the total number of points in the current grid, z i Represents the z-axis coordinate of the i-th point in the current grid, represents the average value of all point coordinates, v i Represent the vector from the point to the center point, perform SVD decomposition on the covariance to obtain the eigenvalues ​​and eigenvectors, define the line variable as the ratio between the two main eigenvalues, and use it to describe the linearity and planarity of the point; define the angle as the angle between the eigenvector and the unit normal vector, and use it to describe the deviation. Finally, complete the screening of seed points under the constraints of multiple features.

5. The method for extracting ground features based on multi-source data according to claim 4 is characterized in that: In step S3, before performing plane fitting on the seed points, a plane model is first given: Ax+By+Cz+D=0 Then, the Householder QR decomposition is used to solve the linear equation system Ax=b. For each column of the matrix A, the Householder matrix is ​​constructed where w j is the vector extracted from the jth column of matrix A and the processed column, I is the identity matrix, Householder transformation is applied to matrix A, matrix A is converted into an upper triangular matrix R, and an orthogonal matrix Q is constructed at the same time; The decomposition can be expressed as A = QR, using the transpose of the orthogonal matrix Q, calculate y = Q T b, where y is the transformed constant term vector; since R is an upper triangular matrix, back substitution is applied to solve x from back to front; Among them, r ij is the element of matrix R, y i is the i-th element of the vector y, and the solved x value is the parameter in the plane equation.

6. The method for extracting ground features based on multi-source data according to claim 5, characterized in that: In step S4, after obtaining the regional plane model, first traverse the points in the region and calculate the distance value between the point and the plane: If it is within the set threshold range, the plane point is counted, otherwise it is recorded as a point to be restored; if there are not enough seed points in the area, resulting in plane fitting failure, all points in the area are recorded as points to be restored. Finally, the plane models of all areas are traversed, and the distance values ​​of all points to be restored between the plane models are counted. If the distance is less than the set threshold, the point is recorded by index and counted as a plane point in the final result. When the ground model is merged, the segmentation between the ground area and the non-ground area is finally completed, and the ground extraction is completed.

Images