Three-dimensional point cloud automatic correction method and system based on deep learning

Through the deep learning-based method, the data connection problem in the process of correction in the bias in the existing technology is solved, and efficient and accurate three-dimensional point cloud correction is achieved.

CN119090782APending Publication Date: 2024-12-06DONGFENG MOTOR GRP
View PDF 6 Cites 0 Cited by

Patent Information

Application Number
CN202410963890.X
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-18
Publication Date
2024-12-06

AI Technical Summary

Technical Problem

The lack of fully automated three-dimensional point cloud deviation correction method in the prior art has resulted in the problem of smooth connection between data after correction and process data after correction cannot be effectively solved.

Method used

A deep learning-based method is adopted to match three-dimensional point cloud data through a trajectory matching algorithm, and a deep learning self-supervised network is used to perform feature point matching and bias correction matrix calculation to achieve automated bias correction of three-dimensional point cloud data.

Benefits of technology

It realizes high degree of automation and high precision three-dimensional point cloud deviation correction, reduces manual intervention, improves production efficiency, and is suitable for various three-dimensional point cloud data fields.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119090782A_ABST
    Figure CN119090782A_ABST
Patent Text Reader

Abstract

The invention discloses a three-dimensional point cloud automatic correction method and system based on deep learning. Performing track matching on two parts of three-dimensional point cloud data collected at different moments in the same physical world one by one through a track matching algorithm to obtain multiple pairs of point cloud data sets; performing two-dimensional conversion on data in each pair of point cloud data sets to obtain a plurality of pairs of point cloud projection drawings; performing feature point matching on each pair of point cloud projection images by using a deep learning self-supervised network to obtain multiple pairs of homonymy points; selecting a reference point from the homonymy points according to the linear relation of the two feature points in each pair of homonymy points; calculating a correction matrix according to the reference points; and correcting the three-dimensional point cloud information collected at different moments in the same physical world by using the correction matrix. According to the method, feature points can be automatically identified, feature points are matched, key point matching is carried out by using data characteristic factors, algorithm evaluation is carried out, and a data correction matrix is output, so that the acquisition cost and the manual interaction correction cost are reduced.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the technical field of three-dimensional information reconstruction, and specifically relates to a three-dimensional point cloud automatic correction method and system based on deep learning. Background Art

[0002] In the business related to high-precision map production and geographic information regional mapping, it is necessary to splice and fuse data at the same place at different times. After fusion, it can ensure that the data area type data can be fully modeled / drawn; however, the relevant characteristics in the original data determine the existence of errors, so correction is required during splicing; the pre-dependence of correction is the relevant information of the registration of homonymous points. The matching objects of homonymous points can be homologous, such as RGB images and RGB images, point cloud projection images and point cloud projection images, depth images and depth images, etc.; similarly, the matching objects of homonymous points can also be multi-source, such as homonymous points in point cloud and RGB images, homonymous points in point cloud and RGB images, homonymous points between RGB images and depth images, etc. The biggest problem with correction is the smooth connection between normal data after correction and correction process data; therefore, there is no fully automated correction in the industry at present. Summary of the invention

[0003] In order to achieve fully automatic deviation correction, the present invention proposes a three-dimensional point cloud automatic deviation correction method and system based on deep learning.

[0004] A three-dimensional point cloud automatic deviation correction method based on deep learning to achieve one of the purposes of the present invention includes:

[0005] The trajectory matching algorithm is used to match the two 3D point cloud data collected at different times in the same physical world one by one, and multiple pairs of point cloud data sets are obtained;

[0006] Perform two-dimensional transformation on the data in each pair of point cloud data sets to obtain multiple pairs of point cloud projection images;

[0007] Use a deep learning self-supervised network to match the feature points of each pair of point cloud projections to obtain multiple pairs of points with the same name;

[0008] Select the reference points from the same-name points according to the linear relationship between the two feature points in each pair of same-name points;

[0009] A correction matrix is ​​calculated based on the above-mentioned reference points; and the correction matrix is ​​used to correct the three-dimensional point cloud information collected at different times in the same physical world.

[0010] Furthermore, the method for obtaining each pair of point cloud data sets includes:

[0011] Extracting three-dimensional point cloud data with overlapping data space positions from the three-dimensional point cloud data information collected at different times in the same physical world;

[0012] All the extracted three-dimensional point cloud data are traversed. When the spatial positions of the three-dimensional point cloud data from two different sources are both less than the set distance from a certain driving trajectory point, the three-dimensional point cloud information of two different sources at the set distance before and after the driving direction of the driving trajectory point is obtained, and then a pair of point cloud data sets are obtained.

[0013] The technical effect of this step is to ensure that the three-dimensional point cloud information of interest is sufficiently correlated with the driving status of the vehicle so that it can be used for subsequent trajectory correction.

[0014] Furthermore, methods for obtaining multiple pairs of points with the same name include:

[0015] Extract feature points of each pair of point cloud projection images using a first deep learning self-supervised network;

[0016] A second deep learning self-supervised network is used to perform pairwise matching of the feature points in each pair of point cloud projection images, and multiple pairs of points with the same name are obtained according to the matching probability.

[0017] Furthermore, the method further includes screening the obtained multiple pairs of points with the same name, and the screening method includes:

[0018] Back-project all points of the same name into three-dimensional space;

[0019] Calculate the rotation matrix of the two points in each pair of points with the same name, and remove them when the angle of the rotation matrix is ​​greater than the set threshold;

[0020] The remaining points with the same name are divided into multiple linear regions according to the linear relationship; for the points with the same name in the same linear region, the points with abnormal linear relationship are eliminated.

[0021] Furthermore, in the above screening method, since the trees on both sides of the road surface vary greatly, but the road surface targets hardly change, before back-projecting all the points with the same name into the three-dimensional space, it also includes: eliminating the points with the same name that are not on the road surface, so as to reduce the amount of calculation of subsequent back-projection and rotation matrices, thereby improving the efficiency and accuracy of the method described in the present invention.

[0022] Furthermore, in the above screening method, before back-projecting all the homonymous points into the three-dimensional space, the method also includes: sorting the matching probability values ​​of the homonymous points, and outputting the top N homonymous point pairs, wherein the matching probability values ​​are output by the second deep learning self-supervised network; the technical effect of retaining homonymous points with higher matching probability values ​​is that the higher the matching probability, the greater the possibility that it is a homonymous point, and points with lower matching probability are eliminated to improve the efficiency and accuracy of the method described in the present invention.

[0023] Furthermore, the method for obtaining a plurality of reference points also includes:

[0024] Calculate the linear relationship value of each pair of same-name points; when the value of the linear relationship is less than the specified error value, it is considered to be a reference point; the remaining same-name points are recorded as matrix calculation points. The linear correlation is such as slope and intercept; since the data scanned by the sensor in a short period of time usually does not undergo large deformation or inflection points, it can be assumed that there is a linear relationship between the data of a certain distance formed within a certain time range. Therefore, the technical effect of this step is: calculate the correction matrix based on the same-name points with smaller linear relationships, so that the process of calculating the correction matrix converges faster.

[0025] Furthermore, the method of using the correction matrix to correct the three-dimensional point cloud information collected at different times in the same physical world includes:

[0026] Calculate the correction matrix and the error average value based on the reference points;

[0027] At the edge joint position, the calculated error average value is evenly distributed to adjacent image segments to reduce misalignment or blur at the joint, thereby achieving better visual continuity.

[0028] Furthermore, the method of using the correction matrix to correct the three-dimensional point cloud information collected at different times in the same physical world also includes:

[0029] The 3D point cloud data within the set range near the reference point is not transformed so that the connection between the data will not be distorted or otherwise deformed.

[0030] A three-dimensional point cloud automatic deviation correction system based on deep learning to achieve the second purpose of the present invention includes:

[0031] Point cloud data set acquisition module: used to perform trajectory matching on two sets of three-dimensional point cloud data collected at different times in the same physical world one by one through a trajectory matching algorithm to obtain multiple pairs of point cloud data sets;

[0032] Point cloud projection map acquisition module: used to perform two-dimensional conversion on the data in each pair of point cloud data sets to obtain multiple pairs of point cloud projection maps;

[0033] Homonymous point acquisition module: used to match feature points of each pair of point cloud projection images using a deep learning self-supervised network to obtain multiple pairs of homonymous points;

[0034] Reference point acquisition module: used to select reference points from the same-name points according to the linear relationship between the two feature points in each pair of same-name points;

[0035] Correction module: used to calculate the correction matrix based on the above-mentioned reference points; use the correction matrix to correct the three-dimensional point cloud information collected at different times in the same physical world.

[0036] Furthermore, the point cloud data set acquisition module includes:

[0037] A three-dimensional point cloud data acquisition module is used to extract three-dimensional point cloud data with overlapping data space positions from the three-dimensional point cloud data information collected at different times in the same physical world;

[0038] Three-dimensional point cloud data screening module: used to traverse all the extracted three-dimensional point cloud data. When the spatial positions of the three-dimensional point cloud data from two different sources are both less than the set distance from a certain driving trajectory point, the three-dimensional point cloud information of two different sources at a set distance before and after the driving direction of the driving trajectory point is obtained, and then a pair of point cloud data sets are obtained.

[0039] Furthermore, the same-name point acquisition module includes:

[0040] Feature point extraction module: used to extract feature points of each pair of point cloud projection images using a first deep learning self-supervised network;

[0041] Feature point matching module: used to use the second deep learning self-supervised network to match the feature points in each pair of point cloud projection images, and obtain multiple pairs of points with the same name according to the matching probability.

[0042] Furthermore, a homonymous point screening module is included, which is used to screen the obtained multiple pairs of homonymous points. The screening method includes:

[0043] Back-project all points of the same name into three-dimensional space;

[0044] Calculate the rotation matrix of the two points in each pair of points with the same name, and remove them when the angle of the rotation matrix is ​​greater than the set threshold;

[0045] The remaining points with the same name are divided into multiple linear regions according to the linear relationship; for the points with the same name in the same linear region, the points with abnormal linear relationship are eliminated.

[0046] Furthermore, in the reference point acquisition module, the method of selecting the reference point includes:

[0047] Calculate the value of the linear relationship of each pair of points with the same name; when the value of the linear relationship is less than the specified error value, it is considered to be a reference point;

[0048] Other points with the same name except the reference point are recorded as matrix calculation points.

[0049] Furthermore, in the correction module, the method for correcting the three-dimensional point cloud information collected at different times in the same physical world includes:

[0050] Calculate the correction matrix and the error average value based on the reference points;

[0051] At the edge position, the calculated error average value is evenly distributed to adjacent image segments.

[0052] Furthermore, the method of using the correction matrix to correct the three-dimensional point cloud information collected at different times in the same physical world also includes:

[0053] The 3D point cloud data within the set range near the reference point is not transformed.

[0054] The beneficial effects of the present invention include:

[0055] High degree of automation: The invention can automatically perform deviation correction processing on three-dimensional point cloud data, greatly reducing the time cost of manual intervention and operation and improving production efficiency.

[0056] High correction accuracy: Through deep learning technology, the invention can accurately identify and correct deviations in point cloud data to obtain a higher-precision three-dimensional model.

[0057] Wide scope of application: The invention can be applied not only to various three-dimensional point cloud data, but also to different fields, such as reverse engineering, three-dimensional modeling, robot vision, etc.

[0058] Strong scalability: With the development of deep learning technology, the correction method and device of the invention can improve its correction effect and performance by continuously updating and optimizing the model.

[0059] High user-friendliness: The correction method and device of the invention have a friendly user interface, allowing users to easily operate and manage, while also providing a wealth of interfaces and function libraries for users to carry out secondary development and customization. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] Figure 1 It is a flow chart of an embodiment of the method of the present invention. DETAILED DESCRIPTION

[0061] The following specific implementations are used to explain the technical solutions of the claims of the present invention so that those skilled in the art can understand the claims. The protection scope of the present invention is not limited to the following specific implementation structures. The technical solutions of the claims of the present invention made by those skilled in the art but different from the following specific implementations are also within the protection scope of the present invention.

[0062] Example 1

[0063] A 3D point cloud automatic correction method based on deep learning, such as Figure 1 As shown, the following steps are included:

[0064] Step 1. Data matching

[0065] 1.1. First, obtain the location information, geometric information and filter distance information of two 3D point cloud data collected from different sources at the same place, and then extract the 3D point cloud information with overlapping data spatial positions;

[0066] 1.2. Use the trajectory information matching algorithm to perform the following matching operations on the extracted point cloud information to obtain multiple pairs of point cloud data sets;

[0067] When the spatial position of the three-dimensional point cloud information from two different sources is close to the vehicle's driving trajectory information (such as the physical distance is less than 2 lane widths), the three-dimensional point cloud information of the vehicle's driving trajectory information from two different sources 10m away from each other along the vehicle's driving direction is obtained, thereby obtaining a pair of point cloud data sets.

[0068] Step 2. Data conversion

[0069] Perform projection center conversion on each pair of point cloud data sets obtained in step 1 above, convert the front view into a top view, remove the elevation information of each 3D point from the perspective of the top view, and then normalize the other two dimensional information to the 2D plane, thereby forming two point cloud projection images (multiple pairs of point cloud data sets form multiple pairs of point cloud projection images); at the same time, record the elevation information corresponding to each point on the 2D plane.

[0070] 3. Feature point registration

[0071] The following is an explanation of an embodiment using one pair of point cloud projection images obtained in step 2;

[0072] 3.1. Feature point search: Use a deep learning self-supervised network framework (such as SuperPoint or Matcher) to extract feature points from a pair of point cloud projection images. The training images of this algorithm do not need to be labeled for direct training, and compared with traditional feature matching algorithms of the same name such as SIFT, Harris and ORB algorithms, which are all based on features to find related corner points, the generalization ability is greatly improved; the key search algorithm that is currently used better is SuperPoint. This algorithm first uses a virtual three-dimensional object as a data set, trains the network to extract corner points, and obtains a trained network. Then, the trained initial network is used to extract corner points from real scene images. This step is called interest point self-labeling; finally, the real scene image is geometrically transformed to obtain a new image, so that there is a pair of images with a known posture relationship. The real scene image and the image that has been geometrically transformed to the real scene image are input into the trained network to extract feature points.

[0073] 3.2. Feature point matching: Use the deep learning self-supervised network SuperGlue to match the feature points of the point cloud projection map. Matched pairs of feature points are called homonymous points (i.e., points representing the same geographic feature in different data sets); the input of the SuperGlue algorithm is the coordinates of the feature points of the two images and their corresponding feature points, and the output is the correspondence between homonymous points, that is, the matching probability between any two feature points.

[0074] 3.3. Algorithm evaluation of feature point search and matching of homonymous points: Since the feature point search and matching algorithms of homonymous points are both self-supervised algorithms, the public data hpatches dataset is used for algorithm evaluation and detection to optimize the parameters of the two algorithms;

[0075] 4. Selection of point pairs with the same name

[0076] Since each pair of point cloud projection images will have several points of the same name in the above feature point matching process, it is very important to select the correct point cloud pair. Therefore, the following strategy is used for screening:

[0077] 4.1. Since point clouds from different sources may be collected at different times and seasons, the trees on both sides of the road vary greatly, but the road targets hardly change. Therefore, the same-name points on the road can be selected for matching, and the model of the extracted road surface can be used to filter out the same-name points on non-road surfaces.

[0078] 4.2. Output the first 10 pairs of points with the same name and the matching probability values, where the matching probability values ​​are output by the SuperGlue algorithm model; retain the points with higher probability values;

[0079] 4.3. Use the elevation information in step 2 to back-project the same-name points into 3D space and calculate the rotation matrix between each same-name point. This rotation matrix represents the rotation transformation from one point cloud dataset to another point cloud dataset.

[0080] 4.4. Due to the characteristics of the sensor, the data of the LiDAR sensor will not undergo large deformation or inflection points within a certain period of time, and the data itself has a series of evaluation methods, such as compliance accuracy evaluation, so the data of a certain distance formed within a certain period of time has a linear relationship; perform linear correlation calculations on adjacent homonymous points, calculate the angles of the rotation matrices of these adjacent homonymous points, remove adjacent homonymous points whose angle differences are greater than the set threshold, and finally the remaining adjacent homonymous points form multiple linear regions of regional linear correlations;

[0081] 4.5. Eliminate the points with the same name that have abnormal distances (not conforming to the linear correlation) in the same linear region;

[0082] 4.6. After implementing the above-mentioned process of eliminating points with the same name, two points with the highest matching probability among the remaining points with the same name are selected for output, that is, after performing the above-mentioned steps 3 to 4.6 on a pair of point cloud projection images, two points with the same name are finally obtained.

[0083] 5. Repeat steps 3 to 4 for each pair of point cloud projections, classify and organize the obtained pairs of points with the same name, and record the points with the same name whose linear correlation (such as distance) is less than the specified error value or close to 0 as reference points, and record the points with the same name whose linear correlation (such as distance) exceeds the error value as matrix calculation points;

[0084] 5. Data Correction:

[0085] 5.1. Delineate a correction area, where the points in the correction area are composed of the above-mentioned reference points;

[0086] 5.2. The correction matrix and error average calculation are performed based on the reference points in the correction area. The correction matrix describes the geometric relationship of how to transform an image (or part of an image) into another image (or an ideal coordinate system). At the same time, by considering the position deviation of all points of the same name, the "error average" is calculated. This value is used to evaluate or adjust the accuracy of the coordinate transformation, similar to the rotation of the homography matrix Homographic Adaptation; the connection of the edge position is compensated for the offset by averaging the error to each segment of the adjacent area, that is, at the edge position, the calculated error value is evenly distributed to the adjacent image segments, and the offset of these segments is adjusted (i.e., fine-tuning) to reduce the misalignment or blur at the splicing point, thereby achieving better visual continuity; the edge position is the overlapping part between adjacent flight strips or scanning areas; these overlapping areas are the key locations where the point cloud data obtained from different scans or flight flights are connected to each other, which is crucial to ensuring the consistency and integrity of the entire point cloud data set. After determining the area that needs to be adjusted, the error value is distributed to adjacent image segments using the principle of average distribution, which means dividing the average error value by the number of adjacent segments to obtain the error value that each segment should share, which usually involves adjusting the transformation parameters such as rotation and translation of the segment.

[0087] The matrix calculation points are used to optimize the correction matrix obtained according to the same-name points in the correction area, and the accuracy of point cloud registration is further improved by adjusting the parameters of the correction matrix;

[0088] 5.3. The data in the area near the benchmark (such as 5m) will not be transformed during the correction process to ensure that the connection between the data will not be distorted or otherwise deformed;

[0089] 5.4. Correct the point cloud data of the three-dimensional point cloud information with overlapping data space positions described in step 1.1 by using the correction matrix and offset;

[0090] 6. Track Correction

[0091] Apply the correction matrix and offset in step 5.3 to the trajectory points corresponding to the point cloud, refresh the trajectory points, and ensure the correctness of subsequent data organization;

[0092] 7. Integrated operation:

[0093] 7.1. Integration and Optimization: Integrate the three functions of data matching, correction algorithm and correction parameter management, and optimize the system. Ensure data flow and collaboration between functional modules, consider performance and efficiency optimization, and achieve the goal of automated processing.

[0094] 7.2. Testing and deployment: Before conducting system testing, strict unit testing and integration testing are carried out to ensure the correctness of each functional module. After passing the test, the system is deployed to the production environment for actual application and operation.

[0095] It should be understood that the order of execution of the steps in the above embodiment does not necessarily mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present invention.

[0096] Example 2

[0097] A three-dimensional point cloud automatic correction method based on deep learning, comprising:

[0098] The trajectory matching algorithm is used to match the two 3D point cloud data collected at different times in the same physical world one by one, and multiple pairs of point cloud data sets are obtained;

[0099] Perform two-dimensional conversion on the data in each pair of point cloud data sets to obtain multiple pairs of point cloud projection images; and save the elevation data of each point cloud data for subsequent back-projection of the point cloud data into three-dimensional space;

[0100] Use a deep learning self-supervised network to match the feature points of each pair of point cloud projections to obtain multiple pairs of points with the same name;

[0101] Select the reference points from the same-name points according to the linear relationship between the two feature points in each pair of same-name points;

[0102] A correction matrix is ​​calculated based on the above-mentioned reference points; and the correction matrix is ​​used to correct the three-dimensional point cloud information collected at different times in the same physical world.

[0103] The correction matrix is ​​used to convert point cloud data acquired from different perspectives or under different conditions into a unified coordinate system, and includes a rotation matrix and a translation vector, which are used to rotate and translate the source point cloud so that it can best match the target point cloud in spatial position. The rotation matrix is ​​used to rotate the source point cloud around one or more axes to adjust its orientation so that it is aligned with the target point cloud in direction; the translation vector is responsible for translating the source point cloud along a certain direction in space to adjust its position so that it coincides with or is as close as possible to the target point cloud in spatial position.

[0104] In some embodiments, a method for acquiring each pair of point cloud data sets includes:

[0105] Extracting three-dimensional point cloud data with overlapping data space positions from the three-dimensional point cloud data information collected at different times in the same physical world;

[0106] All the extracted three-dimensional point cloud data are traversed. When the spatial positions of the three-dimensional point cloud data from two different sources are both less than the set distance from a certain driving trajectory point, the three-dimensional point cloud information of two different sources at the set distance before and after the driving direction of the driving trajectory point is obtained, and then a pair of point cloud data sets are obtained.

[0107] In some embodiments, the method of obtaining multiple pairs of points with the same name includes:

[0108] The first deep learning self-supervised network is used to extract the feature points of each pair of point cloud projection images; such as SuperPoint or Matcher algorithm; the training images of this algorithm do not need to be labeled for direct training, and compared with traditional homonymous point feature matching algorithms such as SIFT, Harris and ORB algorithms, which are all based on features to find related corner points, the generalization ability is greatly improved;

[0109] Use a second deep learning self-supervised network to match each pair of feature points in the point cloud projection image, and obtain multiple pairs of points with the same name according to the matching probability; the second deep learning self-supervised network may be a deep learning self-supervised network using the SuperGlue algorithm.

[0110] In some embodiments, the obtained multiple pairs of points with the same name are screened, and the screening method includes:

[0111] Back-project all points of the same name into three-dimensional space;

[0112] Calculate the rotation matrix of the two points in each pair of points with the same name, and remove them when the angle of the rotation matrix is ​​greater than the set threshold;

[0113] The remaining homonymous points are divided into multiple linear regions according to the linear relationship. For the homonymous points in the same linear region, the homonymous points with abnormal linear relationships are eliminated. If the linear relationship is the slope or intercept, the homonymous points with slopes or intercepts that are obviously different from those of other homonymous points in the same linear region are eliminated. The standard judgment for abnormalities is determined based on actual conditions.

[0114] In some embodiments, the method of obtaining a plurality of reference points further includes:

[0115] The linear relationship value of each pair of points with the same name is calculated; when the value of the linear relationship is less than the specified error value, it is considered to be a reference point.

[0116] In some embodiments, the method of using the correction matrix to correct the three-dimensional point cloud information collected at different times in the same physical world includes:

[0117] Calculate the correction matrix and the error average value based on the reference points;

[0118] Correcting the three-dimensional point cloud information collected at different times in the same physical world based on the correction matrix;

[0119] At the edge position, the calculated error average value is evenly distributed to adjacent image segments.

[0120] In some embodiments, the method of using the correction matrix to correct the three-dimensional point cloud information collected at different times in the same physical world further includes:

[0121] The 3D point cloud data within the set range near the reference point is not transformed.

[0122] Example 3

[0123] A three-dimensional point cloud automatic correction system based on deep learning, comprising:

[0124] Point cloud data set acquisition module: used to perform trajectory matching on two sets of three-dimensional point cloud data collected at different times in the same physical world one by one through a trajectory matching algorithm to obtain multiple pairs of point cloud data sets;

[0125] Point cloud projection map acquisition module: used to perform two-dimensional conversion on the data in each pair of point cloud data sets to obtain multiple pairs of point cloud projection maps;

[0126] Homonymous point acquisition module: used to match feature points of each pair of point cloud projection images using a deep learning self-supervised network to obtain multiple pairs of homonymous points;

[0127] Reference point acquisition module: used to select reference points from the same-name points according to the linear relationship between the two feature points in each pair of same-name points;

[0128] Correction module: used to calculate the correction matrix based on the above-mentioned reference points; use the correction matrix to correct the three-dimensional point cloud information collected at different times in the same physical world.

[0129] In some embodiments, the point cloud data set acquisition module includes:

[0130] A three-dimensional point cloud data acquisition module is used to extract three-dimensional point cloud data with overlapping data space positions from the three-dimensional point cloud data information collected at different times in the same physical world;

[0131] Three-dimensional point cloud data screening module: used to traverse all the extracted three-dimensional point cloud data. When the spatial positions of the three-dimensional point cloud data from two different sources are both less than the set distance from a certain driving trajectory point, the three-dimensional point cloud information of two different sources at a set distance before and after the driving direction of the driving trajectory point is obtained, and then a pair of point cloud data sets are obtained.

[0132] In some embodiments, the same-name point acquisition module includes:

[0133] Feature point extraction module: used to extract feature points of each pair of point cloud projection images using the first deep learning self-supervised network; such as SuperPoint or Matcher algorithm; the training images of this algorithm do not need to be labeled for direct training, and compared with traditional homonymous point feature matching algorithms such as SIFT, Harris and ORB algorithms, which are all based on features to find related corner points, the generalization ability is greatly improved;

[0134] Feature point matching module: used to use the second deep learning self-supervised network to match the feature points in each pair of point cloud projection images, and obtain multiple pairs of points with the same name according to the matching probability, such as a deep learning self-supervised grid using the SuperGlue algorithm.

[0135] In some embodiments, a same-name point screening module is further included, which is used to screen the obtained multiple pairs of same-name points. The screening method includes:

[0136] Back-project all points of the same name into three-dimensional space;

[0137] Calculate the rotation matrix of the two points in each pair of points with the same name, and remove them when the angle of the rotation matrix is ​​greater than the set threshold;

[0138] The remaining points with the same name are divided into multiple linear regions according to the linear relationship; for the points with the same name in the same linear region, the points with abnormal linear relationship are eliminated.

[0139] In some embodiments, in the reference point acquisition module, the method of selecting the reference point includes:

[0140] The linear relationship value of each pair of points with the same name is calculated; when the value of the linear relationship is less than the specified error value, it is considered to be a reference point; other points with the same name except the reference point are recorded as matrix calculation points.

[0141] In some embodiments, in the correction module, the method for correcting the three-dimensional point cloud information collected at different times in the same physical world includes:

[0142] Calculate the correction matrix and the error average value based on the reference points;

[0143] Correcting the three-dimensional point cloud information collected at different times in the same physical world based on the correction matrix;

[0144] At the edge position, the calculated error average value is evenly distributed to adjacent image segments.

[0145] In some embodiments, the method of using the correction matrix to correct the three-dimensional point cloud information collected at different times in the same physical world further includes:

[0146] The 3D point cloud data within the set range near the reference point is not transformed.

[0147] Example 4

[0148] A computer program product, comprising a computer program / instruction, which, when executed by a processor, implements the steps of the deep learning-based three-dimensional point cloud automatic correction method described in claims 1 to 7.

[0149] Example 5

[0150] A non-transitory computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements any step of the deep learning-based three-dimensional point cloud automatic deviation correction method.

[0151] The contents not described in detail in this specification belong to the prior art known to professional and technical personnel in this field.

Claims

1. A three-dimensional point cloud automatic correction method based on deep learning, characterized in that: include: The trajectory matching algorithm is used to match the two 3D point cloud data collected at different times in the same physical world one by one, and multiple pairs of point cloud data sets are obtained; Perform two-dimensional transformation on the data in each pair of point cloud data sets to obtain multiple pairs of point cloud projection images; Use a deep learning self-supervised network to match the feature points of each pair of point cloud projections to obtain multiple pairs of points with the same name; Select the reference points from the same-name points according to the linear relationship between the two feature points in each pair of same-name points; A correction matrix is ​​calculated based on the above-mentioned reference points; and the correction matrix is ​​used to correct the three-dimensional point cloud information collected at different times in the same physical world.

2. The three-dimensional point cloud automatic deviation correction method based on deep learning according to claim 1, characterized in that: The acquisition method of each pair of point cloud data sets includes: Extracting three-dimensional point cloud data with overlapping data space positions from the three-dimensional point cloud data information collected at different times in the same physical world; All the extracted three-dimensional point cloud data are traversed. When the spatial positions of the three-dimensional point cloud data from two different sources are both less than the set distance from a certain driving trajectory point, the three-dimensional point cloud information of two different sources at the set distance before and after the driving direction of the driving trajectory point is obtained, and then a pair of point cloud data sets are obtained.

3. The three-dimensional point cloud automatic deviation correction method based on deep learning according to claim 1, characterized in that: Methods for obtaining multiple pairs of points with the same name include: Extract feature points of each pair of point cloud projection images using a first deep learning self-supervised network; A second deep learning self-supervised network is used to perform pairwise matching of the feature points in each pair of point cloud projection images, and multiple pairs of points with the same name are obtained according to the matching probability.

4. The three-dimensional point cloud automatic deviation correction method based on deep learning according to claim 1, characterized in that: It also includes screening the obtained multiple pairs of points with the same name, and the screening methods include: Back-project all points of the same name into three-dimensional space; Calculate the rotation matrix of the two feature points in each pair of points with the same name, and remove them when the angle of the rotation matrix is ​​greater than the set threshold; The remaining points with the same name are divided into multiple linear regions according to the linear relationship; for the points with the same name in the same linear region, the points with abnormal linear relationship are eliminated.

5. The three-dimensional point cloud automatic deviation correction method based on deep learning according to claim 4, characterized in that: Methods for obtaining multiple reference points include: Calculate the value of the linear relationship for each pair of points with the same name; When the value of the linear relationship is less than the specified error value, it is considered to be a reference point.

6. The three-dimensional point cloud automatic deviation correction method based on deep learning according to claim 1, characterized in that: The method of using the correction matrix to correct the three-dimensional point cloud information collected at different times in the same physical world includes: Calculate the correction matrix and the error average value based on the reference points; Correcting the three-dimensional point cloud information collected at different times in the same physical world based on the correction matrix; At the edge position, the calculated error average value is evenly distributed to adjacent image segments.

7. The three-dimensional point cloud automatic deviation correction method based on deep learning according to claim 1, characterized in that: The method of using the correction matrix to correct the three-dimensional point cloud information collected at different times in the same physical world also includes: The 3D point cloud data within the set range near the reference point is not transformed.

8. A three-dimensional point cloud automatic correction system based on deep learning, characterized in that: include: Point cloud data set acquisition module: used to perform trajectory matching on two sets of three-dimensional point cloud data collected at different times in the same physical world one by one through a trajectory matching algorithm to obtain multiple pairs of point cloud data sets; Point cloud projection map acquisition module: used to perform two-dimensional conversion on the data in each pair of point cloud data sets to obtain multiple pairs of point cloud projection maps; Homonymous point acquisition module: used to match the feature points of each pair of point cloud projection images using a deep learning self-supervised network to obtain multiple pairs of homonymous points; Reference point acquisition module: used to select reference points from the same-name points according to the linear relationship between the two feature points in each pair of same-name points; Correction module: used to calculate the correction matrix based on the above-mentioned reference points; use the correction matrix to correct the three-dimensional point cloud information collected at different times in the same physical world.

9. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instruction is executed by a processor, the steps of the deep learning-based three-dimensional point cloud automatic correction method described in claims 1 to 7 are implemented.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the deep learning-based three-dimensional point cloud automatic correction method as described in any one of claims 1 to 7 are implemented.

Citation Information

Patent Citations

  • Full-automatic point cloud data offset correction method and system

    CN114155168A

  • Self-supervised feature point matching method of fisheye image and storage medium thereof

    CN115410014A

  • Deep learning-based feature point detection network and cross-resolution image matching method

    CN115439669A

  • Inertial navigation error correction method based on indoor walking constraint terrain matching

    CN116202555A

  • Positioning track deviation rectifying method and device

    CN116883490A