Training data optimization method, device and computer equipment for point cloud registration model

Abstract

The present invention relates to the technical field of point cloud data processing, and specifically to a method, device and computer device for optimizing training data of a point cloud registration model; the method includes obtaining a first training set and a second training set, where the first training set includes a plurality of point cloud sample data with pseudo labels, and the second training set includes a plurality of point cloud sample data with true labels; the label corresponds to the transformation matrix between the source point cloud and the target point cloud in the point cloud sample data; in the stage of manually correcting training samples, point cloud sample data is selected according to a first replacement principle; in the stage of self-correcting training samples, point cloud sample data is selected according to a second replacement principle; the present application not only improves the accuracy of point cloud data alignment, but also enhances the feature discrimination degree through the fusion of structural and semantic information, making the registration result more reliable.

Description

Technical Field

[0001] The present invention relates to the technical field of point cloud data processing, and in particular to a method, device and computer device for optimizing training data of a point cloud registration model. Background Art

[0002] Point cloud registration is a process of aligning multiple frames of three-dimensional point cloud data obtained from different perspectives or sensors in the same coordinate system, so that they represent the same scene in the same coordinate system.

[0003] Currently, the most popular point cloud registration methods are based on deep learning. In the point cloud registration task, it is difficult to obtain high-precision data labels when using deep learning technology for data training. The deep learning-based point cloud registration methods mainly include self-supervised learning SuperLine3D and weakly supervised learning. The use of self-supervised learning methods can significantly reduce the dependence on labeled data. However, a major challenge of this method is that the design of self-supervised signals is quite complex, which requires designing auxiliary tasks that can appropriately reflect the requirements of point cloud registration to generate effective training signals. If these self-supervised tasks fail to accurately capture the key elements of point cloud registration, the model may not learn the necessary geometric features. In addition, without explicit guidance signals, the model training in the self-supervised learning framework often has a slow convergence speed and may encounter stability problems in the initial stage of training. Moreover, the final point cloud registration accuracy highly depends on the initially estimated pose, and most self-supervised point cloud registration algorithms have a low tolerance for deviations in the initial point cloud pose, and a large initial error may make it difficult to find the correct transformation parameters during the optimization process.

[0004] The use of weakly supervised learning methods can utilize unlabeled data or only a small amount of high-quality labels for model training, thus greatly reducing the cost of data annotation. However, since only a limited number of labeled samples are utilized, the model may perform excellently only in known or similar scenarios, but its generalization performance may decline when facing unknown scenarios. In addition, the labels used in weakly supervised learning may contain noise. If there are large differences between these approximate labels and the actual labels, the model may learn incorrect alignment strategies. When processing point cloud data sets containing complex or large-scale transformations, due to the lack of sufficient detailed information, the model may only be able to capture the local features of the point cloud and it is difficult to understand the global features or cross-object geometric relationships.

[0005] In summary, the existing deep learning-based point cloud registration models cannot accurately select the true labels of point cloud data sets, resulting in a huge amount of manual annotation. Summary of the Invention

[0006] Based on the problems existing in the prior art, the present invention proposes a method, device and computer equipment for optimizing training data of a point cloud registration model. By adding an active learning framework to the training process of the point cloud registration model, it can select some true labels during the training of the point cloud registration model, and construct a training set through the pseudo-label method, so as to reduce the manual annotation workload and achieve a registration effect close to using 100% of the labels.

[0007] In the first aspect of the present application, the present application provides a method for optimizing training data of a point cloud registration model, the method comprising:

[0008] Obtain a first training set and a second training set, the first training set includes a plurality of point cloud sample data with pseudo-labels, and the second training set includes a plurality of point cloud sample data with true labels; the label corresponds to the transformation matrix between the source point cloud and the target point cloud in the point cloud sample data;

[0009] In the stage of manually correcting training samples, use the point cloud sample data of the third training set as the input of the point cloud registration model, and output the feature matrix, estimated transformation matrix and estimated correspondence of the point cloud sample data; the third training set is the first training set or a data set obtained by replacing the point cloud sample data of the first training set based on the first replacement principle with the second training set; the first replacement principle is determined based on the quality of the point cloud sample data, and the quality of the point cloud sample data is determined by the feature matrix of the point cloud sample data or by the pseudo-label transformation matrix and estimated correspondence of the point cloud sample data;

[0010] In the stage of self-correcting training samples, use the point cloud sample data of the fourth training set as the input of the point cloud registration model, and output the feature matrix, estimated transformation matrix and estimated correspondence of the point cloud sample data; the fourth training set is the first training set after the last round of replacement or a data set obtained by replacing the point cloud sample data of the first training set based on the second replacement principle; the second replacement principle is determined based on the pseudo-label score and estimated relationship score of the point cloud sample data, the pseudo-label score of the point cloud sample data is obtained by calculating the compatibility score from the pseudo-label correspondence after the action of the pseudo-label transformation matrix, and the estimated relationship score of the point cloud data sample is obtained by calculating the compatibility score from the estimated correspondence after the action of the estimated transformation matrix.

[0011] In the second aspect of the present application, the present application further provides a device for optimizing training data of a point cloud registration model, comprising:

[0012] A point cloud data acquisition module, configured to obtain a first training set and a second training set, the first training set includes a plurality of point cloud sample data with pseudo-labels, and the second training set includes a plurality of point cloud sample data with true labels;

[0013] The first training and correction module is used to take the point cloud sample data of the third training set as the input of the point cloud registration model during the artificial correction training sample stage, and output the feature matrix, estimated transformation matrix, and estimated correspondence relationship of the point cloud sample data; the third training set is the first training set or a data set obtained by replacing the point cloud sample data of the first training set with the second training set based on the first replacement principle; the first replacement principle is determined based on the quality of the point cloud sample data, and the quality of the point cloud sample data is determined by the feature matrix of the point cloud sample data or by the pseudo-label transformation matrix and estimated correspondence relationship of the point cloud sample data;

[0014] The second training and correction module is used to take the point cloud sample data of the fourth training set as the input of the point cloud registration model, and output the feature matrix, estimated transformation matrix, and estimated correspondence relationship of the point cloud sample data; the fourth training set is the first training set after the last round of replacement or a data set obtained by replacing the point cloud sample data of the first training set based on the second replacement principle; the second replacement principle is determined based on the pseudo-label score and estimated relationship score of the point cloud sample data, the pseudo-label score of the point cloud sample data is obtained by calculating the compatibility score from the pseudo-label correspondence relationship after the action of the pseudo-label transformation matrix, and the estimated relationship score of the point cloud data sample is obtained by calculating the compatibility score from the estimated correspondence relationship after the action of the estimated transformation matrix.

[0015] In the third aspect of the present application, the present application also provides a computer device, the device includes:

[0016] One or more processors;

[0017] One or more memories for storing executable instructions of the one or more processors;

[0018] Wherein, the one or more processors are configured to execute the steps of the first aspect of the present application.

[0019] Advantages of the present invention:

[0020] This application generates pseudo-labels for training samples without obtained labels through a pseudo-label generator module, which is used to reduce the use of real labels and supervise training with pseudo-labels. This application can generate reliable labels using image information, reduce the annotation problem of a large amount of point cloud data, and optimize the labels through layer-by-layer iteration, so as to generate accurate labels through a strict evaluation system. In the initial training stage, the model only uses pseudo-labels and evaluates the sample quality by calculating the cosine similarity between point cloud features, selects the samples with the lowest confidence and replaces them with real labels for manual correction. As the training progresses, the model iterates continuously, and in each round, the samples with the lowest confidence are selected for real label replacement and correction until the training reaches the predetermined number of rounds. Entering the model self-correction stage, the score is calculated by comparing the estimated matrix and the pseudo-label matrix, and the matrix with a higher score is selected to update the training labels of the samples. This strategy effectively utilizes the limited real label resources and dynamically optimizes the model performance, ultimately achieving high-precision point cloud registration. Beneficial to high-precision and high-efficiency point cloud registration energy, the demand for manual annotation is significantly reduced through an active learning strategy, while the generalization ability and robustness of the model are enhanced. This application not only improves the accuracy of point cloud data alignment, but also enhances the feature discrimination through the fusion of structural and semantic information, making the registration result more reliable. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 is the operation diagram of the training data optimization scheme of the point cloud registration model in the embodiment of this application;

[0022] Figure 2 is the flowchart of the training data optimization method of the point cloud registration model in the embodiment of this application;

[0023] Figure 3 is the schematic diagram of outlier filtering in the preferred embodiment of this application;

[0024] Figure 4 is the schematic diagram of the structure of the training data optimization device of the point cloud registration model in the embodiment of this application;

[0025] Figure 5 is the schematic diagram of the structure of the computer device in the embodiment of this application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0026] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0027] Point cloud registration refers to the process of aligning point cloud data from different sources (e.g., 3D scan data obtained from different perspectives or time periods) into the same coordinate system. Given two frames of point cloud data from the same specific scene, they are respectively called the source point cloud and the target point cloud. There is an overlapping area between these two frames of point clouds in the scene. Our goal is to use the information in these overlapping areas to calculate a 4×4 transformation matrix. This transformation matrix can accurately map the points in the source point cloud to the corresponding positions in the target point cloud, thus achieving the alignment or registration of the point cloud. In this process, some algorithms are usually needed to identify the matching point pairs in the overlapping area and estimate the transformation matrix based on these matching point pairs. These algorithms may include the Iterative Closest Point (ICP) algorithm, feature point matching algorithms, etc. Once the transformation matrix is obtained, it can be applied to the entire source point cloud to achieve precise alignment with the target point cloud. However, in this process, for the task of 3D point cloud registration that requires a large number of real labels, the existing deep learning-based point cloud registration models cannot accurately select the real labels of the point cloud dataset, resulting in a huge amount of manual annotation; based on this, this application proposes a point cloud registration method, device, and computer device based on active learning, thereby reducing the manual annotation workload, optimizing the training data of the point cloud registration model, and improving the accuracy of the point cloud registration model.

[0028] Figure 1 is the operation diagram of the training data optimization scheme of the point cloud registration model in the embodiment of this application; as Figure 1As shown in the figure, the present application introduces a pseudo-label generator module for image matching correspondence, which generates pseudo-labels for training samples without obtained labels, reduces the use of real labels, and uses the pseudo-labels to supervise the training of the point cloud registration model. Then, through the active learning framework of the embodiment of the present invention, the semi-supervised data labels are manually corrected and self-corrected. It can use fewer real labels to train the data, dynamically update the data labels during the training process, and finally achieve a training effect close to using 100% real labels. Among them, the pseudo-label generator uses the image matching of the source image and the target image to obtain the corresponding 2D correspondence and 3D projection to generate pseudo-labels, and then gradually introduces a small amount of real labels for correction by using the active learning framework. In the initial training stage, the point cloud registration model only uses pseudo-labels, and evaluates the sample quality by calculating the feature similarity or inlier rate between point cloud features, selects the samples with the lowest confidence and replaces them with real labels for manual correction. As the training progresses, the point cloud registration model iterates continuously, and in each round, the samples with the lowest confidence are selected for real label replacement and correction until the training reaches the predetermined number of rounds. Entering the model self-correction stage, the score is calculated by comparing the estimated matrix and the pseudo-label matrix, and the matrix with a higher score is selected to update the training label of the sample. This strategy effectively utilizes the limited real label resources and dynamically optimizes the model performance, and finally realizes high-precision point cloud registration.

[0029] Figure 2 is the flowchart of the training data optimization method for the point cloud registration model in the embodiment of the present application. As Figure 2 shown, the method includes:

[0030] 101. Obtain a first training set and a second training set. The first training set includes a plurality of point cloud sample data with pseudo-labels, and the second training set includes a plurality of point cloud sample data with real labels; the label corresponds to the transformation matrix between the source point cloud and the target point cloud in the point cloud sample data;

[0031] In some embodiments of the present application, the point cloud sample data of the first training set and the second training set are respectively labeled. Here, the purpose of labeling can be to distinguish the points belonging to different objects in the point cloud sample data. Among them, the point cloud sample data refers to the registration relationship between the source point cloud and the target point cloud in different point cloud images; the point cloud sample data of the first training set is labeled with pseudo-labels, and the point cloud sample data set of the second training set is labeled with real labels; whether it is a pseudo-label or a real label, the label here is the point cloud transformation matrix between the source point cloud and the target point cloud in the point cloud sample data, and the point cloud transformation matrix is used to describe the translation and rotation operations of the point cloud in three-dimensional space. Usually, a complete transformation matrix includes two parts: rotation and translation, and can be decomposed into a rotation matrix and a horizontal vector.

[0032] In some embodiments of the present application, the annotation method of the training set can be that the user manually annotates the first training set and the second training set, or an annotation algorithm can be used to automatically annotate the first training set and the second training set.

[0033] In a preferred embodiment of the present application, the pseudo-labels of the first training set are obtained by annotating with a pseudo-label generator. The acquisition method of the first training set includes obtaining the image corresponding to the source point cloud and the image corresponding to the target point cloud, performing feature extraction and image matching through a pre-trained model to obtain the correspondence of 2D images; determining the correspondence mapped to the 3D point cloud through the correspondence of 2D images; according to the correspondence after mapping between the source point cloud and the target point cloud, solving the optimal pose transformation matrix between the two point clouds through the random sample consensus algorithm as the pseudo-label transformation matrix; the pseudo-label transformation matrix is used as the pseudo-label of the first training set, and together with the source point cloud and the target point cloud, it constitutes the point cloud sample data of the first training set.

[0034] Among them, determining the correspondence mapped to the 3D point cloud through the correspondence of 2D images includes corresponding the 3D point cloud obtained by projection to the key points of the 2D image, mapping the key points on the 2D image to the real 3D point cloud space, screening out low-quality matching point pairs through a confidence threshold, and finally retaining the key point pairs of the 2D-3D image with high confidence.

[0035] For example, in this embodiment, the source image and the target image can be matched through a pre-trained model to obtain the 2D correspondence pixel pairs (x i , y i ) s and (x i , y i ) t of two images of the same scene, where the superscript s represents the source image, the superscript t represents the target image, and the subscript i represents the index of the point cloud; then project the 2D correspondence into the 3D space. In this projection calculation process, the internal parameters and external parameters of the camera need to be obtained, and the positions of the 3D points P1 (source point) and P2 (target point) are calculated using the pixel matching points and camera parameters; specifically, assuming the point (x1, y1) s in the 2D source image, establish the equation: (x1, y1) s = K[R|t]P1; where K is the 3×3 camera internal parameter matrix and [R|t] is the 3×4 external parameter matrix; according to the pixel coordinates, first add one dimension to the 2D coordinates to become homogeneous coordinates (x1, y1, 1) s , and inversely solve the mapped 3D point P1(x1, y1, z1, w1), and then normalize the P1 point Take the first three normalized coordinates as P1(x1, y1, z1), thus realizing the projection of 2D pixel points to 3D points. Then find the other point (x i , y i ) t of the mapped 3D point for the pixel pair in the target image. (x i , y i ) t = K[R|t]P2. First, transform (x i , y i ) t into homogeneous coordinates (x1, y1, 1) t , and inversely solve the mapped 3D point P2(x2, y2, z2, w2). Then normalize the P2 point Take the first three normalized coordinates as P2(x2, y2, z2), thus obtaining a pair of 3D correspondence relationships. Then, use RANSAC to solve the pseudo-labels of the point cloud sample data based on the obtained 3D correspondence relationships, and finally form a pseudo-label set.

[0036] 102. In the stage of manually correcting the training samples, use the point cloud sample data of the third training set as the input of the point cloud registration model, and output the feature matrix, estimated transformation matrix, and estimated correspondence relationship of the point cloud sample data; the third training set is the first training set or a data set obtained by replacing the point cloud sample data of the first training set with the second training set based on the first replacement principle; the first replacement principle is determined based on the quality of the point cloud sample data, and the quality of the point cloud sample data is determined by the feature matrix of the point cloud sample data or by the pseudo-label transformation matrix and estimated correspondence relationship of the point cloud sample data;

[0037] In some embodiments of the present application, the point cloud registration model can be an existing point cloud registration model; for example, use a deep neural network (such as a convolutional neural network, a recurrent neural network, etc.) to perform feature extraction and representation learning on point cloud data. These networks can capture the complex distribution and geometric features of point cloud data, providing strong support for subsequent registration tasks. Specifically, it can be the DeepGMR model, or PointNetLK, FlowNet3D, etc. The present invention does not limit the specific point cloud registration model, as long as it is a point cloud registration model that needs to use deep learning technology for data training, it can be implemented.

[0038] In some embodiments of the present application, in the stage of manually correcting training samples, a small amount of point cloud sample data with true labels is selected, and a large amount of point cloud sample data with pseudo labels is selected. These point cloud sample data are fed into the KPConv-FPN network for multi-level downsampling to obtain a series of point clouds with different resolutions. At each level of downsampling, the KPConv layer is used to extract features for each point. Then, at the coarsest resolution level, the downsampled points are used as superpoints, and relevant learning features are learned. Then, geometric embedding of the superpoints is performed based on the geometric structure of pairwise distances and triple angles. Then, this superpoint correspondence is sent to the LGR (Local-to-Global Registration) estimator, and through the estimation of the local stage and the global stage, the speed and accuracy of point cloud registration are quickly improved.

[0039] In some embodiments of the present application, the first replacement principle includes removing the point cloud sample data with lower quality in the first training set and replacing it with the point cloud sample data in the second training set. In the stage of manually correcting training samples, the point cloud sample data in the second training set can be replaced successively. For example, when training the point cloud registration model for the first time, 100 point cloud sample data in the first training set are used, and the point cloud sample data in the second training set are not used. When training the point cloud registration model for the second time, 95 point cloud sample data in the first training set are used, and 5 point cloud sample data in the second training set are used. Among the 100 point cloud sample data in the first training set, the 5 point cloud sample data to be replaced have lower sample quality. Similarly, when training the point cloud registration model for the third time, 90 point cloud sample data in the first training set are used, and 10 point cloud sample data in the second training set are used. This method can replace the point cloud sample data with lower pseudo label confidence with the point cloud sample data with true labels. The model is iteratively trained, and in each round, the sample with the lowest confidence is selected for true label replacement and correction until the training reaches the predetermined number of rounds. This strategy effectively utilizes the limited true label resources.

[0040] In some embodiments of the present application, the quality of the point cloud sample data is determined by the feature matrix of the point cloud sample data, including calculating the similarity distance of the point cloud sample data according to the feature matrices of the source point cloud and the target point cloud in the point cloud sample data. The similarity distance of the point cloud sample data characterizes the quality of the point cloud sample data.

[0041] For example, when training the point cloud registration model in the first round, this embodiment selects the first training set and inputs it into the point cloud registration model, that is, 0% of the true labels in the label distribution of the training samples, and the other 100% are pseudo labels for training. Then the point cloud registration model will output a source point cloud feature matrix F P and a target point cloud feature matrix FQ ; Calculate the cosine similarity D between the point cloud feature matrices of the i-th pair of samples i (F P , F Q ); Thus, the quality scores of the point cloud sample data are obtained. Then, the scores of all point cloud sample data are sorted, and the K point cloud sample data with the smallest scores are selected. Replace the true labels in the second training set of point cloud sample data with the pseudo-labels of these K point cloud sample data for manual correction. Similarly, continue to send the replaced point cloud sample data into the point cloud registration model for the second training. For the point cloud sample data with pseudo-labels, calculate the cosine similarity between the source point cloud and the target point cloud features of the samples, and continue to select the K point cloud sample data with the smallest scores. Replace the true labels with the pseudo-labels of these point cloud sample data for manual correction. Until the number of training rounds is greater than 5 rounds, this embodiment believes that enough true labels have been added to maintain the training of this model.

[0042] In some embodiments of the present application, the quality of the point cloud sample data is determined by the estimated correspondence of the point cloud sample data and the pseudo-label transformation matrix, including calculating the inlier number of the point cloud sample data according to the pseudo-label transformation matrix of the source point cloud and the target point cloud in the point cloud sample data; calculating the inlier rate of the point cloud sample data according to the ratio of the inlier number of the point cloud sample data to the number of estimated correspondences; the inlier rate of the point cloud sample data characterizes the quality of the point cloud sample data.

[0043] The calculating the inlier number of the point cloud sample data according to the pseudo-label transformation matrix of the source point cloud and the target point cloud in the point cloud sample data includes if is satisfied, then increment an inlier; where, R represents the rotation matrix in the pseudo-label transformation matrix, t represents the horizontal vector in the pseudo-label transformation matrix, p i represents the i-th source point cloud of the point cloud sample data, q i represents the i-th target point cloud of the point cloud sample data, τ represents a preset threshold, i ∈ Ω pq , Ω pq represents the set of point cloud pairs of the point cloud sample data; the pseudo-label matrix of the point cloud sample data is represented as (R, t).

[0044] For example, when the point cloud sample data is input into the point cloud registration model, the feature matrix, estimated transformation matrix, and estimated correspondence of the point cloud sample data will be output; this embodiment uses the estimated correspondence of the point cloud sample data to form a point cloud correspondence set Ω, where, (p i , q i ∈ Ω), P = {p1, p2,..., p n} and Q = {q1, q2,..., q n}, where n represents the number of point cloud correspondence relationships; by traversing the correspondence relationship set Ω, if it satisfies Then let the inlier number N inliers Increment by 1. By the ratio of the inlier number to the number of point cloud correspondence relationships, the inlier rate of the point cloud sample data can be obtained; the inlier rate refers to the ratio of the number of correct matches (i.e., inliers) to the number of all matches (including correct and incorrect matches) during the feature point matching process. It reflects the accuracy and stability of the matching algorithm; the inlier rate in this embodiment characterizes the sample quality of the point cloud sample data. By using the inlier rate in this embodiment, the label reliability of the point cloud sample data can be evaluated more strictly, making the labels of the subsequent trained model more accurate.

[0045] In some preferred embodiments of the present application, the present invention filters the outliers using a filter. First, the estimated transformation matrix T1 output by the model is applied to the estimated correspondence relationships, and then the source point cloud and the target point cloud after the application are mapped to the same spherical coordinate system, as Figure 3 shown. Define the center of the point cloud as O. For the source point cloud p i , Opi is the distance from the center to the source point cloud. With an r threshold, a spherical shell with a small ball radius of Opi - r and a large ball radius of Opi + r is selected, and the points q i of the target point cloud within the spherical shell range are selected, where q i points satisfy min{||R1·p i + t1 - q i || 2}, then the inlier correspondence relationship is retained and represented as (p i , q i ) s , otherwise it is an outlier correspondence relationship, and the outlier correspondence relationship is filtered out. Similarly, the pseudo-label transformation matrix T2 is applied to the pseudo-label correspondence relationships, the source point cloud and the target point cloud after the application are mapped to the same spherical coordinate system, and by a method similar to the above filtering method, the outlier relationships are filtered out, and finally the inlier correspondence relationship (p i , q i ) t is obtained. Through this filtering method, the number of operations can be significantly reduced, the outlier correspondence relationships can be quickly screened out, thereby improving the operation efficiency.

[0046] 103. In the self-correction training sample stage, the point cloud sample data of the fourth training set is used as the input of the point cloud registration model, and the feature matrix, estimated transformation matrix, and estimated correspondence relationship of the point cloud sample data are output; the fourth training set is the first training set after the last round of replacement or the data set after replacing the point cloud sample data of the first training set based on the second replacement principle; the second replacement principle is determined based on the pseudo-label score and estimated relationship score of the point cloud sample data, the pseudo-label score of the point cloud sample data is obtained by calculating the compatibility score from the pseudo-label correspondence relationship after the action of the pseudo-label transformation matrix, and the estimated relationship score of the point cloud data sample is obtained by calculating the compatibility score from the estimated correspondence relationship after the action of the estimated transformation matrix.

[0047] In the embodiment of the present application, in the self-correction training sample stage, since after the artificial correction training sample stage, there are already some point cloud sample data with true labels and point cloud sample data with pseudo-labels; by comparing the estimated matrix and the pseudo-label matrix to calculate scores, the matrix with the higher score is selected to update the training label of the sample. This strategy effectively utilizes the limited true label resources and dynamically optimizes the model performance, and finally achieves high-precision point cloud registration.

[0048] In some embodiments of the present application, the second replacement principle includes that if the pseudo-label score of the point cloud sample data is greater than the estimated relationship score, the pseudo-label of the point cloud sample data remains unchanged; if the pseudo-label score of the point cloud sample data is not greater than the estimated relationship score, the estimated relationship score value is used as the pseudo-label of the corresponding point cloud sample data. Specifically, in the last round of training the point cloud registration model in the artificial correction training sample stage, there are still some pseudo-labels with low confidence. Therefore, in this embodiment, by comparing the estimated matrix and the pseudo-label matrix to calculate scores, the matrix with the higher score is selected to update the training label of the sample. This strategy effectively utilizes the limited true label resources and dynamically optimizes the model performance, and finally achieves high-precision point cloud registration.

[0049] In some embodiments of the present application, the calculation method of the compatibility score includes determining the correspondence relationship between the same source point cloud and the corresponding target point cloud in different point cloud sample data; mapping the correspondence relationship between the same source point cloud and the corresponding target point cloud into connection nodes; calculating the compatibility distance between the connection nodes according to the difference between the similarity distances between different source point clouds and the similarity distances between the corresponding target point clouds; calculating the compatibility score between the connection nodes according to the compatibility distance between the connection nodes.

[0050] In this embodiment, the correspondence relationship between the i-th source point cloud p i of the point cloud sample data and the i-th target point cloud q i is mapped into a connection node ci; by calculating the source point cloud pi The similarity distance between the source point cloud p j and the similarity distance between the target point cloud q i and the target point cloud q j ; The difference between their similarity distances can reflect the compatibility distance between the connection node ci and the connection node cj; This compatibility distance measures the compatibility of different point clouds in the point cloud image, that is, in the point cloud sample data A, the similarity distance between the i-th source point cloud p i and the j-th source point cloud p j is a, then correspondingly, in the point cloud sample data B, the similarity distance between the i-th target point cloud q i and the j-th target point cloud q j should also be a or close to a. The smaller this compatibility distance is, the more accurate the extracted connection relationship is.

[0051] In the embodiment of the present application, as Figure 4 shown, the present application also provides an optimization device 200 for the training data of a point cloud registration model, including:

[0052] A point cloud data acquisition module 201, configured to acquire a first training set and a second training set, where the first training set includes a plurality of point cloud sample data with pseudo-labels, and the second training set includes a plurality of point cloud sample data with true labels;

[0053] A first training correction module 202, configured to use the point cloud sample data of the third training set as the input of the point cloud registration model during the artificial correction training sample stage, and output the feature matrix, estimated transformation matrix, and estimated correspondence relationship of the point cloud sample data; The third training set is the first training set or a data set obtained by replacing the point cloud sample data of the first training set based on the second training set according to the first replacement principle; The first replacement principle is determined based on the quality of the point cloud sample data, and the quality of the point cloud sample data is determined by the feature matrix of the point cloud sample data or by the pseudo-label transformation matrix and the estimated correspondence relationship of the point cloud sample data;

[0054] A second training correction module 203, configured to use the point cloud sample data of the fourth training set as the input of the point cloud registration model, and output the feature matrix, estimated transformation matrix, and estimated correspondence relationship of the point cloud sample data; The fourth training set is the first training set after the last round of replacement or a data set obtained by replacing the point cloud sample data of the first training set according to the second replacement principle; The second replacement principle is determined based on the pseudo-label score and the estimated relationship score of the point cloud sample data. The pseudo-label score of the point cloud sample data is obtained by calculating the compatibility score of the pseudo-label correspondence relationship after the action of the pseudo-label transformation matrix, and the estimated relationship score of the point cloud data sample is obtained by calculating the compatibility score of the estimated correspondence relationship after the action of the estimated transformation matrix.

[0055] In an embodiment of the present application, the present application further provides a computer device, as Figure 5 shown, the device includes:

[0056] One or more processors;

[0057] One or more memories for storing executable instructions of the one or more processors;

[0058] Wherein, the one or more processors are configured to execute the steps of the first aspect of the present application.

[0059] The training data optimization device of the point cloud registration model provided in this embodiment can execute the method described in any of the above embodiments, and its execution manner and beneficial effects are similar, which will not be elaborated here.

[0060] Specifically, the above-mentioned processor 310 may include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present application.

[0061] The memory 320 may include a mass storage for information or instructions. By way of example and not limitation, the memory 320 may include a hard disk drive (HDD), a floppy disk drive, a flash memory, an optical disc, a magneto-optical disc, a magnetic tape, or a universal serial bus (USB) drive, or a combination of two or more of these. In a suitable case, the memory 320 may include a removable or non-removable (or fixed) medium. In a suitable case, the memory 320 may be internal or external to the integrated gateway device. In a particular embodiment, the memory 320 is a non-volatile solid state memory. In a particular embodiment, the memory 320 includes a read-only memory (ROM). In a suitable case, the ROM may be a mask-programmed ROM, a programmable ROM (PROM), an erasable PROM (EPROM), an electrically erasable PROM (EEPROM), an electrically alterable ROM (EAROM), or a flash memory, or a combination of two or more of these.

[0062] The processor 310 reads and executes the computer program instructions stored in the memory 320 to perform the steps of the training data of the point cloud registration model provided by the embodiments of the present application.

[0063] In one example, the computer device may further include a transceiver 330 and a bus 340. Among them, as Figure 5 shown, the processor 310, the memory 320, and the transceiver 330 are connected through the bus 340 and complete communication with each other.

[0064] The bus 340 includes hardware, software, or both. By way of example and not limitation, the bus may include an Accelerated Graphics Port (AGP) or other graphics bus, an Extended Industry Standard Architecture (EISA) bus, a Front Side BUS (FSB), a Hyper Transport (HT) interconnect, an Industrial Standard Architecture (ISA) bus, an InfiniBand interconnect, a Low Pin Count (LPC) bus, a memory bus, a MicroChannel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local Bus (VLB) bus, or other suitable buses or a combination of two or more of these. In a suitable case, the bus 340 may include one or more buses. Although the embodiments of the present application describe and illustrate specific buses, the present application contemplates any suitable bus or interconnect.

[0065] The embodiments of the present application also provide a computer-readable storage medium, which may store a computer program. When the computer program is executed by a processor, the processor implements the point cloud registration model training method provided by the embodiments of the present application.

[0066] The above storage medium may include, for example, a memory 320 for computer program instructions, and the above instructions can be executed by a processor 310 of the point cloud registration model training device to complete the point cloud registration model training method provided by the embodiments of the present application. Optionally, the storage medium may be a non-transitory computer-readable storage medium. For example, the non-transitory computer-readable storage medium may be a ROM, a random access memory (RAM), a compact disc read-only memory (CD-ROM), magnetic tape, a floppy disk, and an optical data storage device, etc. The above computer program may be written in any combination of one or more programming languages to write program code for performing the operations of the embodiments of the present application. The programming languages include object-oriented programming languages such as Java, C++, etc., and also include conventional procedural programming languages such as the "C" language or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, executed as an independent software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0067] Those of ordinary skill in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing relevant hardware through a program, and the program can be stored in a computer-readable storage medium. The storage medium may include: ROM, RAM, a magnetic disk, or an optical disc, etc.

[0068] Although the embodiments of the present invention have been shown and described, for those of ordinary skill in the art, it can be understood that various changes, modifications, substitutions, and variations can be made to these embodiments without departing from the principles and spirit of the present invention. The scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A training data optimization method for a point cloud registration model, characterized in that: The method comprises: Acquire a first training set and a second training set, wherein the first training set includes a plurality of point cloud sample data with pseudo labels, and the second training set includes a plurality of point cloud sample data with real labels; the labels correspond to a transformation matrix between a source point cloud and a target point cloud in the point cloud sample data; In the manual correction training sample stage, the point cloud sample data of the third training set is used as the input of the point cloud registration model, and the feature matrix, estimated transformation matrix and estimated correspondence relationship of the point cloud sample data are output; the third training set is the first training set or a data set after the point cloud sample data of the first training set is replaced by the second training set based on the first replacement principle; wherein the first replacement principle is determined based on the quality of the point cloud sample data, and the quality of the point cloud sample data is determined by the feature matrix of the point cloud sample data or by the pseudo-label transformation matrix and estimated correspondence relationship of the point cloud sample data; In the self-correction training sample stage, the point cloud sample data of the fourth training set is used as the input of the point cloud registration model, and the feature matrix, estimated transformation matrix and estimated correspondence relationship of the point cloud sample data are output; the fourth training set is the first training set after the last round of replacement or the data set after replacing the point cloud sample data of the first training set based on the second replacement principle; wherein the second replacement principle is determined based on the pseudo-label score and the estimated relationship score of the point cloud sample data, the pseudo-label score of the point cloud sample data is obtained by calculating the compatibility score of the pseudo-label correspondence relationship after the pseudo-label transformation matrix acts on it, and the estimated relationship score of the point cloud data sample is obtained by calculating the compatibility score of the estimated correspondence relationship after the estimated transformation matrix acts on it.

2. The method for optimizing training data of a point cloud registration model according to claim 1, characterized in that: The method for obtaining the first training set includes obtaining an image corresponding to a source point cloud and an image corresponding to a target point cloud, performing feature extraction and image matching through a pre-trained model to obtain a correspondence relationship between 2D images; determining a correspondence relationship mapped to a 3D point cloud through the correspondence relationship between the 2D images; according to the correspondence relationship between the source point cloud and the target point cloud after mapping, solving the optimal posture transformation matrix between the two point clouds as a pseudo-label transformation matrix through a random sample consistency algorithm; the pseudo-label transformation matrix is ​​used as a pseudo-label of the first training set, and constitutes the point cloud sample data of the first training set together with the source point cloud and the target point cloud.

3. The training data optimization method for a point cloud registration model according to claim 1, characterized in that: The first replacement principle includes removing the point cloud sample data with lower quality in the first training set and replacing them with the point cloud sample data in the second training set.

4. The training data optimization method for a point cloud registration model according to claim 1 or 3, characterized in that: The quality of the point cloud sample data is determined by the feature matrix of the point cloud sample data, including calculating the similarity distance of the point cloud sample data according to the feature matrices of the source point cloud and the target point cloud in the point cloud sample data; the similarity distance of the point cloud sample data represents the quality of the point cloud sample data.

5. The method for optimizing training data of a point cloud registration model according to claim 1 or 3, characterized in that: The quality of the point cloud sample data is determined by the estimated correspondence relationship and the pseudo-label transformation matrix of the point cloud sample data, including calculating the number of inliers of the point cloud sample data according to the pseudo-label transformation matrix of the source point cloud and the target point cloud in the point cloud sample data; calculating the inlier rate of the point cloud sample data according to the ratio of the number of inliers of the point cloud sample data to the number of estimated correspondence relationships; The inlier rate of the point cloud sample data represents the quality of the point cloud sample data.

6. The method for optimizing training data of a point cloud registration model according to claim 5, characterized in that: The number of internal points of the point cloud sample data is calculated based on the pseudo-label transformation matrix of the source point cloud and the target point cloud in the point cloud sample data, including if it satisfies Then add an inner point; where R represents the rotation matrix in the pseudo-label transformation matrix, t represents the horizontal vector in the pseudo-label transformation matrix, and p i represents the i-th source point cloud of the point cloud sample data, q i represents the i-th target point cloud of the point cloud sample data, τ represents the preset threshold, i∈Ω pq ,Ω pq A collection of point cloud pairs representing point cloud sample data.

7. The method for optimizing training data of a point cloud registration model according to claim 1, characterized in that: The second replacement principle includes keeping the pseudo label of the point cloud sample data unchanged if the pseudo label score of the point cloud sample data is greater than the estimated relationship score; If the pseudo-label score of the point cloud sample data is not greater than the estimated relationship score, the estimated transformation matrix is ​​used as the pseudo-label of the corresponding point cloud sample data.

8. The method for optimizing training data of a point cloud registration model according to claim 1 or 7, characterized in that: The method for calculating the compatibility score includes determining the correspondence between the same source point cloud and the corresponding target point cloud in different point cloud sample data; mapping the correspondence between the same source point cloud and the corresponding target point cloud into connection nodes; calculating the compatibility distance between the connection nodes based on the difference between the similarity distance between different source point clouds and the similarity distance between the corresponding target point clouds; and calculating the compatibility score between the connection nodes based on the compatibility distance between the connection nodes.

9. A training data optimization device for a point cloud registration model, characterized in that: include: A point cloud data acquisition module, used to acquire a first training set and a second training set, wherein the first training set includes a plurality of point cloud sample data with pseudo labels, and the second training set includes a plurality of point cloud sample data with real labels; A first training correction module is used to use the point cloud sample data of the third training set as the input of the point cloud registration model in the manual correction training sample stage, and output the feature matrix, estimated transformation matrix and estimated correspondence relationship of the point cloud sample data; the third training set is the first training set or a data set after the point cloud sample data of the first training set is replaced by the second training set based on the first replacement principle; The first replacement principle is determined based on the quality of the point cloud sample data, wherein the quality of the point cloud sample data is determined by a feature matrix of the point cloud sample data or by a pseudo-label transformation matrix of the point cloud sample data and an estimated correspondence relationship; A second training correction module is used to use the point cloud sample data of the fourth training set as the input of the point cloud registration model, and output the feature matrix, estimated transformation matrix and estimated correspondence relationship of the point cloud sample data; the fourth training set is the first training set after the last round of replacement or a data set after the point cloud sample data of the first training set is replaced based on the second replacement principle; The second replacement principle is determined based on the pseudo-label score and estimated relationship score of the point cloud sample data. The pseudo-label score of the point cloud sample data is obtained by calculating the compatibility score of the pseudo-label correspondence relationship after the pseudo-label transformation matrix, and the estimated relationship score of the point cloud data sample is obtained by calculating the compatibility score of the estimated correspondence relationship after the estimated transformation matrix.

10. A computer device, characterized in that: The device comprises: One or more processors; one or more memories for storing instructions executable by the one or more processors; wherein the one or more processors are configured to perform the steps of any one of the methods of claims 1-8.

Images