Training data optimization method and device for point cloud registration model and computer equipment

Abstract

The invention relates to the technical field of point cloud data processing, in particular to a training data optimization method and device of a point cloud registration model and computer equipment. The method comprises the steps that a first training set and a second training set are acquired, the first training set comprises multiple pieces of point cloud sample data with pseudo labels, and the second training set comprises multiple pieces of point cloud sample data with real labels; the label corresponds to a transformation matrix between a source point cloud and a target point cloud in the point cloud sample data; in the manual training sample correction stage, point cloud sample data are selected through a first replacement principle; in the training sample self-correction stage, point cloud sample data are selected through a second replacement principle; according to the method, the accuracy of point cloud data alignment is improved, the feature distinction degree is improved through fusion of the structure and semantic information, and the registration result is more reliable.

Description

Technical Field

[0001] The present invention relates to the technical field of point cloud data processing, and in particular to a training data optimization method, device and computer equipment for a point cloud registration model. Background Art

[0002] Point cloud registration is the process of aligning multiple frames of 3D point cloud data acquired from different perspectives or sensors in the same coordinate system so that they represent the same scene in the same coordinate system.

[0003] At present, the most popular point cloud registration method is based on deep learning. In the point cloud registration task, it is difficult to obtain high-precision data labels when using deep learning technology for data training. Point cloud registration methods based on deep learning are mainly divided into self-supervised learning SuperLine3D and weakly supervised learning. The use of self-supervised learning methods can significantly reduce the dependence on labeled data. However, a major challenge of this method is that the design of self-supervised signals is quite complex. It requires the design of auxiliary tasks that can properly reflect the needs of point cloud registration to generate effective training signals. If these self-supervised tasks fail to accurately capture the key elements of point cloud registration, the model may not be able to learn the necessary geometric features. In addition, in the absence of clear guidance signals, model training under the self-supervised learning framework often converges slowly and may encounter stability problems in the early stages of training. Furthermore, the final point cloud registration accuracy is highly dependent on the initial estimated pose, and most self-supervised point cloud registration algorithms have a low tolerance for deviations in the initial point cloud pose. Large initial errors may make it difficult to find the correct transformation parameters during the optimization process.

[0004] Weakly supervised learning methods can use unlabeled data or only require a small amount of high-quality labels for model training, which greatly reduces the cost of data labeling. However, since only limited labeled samples are used, the model may only perform well in known or similar scenes, but its generalization performance may decline when facing unknown scenes. In addition, the labels used in weakly supervised learning may be noisy. If there is a large difference between these approximate labels and the actual labels, the model may learn incorrect alignment strategies. When processing point cloud datasets containing complex or large-scale transformations, due to the lack of sufficient detailed information, the model may only capture the local features of the point cloud, and it is difficult to understand global features or geometric relationships across objects.

[0005] In summary, the existing deep learning-based point cloud registration models cannot accurately select the true labels of point cloud datasets, resulting in a huge amount of manual annotation. Summary of the invention

[0006] Based on the problems existing in the prior art, the present invention proposes a training data optimization method, device and computer equipment for a point cloud registration model. The active learning framework is added to the training process of the point cloud registration model, so that it can select some real labels in the process of training the point cloud registration model, and construct a training set by pseudo-labeling, so that the workload of manual labeling can be reduced, and the registration effect close to that of using 100% labels can be achieved.

[0007] In a first aspect of the present application, the present application provides a training data optimization method for a point cloud registration model, the method comprising:

[0008] Acquire a first training set and a second training set, wherein the first training set includes a plurality of point cloud sample data with pseudo labels, and the second training set includes a plurality of point cloud sample data with real labels; the labels correspond to a transformation matrix between a source point cloud and a target point cloud in the point cloud sample data;

[0009] In the manual correction training sample stage, the point cloud sample data of the third training set is used as the input of the point cloud registration model, and the feature matrix, estimated transformation matrix and estimated correspondence relationship of the point cloud sample data are output; the third training set is the first training set or a data set after the point cloud sample data of the first training set is replaced by the second training set based on the first replacement principle; wherein the first replacement principle is determined based on the quality of the point cloud sample data, and the quality of the point cloud sample data is determined by the feature matrix of the point cloud sample data or by the pseudo-label transformation matrix and estimated correspondence relationship of the point cloud sample data;

[0010] In the self-correction training sample stage, the point cloud sample data of the fourth training set is used as the input of the point cloud registration model, and the feature matrix, estimated transformation matrix and estimated correspondence relationship of the point cloud sample data are output; the fourth training set is the first training set after the last round of replacement or the data set after replacing the point cloud sample data of the first training set based on the second replacement principle; wherein the second replacement principle is determined based on the pseudo-label score and the estimated relationship score of the point cloud sample data, the pseudo-label score of the point cloud sample data is obtained by calculating the compatibility score of the pseudo-label correspondence relationship after the pseudo-label transformation matrix acts on it, and the estimated relationship score of the point cloud data sample is obtained by calculating the compatibility score of the estimated correspondence relationship after the estimated transformation matrix acts on it.

[0011] In a second aspect of the present application, the present application further provides a training data optimization device for a point cloud registration model, comprising:

[0012] A point cloud data acquisition module, used to acquire a first training set and a second training set, wherein the first training set includes a plurality of point cloud sample data with pseudo labels, and the second training set includes a plurality of point cloud sample data with real labels;

[0013] A first training correction module is used to use the point cloud sample data of the third training set as the input of the point cloud registration model in the manual correction training sample stage, and output the feature matrix, estimated transformation matrix and estimated correspondence relationship of the point cloud sample data; the third training set is the first training set or a data set after the point cloud sample data of the first training set is replaced by the second training set based on a first replacement principle; wherein the first replacement principle is determined based on the quality of the point cloud sample data, and the quality of the point cloud sample data is determined by the feature matrix of the point cloud sample data or by the pseudo-label transformation matrix and the estimated correspondence relationship of the point cloud sample data;

[0014] The second training correction module is used to use the point cloud sample data of the fourth training set as the input of the point cloud registration model, and output the feature matrix, estimated transformation matrix and estimated correspondence relationship of the point cloud sample data; the fourth training set is the first training set after the last round of replacement or the data set after replacing the point cloud sample data of the first training set based on the second replacement principle; wherein the second replacement principle is determined based on the pseudo-label score and the estimated relationship score of the point cloud sample data, the pseudo-label score of the point cloud sample data is obtained by calculating the compatibility score of the pseudo-label correspondence relationship after the pseudo-label transformation matrix acts on it, and the estimated relationship score of the point cloud data sample is obtained by calculating the compatibility score of the estimated correspondence relationship after the estimated transformation matrix acts on it.

[0015] In a third aspect of the present application, the present application further provides a computer device, the device comprising:

[0016] one or more processors;

[0017] one or more memories for storing the one or more processor-executable instructions;

[0018] Wherein, the one or more processors are configured to execute the steps of the first aspect of the present application.

[0019] Beneficial effects of the present invention:

[0020] This application generates pseudo labels for unlabeled training samples through a pseudo label generator module; it is used to reduce the use of real labels and supervise training using pseudo labels. This application can use image information to generate reliable labels, reduce the labeling problem of a large amount of point cloud data, and optimize labels through layer-by-layer iteration, so as to generate accurate labels through a strict evaluation system. In the initial training stage, the model only uses pseudo labels, and evaluates the sample quality by calculating the cosine similarity between point cloud features, selects the sample with the lowest confidence to replace it with the real label, and performs manual correction. As the training progresses, the model is continuously iterated, and each round selects the sample with the lowest confidence to replace and correct the real label until the training reaches a predetermined number of rounds. Entering the model self-correction stage, the score is calculated by comparing the estimated matrix and the pseudo label matrix, and the matrix with a higher score is selected to update the training label of the sample. This strategy effectively utilizes the limited real label resources, while dynamically optimizing the model performance, and finally achieving high-precision point cloud registration. It is beneficial to the high-precision and high-efficiency point cloud registration, and the need for manual labeling is significantly reduced through the active learning strategy, while enhancing the generalization ability and robustness of the model. This application not only improves the accuracy of point cloud data alignment, but also improves feature discrimination by fusing structural and semantic information, making the alignment results more reliable. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 It is an operation diagram of the training data optimization scheme of the point cloud registration model in the embodiment of the present application;

[0022] Figure 2 is a flow chart of a training data optimization method for a point cloud registration model in an embodiment of the present application;

[0023] Figure 3 It is a schematic diagram of external point filtering in the preferred implementation of the present application;

[0024] Figure 4 It is a structural schematic diagram of a training data optimization device for a point cloud registration model according to an embodiment of the present application;

[0025] Figure 5 It is a schematic diagram of the computer device structure of an embodiment of the present application. DETAILED DESCRIPTION

[0026] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0027] Point cloud registration refers to the process of aligning point cloud data from different sources (for example, 3D scan data acquired from different viewing angles or time periods) into the same coordinate system. Given two frames of point cloud data from the same specific scene, they are called the source point cloud and the target point cloud. There are overlapping areas in the scene between these two frames of point clouds. Our goal is to use the information of these overlapping areas to calculate a 4×4 transformation matrix. This transformation matrix can accurately map the points in the source point cloud to the corresponding positions in the target point cloud, thereby achieving point cloud alignment or registration. In this process, some algorithms are usually required to identify matching point pairs in the overlapping area and estimate the transformation matrix based on these matching point pairs. These algorithms may include the Iterative Closest Point (ICP) algorithm, feature point matching algorithm, etc. Once the transformation matrix is ​​obtained, it can be applied to the entire source point cloud to achieve accurate alignment with the target point cloud. However, in this process, a large number of real-label three-dimensional point cloud matching tasks are required. The existing deep learning-based point cloud registration model cannot accurately select the real labels of the point cloud data set, resulting in a huge amount of manual labeling. Based on this, the present application proposes a point cloud registration method, device and computer equipment based on active learning, thereby reducing the workload of manual labeling, optimizing the training data of the point cloud registration model, and improving the accuracy of the point cloud registration model.

[0028] Figure 1 is a running diagram of the training data optimization scheme of the point cloud registration model in the embodiment of the present application; Figure 1As shown, the present application introduces a pseudo-label generator module for image matching correspondence, which generates pseudo-labels for unlabeled training samples. It is used to reduce the use of real labels and supervise the training of the point cloud registration model using pseudo-labels. Then, through the active learning framework of the embodiment of the present invention, the semi-supervised data labels are manually corrected and self-corrected. It is possible to train the data with fewer real labels, dynamically update the data labels during the training process, and finally achieve a training effect close to that of using 100% real labels. Among them, the pseudo-label generator uses the image matching of the source image and the target image to obtain the corresponding 2D correspondence and 3D projection to generate pseudo-labels, and then gradually introduces a small amount of real labels for correction using the active learning framework. In the initial training stage, the point cloud registration model only uses pseudo-labels, and evaluates the sample quality by calculating the feature similarity or inlier rate between the point cloud features, selects the sample with the lowest confidence and replaces it with the real label for manual correction. As the training proceeds, the point cloud registration model is continuously iterated, and the sample with the lowest confidence is selected in each round for real label replacement and correction until the training reaches a predetermined number of rounds. Entering the model self-correction stage, the score is calculated by comparing the estimated matrix and the pseudo-label matrix, and the matrix with a higher score is selected to update the training label of the sample. This strategy effectively utilizes the limited real label resources, while dynamically optimizing the model performance, and ultimately achieves high-precision point cloud registration.

[0029] Figure 2 is a flow chart of a training data optimization method for a point cloud registration model in an embodiment of the present application, such as Figure 2 As shown, the method includes:

[0030] 101. Obtain a first training set and a second training set, wherein the first training set includes a plurality of point cloud sample data with pseudo labels, and the second training set includes a plurality of point cloud sample data with real labels; the labels correspond to a transformation matrix between a source point cloud and a target point cloud in the point cloud sample data;

[0031] In some embodiments of the present application, the point cloud sample data of the first training set and the second training set are labeled respectively. Here, the purpose of the labeling can be to distinguish the points belonging to different objects in the point cloud sample data, wherein the point cloud sample data refers to the registration relationship between the source point cloud and the target point cloud in different point cloud images; the point cloud sample data of the first training set is labeled with pseudo labels, and the point cloud sample data set of the second training set is labeled with real labels; whether it is a pseudo label or a real label, the label here is the point cloud transformation matrix between the source point cloud and the target point cloud in the point cloud sample data, and the point cloud transformation matrix is ​​used to describe the translation and rotation operations of the point cloud in three-dimensional space. Usually, a complete transformation matrix includes two parts, rotation and translation, which can be decomposed into a rotation matrix and a horizontal vector.

[0032] In some embodiments of the present application, the training sets may be annotated by a user manually annotating the first training set and the second training set, or may be annotated automatically by a labeling algorithm.

[0033] In a preferred embodiment of the present application, the pseudo labels of the first training set are obtained by annotating with a pseudo label generator, and the method for obtaining the first training set includes obtaining an image corresponding to the source point cloud and an image corresponding to the target point cloud, performing feature extraction and image matching through a pre-trained model to obtain the correspondence between the 2D images; determining the correspondence mapped to the 3D point cloud through the correspondence between the 2D images; according to the correspondence between the source point cloud and the target point cloud after mapping, solving the optimal pose transformation matrix between the two point clouds through a random sample consensus algorithm as a pseudo label transformation matrix; the pseudo label transformation matrix is ​​used as the pseudo label of the first training set, and constitutes the point cloud sample data of the first training set together with the source point cloud and the target point cloud.

[0034] Among them, the corresponding relationship mapped to the 3D point cloud is determined by the corresponding relationship of the 2D image, including corresponding the 3D point cloud obtained by projection with the key points of the 2D image, corresponding the key points on the 2D image to the real 3D point cloud space, filtering out low-quality matching point pairs through the confidence threshold, and finally retaining the key point pairs of the 2D-3D image with high confidence.

[0035] For example, in this embodiment, the source image and the target image can be matched by a pre-trained model to obtain the 2D corresponding pixel pairs (x i ,y i ) s and (x i ,y i ) t , where the superscript s represents the source image, the superscript t represents the target image, and the subscript i represents the index of the point cloud; then the 2D correspondence is projected into the 3D space. In this projection calculation process, it is necessary to obtain the intrinsic and extrinsic parameters of the camera, and use the pixel matching points and camera parameters to calculate the positions of the 3D points P1 (source point) and P2 (target point); specifically, assuming that the point (x1, y1) in the 2D source image s , establish the equation: (x1, y1) s =K[R|t]P1; K is the camera internal parameter matrix 3×3, [R|t] is the external parameter matrix 3×4; according to the pixel coordinates, first add one dimension to the 2D coordinates to become homogeneous coordinates (x1, y1, 1) s , inversely solve the mapped 3D point P1 (x1, y1, z1, w1), and then normalize the P1 point Take the first three normalized coordinates as P1 (x1, y1, z1), so that the projection of the 2D pixel point to the 3D point is realized, and then find the other point (x i ,y i ) t The mapped 3D point (x i ,y i ) t =K[R|t]P2, first transform (x i ,y i ) t is the homogeneous coordinate (x1, y1, 1) t , inversely solve the mapped 3D point P2 (x2, y2, z2, w2), and then normalize the P2 point Take the first three normalized coordinates as P2(x2, y2, z2), so that we get a pair of 3D correspondences, and then use RANSAC to solve the pseudo-labels of the point cloud sample data based on the 3D correspondences, and finally form a pseudo-label set.

[0036] 102. In the manual correction training sample stage, the point cloud sample data of the third training set is used as the input of the point cloud registration model, and the feature matrix, estimated transformation matrix and estimated correspondence relationship of the point cloud sample data are output; the third training set is the first training set or a data set after the point cloud sample data of the first training set is replaced by the second training set based on the first replacement principle; wherein the first replacement principle is determined based on the quality of the point cloud sample data, and the quality of the point cloud sample data is determined by the feature matrix of the point cloud sample data or by the pseudo-label transformation matrix and the estimated correspondence relationship of the point cloud sample data;

[0037] In some embodiments of the present application, the point cloud registration model may be an existing point cloud registration model; for example, a deep neural network (such as a convolutional neural network, a recurrent neural network, etc.) is used to perform feature extraction and representation learning on point cloud data. These networks can capture the complex distribution and geometric features of point cloud data, and provide strong support for subsequent registration tasks. Specifically, it may be a DeepGMR model, or it may be PointNetLK, FlowNet3D, etc. The present invention does not limit the specific point cloud registration model, and any point cloud registration model that requires data training using deep learning technology can be implemented.

[0038] In some embodiments of the present application, in the manual correction training sample stage, a small amount of point cloud sample data with real labels is selected, and a large amount of point cloud sample data with pseudo labels is selected. These point cloud sample data are sent to the KPConv-FPN network for multi-level downsampling to obtain a series of point clouds with different resolutions. At each downsampling level, the KPConv layer is used to extract features of each point. Then, at the coarsest resolution level, the downsampled points are used as super points, and the relevant learning features are learned; then, the super points are geometrically embedded based on the geometric structure of pairwise distances and ternary angles. This super point correspondence is then sent to the LGR (Local-to-Global Registration) estimator, and the speed and accuracy of point cloud registration are rapidly improved through local and global stage estimation.

[0039] In some embodiments of the present application, the first replacement principle includes removing point cloud sample data with lower quality in the first training set and replacing it with point cloud sample data in the second training set. During the manual correction of training samples, the point cloud sample data of the second training set can be replaced one by one; for example, when the point cloud registration model is trained for the first time, 100 point cloud sample data of the first training set are used, and the point cloud sample data of the second training set are not used; when the point cloud registration model is trained for the second time, 95 point cloud sample data of the first training set are used, and 5 point cloud sample data of the second training set are used; among them, the 5 point cloud sample data replaced in the first training set have lower sample quality among the 100 point cloud sample data; similarly, when the point cloud registration model is trained for the third time, 90 point cloud sample data of the first training set are used, and 10 point cloud sample data of the second training set are used; this method can replace the point cloud sample data with low pseudo-label confidence with the point cloud sample data with real labels, and the model is continuously iterated, and the sample with the lowest confidence is selected in each round for real label replacement and correction until the training reaches the predetermined number of rounds; this strategy effectively utilizes limited real label resources.

[0040] In some embodiments of the present application, the quality of the point cloud sample data is determined by a feature matrix of the point cloud sample data, including calculating a similarity distance of the point cloud sample data based on the feature matrices of a source point cloud and a target point cloud in the point cloud sample data; the similarity distance of the point cloud sample data characterizes the quality of the point cloud sample data.

[0041] For example, in the first round of point cloud registration model training, this embodiment selects the first training set to input into the point cloud registration model, that is, the label distribution in the training sample is 0% real label, and the rest is 100% pseudo label for training. Then the point cloud registration model will output a source point cloud feature matrix F P And the target point cloud feature matrix FQ ; Calculate the cosine similarity D between the point cloud feature matrices of the i-th pair of samples i (F P ,F Q ); thereby obtaining the quality score of the point cloud sample data, and then sorting the scores of all the point cloud sample data, selecting the K point cloud sample data with the smallest scores, and replacing the real labels in the point cloud sample data of the second training set with the pseudo labels of these K point cloud sample data, and performing manual correction. Similarly, continue to send the replaced point cloud sample data to the point cloud registration model for a second training. For the point cloud sample data with pseudo labels, calculate the cosine similarity between the source point cloud and target point cloud features of the sample, and continue to select the K point cloud sample data with the smallest scores. Replace the real labels with the pseudo labels of these point cloud sample data, and perform manual correction. Until the number of training rounds is greater than 5, this embodiment believes that enough real labels have been added to maintain the training of this model.

[0042] In some embodiments of the present application, the quality of the point cloud sample data is determined by the estimated correspondence relationship and the pseudo-label transformation matrix of the point cloud sample data, including calculating the number of inliers of the point cloud sample data based on the pseudo-label transformation matrix of the source point cloud and the target point cloud in the point cloud sample data; calculating the inlier rate of the point cloud sample data based on the ratio of the number of inliers of the point cloud sample data to the number of estimated correspondence relationships; the inlier rate of the point cloud sample data represents the quality of the point cloud sample data.

[0043] The number of internal points of the point cloud sample data is calculated based on the pseudo-label transformation matrix of the source point cloud and the target point cloud in the point cloud sample data, including if it satisfies Then add an inner point; where R represents the rotation matrix in the pseudo-label transformation matrix, t represents the horizontal vector in the pseudo-label transformation matrix, and p i represents the i-th source point cloud of the point cloud sample data, q i represents the i-th target point cloud of the point cloud sample data, τ represents the preset threshold, i∈Ω pq ,Ω pq Represents a set of point cloud pairs of point cloud sample data; the pseudo label matrix of point cloud sample data is represented as (R, t).

[0044] For example, when point cloud sample data is input into the point cloud registration model, the feature matrix, estimated transformation matrix and estimated correspondence relationship of the point cloud sample data are output; this embodiment uses the estimated correspondence relationship of the point cloud sample data to form a point cloud correspondence relationship set Ω, where (p i ,q i ∈Ω), P={p1, p2,...,p n} and Q = {q1, q2, ..., q n}, n represents the number of point cloud correspondences; by traversing the corresponding relationship set Ω, if it satisfies Then let the number of interior points N inliers Add 1, and the inlier rate of the point cloud sample data can be obtained by the ratio of the number of inliers to the number of point cloud correspondences; the inlier rate refers to the ratio of the number of correct matches (i.e., inliers) to the number of all matches (including correct matches and incorrect matches) during the feature point matching process. It reflects the accuracy and stability of the matching algorithm; the inlier rate in this embodiment characterizes the sample quality of the point cloud sample data. This embodiment adopts the inlier rate method to more strictly evaluate the label reliability of the point cloud sample data, making the model labels of subsequent training more accurate.

[0045] In some preferred embodiments of the present application, the present invention uses a filter to filter the external points. First, the estimated transformation matrix T1 output by the model is applied to the estimated correspondence relationship, and then the source point cloud and the target point cloud after the action are mapped to the same spherical coordinate system, such as Figure 3 As shown, the center of the point cloud is defined as O. For the source point cloud p i , Opi is the distance from the center to the source point cloud, with r as the threshold, Opi-r is the radius of the small ball, Opi+r is the spherical shell of the large ball radius, and point q of the target point cloud within the spherical shell is selected i , where q i Point satisfies min{||R1·p i +t1-q i || 2}, the internal point correspondence is retained and expressed as (p i ,q i ) s , otherwise it is an external point correspondence, and the external point correspondence is filtered out. Similarly, the pseudo-label transformation matrix T2 is applied to the pseudo-label correspondence, and the source point cloud and the target point cloud are mapped to the same spherical coordinate system. The external point relationship is filtered out by a method similar to the above filtering method, and finally the internal point correspondence (p i ,q i ) t Through this filtering method, the number of operations can be significantly reduced, and the correspondence of outliers can be quickly screened out, thereby improving the operation efficiency.

[0046] 103. In the self-correction training sample stage, the point cloud sample data of the fourth training set is used as the input of the point cloud registration model, and the feature matrix, estimated transformation matrix and estimated correspondence relationship of the point cloud sample data are output; the fourth training set is the first training set after the last round of replacement or the data set after replacing the point cloud sample data of the first training set based on the second replacement principle; wherein the second replacement principle is determined based on the pseudo-label score and the estimated relationship score of the point cloud sample data, the pseudo-label score of the point cloud sample data is obtained by calculating the compatibility score of the pseudo-label correspondence relationship after the pseudo-label transformation matrix acts on it, and the estimated relationship score of the point cloud data sample is obtained by calculating the compatibility score of the estimated correspondence relationship after the estimated transformation matrix acts on it.

[0047] In an embodiment of the present application, in the self-correction training sample stage, after the manual correction training sample stage is completed, some point cloud sample data with real labels and point cloud sample data with pseudo-labels are already included; the scores are calculated by comparing the estimated matrix and the pseudo-label matrix, and the matrix with higher scores is selected to update the training labels of the samples. This strategy effectively utilizes limited real label resources, while dynamically optimizing model performance, and ultimately achieves high-precision point cloud alignment.

[0048] In some embodiments of the present application, the second replacement principle includes that if the pseudo-label score of the point cloud sample data is greater than the estimated relationship score, the pseudo-label of the point cloud sample data is kept unchanged; if the pseudo-label score of the point cloud sample data is not greater than the estimated relationship score, the estimated relationship score value is used as the pseudo-label of the corresponding point cloud sample data. Specifically, in the last round of training the point cloud registration model in the manual correction training sample stage, there are still some pseudo-labels with low confidence. Therefore, this embodiment calculates the score by comparing the estimated matrix and the pseudo-label matrix, and selects the matrix with a higher score to update the training label of the sample. This strategy effectively utilizes limited real label resources, while dynamically optimizing model performance, and ultimately achieves high-precision point cloud registration.

[0049] In some embodiments of the present application, the method for calculating the compatibility score includes determining the correspondence between the same source point cloud and the corresponding target point cloud in different point cloud sample data; mapping the correspondence between the same source point cloud and the corresponding target point cloud into connecting nodes; calculating the compatibility distance between the connecting nodes based on the difference between the similarity distance between different source point clouds and the similarity distance between the corresponding target point clouds; and calculating the compatibility score between the connecting nodes based on the compatibility distance between the connecting nodes.

[0050] In this embodiment, the i-th source point cloud p of the point cloud sample data is i and the i-th target point cloud q i The corresponding relationship between them is mapped into a connection node ci; by calculating the source point cloud pi With the source point cloud p j The similarity distance between them, and the target point cloud q i With the target point cloud q j The similarity distance between them; the difference of their similarity distances can reflect the compatibility distance between the connecting node ci and the connecting node cj; this compatibility distance measures the compatibility of different point clouds in the point cloud image, that is, in the point cloud sample data A, the i-th source point cloud p i With the j-th source point cloud p j The similarity distance is a, then correspondingly, in the point cloud sample data B, the i-th target point cloud q i and the j-th target point cloud q j The similarity distance should also be a or close to a. The smaller the compatibility distance, the more accurate the extracted connection relationship.

[0051] In the embodiments of the present application, Figure 4 As shown, the present application also provides a training data optimization device 200 for a point cloud registration model, comprising:

[0052] The point cloud data acquisition module 201 is used to acquire a first training set and a second training set, wherein the first training set includes a plurality of point cloud sample data with pseudo labels, and the second training set includes a plurality of point cloud sample data with real labels;

[0053] The first training correction module 202 is used to use the point cloud sample data of the third training set as the input of the point cloud registration model in the manual correction training sample stage, and output the feature matrix, estimated transformation matrix and estimated correspondence relationship of the point cloud sample data; the third training set is the first training set or a data set after the point cloud sample data of the first training set is replaced by the second training set based on the first replacement principle; wherein the first replacement principle is determined based on the quality of the point cloud sample data, and the quality of the point cloud sample data is determined by the feature matrix of the point cloud sample data or by the pseudo-label transformation matrix and estimated correspondence relationship of the point cloud sample data;

[0054] The second training correction module 203 is used to use the point cloud sample data of the fourth training set as the input of the point cloud registration model, and output the feature matrix, estimated transformation matrix and estimated correspondence relationship of the point cloud sample data; the fourth training set is the first training set after the last round of replacement or the data set after replacing the point cloud sample data of the first training set based on the second replacement principle; wherein the second replacement principle is determined based on the pseudo-label score and the estimated relationship score of the point cloud sample data, the pseudo-label score of the point cloud sample data is obtained by calculating the compatibility score of the pseudo-label correspondence relationship after the pseudo-label transformation matrix acts on it, and the estimated relationship score of the point cloud data sample is obtained by calculating the compatibility score of the estimated correspondence relationship after the estimated transformation matrix acts on it.

[0055] In the embodiment of the present application, the present application also provides a computer device, such as Figure 5 As shown, the device includes:

[0056] one or more processors;

[0057] one or more memories for storing the one or more processor-executable instructions;

[0058] Wherein, the one or more processors are configured to execute the steps of the first aspect of the present application.

[0059] The training data optimization device for the point cloud registration model provided in this embodiment can execute the method described in any of the above embodiments, and its execution method and beneficial effects are similar and will not be repeated here.

[0060] Specifically, the processor 310 may include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or may be configured to implement one or more integrated circuits of the embodiments of the present application.

[0061] The memory 320 may include a large capacity memory for information or instructions. By way of example and not limitation, the memory 320 may include a hard disk drive (HDD), a floppy disk drive, a flash memory, an optical disk, a magneto-optical disk, a tape, or a universal serial bus (USB) drive, or a combination of two or more of these. Where appropriate, the memory 320 may include a removable or non-removable (or fixed) medium. Where appropriate, the memory 320 may be inside or outside the integrated gateway device. In a particular embodiment, the memory 320 is a non-volatile solid-state memory. In a particular embodiment, the memory 320 includes a read-only memory (ROM). Where appropriate, the ROM may be a mask-programmed ROM, a programmable ROM (PROM), an erasable PROM (EPROM), an electrically erasable PROM (Electrically Erasable Programmable ROM, EEPROM), an electrically rewritable ROM (EAROM), or a flash memory, or a combination of two or more of these.

[0062] The processor 310 reads and executes the computer program instructions stored in the memory 320 to perform the steps of training data of the point cloud registration model provided in the embodiment of the present application.

[0063] In one example, the computer device may further include a transceiver 330 and a bus 340. Figure 5 As shown, the processor 310, the memory 320 and the transceiver 330 are connected via a bus 340 and communicate with each other.

[0064] The bus 340 includes hardware, software, or both. For example, but not limitation, the bus may include an Accelerated Graphics Port (AGP) or other graphics bus, an Extended Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), a Hyper Transport (HT) interconnect, an Industrial Standard Architecture (ISA) bus, an InfiniBand interconnect, a Low Pin Count (LPC) bus, a Memory bus, a MicroChannel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local Bus (VLB) bus, or other suitable buses or a combination of two or more of these. Where appropriate, the bus 340 may include one or more buses. Although embodiments of the present application describe and illustrate a particular bus, the present application contemplates any suitable bus or interconnect.

[0065] An embodiment of the present application also provides a computer-readable storage medium, which can store a computer program. When the computer program is executed by a processor, the processor implements the point cloud registration model training method provided in the embodiment of the present application.

[0066] The above-mentioned storage medium may, for example, include a memory 320 of computer program instructions, and the above-mentioned instructions may be executed by the processor 310 of the point cloud registration model training device to complete the point cloud registration model training method provided in the embodiment of the present application. Optionally, the storage medium may be a non-temporary computer-readable storage medium, for example, the non-temporary computer-readable storage medium may be a ROM, a random access memory (Random Access Memory, RAM), a compact disc read-only memory (Compact Disc ROM, CD-ROM), a magnetic tape, a floppy disk, and an optical data storage device, etc. The above-mentioned computer program may be written in any combination of one or more programming languages ​​to perform the program code of the operation of the embodiment of the present application, and the programming language includes an object-oriented programming language, such as Java, C++, etc., and also includes a conventional procedural programming language, such as "C" language or a similar programming language. The program code may be executed entirely on the user computing device, partially on the user device, as an independent software package, partially on the user computing device, partially on the remote computing device, or entirely on the remote computing device or server.

[0067] A person skilled in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing the relevant hardware through a program, and the program can be stored in a computer-readable storage medium, which can include: ROM, RAM, disk or CD, etc.

[0068] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A training data optimization method for a point cloud registration model, characterized in that: The method comprises: Acquire a first training set and a second training set, wherein the first training set includes a plurality of point cloud sample data with pseudo labels, and the second training set includes a plurality of point cloud sample data with real labels; the labels correspond to a transformation matrix between a source point cloud and a target point cloud in the point cloud sample data; In the manual correction training sample stage, the point cloud sample data of the third training set is used as the input of the point cloud registration model, and the feature matrix, estimated transformation matrix and estimated correspondence relationship of the point cloud sample data are output; the third training set is the first training set or a data set after the point cloud sample data of the first training set is replaced by the second training set based on the first replacement principle; wherein the first replacement principle is determined based on the quality of the point cloud sample data, and the quality of the point cloud sample data is determined by the feature matrix of the point cloud sample data or by the pseudo-label transformation matrix and estimated correspondence relationship of the point cloud sample data; In the self-correction training sample stage, the point cloud sample data of the fourth training set is used as the input of the point cloud registration model, and the feature matrix, estimated transformation matrix and estimated correspondence relationship of the point cloud sample data are output; the fourth training set is the first training set after the last round of replacement or the data set after replacing the point cloud sample data of the first training set based on the second replacement principle; wherein the second replacement principle is determined based on the pseudo-label score and the estimated relationship score of the point cloud sample data, the pseudo-label score of the point cloud sample data is obtained by calculating the compatibility score of the pseudo-label correspondence relationship after the pseudo-label transformation matrix acts on it, and the estimated relationship score of the point cloud data sample is obtained by calculating the compatibility score of the estimated correspondence relationship after the estimated transformation matrix acts on it.

2. The method for optimizing training data of a point cloud registration model according to claim 1, characterized in that: The method for obtaining the first training set includes obtaining an image corresponding to a source point cloud and an image corresponding to a target point cloud, performing feature extraction and image matching through a pre-trained model to obtain a correspondence relationship between 2D images; determining a correspondence relationship mapped to a 3D point cloud through the correspondence relationship between the 2D images; according to the correspondence relationship between the source point cloud and the target point cloud after mapping, solving the optimal posture transformation matrix between the two point clouds as a pseudo-label transformation matrix through a random sample consistency algorithm; the pseudo-label transformation matrix is ​​used as a pseudo-label of the first training set, and constitutes the point cloud sample data of the first training set together with the source point cloud and the target point cloud.

3. The training data optimization method for a point cloud registration model according to claim 1, characterized in that: The first replacement principle includes removing the point cloud sample data with lower quality in the first training set and replacing them with the point cloud sample data in the second training set.

4. The training data optimization method for a point cloud registration model according to claim 1 or 3, characterized in that: The quality of the point cloud sample data is determined by the feature matrix of the point cloud sample data, including calculating the similarity distance of the point cloud sample data according to the feature matrices of the source point cloud and the target point cloud in the point cloud sample data; the similarity distance of the point cloud sample data represents the quality of the point cloud sample data.

5. The method for optimizing training data of a point cloud registration model according to claim 1 or 3, characterized in that: The quality of the point cloud sample data is determined by the estimated correspondence relationship and the pseudo-label transformation matrix of the point cloud sample data, including calculating the number of inliers of the point cloud sample data according to the pseudo-label transformation matrix of the source point cloud and the target point cloud in the point cloud sample data; calculating the inlier rate of the point cloud sample data according to the ratio of the number of inliers of the point cloud sample data to the number of estimated correspondence relationships; The inlier rate of the point cloud sample data represents the quality of the point cloud sample data.

6. The method for optimizing training data of a point cloud registration model according to claim 5, characterized in that: The number of internal points of the point cloud sample data is calculated based on the pseudo-label transformation matrix of the source point cloud and the target point cloud in the point cloud sample data, including if it satisfies Then add an inner point; where R represents the rotation matrix in the pseudo-label transformation matrix, t represents the horizontal vector in the pseudo-label transformation matrix, and p i represents the i-th source point cloud of the point cloud sample data, q i represents the i-th target point cloud of the point cloud sample data, τ represents the preset threshold, i∈Ω pq ,Ω pq A collection of point cloud pairs representing point cloud sample data.

7. The method for optimizing training data of a point cloud registration model according to claim 1, characterized in that: The second replacement principle includes keeping the pseudo label of the point cloud sample data unchanged if the pseudo label score of the point cloud sample data is greater than the estimated relationship score; If the pseudo-label score of the point cloud sample data is not greater than the estimated relationship score, the estimated transformation matrix is ​​used as the pseudo-label of the corresponding point cloud sample data.

8. The method for optimizing training data of a point cloud registration model according to claim 1 or 7, characterized in that: The method for calculating the compatibility score includes determining the correspondence between the same source point cloud and the corresponding target point cloud in different point cloud sample data; mapping the correspondence between the same source point cloud and the corresponding target point cloud into connection nodes; calculating the compatibility distance between the connection nodes based on the difference between the similarity distance between different source point clouds and the similarity distance between the corresponding target point clouds; and calculating the compatibility score between the connection nodes based on the compatibility distance between the connection nodes.

9. A training data optimization device for a point cloud registration model, characterized in that: include: A point cloud data acquisition module, used to acquire a first training set and a second training set, wherein the first training set includes a plurality of point cloud sample data with pseudo labels, and the second training set includes a plurality of point cloud sample data with real labels; A first training correction module is used to use the point cloud sample data of the third training set as the input of the point cloud registration model in the manual correction training sample stage, and output the feature matrix, estimated transformation matrix and estimated correspondence relationship of the point cloud sample data; the third training set is the first training set or a data set after the point cloud sample data of the first training set is replaced by the second training set based on the first replacement principle; The first replacement principle is determined based on the quality of the point cloud sample data, wherein the quality of the point cloud sample data is determined by a feature matrix of the point cloud sample data or by a pseudo-label transformation matrix of the point cloud sample data and an estimated correspondence relationship; A second training correction module is used to use the point cloud sample data of the fourth training set as the input of the point cloud registration model, and output the feature matrix, estimated transformation matrix and estimated correspondence relationship of the point cloud sample data; the fourth training set is the first training set after the last round of replacement or a data set after the point cloud sample data of the first training set is replaced based on the second replacement principle; The second replacement principle is determined based on the pseudo-label score and estimated relationship score of the point cloud sample data. The pseudo-label score of the point cloud sample data is obtained by calculating the compatibility score of the pseudo-label correspondence relationship after the pseudo-label transformation matrix, and the estimated relationship score of the point cloud data sample is obtained by calculating the compatibility score of the estimated correspondence relationship after the estimated transformation matrix.

10. A computer device, characterized in that: The device comprises: One or more processors; one or more memories for storing instructions executable by the one or more processors; wherein the one or more processors are configured to perform the steps of any one of the methods of claims 1-8.

Images