Three-dimensional point cloud registration method and device, equipment and medium

Abstract

The invention discloses a three-dimensional point cloud registration method, device and equipment and a medium, and relates to the technical field of medical image processing, and the method comprises the steps: firstly extracting local regions of point clouds of different scales, and enabling the local regions to strongly correspond to solve the problem of noise points of the point clouds of patient organization; and constructing a virtual corresponding relationship between the source point cloud and the target point cloud to solve the objective difference problem between the source point cloud and the target point cloud. The calculation amount of the feature fusion module is reduced, the parameter amount is low, the network calculation amount is reduced while the relation between the source point cloud and the target point cloud is effectively analyzed, and the problems of large parameter amount and high deployment cost are solved. The corresponding relation filter is independent of the network, network errors are reduced, the registration error problem caused by wrong corresponding points is solved, and meanwhile extra calculation amount is not introduced.

Description

Technical Field

[0001] The present invention relates to the technical field of medical image processing, and particularly to a three-dimensional point cloud registration method, device, equipment and medium based on deep learning and applicable to surgical scenarios. Background Technique

[0002] Point cloud registration has relatively wide applications in surgical scenarios such as cranio-maxillofacial surgery and neurosurgery. Compared with traditional two-dimensional images, three-dimensional point clouds have stronger geometric and structural information, avoiding problems such as illumination and pose changes in 2D images. Generally speaking, three-dimensional point cloud registration in surgical scenarios needs to meet three core requirements:

[0003] High-precision rigid alignment: It is necessary to align the patient tissue model reconstructed from preoperative CT / MRI with the point cloud (such as laser scanning) collected in real time during the operation, and high-precision registration should be ensured to guarantee the safety of the operation;

[0004] Strong anti-interference ability: There are dynamic occlusions in the intraoperative environment (such as surgical instruments or the doctor's hand movements blocking the patient's tissue, and blood stains contaminating the point cloud) and the patient's pose movement, and traditional registration methods face severe challenges;

[0005] Fast processing ability: The surgical scenario has high requirements for latency. Generally speaking, the speed that the human eye can adapt to is not less than 30 frames per second. When it is lower than this threshold, the safety of the operation may be reduced.

[0006] When traditional methods are used to process point cloud registration, they often need to rely on an initial good corresponding relationship between the source point cloud and the target point cloud, which is difficult to achieve in practical applications. In addition, there will be mechanical occlusions and pose changes in the surgical scenario, and traditional methods are also easily affected by this problem and have poor anti-interference ability.

[0007] In recent years, with the proposal of deep learning, new breakthroughs have been continuously made in its achievements on two-dimensional images. More and more researchers have begun to devote themselves to applying deep learning technology to the processing of three-dimensional point clouds. According to whether there is a corresponding relationship, it can be divided into methods without corresponding points and methods with corresponding points. The former estimates the motion parameters by searching for the differences in the global features of the two point clouds, and most of the latter research is basically carried out based on the process of traditional methods.

[0008] The method based on non-corresponding points essentially extracts global information from the source point cloud and the target point cloud, but usually ignores local information. This may lead to a serious problem, that is, the influence of outliers on the extraction of global information is too large. On the contrary, the differential information in the point cloud registration task is usually composed of multiple groups of local information, and the generalization ability of these local features is often stronger. Although the method based on local correspondence has achieved certain results, there are still the following limitations:

[0009] 1. Due to the reasons of 3D point cloud acquisition equipment, there are objective differences between the source point cloud and the target point cloud. Direct registration from the source point cloud and the target point cloud may lead to insufficient extraction of registration accuracy.

[0010] 2. The point cloud of the patient's tissue in the intraoperative scene may have large - area occlusion, be complex and changeable, and is more likely to introduce noise points.

[0011] 3. Mainstream deep - learning models have large computational amounts, large numbers of parameters, high complexity, high deployment costs on general medical computer equipment, may also lead to slow processing speeds, and cannot meet real - time requirements.

[0012] 4. When calculating the rotation - translation matrix, the influence of incorrect corresponding points may lead to low registration accuracy, but filtering incorrect corresponding points by deep - learning methods will introduce complexity.

[0013] Therefore, a 3D point cloud registration system based on deep learning should meet the following requirements: (1) reduce the objective differences between the source point cloud and the target point cloud; (2) consider the complex intraoperative scene and be able to effectively filter noise; (3) reduce the computational amount, the number of parameters and the complexity of the model while ensuring registration accuracy; (4) filter incorrect corresponding points as much as possible while ensuring the computational efficiency of the model. Summary of the Invention

[0014] In view of the above problems, the present invention provides a 3D point cloud registration method, device, equipment and medium for overcoming the above problems or at least partially solving the above problems. It is applicable to high - precision point cloud alignment in scenarios with occlusion (such as surgical instruments and bloodstain interference) and intraoperative patient pose changes (such as translation / rotation) in surgical navigation.

[0015] The present invention provides the following solutions:

[0016] A 3D point cloud registration method, comprising:

[0017] Obtain a source point cloud and a target point cloud; the source point cloud includes the occluded point cloud of the patient's tissue collected during the operation, and the target point cloud includes the complete point cloud of the patient's tissue collected before the operation;

[0018] Use a 3D point cloud initial feature extraction module to respectively extract features from the source point cloud and the target point cloud to obtain the initial features of the source point cloud and the initial features of the target point cloud;

[0019] Use self - attention and cross - attention to respectively enhance the initial features of the source point cloud and the initial features of the target point cloud and cross - fuse the enhanced features to generate cross - fused features;

[0020] Generate a soft matching matrix based on the cross - fused features and the source point cloud;

[0021] Generate a virtual source point cloud using the soft matching matrix and the source point cloud with a learnable multi-class support vector machine;

[0022] Establish an initial correspondence between the virtual source point cloud and the target point cloud to obtain corresponding point pairs;

[0023] Calculate the second-order similarity measure scores of the corresponding point pairs, calculate the number of jointly compatible matching points based on the similarity measure, and complete the screening of the corresponding point pairs in sequence;

[0024] According to the information vectors of the corresponding point pairs obtained by screening, generate a target rotation and translation matrix through singular value decomposition, so as to transfer the source point cloud to the coordinate system of the target point cloud through the target rotation and translation matrix.

[0025] Preferably: the three-dimensional point cloud initial feature extraction module is used to perform feature dimension elevation on the point cloud, then construct a KD-tree feature distribution map based on the elevated features, and construct information at different scales under the same center point according to the KD-tree feature distribution map.

[0026] Preferably: according to the KD-tree feature distribution map, for each point p in the point cloud P with N points i ∈P, i∈N, construct K-nearest neighbor vectors in local regions with different K values, denoted as F k1 =f1, f2, …, f k1 , F k2 =f1, f2, …, f k2 ;

[0027] Use the Sigmoid function to calculate the corresponding vector scores for the K-nearest neighbor vectors in different regions, that is:

[0028] S k1 =Sigmoid(F k1 )

[0029] S k2 =Sigmoid(F k2 )

[0030] In the formula, S k1 and S k2 are the corresponding vector scores respectively;

[0031] Perform matrix multiplication on the vector scores and the original features, and use the symmetric function Sum for vector dimensionality reduction pooling, as follows:

[0032]

[0033] Complete vector aggregation in the channel dimension, that is:

[0034] F out = Concat(F tk1 , F tk2 )

[0035] In the formula, Concat represents concatenation at the channel level.

[0036] Preferably: A learnable differential function is used to judge the similarity between point features, and then the initial correspondence is constructed.

[0037] Preferably: Construct a first-order binary compatibility matrix C;

[0038] Use a monotonically decreasing kernel function φ(·) or directly use a hard threshold to measure their spatial compatibility:

[0039] SC ij = φ(d ij )

[0040] Define the binary compatibility matrix C:

[0041]

[0042] In the formula, d thr represents a preset distance threshold;

[0043] Construct a second-order spatial similarity measure;

[0044] Seed selection and consensus set expansion;

[0045] Estimate the rigid transformation by singular value decomposition;

[0046] By traversing the candidate transformations corresponding to all seeds, select the optimal one as the target rotation and translation matrix.

[0047] Preferably: Constructing the first-order binary compatibility matrix C includes:

[0048] For each pair of correspondences i and j, assume that they correspond to the point pairs (x i , y i ) and (x j , y j ) in the original point clouds respectively, where x comes from the source point cloud and y comes from the target point cloud, and define their distance difference as:

[0049] d ij = |d(x i , x j ) - d(y i , y j )|

[0050] In the formula, d(.,.) represents Euclidean distance calculation.

[0051] Preferably, the second-order similarity is defined as:

[0052]

[0053] where N is the total correspondence. For a pair of corresponding relationships i and j that are inlier, there is C ij = 1, C ij That is, the distance difference between i and j is less than the threshold;

[0054] Using methods such as global spectral decomposition, select some corresponding relationships with high consistency scores from the SC 2 matrix as seeds, and for each seed, select the top k corresponding relationships with the highest similarity to it in the SC 2 metric space to form a local consensus set;

[0055] For each consensus set, use the weighted singular value decomposition method to estimate the rigid transformation:

[0056]

[0057] where the weight w ij Based on score setting, reflecting the reliability of each corresponding relationship.

[0058] A three-dimensional point cloud registration device for performing the above three-dimensional point cloud registration method, the device includes:

[0059] A point cloud acquisition unit for acquiring a source point cloud and a target point cloud; the source point cloud includes the occluded point cloud of the patient's diseased tissue collected during the operation, and the target point cloud includes the complete point cloud of the patient's diseased tissue collected before the operation;

[0060] An initial feature extraction unit for respectively extracting features from the source point cloud and the target point cloud by using a three-dimensional point cloud initial feature extraction module to obtain the initial features of the source point cloud and the initial features of the target point cloud;

[0061] An information cross-fusion unit for respectively strengthening the initial features of the source point cloud and the initial features of the target point cloud by using self-attention and cross-attention and cross-fusing the strengthened features to generate cross-fused features;

[0062] A soft matching matrix generation unit for generating a soft matching matrix based on the cross-fused features and the source point cloud;

[0063] A virtual point cloud generation unit for generating a virtual source point cloud based on the soft matching matrix and the source point cloud by using a learnable multi-class support vector machine;

[0064] A corresponding point pair acquisition unit, configured to establish an initial correspondence relationship between the virtual source point cloud and the target point cloud to obtain corresponding point pairs;

[0065] A corresponding point pair screening unit, configured to calculate the second-order similarity measure score of the corresponding point pairs, calculate the number of mutually compatible matching points based on the similarity measure, and sequentially complete the screening of the corresponding point pairs;

[0066] A rotation and translation matrix generation unit, configured to generate a target rotation and translation matrix through singular value decomposition according to the information vector of the corresponding point pairs obtained by screening, so as to transfer the source point cloud to the coordinate system of the target point cloud through the target rotation and translation matrix.

[0067] A three-dimensional point cloud registration device, the device includes a processor and a memory:

[0068] The memory is used to store program codes and transmit the program codes to the processor;

[0069] The processor is configured to execute the above-mentioned three-dimensional point cloud registration method according to the instructions in the program codes.

[0070] A computer-readable storage medium, the computer-readable storage medium is used to store program codes, and the program codes are used to execute the above-mentioned three-dimensional point cloud registration method.

[0071] According to the specific embodiments provided by the present invention, the present invention discloses the following technical effects:

[0072] A three-dimensional point cloud registration method, device, equipment and medium provided by an embodiment of the present application. The method first extracts local regions of point clouds at different scales, and strong correspondence in the local regions is used to solve the problem of noise points in the point cloud of diseased tissues. A virtual correspondence relationship between the source point cloud and the target point cloud is constructed to solve the objective difference problem between the source point cloud and the target point cloud. The calculation amount of the feature fusion module decreases, and the number of parameters is low. While effectively analyzing the relationship between the source point cloud and the target point cloud, the network calculation amount is reduced, and the problems of large number of parameters and high deployment cost are solved. The correspondence filter is independent of the network, reduces network errors, solves the registration error problem caused by incorrect corresponding points, and does not introduce additional calculation amount.

[0073] Of course, it is not necessary for any product implementing the present invention to simultaneously achieve all the above-mentioned advantages. Description of the Drawings

[0074] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required in the embodiments. Obviously, the drawings in the following description are only some embodiments of the present invention, and those of ordinary skill in the art can obtain other drawings based on these drawings without creative efforts.

[0075] Figure 1 is a flowchart of the three-dimensional point cloud registration method provided by an embodiment of the present invention;

[0076] Figure 2 is a main flowchart block diagram provided by an embodiment of the present invention;

[0077] Figure 3 is a flowchart of initial feature extraction provided by an embodiment of the present invention;

[0078] Figure 4 is a flowchart of corresponding point filtering provided by an embodiment of the present invention;

[0079] Figure 5 is a schematic diagram of a three-dimensional point cloud registration device provided by an embodiment of the present invention;

[0080] Figure 6 is a schematic diagram of a three-dimensional point cloud registration device provided by an embodiment of the present invention. Specific Embodiments

[0081] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, rather than all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art belong to the scope of protection of the present invention.

[0082] See Figure 1 , a three-dimensional point cloud registration method provided by an embodiment of the present invention. As Figure 1 shown, the method may include:

[0083] S101: Obtain a source point cloud and a target point cloud; the source point cloud includes the occluded point cloud of the patient's diseased tissue collected during the operation, and the target point cloud includes the complete point cloud of the patient's diseased tissue collected before the operation;

[0084] S102: Use the three-dimensional point cloud initial feature extraction module to perform feature extraction on the source point cloud and the target point cloud respectively to obtain the initial features of the source point cloud and the initial features of the target point cloud; specifically, in implementation, the embodiments of the present application may provide the three-dimensional point cloud initial feature extraction module for dimension elevation of the point cloud, then constructing a KD-tree feature distribution map based on the elevated features, and constructing information at different scales under the same center point according to the KD-tree feature distribution map.

[0085] Specifically, according to the KD-tree feature distribution map, for each point p in the point cloud P with N points i ∈P, i∈N, local regions with different K values are selected to construct K-nearest neighbor vectors, which are respectively denoted as F k1 = f1, f2, …, f k1 , F k2 = f1, f2, …, f k2 ;

[0086] The K-nearest neighbor vectors in different regions are respectively calculated using Sigmoid to obtain the corresponding vector scores, that is:

[0087] S k1 = Sigmoid(F k1 )

[0088] S k2 = Sigmoid(F k2 )

[0089] In the formula, S k1 and S k2 are the corresponding vector scores respectively;

[0090] The matrix multiplication is performed using the vector scores and the original features, and the symmetric function Sum is used for vector dimensionality reduction pooling, as follows:

[0091]

[0092]

[0093] Vector aggregation is completed in the channel dimension, that is:

[0094] F out = Concat(F tk1 , F tk2 )

[0095] In the formula, Concat represents the connection at the channel level.

[0096] S103: Use self-attention and cross-attention to respectively enhance the initial features of the source point cloud and the target point cloud, and cross-fuse the enhanced features to generate cross-fused features;

[0097] S104: Generate a soft matching matrix based on the cross-fused features and the source point cloud;

[0098] S105: Based on the soft matching matrix and the source point cloud, use a learnable multi-class support vector machine to generate a virtual source point cloud;

[0099] S106: Establish an initial correspondence relationship between the virtual source point cloud and the target point cloud to obtain corresponding point pairs; specifically, in implementation, the embodiments of the present application can provide a learnable differential function to judge the similarity between point features, and then construct the initial correspondence relationship.

[0100] S107: Calculate the second-order similarity measure score of the corresponding point pairs, calculate the number of mutually compatible matching points based on the similarity measure, and complete the screening of the corresponding point pairs in sequence; specifically, in implementation, the embodiments of the present application can provide to construct a first-order binary compatibility matrix C;

[0101] Use a monotonically decreasing kernel function φ(·) or directly use a hard threshold to measure their spatial compatibility:

[0102] SC ij = φ(d ij )

[0103] Define the binary compatibility matrix C:

[0104]

[0105] In the formula, d thr represents a preset distance threshold;

[0106] Construct a second-order spatial similarity measure;

[0107] Seed selection and consensus set expansion;

[0108] Singular value decomposition to estimate the rigid transformation;

[0109] By traversing the candidate transformations corresponding to all seeds, select the optimal one as the target rotation and translation matrix.

[0110] Specifically, constructing the first-order binary compatibility matrix C includes:

[0111] For each pair of correspondences i and j, assume that they respectively correspond to the point pairs (x i , y i ) and (x j , y j ) in the original point cloud, where x comes from the source point cloud and y comes from the target point cloud, and define their distance difference as:

[0112] d ij = |d(x i , x j ) - d(y i , y j )|

[0113] In the formula, d(.,.) represents the Euclidean distance calculation.

[0114] Define the second-order similarity as follows:

[0115]

[0116] In the formula, N is the total correspondence. For a pair of corresponding relations i and j that are inlier points, there is C ij = 1, C ij That is, the distance difference between i and j is less than the threshold;

[0117] Using methods such as global spectral decomposition, select some corresponding relations with high consistency scores from the SC 2 matrix as seeds. For each seed, select the top k corresponding relations with the highest similarity to it in the SC 2 metric space to form a local consensus set;

[0118] For each consensus set, use the weighted singular value decomposition method to estimate the rigid transformation:

[0119]

[0120] In the formula, the weight w ij Based on score setting, which reflects the reliability of each corresponding relation.

[0121] S108: According to the information vectors of the corresponding point pairs obtained by screening, generate the target rotation and translation matrix through singular value decomposition, so as to transfer the source point cloud to the coordinate system of the target point cloud through the target rotation and translation matrix.

[0122] The three-dimensional point cloud registration method provided by the embodiments of the present application first extracts the local regions of point clouds at different scales, and strong correspondence in the local regions is used to solve the problem of noise points in the point cloud of the patient tissue; then constructs the virtual correspondence between the source point cloud and the target point cloud to solve the objective difference problem between the source point cloud and the target point cloud; then passes through a lightweight feature fusion module, while effectively analyzing the relationship between the source point cloud and the target point cloud, reducing the network calculation amount, and solving the problems of large number of parameters and high deployment cost; finally, uses a correspondence filter independent of the network to reduce network errors and does not introduce additional calculation amount, and solves the registration error problem caused by incorrect corresponding points.

[0123] As Figure 2 shown, the three-dimensional point cloud registration method provided by the present application will be introduced in detail below.

[0124] Step 1: Preoperatively collect the complete three-dimensional point cloud of the patient tissue and input it as the target point cloud of the registration system; during the operation, collect the point cloud of the patient tissue of the patient in real time and input it as the source point cloud of the registration system.

[0125] In specific implementation, the complete patient's diseased tissue point cloud is collected before surgery. Before surgery, a three-dimensional information acquisition device is used to collect the point cloud of the patient's complete diseased tissue and perform preprocessing (removing noise points, data normalization, etc.). During the collection process, the device should be aligned with the patient's diseased tissue, and a single-frame three-dimensional image should be collected as much as possible. This step is automatically completed by a robotic arm.

[0126] During surgery, the occluded point cloud of the patient's diseased tissue is collected. During surgery, a three-dimensional information acquisition device is used to collect the incomplete point cloud of the patient's diseased tissue during surgery and perform the same preprocessing as in Step 1. During the collection process, regardless of direction and distance, the first principle is the convenience of the doctor's operation. This step is completed by medical staff operating the robotic arm.

[0127] Step 2: Initial feature extraction of the three-dimensional point cloud. For the intraoperative source point cloud and the preoperative occluded target point cloud, initial feature extraction is performed separately. The processing in this step is similar. Taking the source point cloud as an example, first, the feature dimension of the source point cloud is increased, then a KD-tree is constructed based on this, and information at different scales under the same center point is constructed according to this KD-tree. In specific implementation, first, different-scale regions of interest are selected based on the K-nearest neighbor method with the same center point, and then effective information is extracted and aggregated on different regions respectively; the above operations are performed on all points in the source point cloud, and finally, the output is used as the initial feature of the source point cloud.

[0128] Step 3: After collecting the corresponding initial features of the preoperative source point cloud and the intraoperative target point cloud in Step 3, cross-fusion is performed on the two. Specifically, first, for the self-information vector, the corresponding score is calculated through the Sigmoid function, and the self-information representation is enhanced according to this corresponding score; then, the corresponding scores of the two on each other's information are calculated respectively to complete information fusion. In specific implementation, according to the initial features extracted in Step 3, self-attention is used to process them respectively to obtain enhanced features, and then the enhanced features of the source point cloud and the target point cloud are cross-fused to generate cross features.

[0129] Step 4: According to the fused information and the intraoperative occluded source point cloud, a soft matching matrix is generated. The generation process is generated by a learnable differential function based on a multi-class support vector machine. During the generation process, a soft matching matrix is generated based on the above cross features, continuously approaching the original corresponding matrix.

[0130] Step 5: Based on the soft matching matrix and the original source point cloud, a virtual source point cloud is generated using a learnable multi-class support vector machine.

[0131] Step 6: According to the corresponding matrix, a virtual source point cloud is generated, and then the corresponding point pairs are calculated according to the corresponding information.

[0132] Step 7: Establish an initial correspondence between the virtual source point cloud and the target point cloud, and filter the correspondence using a second-order similarity measure. Calculate the second-order similarity measure scores of the corresponding point pairs, calculate the number of jointly compatible matching points based on the similarity measure, and complete the screening of the corresponding point pairs in sequence.

[0133] Step 8: Generate the rotation and translation matrix. According to the correspondence filtered in Step 7, use singular value decomposition to generate the rotation and translation matrix. Generate the final rotation and translation matrix according to the information vector of the corresponding point pairs. And transfer the occluded source point cloud during the operation to the coordinate system of the preoperative complete patient tissue target point cloud through this matrix.

[0134] As Figure 3 is the working flow chart of the initial feature extraction module provided by this application, which is used for extracting the initial information vector, and the quality of its information extraction has a significant impact on the subsequent registration process. This working flow keeps the preoperative target point cloud and the intraoperative source point cloud consistent. Here, taking the preoperative target point cloud as an example, the 3D point cloud registration system generally includes the following steps.

[0135] Step 1: First, perform information dimensionality elevation on the target point cloud to obtain a high-dimensional information vector, and construct a KD-tree feature distribution map based on this vector.

[0136] Step 2: According to this KD-tree feature distribution map, for each point p in the point cloud P with N points i ∈P, i∈N, there are, select local regions with different K values to construct K-nearest neighbor vectors, which are respectively denoted as F k1 = f1, f2, …, f k1 , F k2 = f1, f2, …, f k2 .

[0137] Step 3: Use the Sigmoid function to calculate the corresponding vector scores for the K-nearest neighbor vectors in different regions respectively, that is:

[0138] S k1 = Sigmoid(F k1 )#(1)

[0139] S k2 = Sigmoid(F k2 )#(2)

[0140] Among them, S k1 and S k2 are the corresponding vector scores respectively. This score reflects the importance of each sub-vector in the information vector. Then, use this vector score to perform matrix multiplication with the original feature, and use the symmetric function Sum for vector dimensionality reduction pooling, that is, strengthen the proportion of important sub-vectors and reduce the influence of general sub-vectors, as follows:

[0141]

[0142] Finally, vector aggregation is completed on the channel dimension, namely:

[0143] F out =Concat(F tk1 ,F tk2 )#(5)

[0144] Among them, Concat represents the connection at the channel level. At this point, the initial information of the source point cloud is extracted and Fs is obtained. out , the initial information of the target point cloud is recorded as Ft out .

[0145] like Figure 4 This is a corresponding point filtering module based on second-order similarity measurement provided by this application, which is used to filter the corresponding point relationship used for point cloud registration, and includes the following steps:

[0146] Step 1: Construct preliminary correspondence. Construct preliminary correspondence based on the feature descriptors of the source point cloud and the target point cloud generated in the previous stage. Specifically, use a learnable differential function to determine the similarity between point features and then construct a correspondence.

[0147] Step 2: Construct the first-order binary compatibility matrix C. For each pair of correspondences i and j, assume that they correspond to the point pairs (x i ,y i ) and (x j ,y j ), where x comes from the source point cloud and y comes from the target point cloud, and their distance difference is defined as:

[0148] d ij =|d(x i ,x j )-d(y i ,y j )|#(6)

[0149] Among them, d(.,.) represents the Euclidean distance calculation. Next, a monotonically decreasing kernel function φ(·) or a hard threshold is used directly to measure their spatial compatibility:

[0150] SC ij =φ(d ij )#(7)

[0151] Then, define the binary compatibility matrix C:

[0152]

[0153] Among them, dthr represents a preset distance threshold.

[0154] Step 3: Construct the second-order spatial similarity measure. The second-order spatial compatibility metric considers whether the corresponding relationships i and j have many other mutually compatible corresponding relationships. Specifically, define the second-order similarity as:

[0155]

[0156] where N is the total number of corresponding relationships. Intuitively, for a pair of corresponding relationships i and j that are inliers, then C ij = 1, C ij that is, the distance difference between i and j is less than the threshold; at the same time, they have compatible relationships with most other inliers, so has a large value. While false matches usually do not have this global consistency and their second-order values are low.

[0157] Step 4: Seed selection and consensus set expansion. Using methods such as global spectral decomposition, select some corresponding relationships with high consistency scores from the SC 2 matrix as "seeds", and these seeds are more likely to be inliers. Then for each seed, select the top k corresponding relationships with the highest similarity to it in the SC 2 metric space to form a local consensus set. This excludes most outliers at an early stage and reduces the error probability of sampling.

[0158] Step 5: Singular value decomposition to estimate the rigid transformation. For each consensus set, use the weighted singular value decomposition method to estimate the rigid transformation (rotation matrix and translation matrix):

[0159]

[0160] where the weight w ij can be set based on the score, reflecting the reliability of each corresponding relationship.

[0161] Finally, by traversing the candidate transformations corresponding to all seeds, select the optimal one as the final rotation and translation matrix.

[0162] In summary, for the 3D point cloud registration method provided in this application, local regions of point clouds at different scales are first extracted, and strong local correspondences are used to solve the problem of noise points in the point cloud of the patient's tissue. A virtual correspondence relationship between the source point cloud and the target point cloud is constructed to solve the problem of objective differences between the source point cloud and the target point cloud. The computational cost of the feature fusion module is reduced, and the number of parameters is low. While effectively analyzing the relationship between the source point cloud and the target point cloud, the network computational cost is reduced, and the problems of large number of parameters and high deployment cost are solved. The correspondence filter is independent of the network, reducing network errors and solving the registration error problem caused by incorrect corresponding points without introducing additional computational cost.

[0163] See Figure 5 , an embodiment of this application may further provide a 3D point cloud registration device, as Figure 5 shown, for performing the above 3D point cloud registration method. The device may include:

[0164] A point cloud acquisition unit 501, configured to acquire a source point cloud and a target point cloud; the source point cloud includes the occluded point cloud of the patient's tissue collected during the operation, and the target point cloud includes the complete point cloud of the patient's tissue collected before the operation;

[0165] An initial feature extraction unit 502, configured to use a 3D point cloud initial feature extraction module to perform feature extraction on the source point cloud and the target point cloud respectively to obtain an initial feature of the source point cloud and an initial feature of the target point cloud;

[0166] An information cross-fusion unit 503, configured to use self-attention and cross-attention to enhance the initial features of the source point cloud and the target point cloud respectively and cross-fuse the enhanced features to generate cross-fused features;

[0167] A soft matching matrix generation unit 504, configured to generate a soft matching matrix based on the cross-fused features and the source point cloud;

[0168] A virtual point cloud generation unit 505, configured to generate a virtual source point cloud using a learnable multi-class support vector machine based on the soft matching matrix and the source point cloud;

[0169] A corresponding point pair acquisition unit 506, configured to establish an initial correspondence relationship between the virtual source point cloud and the target point cloud to obtain corresponding point pairs;

[0170] A corresponding point pair screening unit 507, configured to calculate the second-order similarity measure score of the corresponding point pairs, calculate the number of jointly compatible matching points based on the similarity measure, and complete the screening of the corresponding point pairs in sequence;

[0171] A rotation and translation matrix generation unit 508, configured to generate a target rotation and translation matrix through singular value decomposition according to the information vectors of the corresponding point pairs obtained by screening, so as to transfer the source point cloud to the coordinate system of the target point cloud through the target rotation and translation matrix.

[0172] An embodiment of the present application may further provide a three-dimensional point cloud registration device, and the device includes a processor and a memory:

[0173] The memory is used to store program codes and transmit the program codes to the processor;

[0174] The processor is configured to execute the steps of the above three-dimensional point cloud registration method according to the instructions in the program codes.

[0175] As Figure 6 shown, a three-dimensional point cloud registration device provided by an embodiment of the present application may include: a processor 10, a memory 11, a communication interface 12, and a communication bus 13. The processor 10, the memory 11, and the communication interface 12 all complete mutual communication through the communication bus 13.

[0176] In an embodiment of the present application, the processor 10 may be a central processing unit (CPU), an application specific integrated circuit, a digital signal processor, a field programmable gate array, or other programmable logic devices, etc.

[0177] The processor 10 may call the program stored in the memory 11. Specifically, the processor 10 may execute the operations in the embodiment of the three-dimensional point cloud registration method.

[0178] The memory 11 is used to store one or more programs, and the programs may include program codes. The program codes include computer operation instructions. In an embodiment of the present application, the memory 11 stores at least programs for implementing the following functions:

[0179] Obtain a source point cloud and a target point cloud; the source point cloud includes the occluded point cloud of the patient's diseased tissue collected during the operation, and the target point cloud includes the complete point cloud of the patient's diseased tissue collected before the operation;

[0180] Use a three-dimensional point cloud initial feature extraction module to respectively extract features from the source point cloud and the target point cloud to obtain the initial features of the source point cloud and the initial features of the target point cloud;

[0181] Use self-attention and cross-attention to respectively strengthen the initial features of the source point cloud and the target point cloud and cross-fuse the strengthened features to generate cross-fused features;

[0182] Generate a soft matching matrix based on the cross-fused features and the source point cloud;

[0183] Generate a virtual source point cloud using the learnable multi-class support vector machine based on the soft matching matrix and the source point cloud;

[0184] Establish an initial correspondence between the virtual source point cloud and the target point cloud to obtain corresponding point pairs;

[0185] Calculate the second-order similarity measure scores of the corresponding point pairs, calculate the number of mutually compatible matching points based on the similarity measure, and complete the screening of the corresponding point pairs in sequence;

[0186] According to the information vectors of the corresponding point pairs obtained by screening, generate a target rotation and translation matrix through singular value decomposition, so as to transfer the source point cloud to the coordinate system of the target point cloud through the target rotation and translation matrix.

[0187] In a possible implementation, the memory 11 may include a program storage area and a data storage area. Among them, the program storage area may store an operating system and application programs required for at least one function (such as file creation function, data reading and writing function), etc.; the data storage area may store data created during use, such as initialization data, etc.

[0188] In addition, the memory 11 may include high-speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device or other volatile solid-state storage devices.

[0189] The communication interface 12 may be an interface of a communication module for connecting to other devices or systems.

[0190] Of course, it should be noted that Figure 6 The structure shown does not constitute a limitation on the three-dimensional point cloud registration device in the embodiments of the present application. In actual applications, the three-dimensional point cloud registration device may include more or fewer components than Figure 6 shown, or combine certain components.

[0191] Embodiments of the present application may also provide a computer-readable storage medium, which is used to store program codes, and the program codes are used to execute the steps of the above three-dimensional point cloud registration method.

[0192] It should be noted that in this text, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprising", "including" or any other variant thereof are intended to cover non-exclusive inclusion, such that a process, method, article or device comprising a series of elements not only includes those elements but also includes other elements not expressly listed, or also includes elements inherent to such process, method, article or device. Without further limitation, an element defined by the statement "comprising an..." does not exclude the presence of additional identical elements in the process, method, article or device comprising the said element.

[0193] From the description of the above embodiments, those skilled in the art can clearly understand that the present application can be implemented by means of software plus a necessary general hardware platform. Based on such an understanding, the technical solution of the present application, in essence or the part that contributes to the prior art, can be embodied in the form of a software product, which can be stored in a storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the methods described in various embodiments or some parts of the embodiments of the present application.

[0194] Each embodiment in this specification is described in a progressive manner, and the same or similar parts among the embodiments can be referred to each other. Each embodiment focuses on the differences from other embodiments. In particular, for a system or a system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the description of the method embodiment. The systems and system embodiments described above are only illustrative, where the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art can understand and implement it without creative efforts.

[0195] The above are only the preferred embodiments of the present invention and are not intended to limit the protection scope of the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention are included in the protection scope of the present invention.

Claims

1. A three-dimensional point cloud registration method, characterized in that: include: Acquire a source point cloud and a target point cloud; the source point cloud includes a patient tissue occlusion point cloud collected during surgery, and the target point cloud includes a complete patient tissue point cloud collected before surgery; Using a three-dimensional point cloud initial feature extraction module to extract features from the source point cloud and the target point cloud to obtain source point cloud initial features and target point cloud initial features; Using self-attention and cross-attention to respectively enhance the initial features of the source point cloud and the initial features of the target point cloud and cross-fuse the enhanced features to generate cross-fused features; Generate a soft matching matrix based on the cross-fusion features and the source point cloud; Based on the soft matching matrix and the source point cloud, a virtual source point cloud is generated using a learnable multi-class support vector machine; Using the virtual source point cloud and the target point cloud to establish an initial correspondence relationship to obtain corresponding point pairs; Calculating the second-order similarity measurement score of the corresponding point pairs, calculating the number of commonly compatible matching points based on the similarity measurement, and completing the screening of corresponding point pairs in sequence; The information vector of the corresponding point pair is obtained by screening, and a target rotation and translation matrix is ​​generated by singular value decomposition, so as to transfer the source point cloud to the coordinate system of the target point cloud through the target rotation and translation matrix.

2. The three-dimensional point cloud registration method according to claim 1, characterized in that: The three-dimensional point cloud initial feature extraction module is used to perform feature dimensionality upgrade on the point cloud, and then construct a KD-tree feature distribution map based on the dimensionality upgraded features, and construct information of different scales under the same center point according to the KD-tree feature distribution map.

3. The three-dimensional point cloud registration method according to claim 2, characterized in that: According to the KD-tree feature distribution graph, for each point p in the point cloud P with N points i ∈P,i∈N selects local areas with different K values ​​to construct K nearest neighbor vectors, which are denoted as F k1 =f1,f2,…,f k1 , F k2 =f1,f2,…,f k2 ; The K nearest neighbor vectors in different regions are calculated using Sigmoid to obtain the corresponding vector scores, namely: S k1 =Sigmoid(F k1 ) S k2 =Sigmoid(F k2 ) In the formula, S k1 and S k2 are the corresponding vector scores respectively; Use the vector score to perform matrix multiplication with the original feature, and use the symmetric function Sum to perform vector dimensionality reduction pooling, as shown below: Vector aggregation is done on the channel dimension, namely: F out =Concat(F tk1 ,F tk2 ) Where Concat represents the connection at the channel level.

4. The three-dimensional point cloud registration method according to claim 1, characterized in that: A learnable differential function is used to determine the similarity between point features, thereby constructing the initial correspondence relationship.

5. The three-dimensional point cloud registration method according to claim 4, characterized in that: Construct the first-order binary compatibility matrix C; Use the monotonically decreasing kernel function φ(·) or directly use a hard threshold to measure their spatial compatibility: SC ij =φ(d ij ) Define the binary compatibility matrix C: Where, d thr Indicates a preset distance threshold; Constructing a second-order spatial similarity measure; Seed selection and consensus set expansion; Singular value decomposition estimates rigid transformation; By traversing the candidate transformations corresponding to all seeds, the best one is selected as the target rotation and translation matrix.

6. The three-dimensional point cloud registration method according to claim 5, characterized in that: Constructing the first-order binary compatibility matrix C includes: For each pair of correspondences i and j, assume that they correspond to the point pairs (x i ,y i ) and (x j ,y j ), where x comes from the source point cloud and y comes from the target point cloud, and their distance difference is defined as: d ij =|d(x i ,x j )-d(y i ,y j )| Where d(.,.) represents the Euclidean distance calculation.

7. The three-dimensional point cloud registration method according to claim 5, characterized in that: The second-order similarity is defined as: Where N is the total correspondence. For a pair of correspondences i and j that are interior points, we have C ij =1, C ij That is, the distance difference between i and j is less than the threshold; Using methods such as global spectral decomposition, from SC 2 Select some correspondences with high consistency scores in the matrix as seeds, and select the corresponding relations in SC 2 The first k correspondences with the highest similarity in the metric space constitute a local consensus set; For each consensus set, a weighted singular value decomposition method is used to estimate the rigid transformation: In the formula, the weight w ij based on The score setting reflects the reliability of each corresponding relationship.

8. A three-dimensional point cloud registration device, characterized in that: Used to perform the three-dimensional point cloud registration method according to any one of claims 1 to 7, the device comprising: A point cloud acquisition unit, used to acquire a source point cloud and a target point cloud; the source point cloud includes a patient tissue occlusion point cloud collected during surgery, and the target point cloud includes a complete patient tissue point cloud collected before surgery; An initial feature extraction unit, used to extract features from the source point cloud and the target point cloud respectively using a three-dimensional point cloud initial feature extraction module to obtain source point cloud initial features and target point cloud initial features; An information cross-fusion unit, used to use self-attention and cross-attention to respectively enhance the initial features of the source point cloud and the initial features of the target point cloud and cross-fuse the enhanced features to generate cross-fusion features; A soft matching matrix generating unit, used for generating a soft matching matrix based on the cross-fusion feature and the source point cloud; A virtual point cloud generating unit, configured to generate a virtual source point cloud based on the soft matching matrix and the source point cloud using a learnable multi-class support vector machine; A corresponding point pair acquisition unit, configured to establish an initial corresponding relationship between the virtual source point cloud and the target point cloud to obtain corresponding point pairs; A corresponding point pair screening unit, used to calculate the second-order similarity measurement score of the corresponding point pair, calculate the number of commonly compatible matching points based on the similarity measurement, and complete the screening of corresponding point pairs in sequence; The rotation and translation matrix generating unit is used to obtain the information vector of the corresponding point pair according to screening, and generate a target rotation and translation matrix through singular value decomposition, so as to transfer the source point cloud to the coordinate system of the target point cloud through the target rotation and translation matrix.

9. A three-dimensional point cloud registration device, characterized in that: The device comprises a processor and a memory: The memory is used to store program code and transmit the program code to the processor; The processor is used to execute the three-dimensional point cloud registration method according to any one of claims 1-7 according to the instructions in the program code.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium is used to store program code, and the program code is used to execute the three-dimensional point cloud registration method according to any one of claims 1 to 7.

Images