Point cloud registration method, apparatus, and medical imaging device

Abstract

This invention discloses a point cloud registration method, apparatus, and medical imaging equipment. The point cloud registration method includes: calculating a coarse registration matrix for a point cloud to be registered and a target point cloud based on a preset algorithm to obtain a first transformation matrix that meets a first preset accuracy condition; the algorithm includes a Gaussian mixture model-based algorithm and / or a multi-task learning algorithm; transforming the point cloud to be registered using the first transformation matrix to obtain a first transformed point cloud; and performing fine registration on the first transformed point cloud and the target point cloud multiple times using an iterative nearest-point algorithm to ensure that the final registration accuracy is less than or equal to a first preset value. This point cloud registration method first performs coarse registration on the two datasets to reduce the difference between them, and then performs fine registration multiple times using an iterative nearest-point algorithm, thereby effectively registering the two datasets, improving registration efficiency, and increasing registration accuracy.

Description

Technical Field

[0001] This invention relates to the field of oral surgery, and more particularly to a point cloud registration method, apparatus, and medical imaging equipment. Background Technology

[0002] Both oral CT (Computed Tomography) and intraoral scanners have their advantages. Intraoral scanners can scan precise crown data, while oral CT can scan invisible bone and root data, but with lower precision. Overlaying the two types of data together can facilitate orthodontic and implant procedures for doctors.

[0003] To facilitate the overlay of two types of data, a novel algorithm for automatic registration of two different modal 3D data has been proposed. This algorithm primarily involves manually selecting three or more sets of corresponding matching points on the two datasets and calculating a rigid transformation matrix using these matching points to complete the registration. However, this registration process requires manual intervention, has a low degree of automation, and demands a high level of expertise from personnel, resulting in low efficiency and difficulty in ensuring the stability of accuracy. Summary of the Invention

[0004] This invention aims to at least partially address one of the technical problems in related technologies. To this end, this invention proposes a point cloud registration method, apparatus, and medical imaging equipment to improve registration efficiency and accuracy.

[0005] In a first aspect, the present invention proposes a point cloud registration method, comprising: performing coarse registration matrix calculation on the point cloud to be registered and the target point cloud based on a preset algorithm to obtain a first transformation matrix that satisfies a first precision preset condition, wherein the preset algorithm includes a Gaussian mixture model algorithm and / or a multi-task learning algorithm; transforming the point cloud to be registered using the first transformation matrix to obtain a first transformed point cloud; and performing fine registration on the first transformed point cloud and the target point cloud multiple times using an iterative nearest point algorithm to make the final registration precision less than or equal to a first preset value.

[0006] Secondly, the present invention proposes a point cloud registration device, including a memory, a processor, and a computer program stored in the memory. When the computer program is executed by the processor, it implements the point cloud registration method of the above embodiments.

[0007] Thirdly, the present invention proposes a medical imaging device, including the point cloud registration device of the above embodiments.

[0008] The point cloud registration method, apparatus, and medical imaging equipment of this invention first perform coarse registration on two datasets to reduce the difference between them; then, they perform fine registration multiple times using the ICP (Iterative Closest Point) algorithm. During the fine registration process, the error precision can be set individually for each iteration, and the error precision is increased progressively. This effectively registers the two datasets, improving registration efficiency and accuracy.

[0009] Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0010] Figure 1 This is a flowchart of a point cloud registration method according to an embodiment of the present invention;

[0011] Figure 2 This is a flowchart of step S1 in the first specific embodiment of the present invention;

[0012] Figure 3 This is a flowchart of step S1 in the second specific embodiment of the present invention;

[0013] Figure 4 This is a flowchart of step S1 in the third specific embodiment of the present invention;

[0014] Figure 5 This is a flowchart of step S1 in the fourth specific embodiment of the present invention;

[0015] Figure 6 This is a flowchart of coarse registration based on a Gaussian mixture model algorithm according to an embodiment of the present invention;

[0016] Figure 7 This is a flowchart of coarse registration based on a multi-task learning algorithm according to an embodiment of the present invention;

[0017] Figure 8 This is a structural diagram of coarse registration based on a multi-task learning algorithm according to an embodiment of the present invention;

[0018] Figure 9 This is a flowchart of step S3 in a specific embodiment of the present invention;

[0019] Figure 10 This is a schematic diagram of the registration result of oral CT point cloud and oral scan point cloud in an example of the present invention;

[0020] Figure 11 This is a cross-sectional schematic diagram of the registration result between oral CT point cloud and oral scan point cloud, as an example of the present invention.

[0021] Figure 12This is a schematic diagram of the point cloud registration device according to an embodiment of the present invention;

[0022] Figure 13 This is a schematic diagram of the structure of a medical imaging device according to an embodiment of the present invention. Detailed Implementation

[0023] Point cloud data from different modalities, such as oral CBCT (cone beam computed tomography) data (volume data) and oral surface data (surface data), exhibit significant differences and small overlap, making it difficult to find corresponding points and resulting in considerable noise. To address this issue, this invention first performs coarse registration on the two types of data (which can be from two different modalities). After the first transformation of coarse registration, the difference between the two data sets decreases, reducing the error in finding corresponding points. Then, fine registration is performed multiple times using the ICP (Iterative Closest Point) algorithm. During fine registration, the error precision can be set individually for each iteration, and the error precision increases progressively. Therefore, it is possible to effectively register two data sets without manual intervention, improving registration efficiency and accuracy.

[0024] Embodiments of the present invention are described in detail below, examples of which are illustrated in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain the present invention, and should not be construed as limiting the present invention.

[0025] The point cloud registration method, apparatus, and medical imaging equipment of the present invention are described below with reference to the accompanying drawings.

[0026] Figure 1 This is a flowchart of a point cloud registration method according to an embodiment of the present invention.

[0027] like Figure 1 As shown, point cloud registration methods include:

[0028] S1, based on a preset algorithm, perform coarse registration matrix calculation on the point cloud to be registered and the target point cloud to obtain a first transformation matrix that meets the first precision preset condition. The preset algorithm includes a Gaussian mixture model algorithm and / or a multi-task learning algorithm.

[0029] In this embodiment, the point cloud to be registered and the target point cloud can be data in two different modalities. For example, the point cloud to be registered can be volume data, and the target point cloud can be area data; or the point cloud to be registered can be area data, and the target point cloud can be volume data. The volume data can be obtained from oral CT data obtained from an oral CBCT or oral CT scanner, and the area data can be obtained from oral surface data obtained from an intraoral scanner.

[0030] Specifically, a tooth model can be extracted from oral CT data using region growing, and then the portion of the tooth model used for registration can be converted into point cloud data. The tooth model can include the crowns, roots, and the area between the crowns and roots of all teeth. Registration can be performed using all teeth as a whole, or the teeth can be segmented first, and registration can be performed using only the data containing the crowns. A 3D tooth model is obtained using an intraoral scanner, and then the portion of the 3D tooth model used for registration can be converted into point cloud data. This 3D tooth model can include the crowns and gingival soft tissue. Registration can be performed using data containing both the crowns and gingival soft tissue, or the soft tissue can be removed, and registration can be performed using the remaining crown portion.

[0031] Region growing is an image segmentation method that groups pixels based on their similarity within the same object region. Starting from an initial region (such as a small neighborhood or even each pixel), it gradually grows the region by merging adjacent pixels or other regions with similar properties until no more pixels or small regions can be merged. The similarity measurement of pixels within a region can include information such as average gray value, texture, and color. Optionally, region growing can be directly called from the 3D image processing library ITK. The process involves first performing threshold segmentation on the oral CT data, and then performing region growing segmentation based on the threshold segmentation.

[0032] It should be noted that coarse registration aims to obtain a first transformation matrix between the point cloud to be registered and the target point cloud. This first transformation matrix has a lower registration accuracy requirement and is mainly to facilitate subsequent fine registration. In the embodiments of this invention, the coarse registration process can be directly implemented using either a Gaussian mixture model algorithm or a multi-task learning algorithm; alternatively, one of the two methods can be randomly selected for registration first. If the registration accuracy does not meet the requirements, the other method is selected for registration, and then the two registration results are compared to determine a better registration result as the final coarse registration result.

[0033] The step of calculating a coarse registration matrix between the point cloud to be registered and the target point cloud based on a preset algorithm to obtain a first transformation matrix that satisfies a first precision preset condition includes: calculating a coarse registration matrix between the point cloud to be registered and the target point cloud based on a Gaussian mixture model algorithm to obtain a first initial matrix; and / or, calculating a coarse registration matrix between the point cloud to be registered and the target point cloud based on a multi-task learning algorithm to obtain a second initial matrix; and using the first initial matrix, or the second initial matrix, or the one with the higher corresponding registration precision between the first initial matrix and the second initial matrix as the first transformation matrix that satisfies the first precision preset condition. This process can be implemented in various ways, as detailed below:

[0034] Method 1: The preset algorithm includes an algorithm based on a Gaussian mixture model;

[0035] The coarse registration matrix is ​​calculated for the point cloud to be registered and the target point cloud based on the Gaussian mixture model algorithm to obtain the first initial matrix; the first initial matrix is ​​used as the first transformation matrix that satisfies the first precision preset condition.

[0036] Specifically, such as Figure 2 As shown, method one may include:

[0037] S21, Based on the Gaussian mixture model algorithm, coarse registration matrix calculation is performed on the point cloud to be registered and the target point cloud to obtain the first initial matrix;

[0038] S22, when the first registration accuracy corresponding to the first initial matrix is ​​less than the second preset value, the first initial matrix is ​​used as the first transformation matrix, wherein the second preset value is greater than the first preset value.

[0039] The second preset value is the registration accuracy corresponding to coarse registration, and the first preset value is the registration accuracy corresponding to fine registration. The values ​​of the first preset value and the second preset value can be calibrated as needed. The value range of the first preset value is [0.005, 0.02], such as 0.01; the value range of the second preset value is [0.1, 1.0], such as 0.5.

[0040] Method 2: The preset algorithm includes a multi-task learning algorithm;

[0041] Based on a multi-task learning algorithm, a coarse registration matrix is ​​calculated for the point cloud to be registered and the target point cloud to obtain a second initial matrix; the second initial matrix with higher registration accuracy is used as the first transformation matrix that satisfies the first accuracy preset condition.

[0042] Specifically, such as Figure 3 As shown, method two may include:

[0043] S31, based on the multi-task learning algorithm, coarse registration matrix calculation is performed on the point cloud to be registered and the target point cloud to obtain the second initial matrix;

[0044] S32, when the second registration accuracy corresponding to the second initial matrix is ​​less than the second preset value, the second initial matrix is ​​used as the first transformation matrix, wherein the second preset value is greater than the first preset value.

[0045] Method 3: The preset algorithm includes Gaussian mixture model algorithm and multi-task learning-based algorithm;

[0046] The coarse registration matrix is ​​calculated for the point cloud to be registered and the target point cloud based on the Gaussian mixture model algorithm to obtain a first initial matrix; the coarse registration matrix is ​​calculated for the point cloud to be registered and the target point cloud based on the multi-task learning algorithm to obtain a second initial matrix; the one with higher registration accuracy between the first initial matrix and the second initial matrix is ​​used as the first transformation matrix that satisfies the first accuracy preset condition.

[0047] Specifically, such as Figure 4 As shown, method three may include:

[0048] S41, based on the Gaussian mixture model algorithm, coarse registration matrix calculation is performed on the point cloud to be registered and the target point cloud to obtain the first initial matrix;

[0049] S42, when the first registration accuracy is greater than or equal to the second preset value, a coarse registration matrix is ​​calculated for the point cloud to be registered and the target point cloud based on the multi-task learning algorithm to obtain the second initial matrix;

[0050] S43, when the second registration accuracy corresponding to the second initial matrix is ​​less than the first registration accuracy, the second initial matrix is ​​used as the first transformation matrix;

[0051] S44, when the second registration accuracy is greater than the first registration accuracy, the first initial matrix is ​​used as the first transformation matrix.

[0052] Based on a multi-task learning algorithm, a coarse registration matrix is ​​calculated for the point cloud to be registered and the target point cloud to obtain a second initial matrix.

[0053] Method 4: The preset algorithm includes Gaussian mixture model algorithm and multi-task learning algorithm;

[0054] A coarse registration matrix is ​​calculated for the point cloud to be registered and the target point cloud based on a multi-task learning algorithm to obtain a second initial matrix; a coarse registration matrix is ​​calculated for the point cloud to be registered and the target point cloud based on a Gaussian mixture model algorithm to obtain a first initial matrix; the one with higher registration accuracy between the first initial matrix and the second initial matrix is ​​used as the first transformation matrix that satisfies the first accuracy preset condition.

[0055] Specifically, such as Figure 5 As shown, method four may include:

[0056] S51, based on the multi-task learning algorithm, coarse registration matrix calculation is performed on the point cloud to be registered and the target point cloud to obtain the second initial matrix;

[0057] S52, when the second registration accuracy corresponding to the second initial matrix is ​​greater than or equal to the second preset value, the coarse registration matrix is ​​calculated for the point cloud to be registered and the target point cloud based on the Gaussian mixture model algorithm to obtain the first initial matrix;

[0058] S53, when the first registration accuracy corresponding to the first initial matrix is ​​less than the second registration accuracy, the first initial matrix is ​​used as the first transformation matrix;

[0059] S54, when the second registration accuracy is less than the first registration accuracy, the second initial matrix is ​​used as the first transformation matrix.

[0060] As an example, such as Figure 6 As shown, the coarse registration matrix calculation based on the Gaussian mixture model algorithm for the point cloud to be registered and the target point cloud can include:

[0061] A1. Use a preset neural network to extract features from the point cloud to be registered and the target point cloud to obtain the first feature information and the second feature information.

[0062] The preset neural network can be composed of a T-Net network and a PointNet network. The T-Net network is used to perform initial transformations on the point cloud, and the PointNet network is used to extract features. Specifically, the T-Net network can learn a 3×3 matrix to perform rotation transformations on the point cloud to be registered and the target point cloud respectively, and learn the relative rotation transformation between the two point clouds to enhance the robustness of the feature information extracted by the PointNet network.

[0063] Specifically, the point cloud to be registered p M×3 and target point cloud p N×3 All points in the array have three-dimensional coordinates, i.e., p M×3 ∈R 3 p N×3 ∈R 3 After feature extraction via a pre-defined neural network, the representation of each point in both the point cloud to be registered and the target point cloud can be expanded from three-dimensional coordinates to J-dimensional coordinates, and the corresponding point clouds can be represented as the first feature information. Second feature information Where M is the number of points in the point cloud to be registered, N is the number of points in the target point cloud, and J can be set as needed. Feature extraction is a mapping process, y = f(x), where f() represents a mapping relationship. The preset neural network can be a deep learning neural network, which can learn useful mappings. Once learned, these useful mappings are fixed as a mapping relationship.

[0064] A2. Represent the first feature information as a first Gaussian mixture model of J normal distributions superimposed, and represent the second feature information as a second Gaussian mixture model of J normal distributions superimposed, where J is the dimension of the midpoint of the first feature information and the second feature information.

[0065] Specifically, the Gaussian probability density function is used to accurately quantify things, decomposing a thing into several models based on the Gaussian probability density function, which is the normal distribution.

[0066] Specifically, the GMM (Gaussian Mixture Model) corresponding to the target point cloud can be represented by the following equation (1), and the GMM parameters of the two point clouds can be obtained according to the following equations (2) and (3) respectively. and In a Gaussian Mixture Model (GMM), each component (normal distribution) represents a feature in one dimension of f. Each feature in one dimension covers every point in the point cloud. Each first Gaussian Mixture Model can represent M points, and each second Gaussian Mixture Model can represent N points. Each point is J-dimensional. Since the number of points M and N in the point cloud is not fixed, point clouds of different sizes can be represented using a GMM with J components.

[0067]

[0068]

[0069]

[0070] Where, μ j Let represent the weight scalar, which is a 3×1 mean vector, ∑ j It is a 3×3 covariance matrix.

[0071] A3. Based on the first Gaussian mixture model and the second Gaussian mixture model, the first initial matrix is ​​obtained.

[0072] Specifically, see Figure 6 Step A3 may include:

[0073] A31. Obtain the first intermediate transformation matrix based on the first Gaussian mixture model and the second Gaussian mixture model, and construct the first loss function based on the first intermediate transformation matrix. The first intermediate transformation matrix is ​​the transformation matrix that minimizes the difference between the first Gaussian mixture model and the second Gaussian mixture model after transformation by the first intermediate transformation matrix.

[0074] In this example, based on the point cloud represented by the Gaussian Mixture Model (GMM), a rigid transformation matrix T (i.e., the first intermediate transformation matrix) is obtained to minimize the difference between the first and second Gaussian Mixture Models after transformation T. KL divergence can be used to measure the difference between the two model distributions; a smaller KL divergence indicates a smaller difference between the two GMM distributions, and consequently, a smaller difference in the corresponding point cloud.

[0075] A32. When coarse registration is completed based on the first loss function, the first intermediate transformation matrix is ​​used as the first initial matrix.

[0076] A33. When it is determined that coarse registration is incomplete according to the first loss function, the parameters of the preset neural network are updated according to the first loss function, and the process returns to the step of feature extraction.

[0077] In this example, as shown in equation (4), in order to establish the mapping relationship, i.e., to obtain the... satisfy The correspondence, T truth Here are the true transformation matrices for two point clouds to be registered. In T truth The MSE (Mean-Square Error) loss function is constructed between the obtained T and the parameter T to update the parameters of the PointNet network in the preset neural network. The MSE loss function (i.e. the first loss function) is shown in Equation (5).

[0078]

[0079] Loss=||TT truth || 2 (5)

[0080] in, This is the first Gaussian mixture model after the T-transformation. Let KL divergence be the difference between the first Gaussian mixture model and the second Gaussian mixture model after T transformation.

[0081] After obtaining the first loss function, it can be used to determine whether coarse registration is complete. For example, if the first loss function shown in equation (5) is at its minimum or no longer converges, it can be determined that coarse registration is complete; otherwise, it can be determined that coarse registration is incomplete.

[0082] As an example, such as Figure 7 As shown, the coarse registration matrix calculation for the point cloud to be registered and the target point cloud based on the multi-task learning algorithm includes:

[0083] B1. Extract features from the point cloud to be registered and the target point cloud respectively to obtain the third feature information and the fourth feature information. The third feature information and the fourth feature information have the same form.

[0084] In this process, convolution operations in the PointNet neural network can be used to extract features from the point cloud to be registered and the target point cloud, respectively. "Same format" refers to the same data dimensions, such as both features having dimensions [3, 4, 5].

[0085] B2. Construct multiple tasks based on the third and fourth feature information, and obtain the second initial matrix based on the multiple tasks.

[0086] In this example, such as Figure 7 As shown, the feature extraction module can extract features from the point cloud to be registered and the target point cloud separately, obtaining the third feature information fsrc and the fourth feature information ftgt. The third feature information fsrc and the fourth feature information ftgt have the same form. Based on the shared feature extraction module, multiple task branches are set up, such as two branches. Each branch learns different tasks, and the registration purpose is achieved through the coupling of different tasks.

[0087] In a specific example, for two task branches, one is the point cloud reconstruction task and the other is the point cloud registration task, when the point cloud feature information extracted by the feature extraction module has discriminability (the features extracted from unregistered points are inconsistent) and robustness (the features extracted from successfully registered points are consistent), then the transformation matrix can be extracted from these two features with consistent form (point cloud registration task). The basis for this discriminability and robustness is guaranteed by the point cloud reconstruction task.

[0088] Specifically, see Figure 8 The task has two branches, denoted as branch 1 and branch 2. Branch 1 is the point cloud reconstruction task, and branch 2 is the point cloud registration task. Both branches share the feature extraction module. At this point, step B2 may include:

[0089] B21. Calculate the point cloud reconstruction loss based on the third and fourth feature information;

[0090] Branch 1 can calculate the point cloud reconstruction loss using the following formula:

[0091]

[0092] Where S1 and S2 represent the number of points in the point cloud to be registered and the number of points in the target point cloud, respectively. The first term represents the sum of the minimum distances from any point x in the point cloud to be registered to the target point cloud, and the second term represents the sum of the minimum distances from any point y in the target point cloud to the point cloud to be registered.

[0093] B22. Using the inverse synthesis method, the second intermediate transformation matrix is ​​obtained based on the third and fourth feature information, and the second loss function is constructed based on the second intermediate transformation matrix;

[0094] Branch 2, after obtaining the second intermediate transformation matrix T, compares it with the true T. truth Constructing MSE loss L between mse (i.e., the second loss function), L mse =||TT truth || 2 .

[0095] B23. Calculate the sum of the point cloud reconstruction loss and the second loss function;

[0096] Wherein, the sum of the point cloud reconstruction loss and the second loss function is L = L mse +L recon .

[0097] B24. When the coarse registration is completed based on the sum value, the second intermediate transformation matrix is ​​used as the second transformation matrix.

[0098] B25. When the coarse registration is not completed based on the sum value, the point cloud to be registered is transformed using the second intermediate transformation matrix, and the process returns to the feature extraction step.

[0099] If the sum is minimized or converges, the coarse registration can be considered complete; otherwise, the coarse registration can be considered incomplete.

[0100] See in this example. Figure 6 The third feature information fsrc and the fourth feature information ftgt are used as inputs to the point cloud reconstruction module to calculate the point cloud reconstruction loss. The transformation matrix T is calculated using the third feature information fsrc and the fourth feature information ftgt via inverse synthesis. The point cloud to be registered is transformed using T, and the transformed point cloud to be registered and the target point cloud are input into the point cloud reconstruction module to calculate the point cloud reconstruction loss. This step is repeated. Alternatively, after obtaining the transformation matrix T, it can also be performed as follows... Figure 6 As shown, the target point cloud is transformed by T, and the transformed target point cloud and the point cloud to be registered are input into the point cloud reconstruction module to calculate the point cloud reconstruction loss.

[0101] Specifically, branch 1 includes a feature extraction module and a point cloud reconstruction module, wherein the feature extraction module can be expressed as follows: The mapping represents the feature extraction module in the network mapping the N×3 dimensional point cloud P. N×3 Mapped to a fixed D-dimensional feature f D This ensures that regardless of the size of the input point cloud, both the point cloud to be registered and the target point cloud are represented using features of the same dimension. This feature f D It needs to be distinctive and robust, which is achieved through reconstruction loss L. recon Constraints. L recon The input is the output of the point cloud reconstruction module. The point cloud reconstruction module can be used as follows: Mapping representation. The point cloud reconstruction module takes the original input point cloud P as input. N×3 Reconstruction Through the The constraints make and P N×3 The tendency towards consistency, in turn, constrains f D and P N×3 The relationship. and P N×3 Through continuous optimization of L recon To achieve the goal of gradually approaching, L recon The formula is expressed as in equation (6) above. If the point cloud reconstruction loss shown in equation (6) is large, it indicates that the two point clouds are significantly different; if the point cloud reconstruction loss shown in equation (6) is small, it indicates that the point cloud reconstructed by the point cloud reconstruction module is close to the input point cloud.

[0102] Branch 2 with f src and f tgt The difference r between the input and output is used to iteratively calculate the transformation matrix T using the Inverse Compositional (IC) algorithm. An initial default value T0 is set, and T is updated after each iteration by calculating the increment ΔT. After a certain number of iterations, the predicted transformation matrix is ​​obtained. The transformation matrix T obtained in each iteration transforms the input point cloud to be registered, resulting in an updated point cloud to be registered. This updated point cloud is then fed into the feature extraction module to obtain the updated f. src and f tgt These two features then serve as inputs for the next iteration to calculate T. Theoretically, as the number of iterations increases, the input point cloud will become increasingly consistent.

[0103] It should be noted that the coupling relationship of the multi-task learning in this invention is mainly reflected in three aspects: 1) the two branches share the feature extraction module; 2) the L of branch 1 recon The parts respectively contain the reconstruction loss of the point cloud to be registered and the target point cloud, through inverse constraint. Japanese style This allows branch 2 to pass through the flawed but robust f that failed to register. src and f tgt 3) The point cloud to be registered and the target point cloud tend to be consistent under the transformation T; then f src and f tgt The difference r→0. Through L mse Enhanced backpropagation for the feature extraction module The mapping relationship, positive push The formation of the mapping relationship makes the reconstruction task in branch 1 possible. Therefore, the final loss function obtained by the coarse registration task based on the deep learning model is the sum of the above values.

[0104] S2, use the first transformation matrix to transform the point cloud to be registered to obtain the first transformed point cloud.

[0105] Specifically, coarse registration is achieved by transforming one of the two point clouds using a rigid transformation matrix T from the oral scan point cloud to the CBCT point cloud.

[0106] S3, the iterative nearest point algorithm is used multiple times to perform fine registration of the first transformed point cloud and the target point cloud so that the final registration accuracy is less than or equal to the first preset value.

[0107] Specifically, the ICP algorithm is used iteratively to predict the transformation matrix. One of the two sets of data is transformed according to the currently calculated transformation matrix, making the two sets of data increasingly closer until a preset accuracy is reached. During each iteration of the transformation, the error precision (i.e., the point-to-point accuracy) is set separately, and the error precision gradually decreases as the transformation continues. The error precision setting can be based on experience.

[0108] As a possible implementation method, such as Figure 9 As shown, step S3 may include:

[0109] S91, set n error precision values ​​that decrease in succession, wherein the value of the nth error precision is less than or equal to the first preset value, n is a positive integer and n≥2;

[0110] S92, during the first registration, the first transformed point cloud and the target point cloud are registered using the iterative nearest point algorithm based on the first error accuracy;

[0111] S93, during the i-th registration, the second transformed point cloud and the target point cloud obtained from the (i-1)-th registration are registered using the iterative nearest point algorithm based on the i-th error accuracy, where i is a positive integer and 2≤i≤n.

[0112] Specifically, in one example, step S33, registering the second transformed point cloud and the target point cloud using the iterative nearest-point algorithm based on the error accuracy, may include:

[0113] C1. For each point in the second transformed point cloud, calculate its first distance to each point in the target point cloud;

[0114] C2. Obtain the first effective point pair based on the first distance and error accuracy;

[0115] Specifically, for each point in the second transformed point cloud, the distance from it to each point in the target point cloud can be calculated to obtain a set of distances corresponding to each point in the second transformed point cloud. The point pair corresponding to the minimum distance in each set of distances, and whose minimum distance is less than the preset point pair error precision, is taken as the first valid point pair.

[0116] C3. Calculate the second transformation matrix based on the first effective point pair;

[0117] C4. Use the second transformation matrix to transform the second transformation point cloud and update the second transformation point cloud. Return to the step of calculating the first distance until the preset maximum number of iterations is reached, or the difference between the second transformation matrices obtained from two adjacent iterations is less than the third preset value.

[0118] In this example, point-to-point registration is performed based on the second transformed point cloud and the target point cloud. Multiple registration calculations based on the ICP algorithm are performed, with different point-to-point error precisions set for each calculation. The point-to-point error precision value decreases progressively, meaning the required accuracy increases. After completing the previous ICP-based registration process, the new point-to-point error precision is used to proceed to the next ICP-based registration calculation. The final result after matrix transformation is taken as the fine registration result. This improves the registration accuracy.

[0119] In another example, step S33, registering the second transformed point cloud and the target point cloud using the iterative nearest-point algorithm based on the error accuracy, may include:

[0120] D1. For each point in the second transformed point cloud, determine its projection point to each triangular mesh in the target surface, and calculate its second distance to each projection point, where the points in the target point cloud are the vertices of the triangular meshes on the target surface.

[0121] In determining the projection point of point P in the second transformed point cloud onto the triangular mesh in the target surface, the projection point P' of point P onto the target surface can be determined first. Then, the centroid method and the perpendicular vector method are used to determine whether point P' is in the triangular mesh in the target surface. If it is, then it is determined that the projection point of point P exists in the triangular mesh in the target surface. In one embodiment, the target surface can be obtained by reconstructing a surface based on the target point cloud.

[0122] Specifically, in calculating the projection point, it is first determined whether the projection point of point P (a point in the second transformed point cloud) (projected onto the target surface) is within the triangular mesh. If it is within the triangular mesh, the coordinates of the projection point and the distance to the corresponding point are calculated; otherwise, the calculation is skipped. Compared to calculating the distance first and then determining whether it is within the triangular mesh, this method of determining beforehand significantly reduces the amount of computation. Specifically, determining whether the projection point of point P is within the triangular mesh can be achieved using a combination of the centroid method and the perpendicular vector method.

[0123] If the three vertices of the triangle are A, B, and C, and the two initial vectors of point A in the plane are v1 = BA and v2 = CA, according to the centroid method, for any point PP inside the triangle: PP = A + t1*v1 + t2*v2, where t1 and t2 are scalars. PP is inside triangle ABC when the three conditions 0 ≤ t1 ≤ 1, 0 ≤ t2 ≤ 1, and t1 + t2 < 1 are satisfied simultaneously.

[0124] If the projection point of P is PP, then the vector v = P - PP is perpendicular to the triangular mesh ABC. According to the perpendicular vector method, the dot product of two perpendicular vectors is 0, so we can get: v·v1 = 0 and v·v2 = 0.

[0125] Furthermore, we can deduce that:

[0126] t1=(a1*c2-a2*c1) / (b1*c2-c1*c1);

[0127] t2=(a1*c1-a2*b1) / (c1*c1-b1*c2);

[0128] Where a1 = v3·v1, b1 = v1·v1, c1 = v2·v1, a2 = v3·v2, c2 = v2·v2, and vector v3 = PA. If t1 and t2 satisfy the conditions, calculate the coordinates of the projection point PP, and calculate the Euclidean distance between point P and point PP.

[0129] D2. Obtain the second effective point pair based on the second distance and error accuracy;

[0130] Specifically, for each point in the second transformed point cloud, its projection point to each triangular mesh in the target surface is calculated, and the distance to the projection point is also calculated. This yields a set of distances for each point in the second transformed point cloud. The point in the source point cloud corresponding to the point with the smallest distance in each set, provided that the smallest distance is less than a preset point-pair error precision, and the corresponding projection point are taken as the second valid point pair. The target surface is obtained through surface reconstruction based on the target point cloud.

[0131] D3. Calculate the second transformation matrix based on the second effective point pair;

[0132] D4. Use the second transformation matrix to transform the second transformation point cloud and update the second transformation point cloud. Return to the step of calculating the second distance until the preset maximum number of iterations is reached, or the difference between the second transformation matrices obtained from two adjacent iterations is less than the third preset value.

[0133] In this example, point-to-surface registration is performed based on the second transformed point cloud and the target point cloud. Multiple registration calculations based on the ICP algorithm are performed, with different point-to-point error precisions set for each calculation. The precision value decreases progressively, indicating increasingly higher accuracy requirements. After completing the previous ICP-based registration process, the new point-to-point error precision is used to proceed to the next ICP-based registration calculation. The final result after matrix transformation is taken as the fine registration result. This improves registration accuracy.

[0134] Specifically, in one example, registering the first transformed point cloud and the target point cloud using the iterative nearest-point algorithm based on the first error accuracy can include:

[0135] Based on the current point cloud to be registered and the target point cloud, obtain the effective point pairs and calculate the transformation matrix based on the effective point pairs; use the obtained transformation matrix to transform the current point cloud to be registered.

[0136] Return to the previous step until the preset number of iterations or the preset registration accuracy is reached, and use the transformed point cloud as the second transformed point cloud.

[0137] The process of obtaining valid point pairs can be carried out by referring to any of the above methods for obtaining valid point pairs between the second transformed point cloud and the target point cloud. In this process, when the transformation matrix is ​​calculated for the first time, the point cloud to be registered is the first transformed point cloud. When this step is repeated in the next round, the point cloud to be registered is the point cloud after transformation in the previous round.

[0138] It should be noted that each registration process using the iterative nearest point algorithm is a multi-round iterative loop. In this iterative process, effective point pairs are calculated using the current point cloud to be registered and the target point cloud (or the target surface where the target point cloud is located). The transformation matrix is ​​calculated using the effective point pairs. The transformation matrix is ​​used to transform the current point cloud to be registered to obtain a new point cloud to be registered. This process continues until the preset number of iterations or the preset registration accuracy is reached, at which point the current registration is terminated, and the next registration is performed using the new error accuracy. This continues until the registration accuracy meets the final requirement, which is less than or equal to the first preset value.

[0139] It should be noted that the ICP algorithm is a registration algorithm, suitable for situations where the input data are not significantly different. When using the ICP algorithm, an error precision can be set. If the registration process of the ICP algorithm can be completed within the error precision, the registration is successful; otherwise, the registration result is uncontrollable. Generally, it is only used once (with a set error) for fine-tuning. This invention uses the ICP algorithm as a fine-tuning algorithm. By using the ICP algorithm multiple times, with a relatively large initial error precision (e.g., 0.5), it is gradually reduced to the expected error precision (e.g., 0.01) to achieve the desired registration effect.

[0140] In some embodiments of the present invention, before performing coarse registration matrix calculation on the point cloud to be registered and the target point cloud, the method may further include: extracting a tooth model from oral CT data using a region growing method, converting the extracted tooth model into first point cloud data; converting a three-dimensional tooth model obtained using an intraoral scanner into second point cloud data; filtering outliers in the first point cloud data and the second point cloud data respectively, and downsampling the filtered point cloud data; and using one of the two sets of point cloud data obtained after filtering and sampling as the point cloud to be registered and the other as the target point cloud.

[0141] Specifically, after obtaining the first point cloud data and the second point cloud data, the two point clouds can be preprocessed separately. This preprocessing may include: using statistical filters to filter outliers in the first and second point cloud data respectively, and downsampling the filtered point clouds to reduce the magnitude of the point cloud data while maintaining the corresponding two point clouds at the same density.

[0142] In some embodiments of the present invention, before calculating the coarse registration matrix for the point cloud to be registered and the target point cloud, the method may further include: determining a first centroid of the point cloud to be registered and a second centroid of the target point cloud; adjusting the positions of the point cloud to be registered and / or the target point cloud so that the first centroid and the second centroid coincide. This reduces the computational load and prediction difficulty when obtaining the transformation matrix through subsequent registration.

[0143] In a specific example, the present invention applies the above-described method, and the final registration effect of the oral CT data and oral scan data is as follows: Figure 10 , Figure 11 As shown. From Figure 10 , Figure 11 As can be seen, the registration accuracy obtained by using the registration method of the present invention can be less than 0.4 mm.

[0144] In summary, the point cloud registration method of this invention can effectively register oral CT data and oral scan data without manual intervention, improving registration efficiency and solving the registration problem of cross-modal data. Furthermore, when using this method to align cross-modal data, the requirement for uniformity in the coverage areas of the two data parts is relatively low. This not only saves on the cost of data preprocessing but also reduces the occurrence of registration failures or low accuracy due to the loss of important data information caused by preprocessing, thereby improving registration efficiency.

[0145] Based on the point cloud registration method described above, this invention also proposes a point cloud registration device.

[0146] In embodiments of the present invention, such as Figure 12As shown, the point cloud registration device 100 includes a memory 10, a processor 20, and a computer program 30 stored in the memory 10. When the computer program 30 is executed by the processor 20, it implements the point cloud registration method described above.

[0147] Furthermore, the present invention also proposes a medical imaging device.

[0148] In embodiments of the present invention, such as Figure 13 As shown, the medical imaging device 200 includes the point cloud registration device 100 of the above embodiment.

[0149] The medical imaging equipment 200 can be a dental CT, dental CBCT, dental X-ray machine, etc., or an intraoral scanning system, etc. Specifically, the point cloud registration device 100 can be a computer, host computer, mobile terminal (such as mobile phone, tablet computer), etc.

[0150] The medical imaging equipment of this invention, through the aforementioned point cloud registration device, can effectively register oral CT data and oral scan data without manual intervention, improving registration efficiency and solving the registration problem of cross-modal data. Furthermore, when using this method to align cross-modal data, the requirement for uniformity in the coverage areas of the two data parts is relatively low. This not only saves on the cost of data preprocessing but also reduces the occurrence of registration failures or low accuracy due to the loss of important data information caused by preprocessing, thereby improving registration efficiency.

[0151] It should be noted that the logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include: an electrical connection having one or more wires (electronic device), a portable computer disk drive (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Alternatively, the computer-readable medium may be paper or other suitable media on which the program can be printed, since the program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in a computer memory.

[0152] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0153] In the description of this specification, references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0154] In the description of this invention, it should be understood that the terms "center," "longitudinal," "lateral," "length," "width," "thickness," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," "outer," "clockwise," "counterclockwise," "axial," "radial," and "circumferential" indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are used only for the convenience of describing this invention and simplifying the description, and are not intended to indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this invention.

[0155] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this invention, "a plurality of" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0156] In this invention, unless otherwise explicitly specified and limited, the terms "installation," "connection," "linking," and "fixing," etc., should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral part; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; they can refer to the internal communication of two components or the interaction between two components, unless otherwise explicitly limited. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0157] In this invention, unless otherwise explicitly specified and limited, "above" or "below" the second feature can mean that the first feature is in direct contact with the second feature, or that the first feature is in indirect contact with the second feature through an intermediate medium. Furthermore, "above," "over," and "on top" of the second feature can mean that the first feature is directly above or diagonally above the second feature, or simply that the first feature is at a higher horizontal level than the second feature. "Below," "below," and "under" the second feature can mean that the first feature is directly below or diagonally below the second feature, or simply that the first feature is at a lower horizontal level than the second feature.

[0158] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.

Claims

1. A point cloud registration method, characterized in that, include: Based on a preset algorithm, a coarse registration matrix is ​​calculated between the point cloud to be registered and the target point cloud to obtain a first transformation matrix that meets the first precision preset condition. The preset algorithm includes a Gaussian mixture model-based algorithm. The point cloud to be registered is transformed using the first transformation matrix to obtain a first transformed point cloud. The first transformed point cloud and the target point cloud are finely registered multiple times using the iterative nearest point algorithm so that the final registration accuracy is less than or equal to the first preset value. The step of calculating the coarse registration matrix of the point cloud to be registered and the target point cloud based on a preset algorithm to obtain a first transformation matrix that satisfies the first precision preset condition includes: calculating the coarse registration matrix of the point cloud to be registered and the target point cloud based on a Gaussian mixture model algorithm to obtain a first initial matrix; and using the first initial matrix as the first transformation matrix that satisfies the first precision preset condition. The method for calculating the coarse registration matrix of the point cloud to be registered and the target point cloud based on the Gaussian mixture model algorithm to obtain the first initial matrix includes: A preset neural network is used to extract features from the point cloud to be registered and the target point cloud to obtain first feature information and second feature information. The first feature information is represented as a first Gaussian mixture model of J normal distributions superimposed, and the second feature information is represented as a second Gaussian mixture model of J normal distributions superimposed, where J is the dimension of the midpoint of the first feature information and the second feature information; A first initial matrix is ​​obtained based on the first Gaussian mixture model and the second Gaussian mixture model; The step of obtaining the first initial matrix based on the first Gaussian mixture model and the second Gaussian mixture model includes: A first intermediate transformation matrix is ​​obtained based on the first Gaussian mixture model and the second Gaussian mixture model, and a first loss function is constructed based on the first intermediate transformation matrix. The first intermediate transformation matrix is ​​the transformation matrix that minimizes the difference between the first Gaussian mixture model and the second Gaussian mixture model after transformation by the first intermediate transformation matrix. When coarse registration is completed based on the first loss function, the first intermediate transformation matrix is ​​used as the first initial matrix; If it is determined that coarse registration is incomplete based on the first loss function, the parameters of the preset neural network are updated according to the first loss function, and the process returns to the step of feature extraction.

2. The method as described in claim 1, characterized in that, The step of performing coarse registration matrix calculation on the point cloud to be registered and the target point cloud based on a preset algorithm to obtain a first transformation matrix that satisfies the first precision preset condition also includes: The preset algorithm also includes a multi-task learning algorithm; Based on a multi-task learning algorithm, a coarse registration matrix is ​​calculated for the point cloud to be registered and the target point cloud to obtain a second initial matrix. The one with higher registration accuracy between the first initial matrix and the second initial matrix is ​​used as the first transformation matrix that satisfies the first accuracy preset condition.

3. The method as described in claim 2, characterized in that, The coarse registration matrix is ​​calculated for the point cloud to be registered and the target point cloud based on a multi-task learning algorithm, including: Feature extraction is performed on the point cloud to be registered and the target point cloud respectively to obtain third feature information and fourth feature information, wherein the third feature information and the fourth feature information have the same form; Multiple tasks are constructed based on the third and fourth feature information, and the second initial matrix is ​​obtained based on the multiple tasks.

4. The method as described in claim 3, characterized in that, The step of constructing multiple tasks based on the third and fourth feature information, and obtaining the second initial matrix based on the multiple tasks, includes: Calculate the point cloud reconstruction loss based on the third and fourth feature information; The second intermediate transformation matrix is ​​obtained using the inverse synthesis method based on the third and fourth feature information, and a second loss function is constructed based on the second intermediate transformation matrix. Calculate the sum of the point cloud reconstruction loss and the second loss function; When the coarse registration is completed based on the sum value, the second intermediate transformation matrix is ​​used as the second initial matrix; If coarse registration is not completed based on the sum value, the point cloud to be registered is transformed using the second intermediate transformation matrix, and the process returns to the feature extraction step.

5. The method as described in claim 1, characterized in that, The step of repeatedly using the iterative nearest-point algorithm to perform fine registration of the first transformed point cloud and the target point cloud, so that the final registration accuracy is less than or equal to a first preset value, includes: Set n error precision values ​​that decrease in increments, where the value of the nth error precision is less than or equal to the first preset value, n is a positive integer, and n≥2; During the first registration, the first transformed point cloud and the target point cloud are registered using the iterative nearest-point algorithm based on the first error accuracy. During the i-th registration, the second transformed point cloud obtained from the (i-1)-th registration and the target point cloud are registered using the iterative nearest point algorithm based on the i-th error accuracy, where i is a positive integer and 2≤i≤n.

6. The method as described in claim 5, characterized in that, Based on the error accuracy, the iterative nearest-point algorithm is used to register the second transformed point cloud and the target point cloud, including: For each point in the second transformed point cloud, calculate its first distance to each point in the target point cloud; The first effective point pair is obtained based on the first distance and the error accuracy; Calculate the second transformation matrix based on the first pair of valid points; The second transformation point cloud is transformed and updated using the second transformation matrix. The process returns to the step of calculating the first distance until the preset maximum number of iterations is reached, or the difference between the second transformation matrices obtained from two adjacent iterations is less than a third preset value.

7. The method as described in claim 5, characterized in that, Based on the error accuracy, the iterative nearest-point algorithm is used to register the second transformed point cloud and the target point cloud, including: For each point in the second transformed point cloud, determine its projection point to each triangular mesh in the target surface, and calculate its second distance to each projection point, wherein the point in the target point cloud is the vertex of the triangular mesh on the target surface; The second effective point pair is obtained based on the second distance and the error accuracy; Calculate the second transformation matrix based on the second effective point pair; The second transformation point cloud is transformed and updated using the second transformation matrix. The process returns to the step of calculating the second distance until the preset maximum number of iterations is reached, or the difference between the second transformation matrices obtained from two adjacent iterations is less than a third preset value.

8. The method as described in claim 7, characterized in that, When determining the projection point of point P in the second transformed point cloud onto the triangular mesh in the target surface... Determine the projection point P' from point P onto the target surface; Use the centroid method and the perpendicular vector method to determine whether point P' is in the triangular mesh of the target surface; If present, then it is determined that point P, the projection point, exists in the triangular mesh of the target surface.

9. The method as described in claim 1, characterized in that, Before performing coarse registration matrix calculation on the point cloud to be registered and the target point cloud, the method further includes: A tooth model was extracted from oral CT data using the region growing method, and the extracted tooth model was converted into first point cloud data. The three-dimensional model of the teeth obtained using an intraoral scanner is converted into second point cloud data. Outliers in the first point cloud data and the second point cloud data are filtered respectively, and the filtered point cloud data is downsampled. One of the two sets of point cloud data obtained after filtering and sampling is used as the point cloud to be registered, and the other is used as the target point cloud.

10. The method as described in claim 1, characterized in that, Before performing coarse registration matrix calculation on the point cloud to be registered and the target point cloud, the method further includes: Determine the first centroid of the point cloud to be registered and the second centroid of the target point cloud; Adjust the positions of the point cloud to be registered and / or the target point cloud so that the first centroid and the second centroid coincide.

11. The point cloud registration method as described in claim 1, characterized in that, Of the point cloud to be registered and the target point cloud, one is a point cloud obtained from oral CT data, and the other is a point cloud obtained from surface data of the oral cavity scanned by an intraoral scanner.

12. A point cloud registration device, comprising a memory, a processor, and a computer program stored in the memory, characterized in that, When the computer program is executed by the processor, it implements the point cloud registration method as described in any one of claims 1-11.

13. A medical imaging device, characterized in that, Including the point cloud registration device as described in claim 12.

Images