Dual-stage CBCT (cone beam computed tomography) and oral cavity scanning tooth registration method and system

Abstract

The invention discloses a dual-stage CBCT and oral scanning tooth registration method and system, and relates to the field of tooth three-dimensional digital model registration, and the method comprises the following steps: extracting a tooth voxel structure in a CBCT image and a dental crown grid structure in an oral scanning image through a segmentation network; a coarse-to-fine depth registration network is introduced, geometric feature coding and feature matching are carried out on super-points obtained through multi-resolution down-sampling, local matching is carried out on a dense point set in a super-point neighborhood, a rigid transformation matrix is calculated in parallel, and an optimal registration result is screened out from the rigid transformation matrix; and dividing a registration object into a plurality of three-tooth groups containing three adjacent teeth, and performing local iteration alignment based on an initial registration result. The method still has higher robustness and accuracy under the complex conditions of large cross-modal difference, high input point cloud density and the like, is shorter in time consumption, and provides reliable data basis and automatic support for clinical scenes such as orthodontics and implantation.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of tooth three-dimensional digital model registration, in particular to a two-stage CBCT and intraoral scanning tooth registration method and system. BACKGROUND

[0002] With the rapid development of digital dentistry, cone beam computed tomography (CBCT) and intraoral scanning (IOS) are increasingly widely used in clinical diagnosis and treatment. IOS has become an important means to obtain three-dimensional structure of tooth surface due to its high resolution, no radiation and convenient operation, while CBCT can provide complete three-dimensional bone anatomy information and is widely used in implant design, orthodontic planning and apex diagnosis. The fusion of the two information and the high-precision alignment of the IOS and CBCT images through registration technology have become an important direction of digital oral treatment.

[0003] However, CBCT and IOS belong to different modalities, and their acquisition mechanisms and focuses are significantly different. The former mainly reflects the internal bone structure and voxel intensity distribution, while the latter focuses on the mesh expression of the crown surface shape. Therefore, the registration task between such multi-modal data faces multiple challenges such as inconsistent features, scale differences, data noise and occlusions.

[0004] Traditional methods often rely on manual features or simple transformations, resulting in low registration accuracy. Methods that rely on mathematical formula iterative registration, although the principle is clear and the implementation is simple, are highly dependent on the initial alignment position. In recent years, deep learning registration methods have emerged, which have stronger robustness and real-time advantages, can skip the explicit feature matching process and directly predict the transformation matrix, but often sacrifice the accuracy of the final registration, and cannot achieve accurate registration between two modalities with low overlap and large differences, especially in the alignment of detailed structures.

[0005] Therefore, in order to balance the registration accuracy and efficiency and fully integrate the structural advantages of CBCT and IOS data, it is necessary to propose a new method for registration of multi-modal images in oral cavity.

[0006] The above information disclosed in the background section is only used to strengthen the understanding of the background of the present disclosure, and therefore it can include information that does not constitute prior art known to those of ordinary skill in the art. SUMMARY

[0007] The purpose of the present application is to provide a two-stage CBCT and intraoral scanning tooth registration method and system to solve the problems raised in the background.

[0008] To achieve the above purpose, the present application provides the following technical solutions: A method for two-stage CBCT and oral scanning tooth registration, the specific steps comprising: S1, based on a convolutional neural network, a first segmentation model is constructed and trained, the input of which is an oral slice image, and the output is a three-dimensional tooth image, and the three-dimensional tooth image is marked with tooth segmentation contours and voxel information; S2, based on ToothNet network, a second segmentation model is constructed and trained, the input of which is a three-dimensional oral image, and the output is a three-dimensional crown image, and the three-dimensional crown image is marked with crown contour and grid information; S3, based on the first segmentation model and the second segmentation model, the three-dimensional tooth image and the three-dimensional crown image of the detector are obtained, and the voxel information and the grid information are extracted to be converted into point cloud data, and the tooth overall point cloud and the crown overall point cloud are obtained; S4, the teeth of the detector are divided according to the mode of adjacent three teeth as a group and one tooth between adjacent groups, and a plurality of three-tooth groups are obtained, the voxel information and the grid information of each three-tooth group are extracted to be converted into point cloud data, and three-tooth group tooth point cloud and three-tooth group crown point cloud are obtained; S5, the tooth overall point cloud and the crown overall point cloud are input into a pre-trained registration network model, and the overall point cloud is preliminarily registered, so that the three-dimensional crown image is preliminarily aligned in the space of the three-dimensional tooth image; S6, based on the three-dimensional crown image and the three-dimensional tooth image preliminarily aligned in space, the three-tooth group tooth point cloud and the three-tooth group crown point cloud are locally iteratively registered according to the three-tooth group, and the three-dimensional crown image point cloud accurately aligned with the three-dimensional tooth image point cloud after transformation is obtained.

[0009] Further, the oral slice image is obtained based on CBCT technology, which is used to reflect the tooth information inside the oral cavity, the first segmentation model is established based on convolutional neural network, and the convolutional neural network is composed of input layer, convolutional layer, pooling layer, full connection layer and output layer, wherein the activation function in the convolutional layer is function, for the full connection layer, the number of full connection layer neurons is set to 32, the initial neural network learning rate is set to 0.001, and the training rounds are set to 100; The three-dimensional oral image is obtained based on oral scanning technology, which is used to reflect the crown information inside the oral cavity, and the second segmentation model is established based on ToothNet network structure, wherein the second segmentation model extracts the slice edge graph grid information of each three-dimensional oral image, connects the slice edge graph grid information with the three-dimensional oral image grid information, and then puts the connected grid information into the corresponding network to complete the segmentation and identification of the crown.

[0010] Further, S31, the tooth segmentation contour voxel information output by the first segmentation model is converted into point cloud data to obtain the overall tooth point cloud. The logic is as follows: the voxel information of the tooth segmentation contour is processed by threshold segmentation and surface extraction algorithms to generate three-dimensional point cloud data containing the tooth shell structure. The specific method is as follows: the voxel information marked in the three-dimensional tooth image is read, its three-dimensional voxel array is extracted, and the spatial resolution corresponding to each voxel is recorded; a conversion threshold is set, and the voxel information marked in the three-dimensional tooth image is binarized. The specific conversion rules are as follows: In the formula, The coordinates in the 3D tooth image are: The voxel values ​​after voxel binarization. The coordinates in the 3D tooth image are: voxel values, The conversion threshold; Based on the binarized voxel values, isosurface structures are extracted, generating a set of triangular facets and vertices. The triangular meshes within the facet set are then subjected to multiple Laplacian smoothing processes, with a set number of iterations. The coordinates of all vertices in the smoothed triangular meshes are extracted as point cloud data, forming a complete point cloud set for the tooth. The specific expression for the complete point cloud set for the tooth is as follows: In the formula, A collection of point clouds for the entire tooth structure. Let j be the coordinates of the j-th point in the overall point cloud set of the teeth. This represents the total number of points in the overall tooth point cloud set, where j is the index of a point in the overall tooth point cloud set. S32, the method for converting the crown contour mesh information output by the second segmentation model into point cloud data is as follows: extracting the crown structure vertices from the crown contour mesh structure as point cloud data. Specifically, the extraction method is as follows: loading the IOS 3D intraoral scan model file stored in PLY format, extracting the set of crown structure vertices as 3D point cloud data, and constructing the overall crown point cloud set. The specific expression for the overall crown point cloud set is: In the formula, This represents the collection of point clouds representing the entire crown. Let i be the coordinates of the i-th point in the overall point cloud set of the tooth crown. This represents the total number of points in the overall point cloud set of the crown, where i is the index of the coordinates of the points in the overall point cloud set of the crown.

[0011] Furthermore, the specific steps for pre-training the registration network model are as follows: using the 3Dmatch public dataset as training input, perform overall feature extraction and registration on the overall point cloud set of the crown and the overall point cloud set of the tooth; filter the point cloud feature extraction network, superpoint matching network, local point matching network, and optimal registration transformation matrix in the registration network model; and train the registration network model through backpropagation using local matching loss and global transformation loss. The specific training process includes: S51, the point cloud feature extraction network is used to extract features from the point cloud data in the overall point cloud set of the crown and the overall point cloud set of the tooth, respectively. KPConv-FPN convolution is used to perform multi-resolution downsampling of the input point cloud, obtaining sparse and dense point sets for the overall point cloud set of the crown and the overall point cloud set of the tooth, respectively. Specific steps include: a first-layer downsampling, performing FPS downsampling on the transformed point cloud data, retaining 1 / 4 of the original number of points as the convolution center point, and outputting 64-dimensional features; a second-layer downsampling, retaining 1 / 16 of the original number of points, and outputting 128-dimensional features; a third-layer downsampling, retaining 1 / 64 of the original number of points, and outputting 256-dimensional features; using an upsampling module to perform upsampling twice on the third-layer image, restoring the spatial position through interpolation, and performing skip connections with the features of the corresponding downsampling layer, merging them to output 64-dimensional dense features; finally, the sparse and dense point sets and their corresponding features are output, where points within the sparse point set are designated as superpoints. S52 utilizes a superpoint matching network, combining the corresponding features of the obtained sparse point set, Euclidean distance, and geometric embedding features obtained from the angle between the three points. Through a self-attention network and a cross-attention network, feature matching is performed on the two sets of feature descriptors of the final downsampled crown point cloud and tooth point cloud. The Gaussian kernel function is used to obtain superpoint pairs that meet the confidence requirements, and the top G superpoint pairs are selected. S53, based on the three-dimensional coordinates of the selected top G superpoints, using the local geometric region point set in their neighborhood as the unit, at the resolution of the dense point set, extract the local point cloud subsets of the crown and the tooth as a whole, use the local point matching network to perform local point pair matching, and use the weighted singular value decomposition method to calculate the rigid transformation matrix between each pair of local regions, generating G rigid transformation matrices. S54. Select the transformation with the best matching consistency index from the generated G rigid transformation matrices as the global initial transformation matrix, and calculate the best registration transformation matrix between the crown point cloud and the tooth point cloud through interior point screening and iterative optimization. S55, apply the optimal rigid transformation matrix to the overall point cloud of the three-dimensional crown image to initially align the three-dimensional crown image with the three-dimensional tooth image in space.

[0012] Further, in step S51, the steps of multi-resolution downsampling and feature extraction specifically include: randomly selecting an initial point in both the crown point cloud and tooth point cloud data, and using this point as the starting point, using the far-point sampling algorithm to construct a sparse and spatially uniform point set as the center point set of the first convolution layer; Based on the center point set of convolution, a KPConv-FPN convolutional structure is constructed. In the neighborhood of each center point, feature vectors are extracted by kernel point weighted feature aggregation. The KPConv-FPN convolutional structure is based on U-Net, and the center point of each convolutional layer is the point obtained by downsampling of the previous layer. Based on this, a new local neighborhood is constructed for convolution operation, and multi-scale point cloud features are extracted through multiple iterations. The upsampling module upsamples the output of the third downsampling layer twice and restores the spatial position through interpolation.

[0013] Further, in step S52, the geometric embedding feature includes Euclidean distance and spatial angle formed by three points. The geometric embedding feature and the corresponding feature of the sparse point set together constitute a feature descriptor for matching. Finally, feature matching is performed on the two sets of feature descriptors obtained from the downsampled crown point cloud and the tooth point cloud. The feature matching steps include: inputting the feature descriptors into an attention-based matching network, which includes a self-attention algorithm and a cross-attention algorithm, to achieve feature alignment and fusion of the two sets of feature descriptors; the expressions for the self-attention algorithm and the cross-attention algorithm are as follows: In the formula, Here is the expression for the self-attention algorithm. This is the expression for the cross-attention algorithm. These represent the query, key, and value matrices of the self-attention algorithm, respectively. These represent the query, key, and value matrix of the cross-attention algorithm, respectively. For the first feature descriptor in the input feature descriptor Feature dimensions of each feature vector This indicates the number of feature vectors in the input feature descriptor. This is the index of the feature vector in the feature descriptor; The Gaussian kernel function is used to find the superpoint pairs that meet the confidence requirements. The logic for calculating the confidence using the Gaussian kernel function is as follows: calculate the Gaussian correlation score between the superpoints of each pair of crowns and teeth, and perform Softmax normalization on all scores to form a probability distribution, thus obtaining the matching confidence. The specific formula used to calculate the matching confidence is as follows: In the formula, This represents the sparse point cloud of the entire tooth structure, with the 3D coordinates of the t-th superpoint. and Represent the sparse point concentration of the overall point cloud of the tooth crown, and the three-dimensional coordinates of the e-th and u-th superpoints, respectively. It is the scaling parameter of the Gaussian kernel function. This represents the matching confidence between the t-th superpoint and the e-th point in the overall point cloud of the tooth crown, where t is the index of the superpoint in the sparse point set of the overall point cloud of the tooth. `e` is the index of the sparse point concentration superpoint of the overall point cloud of the crown; For any superpoint in the sparse point set of the overall tooth point cloud, sort the matching confidence scores of the superpoint with all superpoints in the overall crown point cloud in descending order. If the maximum matching confidence score is greater than the correlation threshold, then the two superpoints corresponding to the maximum matching confidence score are listed as superpoint pairs. If there are multiple identical maximum matching confidence scores, then randomly select the two superpoints corresponding to the maximum matching confidence score as superpoint pairs. Sort all superpoint pairs in descending order according to the matching confidence score to form an optimal set. Select the top G superpoint pairs in the optimal set.

[0014] Further, in step S53: based on the three-dimensional coordinates corresponding to the selected top G superpoints, the local geometric region point set in their neighborhood is used as the unit, wherein the specific formula for representing the local geometric region is: In the formula, This indicates the concentration of sparse points in the overall point cloud of teeth. A local geometric region is constructed centered on a superpoint. To select a superpoint, the corresponding superpoint should be a point with a dense concentration of points in its neighborhood. To select a superpoint, a dense set of points within the corresponding superpoint's neighborhood is required. Let s be the s-th superpoint within the sparse point set of the overall tooth point cloud, where s is the index of the superpoint within the sparse point set of the overall tooth point cloud; determine the local geometric regions of the overall tooth point cloud and the overall crown point cloud respectively, and denote the point set formed by the points in the local geometric region as the local point cloud subset; The specific steps for local point pair matching using a local point matching network include: Point-to-point feature matching was performed on subsets of the point clouds of the entire crown and the entire tooth. A similarity score was calculated, and a matching matrix was constructed. The matrix with the highest similarity was then selected. Each pair of points is used as a candidate matching point pair, and the corresponding matching confidence level is used as a weighting coefficient to calculate the weighted centroid of the two sets of point clouds. The formula used to calculate the weighted centroid of the overall tooth point cloud and the local point cloud of the overall crown point cloud is as follows: In the formula, For the first The feature vector of the corresponding point in the subset of the local point cloud of the whole tooth, for each candidate matching point pair. express The matching confidence weight of each candidate matching point pair Indicates the first The feature vectors of the corresponding points in the subset of the local point cloud of the entire crown, representing candidate matching point pairs. This represents the weighted centroid of a subset of the local point cloud representing the overall area of ​​the tooth. The weighted centroid of the local point cloud subset representing the overall crown is given by the following formula: For the index of the candidate matching point pair, ; A weighted covariance matrix is ​​constructed based on the obtained weighted centroids, where the formula for calculating the weighted covariance matrix is ​​as follows: In the formula, Indicates the first The weighted covariance matrix of candidate matching pairs; By analyzing the weighted covariance matrix Perform singular value decomposition to calculate the rotation matrix and translation vector, and construct the rigid transformation matrix, where the specific expression of the rigid transformation matrix is: In the formula, For the first The rigid transformation matrix of candidate matching point pairs For the first Rotation matrix of candidate matching point pairs, For the first Translation vectors of candidate matching point pairs; S533, the obtained local rigid transformation matrices are applied to the corresponding local source point cloud regions, and the error residuals obtained after applying each pair of transformations in the global space are calculated. Using the confidence level of local matching as weights, the optimal registration transformation matrix is ​​constructed. The specific formula for the optimal registration transformation matrix is ​​as follows: In the formula, The optimal registration transformation matrix is... Indicates the first The local geometric region of the entire crown corresponding to each superpoint. For the first Each superpoint's matching confidence weight Indicates the application of the transformation matrix. and These are the corresponding points in the partial point cloud sub-sets of the entire tooth and the entire crown, respectively.

[0015] Furthermore, the loss function during the pre-training process of the registration network model is: In the formula, For loss function, For local point pair matching loss function, The global registration transformation loss function is... The hyperparameters are defined as follows: the local point pair matching loss function is calculated using the cross-entropy between the predicted probability of the matched point pair and the true label; the global registration transformation loss function is constructed based on the Euclidean distance between corresponding point pairs in the source and target point clouds, and its specific expression is as follows: In the formula, This represents the Euclidean distance.

[0016] Further, in step S6, the specific steps for performing local iterative registration according to the three-tooth group point cloud and the three-tooth group crown point cloud include: For each point in the crown point cloud of the three dentition groups, find the nearest point in the point cloud space of the three dentition groups and establish a matching point set; by minimizing the spatial distance error between corresponding points, solve for the optimal rigid transformation so that the point cloud corresponding to the crown point cloud of the three dentition groups is precisely aligned with the point cloud of the three dentition groups in position and direction; in each iteration, update the position of the crown point cloud of the three dentition groups, and repeat this process until the set maximum number of iterations is reached. The formula used for local iterative registration is as follows: In the formula, In the point cloud of the third set of teeth, the first The three-dimensional coordinate vector of a point In the dot cloud of the crowns of the third group of teeth The three-dimensional coordinate vector of a point The maximum number of iterations is set. The index is the number of iterations. Finally, a precisely aligned 3D tooth image point cloud and a precisely aligned 3D crown image point cloud are obtained.

[0017] The present invention also provides a two-stage CBCT and oral scan tooth registration system, which is used to perform the above-described two-stage CBCT and oral scan tooth registration method, including: The first modality segmentation module is used to construct and train a first segmentation model based on a convolutional neural network, which takes an oral slice image as input and outputs a three-dimensional tooth image. The three-dimensional tooth image is marked with tooth segmentation contours and their voxel information. The second modality extraction module is used to construct and train a second segmentation model based on the ToothNet network, which takes a three-dimensional oral cavity image as input and outputs a three-dimensional tooth crown image. The three-dimensional tooth crown image is marked with the tooth crown contour and its mesh information. The point cloud information conversion module is used to acquire three-dimensional tooth images and three-dimensional crown images of the inspector based on the first segmentation model and the second segmentation model, and extract their voxel information and mesh information to convert them into point cloud data, so as to obtain the overall point cloud of the tooth and the overall point cloud of the crown. The grouping optimization module is used to divide the teeth of the inspector into multiple three-tooth groups by grouping three adjacent teeth together and separating adjacent groups by one tooth. The voxel information and mesh information of each three-tooth group are extracted and converted into point cloud data to obtain the point cloud of the teeth and the point cloud of the crown of the three-tooth group. The overall preliminary alignment module is used to input the overall point cloud of the tooth and the overall point cloud of the crown into the pre-trained registration network model, perform preliminary registration of the overall point cloud, and make the three-dimensional crown image initially aligned in the space of the three-dimensional tooth image. The two-stage group alignment module is used to perform local iterative registration of the point clouds of the teeth and the point clouds of the crowns of the three teeth according to the three-tooth groups based on the three-tooth groups that are initially aligned in space, so as to obtain the three-dimensional crown image point cloud that is accurately aligned with the point cloud of the three-dimensional tooth image after transformation.

[0018] Compared with the prior art, the beneficial effects of the present invention are: The proposed 3D point cloud registration method employs a two-stage strategy of multi-scale feature alignment and group optimization. The multi-scale feature alignment stage utilizes an end-to-end deep registration network structure, integrating KPConv convolutional kernels, geometric feature embedding, self-attention mechanisms, and cross-attention mechanisms. This enhances the alignment capability of extracted tooth and crown features in both CBCT and oral cavity scanning modalities, enabling preliminary alignment of point clouds from both modalities. The group optimization stage employs a three-tooth group division strategy, allowing for local error correction and arch fitting registration within three-tooth groups. Furthermore, a hierarchical registration strategy model for superpoints and dense points was designed in the feature matching stage. First, coarse pairing is performed between sparse superpoints, and then fine matching is performed on the local geometric regions of dense point sets based on their neighborhoods. For each pair of local regions, point pair matching and weighted singular value decomposition are used to calculate rigid transformation. The best matching consistency is selected globally through multiple pairs of local rigid matrices, which effectively solves the problem of low registration accuracy of deep learning models caused by large differences in point cloud density and geometric shape between the two modes. Meanwhile, the dental arch is divided into multiple tridental groups consisting of three adjacent teeth. After initial overall alignment, local error correction and dental arch fitting registration are further performed on a tridental group basis. The stitching error generated during the scanning process of the oral scan image is fully considered, effectively eliminating the local deviation of the scan point cloud, optimizing details that cannot be considered in the overall feature matching stage, and improving the spatial consistency of multimodal data in the overall dental arch structure. Finally, the two-stage registration process and its supporting feature alignment mechanism proposed in this invention can significantly improve the final accuracy and robustness of CBCT and IOS point cloud registration, reduce the registration failure rate, and require no manual intervention. It has high practical value and promotion prospects in actual oral diagnosis and treatment applications such as orthodontics, implantology, and restoration. Attached Figure Description

[0019] Figure 1 This is a schematic diagram of the overall method flow of the present invention; Figure 2 A schematic diagram of a three-dimensional digital model obtained from an oral scan and its crown contour annotation; Figure 3 The three-dimensional reconstructed digital image of teeth is output from the CBCT oral slice image using the first segmentation model; Figure 4 A schematic diagram of one of the three groups of teeth converted from CBCT to point cloud; Figure 5 This is a schematic diagram of one of the three tooth groups converted from an intraoral scan into a point cloud. Figure 6 A schematic diagram showing the point clouds of the anterior teeth and crowns in the same space for registration; Figure 7 A schematic diagram showing the point cloud of the registered tooth and crown in the same space; Figure 8 This is a schematic diagram of the overall system structure of the present invention. Detailed Implementation

[0020] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to specific embodiments.

[0021] It should be noted that, unless otherwise defined, the technical or scientific terms used in this invention should have the ordinary meaning understood by one of ordinary skill in the art to which this invention pertains. The terms "first," "second," and similar terms used in this invention do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Terms such as "comprising" or "including" mean that the element or object preceding the word encompasses the elements or objects listed following the word and their equivalents, without excluding other elements or objects. Terms such as "connected" or "linked" are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. Terms such as "upper," "lower," "left," and "right" are used only to indicate relative positional relationships; when the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0022] Example: Please see Figures 1-7 The present invention provides a technical solution: A method for registering teeth using a two-stage CBCT and oral scan, comprising the following steps: S1, based on a convolutional neural network, constructs and trains a first segmentation model that takes an oral cavity slice image as input and outputs a three-dimensional tooth image, and the three-dimensional tooth image is marked with tooth segmentation contours and their voxel information.

[0023] Several oral CBCT images were acquired, segmented, and several oral slice images were obtained. The tooth segmentation contours were marked in the oral slice images by manual labeling, and the tooth segmentation voxel information was recorded. A first segmentation model was established. The oral slice images were used as input, and the oral slice images with corresponding tooth segmentation contours and voxel information were used as labels to train the first segmentation model.

[0024] The oral cavity slice images are acquired using CBCT technology and are used to reflect tooth information inside the oral cavity. The first segmentation model is built based on a convolutional neural network, which consists of an input layer, a convolutional layer, a pooling layer, a fully connected layer, and an output layer. The activation function in the convolutional layer is... For the fully connected layer, the function sets the number of neurons in the fully connected layer to 32, the initial neural network learning rate to 0.001, and the number of training rounds to 100. The first segmentation model is based on a convolutional neural network, which consists of an input layer, convolutional layers, pooling layers, fully connected layers, and an output layer. The activation function in the convolutional layers is... function, The specific expression of the function is: in, Indicates the first Each corresponding convolutional layer Then it means the first In the corresponding convolutional layer, the th The first feature map Each feature value.

[0025] S2, based on the ToothNet network, constructs and trains a second segmentation model that takes a 3D oral cavity image as input and outputs a 3D tooth crown image, with the tooth crown contour and its mesh information marked on the 3D tooth crown image.

[0026] Specifically, based on the three-dimensional oral images obtained from oral scans, the crown contours are marked in the three-dimensional images using manual labeling, and the mesh information of the crown contours is recorded. A second segmentation model is then established, using the three-dimensional oral images as input and the corresponding three-dimensional oral images with marked crown contours and their mesh information as labels to train the second segmentation model.

[0027] The three-dimensional oral cavity images are acquired using oral scanning technology and are used to reflect the crown information inside the oral cavity. The second segmentation model is built based on the ToothNet network structure. The second segmentation model extracts the slice edge map mesh information from each three-dimensional oral cavity image, connects the slice edge map mesh information with the three-dimensional oral cavity image mesh information, and then puts the connected mesh information into the corresponding network to complete the segmentation and recognition of the crown. ToothNet is a commonly used network model, and its specific structure and training process are conventional existing technologies. For details, please refer to publication number CN119444706A, patent titled "A Method for Registration of Three-Dimensional Digital Models of Teeth and Extraction of Cervical Lines Based on Oral Scanning and CBCT," which will not be elaborated here.

[0028] S3, based on the first segmentation model and the second segmentation model, acquires the three-dimensional tooth image and the three-dimensional crown image of the inspector, and extracts its voxel information and mesh information to convert them into point cloud data, thus obtaining the overall point cloud of the tooth and the overall point cloud of the crown.

[0029] The voxel information of the tooth segmentation contour output by the first segmentation model is converted into point cloud data to obtain the overall tooth point cloud. The logic is as follows: the voxel information of the tooth segmentation contour is processed by threshold segmentation and surface extraction algorithms to generate three-dimensional point cloud data containing the tooth shell structure. Specifically, the voxel information labeled in the three-dimensional tooth image is read, its three-dimensional voxel array is extracted, and the spatial resolution corresponding to each voxel is recorded; a conversion threshold is set, and the voxel information labeled in the three-dimensional tooth image is binarized. The specific conversion rules are as follows: In the formula, The coordinates in the 3D tooth image are: The voxel values ​​after voxel binarization. The coordinates in the 3D tooth image are: voxel values, The conversion threshold can be set based on expert experience. Based on the binarized voxel values, the isosurface structure is extracted, generating a set of triangular facets and a set of vertices. The triangular meshes within the set of triangular facets are then subjected to multiple Laplacian smoothing processes with a set of iterations. The coordinates of all vertices in the smoothed triangular meshes are extracted as point cloud data to form the overall point cloud set of the tooth.

[0030] Based on the binarized voxel values, isosurface structures are extracted, generating a set of triangular facets and vertices. The triangular meshes within the facet set are then subjected to multiple Laplacian smoothing processes, with a set of iterations. The Marching Cubes algorithm or other isosurface extraction algorithms are used to extract isosurfaces from the binarized voxel data. This algorithm generates the corresponding set of triangular facets (mesh) and records the corresponding vertex set. Smoothing parameters are set, including the number of iterations (e.g., 10, 20, etc.). A Laplacian operator is constructed based on the topological structure of the triangular mesh, specifically determined by the connectivity between vertices and their normal vectors. For each vertex, the average position of all vertices in its neighborhood is calculated, and the vertex position is updated to a certain proportion of the average neighborhood position, specifically by weighting with a smoothing coefficient. These steps are repeated until the set number of iterations is reached.

[0031] The specific expression for the overall point cloud set of teeth is: In the formula, A collection of point clouds for the entire tooth structure. Let j be the coordinates of the j-th point in the overall point cloud set of the teeth. represents the total number of points in the CBCT voxel information point cloud set, and j is the index of a point in the overall tooth point cloud set; S32, the method for converting the crown contour mesh information output by the second segmentation model into point cloud data is as follows: extracting the vertices in the crown contour mesh structure as point cloud data. Specifically, the extraction method is as follows: First, obtain and open the ISO 3D crown file stored in PLY format. PLY files are a commonly used 3D model storage format, containing the vertices, faces, and other relevant information of the mesh. Load the file using appropriate software or libraries, such as 3D modeling software or dedicated point cloud processing tools. After loading the PLY file, focus on its mesh structure. Extract the set of vertices in the mesh; these vertices represent the geometry of the crown contour. Each vertex typically contains 3D coordinate information, which defines the position of the crown contour in 3D space. Ensure that all relevant vertex data is extracted to fully reflect the shape of the crown. Extract the set of vertices from this set as 3D point cloud data to form the overall crown point cloud set. The specific expression for the overall crown point cloud set is: In the formula, This represents the collection of point clouds representing the entire crown. Let i be the coordinates of the i-th point in the overall point cloud set of the tooth crown. This represents the total number of points in the overall point cloud set of the crown, where i is the index of the coordinates of the points in the overall point cloud set of the crown.

[0032] S4. The teeth of the inspector are divided into multiple three-tooth groups according to the method of three adjacent teeth as a group and one tooth between adjacent groups. The voxel information and mesh information of each three-tooth group are extracted and converted into point cloud data to obtain the point cloud of the teeth of the three-tooth group and the point cloud of the crown of the three-tooth group.

[0033] Based on anatomical regularities, the teeth are grouped into sets of three, skipping one tooth between groups. The specific grouping rules are as follows: Select the first tooth: Start with the first tooth in the tooth sequence; Select the next two teeth: After selecting the first tooth, select the next two teeth in sequence; Skip a tooth: After selecting three teeth, skip the next tooth; Repeat the above steps: Continue starting from the next tooth, repeating the process of selecting three teeth and skipping one tooth, until all teeth have been grouped. Using these grouping rules, multiple sets of three teeth are gradually constructed. Each set of three teeth should contain three consecutive teeth, and groups are separated by skipping one tooth.

[0034] S5 inputs the overall point cloud of the tooth and the overall point cloud of the crown into the pre-trained registration network model to perform preliminary registration of the overall point cloud, so that the three-dimensional crown image is initially aligned in the space of the three-dimensional tooth image.

[0035] The pre-trained registration network model is trained using the 3Dmatch public dataset as training input. The 3Dmatch public dataset is a widely used 3D shape matching and registration dataset, primarily used to evaluate and compare the matching capabilities of various algorithms between 3D models. It provides rich 3D data, including various types of models and corresponding registration information.

[0036] Global feature extraction and registration were performed on the overall point cloud datasets of the crown and the teeth. The registration network model was selected using a point cloud feature extraction network, a superpoint matching network, a local point matching network, and an optimal registration transformation matrix. The model was then trained via backpropagation using local matching loss and global transformation loss. The specific training process included: S51, the point cloud feature extraction network is used to extract features from the point cloud data in the overall point cloud set of the crown and the overall point cloud set of the tooth, respectively. KPConv-FPN convolution is used to perform multi-resolution downsampling of the input point cloud, obtaining sparse and dense point sets for the overall point cloud set of the crown and the overall point cloud set of the tooth, respectively. Specific steps include: a first-layer downsampling, performing FPS downsampling on the transformed point cloud data, retaining 1 / 4 of the original number of points as the convolution center point, and outputting 64-dimensional features; a second-layer downsampling, retaining 1 / 16 of the original number of points, and outputting 128-dimensional features; a third-layer downsampling, retaining 1 / 64 of the original number of points, and outputting 256-dimensional features; using an upsampling module to perform upsampling twice on the third-layer image, restoring the spatial position through interpolation, and performing skip connections with the features of the corresponding downsampling layer, merging them to output 64-dimensional dense features; finally, the sparse and dense point sets and their corresponding features are output, where points within the sparse point set are designated as superpoints. S52 uses a superpoint matching network, combining the corresponding features of the obtained sparse point set, Euclidean distance, and geometric embedding features obtained from the angle between the three points, and performs feature matching on the two sets of feature descriptors of the final downsampled crown point cloud and tooth point cloud through a self-attention network and a cross-attention network; the superpoint pairs that meet the confidence requirements are obtained using a Gaussian kernel function, and the top G superpoint pairs are selected. S53, based on the three-dimensional coordinates of the selected top G superpoints, using the local geometric region point set in their neighborhood as the unit, at the resolution of the dense point set, extract the local point cloud subsets of the crown and the tooth as a whole, use the local point matching network to perform local point pair matching, and use the weighted singular value decomposition method to calculate the rigid transformation matrix between each pair of local regions, generating G rigid transformation matrices. S54. Select the transformation with the best matching consistency index from the generated G rigid transformation matrices as the global initial transformation matrix, and calculate the best registration transformation matrix between the crown point cloud and the tooth point cloud through interior point screening and iterative optimization. S55, S55, apply the optimal rigid transformation matrix to the overall point cloud of the 3D crown image to initially align the 3D crown image with the 3D tooth image in space.

[0037] In step S51, the steps of multi-resolution downsampling and feature extraction specifically include: randomly selecting an initial point in the overall point cloud of the crown and the overall point cloud of the tooth, and using this point as the starting point, using the far point sampling algorithm to construct a sparse and spatially uniform point set as the center point set of the first convolution layer; Based on the center point set of convolution, a KPConv-FPN convolutional structure is constructed. In the neighborhood of each center point, feature vectors are extracted by kernel point weighted feature aggregation. The KPConv-FPN convolutional structure is based on U-Net. The center point of each convolutional layer is the point obtained by downsampling of the previous layer. Based on this, a new local neighborhood is constructed for convolution operation, and multi-scale point cloud features are extracted through multiple iterations. The upsampling module upsamples the output of the third downsampling layer twice and restores the spatial position through interpolation.

[0038] Since extracting feature vectors through kernel-weighted feature aggregation and restoring spatial location through interpolation are conventional techniques, the specific processing steps will not be elaborated here.

[0039] In step S52, the geometric embedding features include Euclidean distance and the spatial angle formed by the three points. The geometric embedding features and the corresponding features of the sparse point set together constitute a feature descriptor for matching.

[0040] Euclidean distance is a standard technique and will not be elaborated upon here. The specific formula used to calculate the spatial angle formed by the three points is as follows: In the formula, This represents the three-dimensional coordinates of the t-th superpoint in the sparse point set of the overall tooth point cloud. This represents the three-dimensional coordinates of the e-th superpoint in the sparse point set of the overall point cloud of the tooth crown. express The overall point cloud of the tooth is sparse with concentrated points, and Recent overshoot.

[0041] Finally, feature matching is performed on the two sets of feature descriptors obtained from the downsampled crown point cloud and the tooth point cloud. The feature matching steps include: inputting the feature descriptors into an attention-based matching network, which includes a self-attention algorithm and a cross-attention algorithm, to achieve feature alignment and fusion of the two sets of feature descriptors; the expressions for the self-attention algorithm and the cross-attention algorithm are as follows: In the formula, Here is the expression for the self-attention algorithm. This is the expression for the cross-attention algorithm. These represent the query, key, and value matrices of the self-attention algorithm, respectively. These represent the query, key, and value matrix of the cross-attention algorithm, respectively. For the first feature descriptor in the input feature descriptor Feature dimensions of each feature vector This indicates the number of feature vectors in the input feature descriptor. is the index of the feature vector in the feature descriptor; where, in the cross-attention algorithm, Derived from the source mode, , It originates from the target mode.

[0042] The Gaussian kernel function is used to find the superpoint pairs that meet the confidence requirements. The logic for calculating the confidence using the Gaussian kernel function is as follows: calculate the Gaussian correlation score between the superpoints of each pair of crowns and teeth, and perform Softmax normalization on all scores to form a probability distribution, thus obtaining the matching confidence. The specific formula used to calculate the matching confidence is as follows: In the formula, This represents the sparse point cloud of the entire tooth structure, with the 3D coordinates of the t-th superpoint. and Represent the sparse point concentration of the overall point cloud of the tooth crown, and the three-dimensional coordinates of the e-th and u-th superpoints, respectively. It is the scaling parameter of the Gaussian kernel function. This represents the matching confidence between the sparse points in the overall point cloud of the tooth and the e-th point in the overall point cloud of the t-th superpoint crown. This indicates that the total number of superpoints corresponding to index u is accumulated.

[0043] Where t is the index of the superpoint of the sparse point concentration in the overall point cloud of the tooth. `e` is the index of the sparse point concentration superpoint of the overall point cloud of the crown; For any superpoint in the sparse point set of the overall tooth point cloud, sort the matching confidence scores of the superpoint with all superpoints in the overall crown point cloud in descending order. If the maximum matching confidence score is greater than the correlation threshold, then the two superpoints corresponding to the maximum matching confidence score are listed as superpoint pairs. If there are multiple identical maximum matching confidence scores, then randomly select the two superpoints corresponding to the maximum matching confidence score as superpoint pairs. Sort all superpoint pairs in descending order according to the matching confidence score to form an optimal set. Select the top G superpoint pairs in the optimal set.

[0044] In step S53: Based on the three-dimensional coordinates corresponding to the selected first G superpoints, the local geometric region point set in their neighborhood is used as the unit, where the specific formula for representing the local geometric region is: In the formula, Represents the sparse point set of the overall point cloud of teeth. A local geometric region is constructed centered on a superpoint. To select a superpoint, the corresponding superpoint should be a point with a dense concentration of points in its neighborhood. To select a superpoint, a dense set of points within the corresponding superpoint's neighborhood is required. Let be the s-th superpoint within the sparse point set of the overall tooth point cloud, where s is the index of the superpoint within the sparse point set of the overall tooth point cloud; determine the local geometric regions of the overall tooth point cloud and the overall crown point cloud respectively, and denote the point set formed by the points in the local geometric region as a subset of the local point cloud; where To select the index of the first G superpoints corresponding to a single superpoint within the interval, .

[0045] S532, the specific steps for local point pair matching using a local point matching network include: Point-to-point feature matching was performed on subsets of the point clouds of the entire crown and the entire tooth. A similarity score was calculated, and a matching matrix was constructed. The matrix with the highest similarity was then selected. Each pair of points is used as a candidate matching point pair, and the corresponding matching confidence level is used as a weighting coefficient to calculate the weighted centroid of the two sets of point clouds. The formula used to calculate the weighted centroid of the overall tooth point cloud and the local point cloud of the overall crown point cloud is as follows: In the formula, For the first The feature vector of the corresponding point in the local point cloud subset of the entire tooth, representing a candidate matching point pair. express The matching confidence weight of each candidate matching point pair Indicates the first The feature vector of the corresponding point in the local point cloud subset of the entire crown, representing a candidate matching point pair. This represents the weighted centroid of a subset of the local point cloud representing the overall area of ​​the tooth. The weighted centroid of the local point cloud subset representing the overall crown is given by the following formula: For the index of the candidate matching point pair, ; A weighted covariance matrix is ​​constructed based on the obtained weighted centroids, where the formula for calculating the weighted covariance matrix is ​​as follows: In the formula, Indicates the first The weighted covariance matrix of candidate matching pairs; By analyzing the weighted covariance matrix Singular value decomposition is performed to calculate the rotation matrix and translation vector. The specific method for performing singular value decomposition on the weighted covariance matrix is ​​as follows: [The text abruptly ends here, likely due to an incomplete sentence or missing information.] Singular value decomposition is performed to obtain an orthogonal matrix, and the rotation matrix is ​​calculated using the orthogonal matrix; the translation vector is calculated using the centroid and the rotation matrix.

[0046] Construct a rigid transformation matrix, where the specific expression for the rigid transformation matrix is: In the formula, For the first The rigid transformation matrix of candidate matching point pairs For the first Rotation matrix of candidate matching point pairs, For the first Translation vectors of candidate matching point pairs; S533, the obtained local rigid transformation matrices are applied to the corresponding local source point cloud regions, and the error residuals obtained after applying each pair of transformations in the global space are calculated. Using the confidence level of local matching as weights, the optimal registration transformation matrix is ​​constructed. The specific formula for the optimal registration transformation matrix is ​​as follows: In the formula, The optimal registration transformation matrix is... Indicates the first The local geometric region of the entire crown corresponding to each superpoint. For the first Each superpoint's matching confidence weight Indicates the application of the transformation matrix. and These are the corresponding points in the partial point cloud sub-sets of the entire tooth and the entire crown, respectively.

[0047] The loss function during the pre-training process of the registration network model is: In the formula, For loss function, For local point pair matching loss function, The global registration transformation loss function is... The hyperparameters are defined as follows: the local point pair matching loss function is calculated using the cross-entropy between the predicted probability of the matched point pair and the true label; the global registration transformation loss function is constructed based on the Euclidean distance between corresponding point pairs in the source and target point clouds, and its specific expression is as follows: In the formula, This represents the Euclidean distance.

[0048] S6. Based on the three-dimensional crown image and the three-dimensional tooth image that are initially aligned in space, the point clouds of the three tooth groups and the point clouds of the crowns of the three tooth groups are locally iteratively registered according to the three tooth groups to obtain the three-dimensional crown image point cloud that is accurately aligned with the three-dimensional tooth image point cloud after transformation.

[0049] Local iterative registration is performed according to the three groups of teeth, so that the point cloud of the three-dimensional crown image is corrected for splicing error by referring to the dental arch morphology of the point cloud of the three-dimensional tooth image, and is accurately aligned in space.

[0050] In step S6, the specific steps for performing local iterative registration according to the three-tooth group point cloud and the three-tooth group crown point cloud include: For each point in the crown point cloud of the three dentition groups, find the nearest point in the point cloud space of the three dentition groups and establish a matching point set; by minimizing the spatial distance error between corresponding points, solve for the optimal rigid transformation so that the point cloud corresponding to the crown point cloud of the three dentition groups is precisely aligned with the point cloud of the three dentition groups in position and direction; in each iteration, update the position of the crown point cloud of the three dentition groups, and repeat this process until the set maximum number of iterations is reached. The formula used for local iterative registration is as follows: In the formula, In the point cloud of the third set of teeth, the first The three-dimensional coordinate vector of a point In the dot cloud of the crowns of the third group of teeth The three-dimensional coordinate vector of a point Indicates the first iteration The maximum number of iterations is set. The index is the number of iterations. Finally, a precisely aligned 3D tooth image point cloud and a precisely aligned 3D crown image point cloud are obtained.

[0051] Please see Figure 8 The present invention also provides a two-stage CBCT and oral scan tooth registration system, which is used to perform the above-described two-stage CBCT and oral scan tooth registration method, including: The first modality segmentation module is used to construct and train a first segmentation model based on a convolutional neural network, which takes an oral slice image as input and outputs a three-dimensional tooth image. The three-dimensional tooth image is marked with tooth segmentation contours and their voxel information. The second modality extraction module is used to construct and train a second segmentation model based on the ToothNet network, which takes a three-dimensional oral cavity image as input and outputs a three-dimensional tooth crown image. The three-dimensional tooth crown image is marked with the tooth crown contour and its mesh information. The point cloud information conversion module is used to acquire three-dimensional tooth images and three-dimensional crown images of the inspector based on the first segmentation model and the second segmentation model, and extract their voxel information and mesh information to convert them into point cloud data, so as to obtain the overall point cloud of the tooth and the overall point cloud of the crown. The grouping optimization module is used to divide the teeth of the inspector into multiple three-tooth groups by grouping three adjacent teeth together and separating adjacent groups by one tooth. The voxel information and mesh information of each three-tooth group are extracted and converted into point cloud data to obtain the point cloud of the teeth and the point cloud of the crown of the three-tooth group. The overall preliminary alignment module is used to input the overall point cloud of the tooth and the overall point cloud of the crown into the pre-trained registration network model, perform preliminary registration of the overall point cloud, and make the three-dimensional crown image initially aligned in the space of the three-dimensional tooth image. The two-stage group alignment module is used to perform local iterative registration of the point clouds of the teeth and the point clouds of the crowns of the three teeth according to the three-tooth groups based on the three-tooth groups that are initially aligned in space, so as to obtain the three-dimensional crown image point cloud that is accurately aligned with the point cloud of the three-dimensional tooth image after transformation.

[0052] The above formulas are all dimensionless calculations. The formulas are derived from software simulations based on a large amount of collected data to obtain the most recent real-world results. The preset parameters in the formulas are set by those skilled in the art according to the actual situation.

[0053] The above embodiments can be implemented, in whole or in part, by software, hardware, firmware, or any other combination thereof. When implemented in software, the above embodiments can be implemented, in whole or in part, as a computer program product. Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented by electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution.

[0054] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment, depending on actual needs.

[0055] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application.

Claims

1. A method for registering teeth using a two-stage CBCT and an oral scan, characterized in that, The specific steps include: S1, based on a convolutional neural network, constructs and trains a first segmentation model with an input of oral slice images and an output of a three-dimensional tooth image, and the three-dimensional tooth image is marked with tooth segmentation contours and their voxel information; S2, based on the ToothNet network, constructs and trains a second segmentation model that takes a 3D oral cavity image as input and outputs a 3D tooth crown image, and the 3D tooth crown image is marked with the tooth crown outline and its mesh information; S3, based on the first segmentation model and the second segmentation model, acquire the three-dimensional tooth image and the three-dimensional crown image of the inspector, and extract their voxel information and mesh information to convert them into point cloud data, thus obtaining the overall point cloud of the tooth and the overall point cloud of the crown. S4. The teeth of the inspector are divided into multiple three-tooth groups according to the method of three adjacent teeth as a group and one tooth between adjacent groups. The voxel information and mesh information of each three-tooth group are extracted and converted into point cloud data to obtain the point cloud of the teeth of the three-tooth group and the point cloud of the crown of the three-tooth group. S5. Input the overall point cloud of the tooth and the overall point cloud of the crown into the pre-trained registration network model to perform preliminary registration of the overall point cloud, so that the three-dimensional crown image is initially aligned in the space of the three-dimensional tooth image. S6. Based on the three-dimensional crown image and the three-dimensional tooth image that are initially aligned in space, the point clouds of the three tooth groups and the point clouds of the crowns of the three tooth groups are locally iteratively registered according to the three tooth groups to obtain the three-dimensional crown image point cloud that is accurately aligned with the three-dimensional tooth image point cloud after transformation.

2. The method for registering teeth using a dual-stage CBCT and oral scan according to claim 1, characterized in that: The oral cavity slice images are acquired using CBCT technology and are used to reflect tooth information inside the oral cavity. The first segmentation model is built based on a convolutional neural network, which consists of an input layer, a convolutional layer, a pooling layer, a fully connected layer, and an output layer. The activation function in the convolutional layer is... For the fully connected layer, the function sets the number of neurons in the fully connected layer to 32, the initial neural network learning rate to 0.001, and the number of training rounds to 100. The three-dimensional oral cavity image is acquired based on oral cavity scanning technology and is used to reflect the crown information inside the oral cavity. The second segmentation model is established based on the ToothNet network structure. The second segmentation model extracts the slice edge map mesh information of each three-dimensional oral cavity image, connects the slice edge map mesh information with the three-dimensional oral cavity image mesh information, and then puts the connected mesh information into the corresponding network to complete the segmentation and recognition of the crown.

3. The method for registering teeth using a dual-stage CBCT and oral scan according to claim 2, characterized in that: Step S3 also includes the following steps: S31, the tooth segmentation contour voxel information output by the first segmentation model is converted into point cloud data to obtain the overall tooth point cloud. The logic is as follows: the voxel information of the tooth segmentation contour is processed by threshold segmentation and surface extraction algorithms to generate three-dimensional point cloud data containing the tooth shell structure. The specific method is as follows: the voxel information marked in the three-dimensional tooth image is read, its three-dimensional voxel array is extracted, and the spatial resolution corresponding to each voxel is recorded; a conversion threshold is set, and the voxel information marked in the three-dimensional tooth image is binarized. The specific conversion rules are as follows: In the formula, The coordinates in the 3D tooth image are: The voxel values ​​after voxel binarization. The coordinates in the 3D tooth image are: voxel values, The conversion threshold; Based on the binarized voxel values, isosurface structures are extracted, generating a set of triangular facets and vertices. The triangular meshes within the facet set are then subjected to multiple Laplacian smoothing processes, with a set number of iterations. The coordinates of all vertices in the smoothed triangular meshes are extracted as point cloud data, forming a complete point cloud set for the tooth. The specific expression for the complete point cloud set for the tooth is as follows: In the formula, A collection of point clouds for the entire tooth structure. Let j be the coordinates of the j-th point in the overall point cloud set of the teeth. This represents the total number of points in the overall tooth point cloud set, where j is the index of a point in the overall tooth point cloud set. S32, the method for converting the crown contour mesh information output by the second segmentation model into point cloud data is as follows: extracting the crown structure vertices from the crown contour mesh structure as point cloud data. Specifically, the extraction method is as follows: based on the output 3D crown image, extract the set of crown structure vertices as 3D point cloud data to form the overall crown point cloud set. The specific expression for the overall crown point cloud set is: In the formula, This represents the collection of point clouds representing the entire crown. Let i be the coordinates of the i-th point in the overall point cloud set of the tooth crown. This represents the total number of points in the overall point cloud set of the crown, where i is the index of the coordinates of the points in the overall point cloud set of the crown.

4. The method for registering teeth using a dual-stage CBCT and oral scan according to claim 3, characterized in that: The specific steps for pre-training the registration network model are as follows: using the 3Dmatch public dataset as training input, perform overall feature extraction and registration on the overall point cloud set of the crown and the overall point cloud set of the tooth; filter the optimal registration transformation matrix through the point cloud feature extraction network, superpoint matching network, local point matching network, and the best registration transformation matrix in the registration network model; and train the registration network model through backpropagation using local matching loss and global transformation loss. The specific training process includes: S51, the point cloud feature extraction network is used to extract features from the point cloud data in the overall point cloud set of the crown and the overall point cloud set of the tooth, respectively. KPConv-FPN convolution is used to perform multi-resolution downsampling of the input point cloud, obtaining sparse and dense point sets for the overall point cloud set of the crown and the overall point cloud set of the tooth, respectively. Specific steps include: a first-layer downsampling, performing FPS downsampling on the transformed point cloud data, retaining 1 / 4 of the original number of points as the convolution center point, and outputting 64-dimensional features; a second-layer downsampling, retaining 1 / 16 of the original number of points, and outputting 128-dimensional features; a third-layer downsampling, retaining 1 / 64 of the original number of points, and outputting 256-dimensional features; using an upsampling module to perform two upsampling operations on the output of the third-layer downsampling, restoring the spatial position through interpolation, and performing skip connections with the features of the corresponding downsampling layer, merging them to output 64-dimensional dense features; finally, the sparse and dense point sets and their corresponding features are output, where points within the sparse point set are designated as superpoints. S52 utilizes a superpoint matching network, combining the corresponding features of the obtained sparse point set, Euclidean distance, and geometric embedding features obtained from the angle between the three points. Through a self-attention network and a cross-attention network, feature matching is performed on the two sets of feature descriptors of the final downsampled crown point cloud and tooth point cloud. The Gaussian kernel function is used to obtain superpoint pairs that meet the confidence requirements, and the top G superpoint pairs are selected. S53, based on the three-dimensional coordinates of the selected top G superpoints, using the local geometric region point set in their neighborhood as the unit, at the resolution of the dense point set, extract the local point cloud subsets of the crown and the tooth as a whole, use the local point matching network to perform local point pair matching, and use the weighted singular value decomposition method to calculate the rigid transformation matrix between each pair of local regions, generating G rigid transformation matrices. S54. Select the transformation with the best matching consistency index from the generated G rigid transformation matrices as the global initial transformation matrix, and calculate the best registration transformation matrix between the crown point cloud and the tooth point cloud through interior point screening and iterative optimization. S55, apply the optimal rigid transformation matrix to the overall point cloud of the three-dimensional crown image to initially align the three-dimensional crown image with the three-dimensional tooth image in space.

5. The method for registering teeth using a dual-stage CBCT and oral scan according to claim 4, characterized in that: In step S51, the steps of multi-resolution downsampling and feature extraction specifically include: randomly selecting an initial point in both the crown point cloud and tooth point cloud data, and using this point as the starting point, employing a far-point sampling algorithm to construct a sparse and spatially uniform point set as the center point set of the first convolution layer; Based on the center point set of convolution, a KPConv-FPN convolutional structure is constructed. In the neighborhood of each center point, feature vectors are extracted by kernel point weighted feature aggregation. The KPConv-FPN convolutional structure is based on U-Net, and the center point of each convolutional layer is the point obtained by downsampling of the previous layer. Based on this, a new local neighborhood is constructed for convolution operation, and multi-scale point cloud features are extracted through multiple iterations. The upsampling module upsamples the output of the third downsampling layer twice and restores the spatial position through interpolation.

6. The method for registering teeth using a dual-stage CBCT and oral scan according to claim 4, characterized in that: In step S52, the geometric embedding feature includes Euclidean distance and the spatial angle formed by the three points. The geometric embedding feature and the corresponding feature of the sparse point set together constitute a feature descriptor for matching. The two sets of feature descriptors, the final downsampled crown point cloud and the tooth point cloud, are used for feature matching. The feature matching steps include: inputting the feature descriptors into an attention-based matching network, which includes a self-attention algorithm and a cross-attention algorithm, to achieve feature alignment and fusion of the two sets of feature descriptors; the expressions for the self-attention algorithm and the cross-attention algorithm are as follows: In the formula, Here is the expression for the self-attention algorithm. This is the expression for the cross-attention algorithm. These represent the query, key, and value matrices of the self-attention algorithm, respectively. These represent the query, key, and value matrix of the cross-attention algorithm, respectively. For the first feature descriptor in the input feature descriptor Feature dimensions of each feature vector This indicates the number of feature vectors in the input feature descriptor. This is the index of the feature vector in the feature descriptor; The Gaussian kernel function is used to find the superpoint pairs that meet the confidence requirements. The logic for calculating the confidence using the Gaussian kernel function is as follows: calculate the Gaussian correlation score between the superpoints of each pair of crowns and teeth, and perform Softmax normalization on all scores to form a probability distribution, thus obtaining the matching confidence. The specific formula used to calculate the matching confidence is as follows: In the formula, This represents the sparse point cloud of the entire tooth structure, with the 3D coordinates of the t-th superpoint. and Represent the sparse point concentration of the overall point cloud of the tooth crown, and the three-dimensional coordinates of the e-th and u-th superpoints, respectively. It is the scaling parameter of the Gaussian kernel function. This represents the matching confidence between the t-th superpoint in the sparse point set of the overall tooth point cloud and the e-th point in the overall crown point cloud; where t is the index of the superpoint in the sparse point set of the overall tooth point cloud. `e` is the index of the sparse point concentration superpoint of the overall point cloud of the crown; For any superpoint in the sparse point set of the overall tooth point cloud, sort the matching confidence scores of the superpoint with all superpoints in the overall crown point cloud in descending order. If the maximum matching confidence score is greater than the correlation threshold, then the two superpoints corresponding to the maximum matching confidence score are listed as superpoint pairs. If there are multiple identical maximum matching confidence scores, then randomly select the two superpoints corresponding to the maximum matching confidence score as superpoint pairs. Sort all superpoint pairs in descending order according to the matching confidence score to form an optimal set. Select the top G superpoint pairs in the optimal set.

7. The method for registering teeth using a dual-stage CBCT and oral scan according to claim 6, characterized in that: In step S53: Based on the three-dimensional coordinates corresponding to the selected first G superpoints, the local geometric region point set in their neighborhood is used as the unit, where the specific formula for representing the local geometric region is: In the formula, This indicates the concentration of sparse points in the overall point cloud of teeth. A local geometric region is constructed centered on a superpoint. To select a superpoint, the corresponding superpoint should be a point with a dense concentration of points in its neighborhood. To select a superpoint, a dense set of points within the corresponding superpoint's neighborhood is required. Let be the s-th superpoint within the sparse point set of the overall tooth point cloud, where s is the index of the superpoint within the sparse point set of the overall tooth point cloud; determine the local geometric regions of the overall tooth point cloud and the overall crown point cloud respectively, and denote the point set formed by the points in the local geometric region as the local point cloud subset; The specific steps for local point pair matching using a local point matching network include: Point-to-point feature matching was performed on subsets of the point clouds of the entire crown and the entire tooth. A similarity score was calculated, and a matching matrix was constructed. The matrix with the highest similarity was then selected. Each pair of points is used as a candidate matching point pair, and the corresponding matching confidence level is used as a weighting coefficient to calculate the weighted centroid of the two sets of point clouds. The formula used to calculate the weighted centroid of the overall tooth point cloud and the local point cloud of the overall crown point cloud is as follows: In the formula, For the first The feature vector of the corresponding point in the local point cloud subset of the entire tooth, representing a candidate matching point pair. express The matching confidence weight of each candidate matching point pair Indicates the first The feature vector of the corresponding point in the local point cloud subset of the entire crown, representing a candidate matching point pair. This represents the weighted centroid of a subset of the local point cloud representing the overall area of ​​the tooth. The weighted centroid of the local point cloud subset representing the overall crown is given by the following formula: For the index of the candidate matching point pair, ; A weighted covariance matrix is ​​constructed based on the obtained weighted centroids, where the formula for calculating the weighted covariance matrix is ​​as follows: In the formula, Indicates the first The weighted covariance matrix of candidate matching pairs; By analyzing the weighted covariance matrix Perform singular value decomposition to calculate the rotation matrix and translation vector, and construct the rigid transformation matrix, where the specific expression of the rigid transformation matrix is: In the formula, For the first The rigid transformation matrix of candidate matching point pairs For the first Rotation matrix of candidate matching point pairs, For the first Translation vectors of candidate matching point pairs; The obtained local rigid transformation matrices are applied to the corresponding local source point cloud regions, and the error residuals obtained after applying each pair of transformations in the global space are calculated. Using the confidence level of local matching as weights, the optimal registration transformation matrix is ​​constructed. The specific formula for the optimal registration transformation matrix is ​​as follows: In the formula, The optimal registration transformation matrix is... Indicates the first The local geometric region of the entire crown corresponding to each superpoint. For the first Each superpoint's matching confidence weight Indicates the application of the transformation matrix. and These are the corresponding points in the partial point cloud sub-sets of the entire tooth and the entire crown, respectively.

8. The method for registering teeth using a dual-stage CBCT and oral scan according to claim 7, characterized in that: The loss function during the pre-training process of the registration network model is: In the formula, For loss function, For local point pair matching loss function, The global registration transformation loss function is... The hyperparameters are defined as follows: the local point pair matching loss function is calculated using the cross-entropy between the predicted probability of the matched point pair and the true label; the global registration transformation loss function is constructed based on the Euclidean distance between corresponding point pairs in the source and target point clouds, and its specific expression is as follows: In the formula, This represents the Euclidean distance.

9. The method for registering teeth using a dual-stage CBCT and oral scan according to claim 6, characterized in that: In step S6, the specific steps for performing local iterative registration according to the three-tooth group point cloud and the three-tooth group crown point cloud include: For each point in the crown point cloud of the three dentition groups, find the nearest point in the point cloud space of the three dentition groups and establish a matching point set; by minimizing the spatial distance error between corresponding points, solve for the optimal rigid transformation so that the point cloud corresponding to the crown point cloud of the three dentition groups is precisely aligned with the point cloud of the three dentition groups in position and direction; in each iteration, update the position of the crown point cloud of the three dentition groups, and repeat this process until the set maximum number of iterations is reached. The formula used for local iterative registration is as follows: In the formula, In the point cloud of the third set of teeth, the first The three-dimensional coordinate vector of a point In the dot cloud of the crowns of the third group of teeth The three-dimensional coordinate vector of a point The maximum number of iterations is set. The index is the number of iterations. Finally, a precisely aligned 3D tooth image point cloud and a precisely aligned 3D crown image point cloud are obtained.

10. A dual-stage CBCT and oral scan tooth registration system, characterized in that: The dual-stage CBCT and oral scan tooth registration system is used to perform the dual-stage CBCT and oral scan tooth registration method according to any one of claims 1-9, including: The first modality segmentation module is used to construct and train a first segmentation model based on a convolutional neural network, which takes an oral slice image as input and outputs a three-dimensional tooth image. The three-dimensional tooth image is marked with tooth segmentation contours and their voxel information. The second modality extraction module is used to construct and train a second segmentation model based on the ToothNet network, which takes a three-dimensional oral cavity image as input and outputs a three-dimensional tooth crown image. The three-dimensional tooth crown image is marked with the tooth crown contour and its mesh information. The point cloud information conversion module is used to acquire three-dimensional tooth images and three-dimensional crown images of the inspector based on the first segmentation model and the second segmentation model, and extract their voxel information and mesh information to convert them into point cloud data, so as to obtain the overall point cloud of the tooth and the overall point cloud of the crown. The grouping optimization module is used to divide the teeth of the inspector into multiple three-tooth groups by grouping three adjacent teeth together and separating adjacent groups by one tooth. The voxel information and mesh information of each three-tooth group are extracted and converted into point cloud data to obtain the point cloud of the teeth and the point cloud of the crown of the three-tooth group. The overall preliminary alignment module is used to input the overall point cloud of the tooth and the overall point cloud of the crown into the pre-trained registration network model, perform preliminary registration of the overall point cloud, and make the three-dimensional crown image initially aligned in the space of the three-dimensional tooth image. The two-stage group alignment module is used to perform local iterative registration of the point clouds of the teeth and the point clouds of the crowns of the three teeth according to the three-tooth groups based on the three-tooth groups that are initially aligned in space, so as to obtain the three-dimensional crown image point cloud that is accurately aligned with the point cloud of the three-dimensional tooth image after transformation.

Images