Tooth three-dimensional digital model registration and tooth neck line extraction method based on oral scanning and CBCT

Abstract

The invention discloses a tooth three-dimensional digital model registration and tooth neck line extraction method based on oral scanning and CBCT, and relates to the field of tooth three-dimensional digital model registration. Comprising the following steps: respectively acquiring three-dimensional oral images of oral scanning and CBCT (cone beam computed tomography), and automatically segmenting three-dimensional structures of dental crowns and teeth by utilizing pre-trained V-Net and ToothNet image segmentation models. Alignment and registration of two modal images are realized through a feature extraction network, an information exchange network and a rigid transformation network. The feature extraction network extracts local and global features by using far-point sampling and a dynamic graph convolutional neural network, the self-attention and cross-attention module performs feature enhancement and information exchange between the same mode and different modes, and based on a registration result, a tooth neck line is further extracted and optimized. According to the method, the tooth three-dimensional digital model can be quickly and accurately registered, a high-precision tooth neck line extraction result is provided, the manpower and time cost of a medical institution can be reduced, and the medical efficiency is improved.

Description

Technical Field

[0001] The present invention relates to the field of tooth three-dimensional digital model registration, and in particular to a tooth three-dimensional digital model registration and tooth neck line extraction method based on oral scanning and CBCT. Background Art

[0002] With the continuous advancement of oral medicine and dental technology, intraoral scanning (IOS) and cone-beam computed tomography (CBCT) have become indispensable tools for modern dental diagnosis and treatment. Intraoral scanning technology can quickly and non-invasively obtain a three-dimensional digital model of a patient's teeth, providing high-precision tooth morphology information. However, relying solely on intraoral scanning has certain limitations. For example, when dealing with complex periodontal structures, bone mass assessment, and apical lesions, the images obtained by intraoral scanning may not be sufficient to provide comprehensive anatomical information. At the same time, CBCT technology can provide more detailed three-dimensional images, especially in terms of the positional relationship between bone structure and soft tissue, and can effectively reveal the anatomical characteristics of teeth and their surrounding tissues.

[0003] However, fusing intraoral scans and CBCT images and performing effective registration remains a challenging task. Due to the modality differences between the images acquired by these two technologies, intraoral scans mainly focus on the surface structure of the crown, while CBCT provides more in-depth three-dimensional information. Such modality differences bring difficulties to the registration process, especially in the feature extraction and matching stages. Traditional methods often rely on manual features or simple transformations, resulting in low registration accuracy and susceptibility to noise and artifacts. Therefore, how to efficiently register multimodal images and extract key tooth features (such as the cervical line) has become a hot topic in current research.

[0004] Therefore, in order to solve the above technical problems, it is necessary to provide a new tooth three-dimensional digital model registration method and tooth neck line extraction method.

[0005] In the prior art, the publication number CN114974523A discloses a method for fusing a three-dimensional digital model of a tooth, which specifically includes: obtaining a first three-dimensional digital model representing a crown of a first tooth, a second three-dimensional digital model representing the entire first tooth, and a first segmentation scheme, wherein the first segmentation scheme segments the second three-dimensional digital model into a first part and a second part corresponding to the crown and the root, respectively; performing point cloud registration on the first three-dimensional digital model and the first part of the second three-dimensional digital model; based on the result of the point cloud registration, calculating a set of points on the second three-dimensional digital model that are closest to the points on the edge of the first three-dimensional digital model, and deleting the part of the second three-dimensional digital model corresponding to the first three-dimensional digital model based on the set of the closest points, to obtain the third part of the second three-dimensional digital model; performing surface stitching on the first three-dimensional digital model and the third part of the second three-dimensional digital model to obtain a fused three-dimensional digital model. However, this method has high requirements on the quality and accuracy of the input overall model by fusing multi-dimensional digital models. If any of the models has noise, defects or inaccuracies, it will directly affect the accuracy of the final fused model. At the same time, multiple point cloud registrations and geometric processing are required, which takes a long time to process. Therefore, this solution requires a large amount of computing resources and also reduces the real-time and effectiveness of model fusion.

[0006] The above information disclosed in this Background section is only for enhancement of understanding of the background of the present disclosure and therefore it may contain information that does not constitute the prior art that is already known to one of ordinary skill in the art. Summary of the invention

[0007] The purpose of the present invention is to provide a method for tooth three-dimensional digital model registration and tooth neck line extraction based on oral scanning and CBCT to solve the problems raised in the above background technology.

[0008] To achieve the above object, the present invention provides the following technical solutions:

[0009] A method for tooth three-dimensional digital model registration and tooth neck line extraction based on oral scan and CBCT, the specific steps include:

[0010] S1, collecting a number of three-dimensional oral images and CBCT oral slice images of different tooth morphologies in historical oral scans, marking the crown contour in the three-dimensional oral image of the historical oral scan by manual marking, and recording the grid information of the crown contour, marking the tooth contour in the image CBCT oral slice image by manual marking, and recording the tooth segmentation voxel information, wherein the grid information of the crown contour includes: vertex data, face data, edge data and segmentation label data of the crown in the image; the voxel information of the tooth segmentation includes: coordinate data, category label, gray value and resolution of the voxel in the tooth image;

[0011] S2, establishing a first segmentation model and a second segmentation model, using a three-dimensional oral image with crown contours and grid information marked, and a CBCT oral slice image with tooth segmentation and voxel information marked to train the first segmentation model and the second segmentation model respectively;

[0012] S3, inputting the 3D oral image to be registered in the oral scan and the CBCT oral slice image into the trained first segmentation model and the second segmentation model respectively, outputting the 3D crown image of the first modality and its grid information, the 3D reconstructed tooth image of the second modality and the corresponding voxel information, and converting the obtained grid information and voxel information into point cloud data respectively;

[0013] S4, based on the transformed point cloud data, feature extraction and registration are performed on the two sets of point cloud data, so that the three-dimensional crown image of the first modality is accurately aligned in space with the three-dimensional tooth image of the second modality, and the transformed three-dimensional crown image of the first modality is obtained, and the cervical line is extracted and optimized based on the transformed three-dimensional crown image of the first modality, and finally an accurate cervical line contour is obtained.

[0014] Furthermore, the first segmentation model uses a V-Net structure, and the second segmentation model uses ToothNet to achieve automatic segmentation and recognition of teeth, wherein the second segmentation model extracts feature information of the edge map of each slice, connects the edge map feature information with the original image feature information, and then puts the feature information into the corresponding network to complete the segmentation and recognition of teeth.

[0015] Furthermore, the step S3 further comprises the following steps:

[0016] The logic for converting the obtained mesh information into point cloud data is:

[0017] The transformation of point cloud data is achieved through rotation matrix and translation vector:

[0018] S31, extract the vertex coordinates in the mesh information and generate the initial point cloud data:

[0019] The mesh data contains several mesh faces, and each mesh face is defined by multiple vertices. First, extract all vertices P from the mesh data and represent them as an N×3 matrix:

[0020]

[0021] Where N is the number of vertices, and each row represents the coordinates of a vertex in three-dimensional space (x i ,y i , z i ), where i = 1, 2, ..., N;

[0022] The coordinates of each mesh vertex are transformed by the rotation matrix and translation vector to generate new point cloud data. The specific steps include:

[0023] S32, calculation of the rotation matrix: generate a random number between 0 and 1, and multiply this random number by π / 4 to get the Euler angle θ x ,θ y ,θ z ;

[0024] Use the function euler2mat to convert the Euler angle into a 3×3 rotation matrix R, which is used to describe the rotation of the object around each axis:

[0025] R = euler2mat(θ x ,θ y ,θ z )

[0026] In the formula, θ x ,θ y ,θ z Represents the Euler angle, specifically the rotation angle around the x, y, and z axes, in radians; based on the rotation matrix formula, the rotation matrices around the x, y, and z axes are calculated respectively. Remember r x , R y and R z , then the formula for the final rotation matrix R′ is:

[0027] R′=R x ·R y ·R z

[0028] According to the generated rotation matrix R′, all the extracted vertices P are rotated. The specific formula is:

[0029] P rotated =(R′×P T ) T

[0030] Where P rotated Represents the rotated point cloud, with a shape of N×3, and T represents transpose;

[0031] S33, the step of performing translation transformation on the coordinates of each mesh vertex includes: generating three random numbers between -5 and 5, combining them into a three-dimensional vector G, and then translating the rotated point cloud to move it from the origin to the target position, and the specific formula is:

[0032] P IOS =P rotated +G

[0033] Where PIOS Represents the translated point cloud;

[0034] S34, generating a point cloud data set, for each mesh surface, repeating the above rotation and translation steps to generate a required number of point cloud data, merging all generated points into one point cloud data set, and finally obtaining a complete point cloud set, wherein the point cloud set is a collection of point clouds in all mesh surfaces;

[0035] The logic for converting the obtained voxel information into point cloud data is: extract all voxels with non-zero values, multiply their coordinates by the scaling factor s, convert them into actual physical coordinates, and obtain point cloud data. Then, only the voxels that meet the boundary conditions are retained to obtain the boundary part of interest. The specific expression of the point cloud data set converted from voxel data is:

[0036] P CBCT ={(x i ·s,y i ·s, z i ·s)|(x i ,y i , z i )∈F}

[0037] Where P CBCT represents the point cloud data set converted from CBCT voxel data, and F is the voxel coordinate set that meets the boundary conditions.

[0038] Furthermore, in step S4, the process of registering the two sets of point cloud data includes:

[0039] S41, constructing a feature extraction network, performing feature extraction on the point cloud data of the three-dimensional crown image of the first modality and the three-dimensional tooth image of the second modality, respectively, extracting the surface edge features, textures, geometric structures and global geometric morphological features thereof, and generating corresponding feature descriptors;

[0040] S42, using the information exchange network, respectively performing feature matching on the two groups of feature descriptors, identifying corresponding feature point pairs based on the anatomical features of the crown and the tooth, achieving feature alignment between the two groups of feature descriptors, and outputting the aligned feature descriptors;

[0041] S43, through the rigid transformation network, based on the feature descriptors after feature alignment, calculate the rigid transformation matrix for transforming the three-dimensional crown image of the first modality into the three-dimensional tooth image coordinate system of the second modality, wherein the rigid transformation matrix includes a rotation matrix and a translation vector, so that the three-dimensional crown image of the first modality is accurately aligned with the three-dimensional tooth image of the second modality in space, and obtain the transformed three-dimensional crown image of the first modality.

[0042] Furthermore, the step S41 includes the following steps:

[0043] S411, the step of extracting features from the point cloud data includes: downsampling the point clouds of the three-dimensional crown image of the first modality and the three-dimensional tooth image of the second modality using a far-point sampling algorithm, and extracting features from the downsampled point cloud data of the three-dimensional crown image of the first modality and the three-dimensional tooth image of the second modality using a dynamic graph convolutional neural network with shared weights to obtain a local point cloud feature descriptor;

[0044] The specific logic of the far-point sampling algorithm is: randomly select an initial point from the point cloud data of the three-dimensional crown image of the first modality and the three-dimensional tooth image of the second modality, generate a sampling set, and put the selected initial point into the sampling set, wherein the specific expression of the sampling set is:

[0045] S IOS = {P IOS1}, P IOS1 ∈P IOS

[0046] S CBCT = {P CBCT1}, P CBCT1 ∈P CBCT

[0047] In the formula, S IOS represents the grid point cloud sampling set, S CBCT is the CBCT voxel point cloud sampling set, P IOS1 and P CBCT1 They are the point cloud after grid translation and a point in the point cloud data set converted from the CBCT voxel data;

[0048] Select the point farthest from the selected sampling set from the remaining points and add it to the sampling set, and repeat the above steps until a predetermined number of points are selected.

[0049] Furthermore, the weight-sharing dynamic graph convolutional neural network includes four convolutional layers, a normalization layer, an activation layer, a pooling layer, and one convolutional layer, a normalization layer, a fully connected layer, and an output layer. The convolutional layers are used to gradually extract local and global features of different levels of the three-dimensional crown image of the first modality and the three-dimensional tooth image of the second modality. Each layer of convolution operation starts from local features and gradually extracts higher-dimensional geometric structures, specifically including:

[0050] The first convolutional layer, with 6 input channels and 64 output channels, extracts low-level surface edge features for detecting and aligning the surface edges of the first modality's 3D crown image and the second modality's 3D tooth image before registration:

[0051] X1=ReLU(BatchNorm(W1*X+b1))

[0052] Where X∈R N×3 Represents the original point cloud data, W1 represents the weight matrix of this layer, W1∈R 6×64 , X1 represents the point cloud data after the first convolutional layer, X1∈R N×64 , b1 represents the bias term of this layer, and ReLU represents the activation function;

[0053] The second convolutional layer, with 128 input channels and 64 output channels, further extracts surface texture information, identifies local geometric features and shapes on the first modality 3D crown image and the second modality 3D tooth image, and retains their local features:

[0054] X2=ReLU(BatchNorm(W2*X1+b2))

[0055] Where W2∈R 64×64 , X2∈R N×64 ;

[0056] The third convolutional layer has 128 input channels and 128 output channels, which respectively extracts the deep geometric structure information of the first modality 3D crown image and the second modality 3D tooth image, and recognizes the surface morphology of the crown and the tooth:

[0057] X3=ReLU(BatchNorm(W3*X2+b3))

[0058] Among them, W3∈R 64×128 , X3∈R N×128 ;

[0059] The fourth convolutional layer, with 256 input channels and 256 output channels, captures the global geometric morphology of the entire first modality 3D crown image and the second modality 3D tooth image, respectively, and identifies the overall spatial relationship:

[0060] X4=ReLU(BatchNorm(W4*X3+b4))

[0061] Where W4∈R 128×256 , X4∈R N×256 ;

[0062] Finally, the features at different levels are processed through the fully connected layer to output a feature vector with geometric information of the precise tooth model:

[0063] X final =W fc *X4+b fc

[0064] Where, Xfinal The feature vector W represents the geometric information of the accurate tooth model. fc is the weight matrix of the fully connected layer, b fc is the bias vector of the fully connected layer, W fc ∈R 256×1024 , X final ∈R N×1024 .

[0065] Further, in step S42, the information exchange network includes a self-attention algorithm and a cross-attention algorithm, and the feature alignment and fusion of the two data modalities are completed by the two attention algorithms, specifically including:

[0066] Self-attention mechanism is used to enhance the feature expression within each modality. By calculating the correlation between each point in the point cloud data and other points, it focuses on the most representative features in the tooth image and ensures that the most critical geometric features and boundary information are extracted in each modality.

[0067] The cross-attention mechanism is used to exchange information between different modalities including oral scan images and CBCT images. By calculating the correlation between the features of one modality and another modality, the common features in the two images are captured to enhance the accuracy of fusion.

[0068] Furthermore, in step S43, the logic for obtaining the transformed first modality three-dimensional crown image through the rigid transformation network is:

[0069] Based on the feature-aligned three-dimensional crown image of the first modality and the feature descriptors of the three-dimensional tooth image of the second modality, the feature Euclidean distance between the two modality image point clouds is calculated, and a matching matrix is ​​generated;

[0070] The logic of generating the matching matrix is ​​as follows: calculate the Euclidean distance between the feature points in the source point cloud and the feature points in the target point cloud to obtain the similarity matrix between each feature point, use the softmax function to normalize all distances to obtain the probability distribution, and then obtain the matching matrix M to describe the matching strength between the source point and the target point; through the matching matrix M, select the point pair with the highest similarity between each source point and the target point as the candidate matching point pair, and take the similarity in the matching matrix to calculate the weight of each point pair. The formula for calculating the weight of each point pair is:

[0071] ω k =M[p, q], (p, q)∈TopK(M)

[0072] In the formula, TopK(M) means selecting the best top u matching point pairs according to the similarity matrix M, ω k represents the weight of the kth point pair, where k is the index of the point pair;

[0073] The covariance matrix is ​​calculated using the weights of the matching point pairs: the weighted centroids are calculated for the point cloud data of the first modality three-dimensional crown image and the second modality three-dimensional tooth image respectively; the source point cloud and the reference point cloud are then removed from their respective centroids, and the covariance matrix H is calculated through the decentralized point cloud; the covariance matrix is ​​subjected to singular value decomposition using SVD to calculate the rigid transformation matrix R and the translation matrix t; based on the rigid transformation matrix R and the translation matrix t, the three-dimensional crown image of the first modality is transformed to overlap with the three-dimensional tooth image of the second modality, so as to obtain the transformed three-dimensional crown image of the first modality.

[0074] Furthermore, the method for extracting and optimizing the tooth neck line based on the transformed three-dimensional crown image of the first modality is: extracting the boundary points of the tooth from the transformed three-dimensional crown image of the first modality;

[0075] The nearest points of these boundary points are found in the three-dimensional tooth image of the second modality, and the nearest points are connected to generate a cervical line, the cervical line is processed, outliers in the curve are removed, and the remaining points are used as key points. The shortest path between adjacent key points is determined using a breadth-first search algorithm; the cervical line is interpolated based on the points in the shortest path to obtain an accurate cervical line contour.

[0076] Compared with the prior art, the present invention has the following beneficial effects:

[0077] This method obtains three-dimensional oral images of oral scan and CBCT respectively, and uses the pre-trained V-Net and ToothNet image segmentation models to automatically segment the three-dimensional structure of the crown and tooth. Then, the two modal images are aligned and registered through the feature extraction network, information exchange network and rigid transformation network. The feature extraction network uses far point sampling (FPS) and dynamic graph convolutional neural network (DGCNN) to extract local and global features. By introducing self-attention and cross-attention modules, it can achieve effective information exchange between the same modality and different modalities, further improving the accuracy and stability of registration. Based on this registration result, the cervical line is further extracted and optimized. This method only requires two oral images and only uses one cloud point registration, which saves computing resources. In addition, the feature extraction network combines far point sampling and dynamic graph convolutional neural network, so that the model can better capture local details and global features, so as to more accurately perform feature correspondence and information integration between modalities. It can not only quickly and accurately align the three-dimensional digital model of teeth and provide high-precision cervical line extraction results, but also reduce the manpower and time costs of medical institutions and improve medical efficiency. It is suitable for orthodontic treatment and precise construction of tooth models. BRIEF DESCRIPTION OF THE DRAWINGS

[0078] Figure 1It is a schematic diagram of the overall method flow of the present invention;

[0079] Figure 2 A schematic diagram of the three-dimensional digital model obtained by oral scanning and its crown contour annotation;

[0080] Figure 3 A three-dimensional digital image of a tooth crown in a first mode outputted by a first segmentation model from an intraoral scanned three-dimensional digital image;

[0081] Figure 4 A three-dimensional reconstructed tooth digital image of a second modality outputted by a second segmentation model from a CBCT oral slice image;

[0082] Figure 5 This is a schematic diagram of the tooth neck line extraction effect. DETAILED DESCRIPTION

[0083] In order to make the objectives, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with specific embodiments.

[0084] It should be noted that, unless otherwise defined, the technical terms or scientific terms used in the present invention should be the common meanings understood by people with ordinary skills in the field to which the present invention belongs. The words "first", "second" and similar words used in the present invention do not indicate any order, quantity or importance, but are only used to distinguish different components. "Include" or "comprise" and similar words mean that the elements or objects appearing before the word include the elements or objects listed after the word and their equivalents, without excluding other elements or objects. "Connect" or "connected" and similar words are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect.

[0085] "Up", "down", "left", "right", etc. are only used to indicate relative position relationships. When the absolute position of the object being described changes, the relative position relationship may also change accordingly.

[0086] Example:

[0087] See also Figure 1-Figure 5 , the present invention provides a technical solution:

[0088] A method for tooth three-dimensional digital model registration and tooth neck line extraction based on oral scan and CBCT, the specific steps include:

[0089] S1, collecting a number of three-dimensional oral images and CBCT oral slice images of different tooth morphologies in historical oral scans, marking the crown contour in the three-dimensional oral image of the historical oral scan by manual marking, and recording the grid information of the crown contour, marking the tooth contour in the image CBCT oral slice image by manual marking, and recording the tooth segmentation voxel information, wherein the grid information of the crown contour includes: vertex data, face data, edge data and segmentation label data of the crown in the image; the voxel information of the tooth segmentation includes: coordinate data, category label, gray value and resolution of the voxel in the tooth image;

[0090] The first segmentation model uses the V-Net structure, and the second segmentation model uses ToothNet to achieve automatic segmentation and recognition of teeth. The second segmentation model extracts the feature information of the edge map of each slice, connects the edge map feature information with the original image feature information, and then puts the feature information into the corresponding network to complete the segmentation and recognition of teeth. The V-Net structure and ToothNet are commonly used network models, and the specific structures are not described here.

[0091] S2, establishing a first segmentation model and a second segmentation model, using a three-dimensional oral image with crown contours and grid information marked, and a CBCT oral slice image with tooth segmentation and voxel information marked to train the first segmentation model and the second segmentation model respectively;

[0092] The inputs are three-dimensional oral images from oral scans and CBCT oral slice images. The segmentation model can identify and extract three-dimensional images of crowns and teeth, and obtain image data of crowns and tooth contours. These data are stored in the form of grids to form crown grid information and voxel information for tooth segmentation.

[0093] S3, inputting the 3D oral image to be registered in the oral scan and the CBCT oral slice image into the trained first segmentation model and the second segmentation model respectively, outputting the 3D crown image of the first modality and its grid information, the 3D reconstructed tooth image of the second modality and the corresponding voxel information, and converting the obtained grid information and voxel information into point cloud data respectively;

[0094] The step S3 further comprises the following steps:

[0095] The logic for converting the obtained mesh information into point cloud data is:

[0096] The transformation of point cloud data is achieved through rotation matrix and translation vector:

[0097] S31, extract the vertex coordinates in the mesh information and generate the initial point cloud data:

[0098] The mesh data contains several mesh faces, and each mesh face is defined by multiple vertices. First, extract all vertices P from the mesh data and represent them as an N×3 matrix:

[0099]

[0100] Where N is the number of vertices, and each row represents the coordinates of a vertex in three-dimensional space (x i ,y i ,z i ), where i = 1, 2, ..., N;

[0101] The coordinates of each mesh vertex are transformed by the rotation matrix and translation vector to generate new point cloud data. The specific steps include:

[0102] S32, calculation of the rotation matrix: generate a random number between 0 and 1, and multiply this random number by π / 4 to get the Euler angle θ x ,θ y ,θ z ;

[0103] Use the function euler2mat to convert the Euler angle into a 3×3 rotation matrix R, which is used to describe the rotation of the object around each axis:

[0104] R = euler2mat(θ x ,θ y ,θ z )

[0105] In the formula, θ x ,θ y ,θ z Represents the Euler angle, specifically the rotation angle around the x, y, and z axes, in radians; based on the rotation matrix formula, the rotation matrices around the x, y, and z axes are calculated respectively. Remember that R x , R y and R z , then the formula for the final rotation matrix R′ is:

[0106] R′=R x ·R y ·R z

[0107] Among them, the specific formula of euler2mat function is:

[0108]

[0109] According to the generated rotation matrix R′, all the extracted vertices P are rotated. The specific formula is:

[0110] P rotated=(R′×P T ) T

[0111] Where P rotated Represents the rotated point cloud, with a shape of N×3, and T represents transpose;

[0112] S33, the step of performing translation transformation on the coordinates of each mesh vertex includes: generating three random numbers between -5 and 5, combining them into a three-dimensional vector G, and then translating the rotated point cloud to move it from the origin to the target position, and the specific formula is:

[0113] P IOS =P rotated +G

[0114] Where P IOS Represents the translated point cloud;

[0115] S34, generating a point cloud data set, for each mesh surface, repeating the above rotation and translation steps to generate a required number of point cloud data, merging all generated points into one point cloud data set, and finally obtaining a complete point cloud set, wherein the point cloud set is a collection of point clouds in all mesh surfaces;

[0116] The logic for converting the obtained voxel information into point cloud data is: extract all voxels with non-zero values, multiply their coordinates by the scaling factor s, convert them into actual physical coordinates, and obtain point cloud data. Then, only the voxels that meet the boundary conditions are retained to obtain the boundary part of interest. The specific expression of the point cloud data set converted from voxel data is:

[0117] P CBCT ={(x i ·s,y i ·s, z i ·s)|(x i ,y i , z i )∈F}

[0118] Where P CBCT represents the point cloud data set converted from CBCT voxel data, and F is the voxel coordinate set that meets the boundary conditions.

[0119] S4, based on the transformed point cloud data, feature extraction and registration are performed on the two sets of point cloud data, so that the three-dimensional crown image of the first modality is accurately aligned in space with the three-dimensional tooth image of the second modality, and the transformed three-dimensional crown image of the first modality is obtained, and the cervical line is extracted and optimized based on the transformed three-dimensional crown image of the first modality, and finally an accurate cervical line contour is obtained.

[0120] In step S4, the process of registering two sets of point cloud data includes:

[0121] S41, constructing a feature extraction network, performing feature extraction on the point cloud data of the three-dimensional crown image of the first modality and the three-dimensional tooth image of the second modality, respectively, extracting the surface edge features, textures, geometric structures and global geometric morphological features thereof, and generating corresponding feature descriptors;

[0122] S42, using the information exchange network, respectively performing feature matching on the two groups of feature descriptors, identifying corresponding feature point pairs based on the anatomical features of the crown and the tooth, achieving feature alignment between the two groups of feature descriptors, and outputting the aligned feature descriptors;

[0123] S43, through the rigid transformation network, based on the feature descriptors after feature alignment, calculate the rigid transformation matrix for transforming the three-dimensional crown image of the first modality into the three-dimensional tooth image coordinate system of the second modality, wherein the rigid transformation matrix includes a rotation matrix and a translation vector, so that the three-dimensional crown image of the first modality is accurately aligned with the three-dimensional tooth image of the second modality in space, and obtain the transformed three-dimensional crown image of the first modality.

[0124] The step S41 comprises the following steps:

[0125] S411, the step of extracting features from the point cloud data includes: downsampling the point clouds of the three-dimensional crown image of the first modality and the three-dimensional tooth image of the second modality using a far-point sampling algorithm, and extracting features from the downsampled point cloud data of the three-dimensional crown image of the first modality and the three-dimensional tooth image of the second modality using a dynamic graph convolutional neural network with shared weights to obtain a local point cloud feature descriptor;

[0126] The specific logic of the far-point sampling algorithm is: randomly select an initial point from the point cloud data of the three-dimensional crown image of the first modality and the three-dimensional tooth image of the second modality, generate a sampling set, and put the selected initial point into the sampling set, wherein the specific expression of the sampling set is:

[0127] S IOS = {P IOS1}, P IOS1 ∈P IOS

[0128] S CBCT = {P CBCT1}, P CBCT1 ∈P CBCT

[0129] In the formula, S IOS represents the grid point cloud sampling set, S CBCTis the CBCT voxel point cloud sampling set, P IOS1 and P CBCT1 They are the point cloud after grid translation and a point in the point cloud data set converted from the CBCT voxel data;

[0130] Select the point farthest from the selected sampling set from the remaining points and add it to the sampling set. The formula for calculating the farthest point is:

[0131]

[0132] P iosnew = argmaxd i

[0133] Where, d i P iosi and P iosj The minimum Euclidean distance between iosi -P iosj ||2 means P iosi and P iosj The Euclidean distance between iosi and P iosj is the i-th and j-th point cloud data in the point cloud set after grid translation, T iosnew is the point farthest from the selected sampling set, argmax means taking the independent variable that makes a function reach the maximum value;

[0134] Repeat the above steps until a predetermined number of points are selected. The CBCT voxel point cloud sampling set is established in the same way.

[0135] The shared-weight dynamic graph convolutional neural network includes four convolutional layers, a normalization layer, an activation layer, a pooling layer, a fully connected layer, and an output layer. The convolutional layers are used to gradually extract local and global features of different levels of the three-dimensional crown image of the first modality and the three-dimensional tooth image of the second modality. Each layer of convolution operation starts from local features and gradually extracts higher-dimensional geometric structures, specifically including:

[0136] The first convolutional layer, with 6 input channels and 64 output channels, extracts low-level surface edge features for detecting and aligning the surface edges of the first modality 3D crown image and the second modality 3D tooth image before registration:

[0137] X1=ReLU(BatchNorm(W1*X+b1))

[0138] Where X∈R N×3 Represents the original point cloud data, W1 represents the weight matrix of this layer, W1∈R 6×64 , X1 represents the point cloud data after the first convolutional layer, X1∈R N×64, b1 represents the bias term of this layer, and ReLU represents the activation function;

[0139] The second convolutional layer, with 128 input channels and 64 output channels, further extracts surface texture information, identifies local geometric features and shapes on the first modality 3D crown image and the second modality 3D tooth image, and retains their local features:

[0140] X2=ReLU(BatchNorm(W2*X1+b2))

[0141] Where W2∈R 64×64 , X2∈R N×64 ;

[0142] The third convolutional layer has 128 input channels and 128 output channels, which respectively extracts the deep geometric structure information of the first modality 3D crown image and the second modality 3D tooth image, and recognizes the surface morphology of the crown and the tooth:

[0143] X3=ReLU(BatchNorm(W3*X2+b3))

[0144] Among them, W3∈R 64×128 , X3∈R N×128 ;

[0145] The fourth convolutional layer, with 256 input channels and 256 output channels, captures the global geometric morphology of the entire first modality 3D crown image and the second modality 3D tooth image, respectively, and identifies the overall spatial relationship:

[0146] X4=ReLU(BatchNorm(W4*X3+b4))

[0147] Where W4∈R 128×256 , X4∈R N×256 ;

[0148] Finally, the features at different levels are processed through the fully connected layer to output a feature vector with geometric information of the precise tooth model:

[0149] X final =W fc *X4+b fc

[0150] Where, X final The feature vector W represents the geometric information of the accurate tooth model. fc is the weight matrix of the fully connected layer, b fc is the bias vector of the fully connected layer, W fc ∈R 256×1024 , X final ∈R N×1024 ;

[0151] The point cloud data sets converted from the grid-translated point cloud and the CBCT voxel data are used as inputs of a dynamic graph convolutional neural network with shared weights, and a feature vector with geometric information of the precise tooth model is obtained through the dynamic graph convolutional neural network.

[0152] In step S42, the information exchange network includes a self-attention algorithm and a cross-attention algorithm, and the feature alignment and fusion of the two data modalities are completed by the two attention algorithms, specifically including:

[0153] Self-attention mechanism is used to enhance the feature expression within each modality. By calculating the correlation between each point in the point cloud data and other points, it focuses on the most representative features in the tooth image and ensures that the most critical geometric features and boundary information are extracted in each modality.

[0154] The cross-attention mechanism is used to exchange information between different modalities including oral scan images and CBCT images. By calculating the correlation between the features of one modality and another modality, the common features in the two images are captured to enhance the accuracy of fusion.

[0155] Aggregating global context information of local feature descriptors of the 3D crown image of the first modality and the 3D tooth image of the second modality extracted by a dynamic graph convolutional neural network (DGCNN) through a self-attention module;

[0156] Information is exchanged between the feature descriptors of the three-dimensional crown image of the first modality and the three-dimensional tooth image of the second modality after aggregating global context information through a cross-attention module to enhance the distinguishing ability of the feature descriptors.

[0157] The self-attention and cross-attention modules include:

[0158] The self-attention module calculates the feature vectors in the three-dimensional crown image of the first modality and the three-dimensional tooth image of the second modality respectively, and enhances the feature expression inside each modality image;

[0159] The cross-attention module performs calculations between feature vectors of the three-dimensional crown image of the first modality and the three-dimensional tooth image of the second modality to promote feature alignment and fusion of the two data modalities;

[0160] The attention module specifically includes:

[0161] Perform a linear transformation on each feature vector to generate query, key, and value matrices;

[0162] The attention weight matrix is ​​obtained by calculating the dot product between the query matrix and the key matrix and normalizing it with the softmax function;

[0163] The value matrix is ​​weighted and summed using the attention weight matrix to obtain the enhanced feature representation;

[0164] The results of multiple attention heads are concatenated and transformed through a linear layer to output the final feature descriptor;

[0165] Position encoding and feed-forward neural network are introduced to ensure that the feature vector captures local information while maintaining the integrity of the global context.

[0166] In step S43, the logic for obtaining the transformed first modality three-dimensional crown image through the rigid transformation network is:

[0167] Based on the feature-aligned three-dimensional crown image of the first modality and the feature descriptors of the three-dimensional tooth image of the second modality, the feature Euclidean distance between the two modality image point clouds is calculated, and a matching matrix is ​​generated;

[0168] The logic of generating the matching matrix is ​​as follows: calculate the Euclidean distance between the feature points in the source point cloud and the feature points in the target point cloud to obtain the similarity matrix between each feature point, use the softmax function to normalize all distances to obtain the probability distribution, and then obtain the matching matrix M to describe the matching strength between the source point and the target point; through the matching matrix M, select the point pair with the highest similarity between each source point and the target point as the candidate matching point pair, and take the similarity in the matching matrix to calculate the weight of each point pair. The formula for calculating the weight of each point pair is:

[0169] ω k =M[p,q],(p,q)∈TopK(M)

[0170] In the formula, TopK(M) means selecting the best top u matching point pairs according to the similarity matrix M, ω k represents the weight of the kth point pair, where k is the index of the point pair;

[0171] The covariance matrix is ​​calculated using the weights of the matching point pairs: the weighted centroids are calculated for the point cloud data of the first modality three-dimensional crown image and the second modality three-dimensional tooth image respectively; the source point cloud and the reference point cloud are then removed from their respective centroids, and the covariance matrix H is calculated through the decentralized point cloud; the covariance matrix is ​​subjected to singular value decomposition using the SVD algorithm to calculate the rigid transformation matrix R and the translation matrix t; based on the rigid transformation matrix R and the translation matrix t, the three-dimensional crown image of the first modality is transformed to overlap with the three-dimensional tooth image of the second modality, so as to obtain the transformed three-dimensional crown image of the first modality.

[0172] The method for extracting and optimizing the tooth neck line based on the transformed three-dimensional crown image of the first modality is as follows: extracting the boundary points of the tooth from the transformed three-dimensional crown image of the first modality;

[0173] The nearest points of these boundary points are found in the three-dimensional tooth image of the second modality, and the nearest points are connected to generate a cervical line, the cervical line is processed, outliers in the curve are removed, and the remaining points are used as key points. The shortest path between adjacent key points is determined using a breadth-first search algorithm; the cervical line is interpolated based on the points in the shortest path to obtain an accurate cervical line contour.

[0174] Statistical methods (such as mean and standard deviation) or clustering-based methods (such as DBSCAN) can be used to identify and remove outliers in the curve. Spline interpolation (such as B-spline or Catmull-Rom spline) methods can be used to interpolate key points in the shortest path to obtain a more refined tooth neck line contour. The breadth-first search (BFS) algorithm is a mature existing technology and will not be described in detail here.

[0175] The above formulas are all dimensionless and numerical calculations. The formula is a formula for the most recent real situation obtained by collecting a large amount of data and performing software simulation. The preset parameters in the formula are set by technicians in this field according to actual conditions.

[0176] The above embodiments may be implemented in whole or in part by software, hardware, firmware or any other combination thereof. When implemented by software, the above embodiments may be implemented in whole or in part in the form of a computer program product. Those skilled in the art may appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein may be implemented by electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed by hardware or software methods depends on the specific application and design constraints of the technical solution.

[0177] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, and may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0178] The above description is only a specific implementation manner of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application.

Claims

1. A method for tooth three-dimensional digital model registration and tooth neck line extraction based on oral scan and CBCT, characterized in that: The specific steps include: S1, collecting a number of three-dimensional oral images and CBCT oral slice images of different tooth morphologies in historical oral data, marking the crown contour in the three-dimensional oral image of the historical oral scan by manual marking, and recording the grid information of the crown contour, marking the tooth contour in the image CBCT oral slice image by manual marking, and recording the tooth segmentation voxel information, wherein the grid information of the crown contour includes: the vertex data, face data, edge data and segmentation label data of the crown in the image; the voxel information of the tooth segmentation includes: the coordinate data, category label, gray value and resolution of the voxel in the tooth image; S2, establishing a first segmentation model and a second segmentation model, using a three-dimensional oral image with crown contours and grid information marked, and a CBCT oral slice image with tooth segmentation and voxel information marked to train the first segmentation model and the second segmentation model respectively; S3, inputting the 3D oral image to be registered in the oral scan and the CBCT oral slice image into the trained first segmentation model and the second segmentation model respectively, outputting the 3D crown image of the first modality and its grid information, the 3D reconstructed tooth image of the second modality and the corresponding voxel information, and converting the obtained grid information and voxel information into point cloud data respectively; S4, based on the transformed point cloud data, feature extraction and registration are performed on the two sets of point cloud data, so that the three-dimensional crown image of the first modality is accurately aligned in space with the three-dimensional tooth image of the second modality, and the transformed three-dimensional crown image of the first modality is obtained, and the cervical line is extracted and optimized based on the transformed three-dimensional crown image of the first modality, and finally an accurate cervical line contour is obtained.

2. The method for aligning a three-dimensional digital model of teeth and extracting a tooth neck line based on oral scan and CBCT according to claim 1, characterized in that: The first segmentation model uses a V-Net structure, and the second segmentation model uses ToothNet to achieve automatic segmentation and recognition of teeth. The second segmentation model extracts the feature information of the edge map of each slice, connects the edge map feature information with the original image feature information, and then puts the feature information into the corresponding network to complete the segmentation and recognition of teeth.

3. The method for aligning a three-dimensional digital model of teeth and extracting a tooth neck line based on oral scan and CBCT according to claim 1, characterized in that: The step S3 further comprises the following steps: The logic for converting the obtained mesh information into point cloud data is: The transformation of point cloud data is achieved through rotation matrix and translation vector: S31, extract the vertex coordinates in the mesh information and generate the initial point cloud data: The mesh data contains several mesh faces, and each mesh face is defined by multiple vertices. First, extract all vertices P from the mesh data and represent them as an N×3 matrix: Where N is the number of vertices, and each row represents the coordinates of a vertex in three-dimensional space (x i ,y i , z i ), where i = 1, 2, ..., N; The coordinates of each mesh vertex are transformed by the rotation matrix and translation vector to generate new point cloud data. The specific steps include: S32, calculation of the rotation matrix: generate a random number between 0 and 1, and multiply this random number by π / 4 to get the Euler angle θ x ,θ y ,θ z ; Use the function euler2mat to convert the Euler angle into a 3×3 rotation matrix R, which is used to describe the rotation of the object around each axis: R=euler2mat(θ x ,the y ,the z ) In the formula, θ x ,θ y ,θ z Represents the Euler angle, specifically the rotation angle around the x, y, and z axes, in radians; based on the rotation matrix formula, the rotation matrices around the x, y, and z axes are calculated respectively. Remember that R x , R y and R z , then the formula for the final rotation matrix R′ is: R′=R x ·R y ·R z According to the generated rotation matrix R′, all the extracted vertices P are rotated. The specific formula is: P rotated =(R′×P T ) T Where P rotated Represents the rotated point cloud, with a shape of N×3, and T represents transpose; S33, the step of performing translation transformation on the coordinates of each mesh vertex includes: generating three random numbers between -5 and 5, combining them into a three-dimensional vector G, and then translating the rotated point cloud to move it from the origin to the target position, and the specific formula is: P IOS =P rotated +G Where P IOS Represents the translated point cloud; S34, generating a point cloud data set, for each mesh surface, repeating the above rotation and translation steps to generate a required number of point cloud data, merging all generated points into one point cloud data set, and finally obtaining a complete point cloud set, wherein the point cloud set is a collection of point clouds in all mesh surfaces; The logic for converting the obtained voxel information into point cloud data is: extract all voxels with non-zero values, multiply their coordinates by the scaling factor s, convert them into actual physical coordinates, and obtain point cloud data. Then, only the voxels that meet the boundary conditions are retained to obtain the boundary part of interest. The specific expression of the point cloud data set converted from voxel data is: P CBCT ={(x i ·s,y i ·s,z i ·s)|(x i ,y i ,z i )∈F} Where P CBCT represents the point cloud data set converted from CBCT voxel data, and F is the voxel coordinate set that meets the boundary conditions.

4. The method for aligning a three-dimensional digital model of teeth and extracting a tooth neck line based on oral scan and CBCT according to claim 1, characterized in that: In step S4, the process of registering two sets of point cloud data includes: S41, constructing a feature extraction network, performing feature extraction on the point cloud data of the three-dimensional crown image of the first modality and the three-dimensional tooth image of the second modality, respectively, extracting the surface edge features, textures, geometric structures and global geometric morphological features thereof, and generating corresponding feature descriptors; S42, using the information exchange network, respectively performing feature matching on the two groups of feature descriptors, identifying corresponding feature point pairs based on the anatomical features of the crown and the tooth, achieving feature alignment between the two groups of feature descriptors, and outputting the aligned feature descriptors; S43, through the rigid transformation network, based on the feature descriptors after feature alignment, calculate the rigid transformation matrix for transforming the three-dimensional crown image of the first modality into the three-dimensional tooth image coordinate system of the second modality, wherein the rigid transformation matrix includes a rotation matrix and a translation vector, so that the three-dimensional crown image of the first modality is accurately aligned with the three-dimensional tooth image of the second modality in space, and obtain the transformed three-dimensional crown image of the first modality.

5. The method for aligning a three-dimensional digital model of teeth and extracting a tooth neck line based on oral scan and CBCT according to claim 4, characterized in that: The step S41 comprises the following steps: S411, the step of extracting features from the point cloud data includes: downsampling the point clouds of the three-dimensional crown image of the first modality and the three-dimensional tooth image of the second modality using a far-point sampling algorithm, and extracting features from the downsampled point cloud data of the three-dimensional crown image of the first modality and the three-dimensional tooth image of the second modality using a dynamic graph convolutional neural network with shared weights to obtain a local point cloud feature descriptor; The specific logic of the far-point sampling algorithm is: randomly select an initial point from the point cloud data of the three-dimensional crown image of the first modality and the three-dimensional tooth image of the second modality, generate a sampling set, and put the selected initial point into the sampling set, wherein the specific expression of the sampling set is: S IOS ={P IOS1 },P IOS1 ∈P IOS S CBCT ={P CBCT1 },P CBCT1 ∈P CBCT In the formula, S IOS represents the grid point cloud sampling set, S CBCT is the CBCT voxel point cloud sampling set, P IOS1 and P CBCT1 They are the point cloud after grid translation and a point in the point cloud data set converted from the CBCT voxel data; Select the point farthest from the selected sampling set from the remaining points and add it to the sampling set, and repeat the above steps until a predetermined number of points are selected.

6. The method for aligning a three-dimensional digital model of teeth and extracting a tooth neck line based on oral scan and CBCT according to claim 5, characterized in that: The shared-weight dynamic graph convolutional neural network includes four convolutional layers, a normalization layer, an activation layer, a pooling layer, a fully connected layer, and an output layer. The convolutional layers are used to gradually extract local and global features of different levels of the three-dimensional crown image of the first modality and the three-dimensional tooth image of the second modality. Each layer of convolution operation starts from local features and gradually extracts higher-dimensional geometric structures, specifically including: The first convolutional layer, with 6 input channels and 64 output channels, extracts low-level surface edge features for detecting and aligning the surface edges of the first modality 3D crown image and the second modality 3D tooth image before registration: X1=ReLU(BatchNorm(W1*X+b1)) Where X∈R N×3 Represents the original point cloud data, W1 represents the weight matrix of this layer, W1∈R 6×64 , X1 represents the point cloud data after the first convolutional layer, X1∈R N×64 , b1 represents the bias term of this layer, and ReLU represents the activation function; The second convolutional layer, with 128 input channels and 64 output channels, further extracts surface texture information, identifies local geometric features and shapes on the first modality 3D crown image and the second modality 3D tooth image, and retains their local features: X2=ReLU(BatchNorm(W2*X1+b2)) Where W2∈R 64×64 , X2∈R N×64 ; The third convolutional layer has 128 input channels and 128 output channels, which respectively extracts the deep geometric structure information of the first modality 3D crown image and the second modality 3D tooth image, and recognizes the surface morphology of the crown and the tooth: X3=ReLU(BatchNorm(W3*X2+b3)) Among them, W3∈R 64×128 , X3∈R N×128 ; The fourth convolutional layer, with 256 input channels and 256 output channels, captures the global geometric morphology of the entire first modality 3D crown image and the second modality 3D tooth image, respectively, and identifies the overall spatial relationship: X4=ReLU(BatchNorm(W4*X3+b4)) Where W4∈R 128×156 , X4∈R N×256 ; Finally, the features at different levels are processed through the fully connected layer to output a feature vector with geometric information of the precise tooth model: X final =W fc *X4+b fc Where, X final The feature vector W represents the geometric information of the accurate tooth model. fc is the weight matrix of the fully connected layer, b fc is the bias vector of the fully connected layer, W fc ∈R 256×1024 , X final ∈R N×1024 .

7. The method for aligning a three-dimensional digital model of teeth and extracting a tooth neck line based on oral scan and CBCT according to claim 4, characterized in that: In step S42, the information exchange network includes a self-attention algorithm and a cross-attention algorithm, and the feature alignment and fusion of the two data modalities are completed by the two attention algorithms, specifically including: Self-attention mechanism is used to enhance the feature expression within each modality. By calculating the correlation between each point in the point cloud data and other points, it focuses on the most representative features in the tooth image and ensures that the most critical geometric features and boundary information are extracted in each modality. The cross-attention mechanism is used to exchange information between different modalities including oral scan images and CBCT images. By calculating the correlation between the features of one modality and another modality, the common features in the two images are captured to enhance the accuracy of fusion.

8. The method for aligning a three-dimensional digital model of teeth and extracting a tooth neck line based on oral scan and CBCT according to claim 4, characterized in that: In step S43, the logic for obtaining the transformed first modality three-dimensional crown image through the rigid transformation network is: Based on the feature-aligned three-dimensional crown image of the first modality and the feature descriptors of the three-dimensional tooth image of the second modality, the feature Euclidean distance between the two modality image point clouds is calculated, and a matching matrix is ​​generated; The logic of generating the matching matrix is ​​as follows: calculating the Euclidean distance between the feature points in the source point cloud and the feature points in the target point cloud, obtaining the similarity matrix between each feature point, normalizing all distances using the softmax function, obtaining a probability distribution, and then obtaining a matching matrix M to describe the matching strength between the source point and the target point; Through the matching matrix M, select the point pair with the highest similarity between each source point and target point as the candidate matching point pair, and take the similarity in the matching matrix to calculate the weight of each point pair. The formula for calculating the weight of each point pair is: ω k =M[p,q],(p,q)∈TopK(M) In the formula, TopK(M) means selecting the best top u matching point pairs according to the similarity matrix M, ω k represents the weight of the kth point pair, where k is the index of the point pair; Calculating a covariance matrix using the weights of the matching point pairs: calculating a weighted centroid for the point cloud data of the first modality three-dimensional crown image and the second modality three-dimensional tooth image respectively; Then, the centroids of the source point cloud and the reference point cloud are removed, and the covariance matrix H is calculated through the decentralized point cloud; the covariance matrix is ​​decomposed by the SVD algorithm to calculate the rigid transformation matrix R and the translation matrix t; based on the rigid transformation matrix R and the translation matrix t, the three-dimensional crown image of the first modality is transformed to overlap with the three-dimensional tooth image of the second modality, so as to obtain the transformed three-dimensional crown image of the first modality.

9. The method for tooth three-dimensional digital model registration and tooth neck line extraction based on oral scan and CBCT according to claim 1, characterized in that: The method for extracting and optimizing the tooth neck line based on the transformed three-dimensional crown image of the first modality is as follows: extracting the boundary points of the tooth from the transformed three-dimensional crown image of the first modality; The nearest points of these boundary points are found in the three-dimensional tooth image of the second modality, and the nearest points are connected to generate a cervical line, the cervical line is processed, outliers in the curve are removed, and the remaining points are used as key points. The shortest path between adjacent key points is determined using a breadth-first search algorithm; the cervical line is interpolated based on the points in the shortest path to obtain an accurate cervical line contour.

Images