A method for repairing craniomaxillofacial hard tissue loss based on a universal repair model

Abstract

The present invention provides a method for repairing craniofacial hard tissue loss based on a universal repair model, which relates to the technical field of mandibular defect reconstruction. The method utilizes a plug-and-play conditional attention mechanism and a curvature-aware query generator to construct a universal craniofacial hard tissue repair model. Point cloud data of craniofacial hard tissue is input into the universal craniofacial hard tissue repair model and trained using a weighted loss function to obtain a trained universal craniofacial hard tissue repair model. Denoising is performed on the missing repair point cloud features to obtain denoised missing repair point cloud features. Point cloud meshing is performed using the denoised missing repair point cloud features to obtain a craniofacial hard tissue loss repair result, thereby completing the repair of craniofacial hard tissue loss. The present invention solves the problem of the difficulty in uniformly addressing hard tissue repair and reconstruction in the skull.

Description

Technical Field

[0001] This specification relates to the technical field of jaw defect reconstruction, and in particular to a method for repairing craniomaxillofacial hard tissue loss based on a universal repair model. Background Art

[0002] Currently, AI-based hard tissue repair algorithms are mainly divided into two branches: one is the repair algorithm for missing teeth, and the other is the repair algorithm for missing jaws or skulls. The core difference between these two types of algorithms lies in the data source they rely on. Specifically, tooth repair algorithms are usually based on 3D oral scan data, which is composed of tens of thousands of triangular facets or point clouds, while jaw and skull repair algorithms mostly use three-dimensional CT imaging data. Although tissue repair can be classified as a hard tissue repair problem, there is currently a lack of a universal model that can cover the repair of hard tissues in various parts of the skull. Summary of the Invention

[0003] In view of the above-mentioned deficiencies in the prior art, the present invention provides a method for repairing craniofacial hard tissue loss based on a universal repair model, which solves the problem that hard tissue repair and reconstruction in the skull area is difficult to solve in a unified way.

[0004] In order to achieve the above-mentioned purpose of the invention, the technical solution adopted by the present invention is: a method for repairing craniomaxillofacial hard tissue loss based on a universal repair model, comprising:

[0005] S1: Build a universal craniomaxillofacial hard tissue repair model using a plug-and-play conditional attention mechanism and a curvature-aware query generator;

[0006] S2: Inputting the craniomaxillofacial hard tissue point cloud data into the craniomaxillofacial hard tissue universal repair model, and training using a weighted loss function to obtain a trained craniomaxillofacial hard tissue universal repair model; wherein the trained craniomaxillofacial hard tissue universal repair model is used to process the craniomaxillofacial hard tissue point cloud data to obtain missing repair point cloud features;

[0007] S3: performing denoising processing on the missing repair point cloud features to obtain denoised missing repair point cloud features;

[0008] S4: performing point cloud mesh modeling processing using the denoised missing repair point cloud features to obtain a craniomaxillofacial hard tissue missing repair result, thereby completing the craniomaxillofacial hard tissue missing repair.

[0009] The beneficial effects of this invention include: using a universal restoration model to repair craniofacial hard tissue loss and achieving the desired results. This approach can uniformly address the reconstruction of missing and missing teeth in the field of implant restoration, the reconstruction of jaw defects in the field of craniofacial surgery, and the restoration of skull defects in the field of neurosurgery, solving the entire cranial hard tissue repair and reconstruction challenge with a universal end-to-end deep learning model.

[0010] Furthermore, the universal craniomaxillofacial hard tissue repair model includes:

[0011] A feature extraction module is used to extract features of the dentition to be completed and the dentition of the opposing teeth to be completed, respectively, to obtain corresponding features of the dentition to be completed and the dentition of the opposing teeth;

[0012] The encoder module includes multiple multi-head attention layers and cross-attention layers; wherein the multi-head attention layer is used to encode the features of the dentition to be completed and the features of the opposing teeth to obtain a feature vector of the dentition to be completed and a feature vector of the opposing teeth; the cross-attention layer is used to perform cross-attention calculation on the feature vector of the dentition to be completed and the feature vector of the opposing teeth based on a plug-and-play conditional attention mechanism to obtain a cross-attention result;

[0013] A query generator module, comprising a curvature-aware query generator, configured to perform linear projection on the cross-attention result to obtain a query embedding vector;

[0014] The decoder module is used to calculate the feature vector of the to-be-completed dentition and the query embedding vector to obtain missing prediction data; the multi-scale point cloud generation module performs multi-scale point cloud generation calculation based on the missing prediction data to obtain missing repair point cloud features.

[0015] By introducing the encoder module of the multi-head attention layer, the complex geometric features of the dentition to be completed can be effectively captured, improving the accuracy and efficiency of feature extraction. The linear transformation unit converts the input features into a query matrix, a key matrix, and a value matrix. The attention weight unit further enhances the model's attention to key features by calculating the attention weights. The multi-head attention unit can integrate feature information from different angles and improve the expressiveness of features by splicing multiple attention weight results. The geometric perception unit further enhances the model's perception of geometric features through the maximum pooling operation, making the final generated feature vector of the dentition to be completed more accurate and robust. This enables the model to better capture local and global features when dealing with craniomaxillofacial hard tissue repair tasks, thereby improving the accuracy and naturalness of the repair results.

[0016] Furthermore, the multi-head attention layer in the encoder module includes:

[0017] A linear transformation unit, configured to perform linear transformation on the features of the dentition to be completed and the features of the opposing teeth to obtain a coding query matrix, a coding key matrix, and a coding value matrix;

[0018] an attention weight unit, configured to calculate an attention weight based on the encoding query matrix, the encoding key matrix, and the encoding value matrix to obtain an encoding attention weight result;

[0019] Multi-head attention unit, used to splice multiple encoding attention weight results to obtain the encoding splicing vector;

[0020] The geometric perception unit is used to perform maximum pooling on the features of the dentition to be completed and the features of the opposing teeth, and connect them with the encoded splicing vector to obtain the feature vector of the dentition to be completed and the feature vector of the opposing teeth respectively.

[0021] By introducing the cross-attention layer, the interactive information between the feature vectors of the dentition to be completed and the feature vectors of the opposing teeth can be effectively captured, further improving the accuracy of feature fusion. Specifically, the linear transformation unit converts the feature vectors of the dentition to be completed and the feature vectors of the opposing teeth into a cross-query matrix, a cross-key matrix, and a cross-value matrix, respectively. The attention weight unit can dynamically adjust the correlation strength between features by calculating the cross-attention weights. The cross-attention unit can synthesize feature information from multiple angles and generate more expressive cross-attention results by splicing multiple cross-attention weight results. This enables the model to better understand the geometric and functional relationship between the area to be completed and the opposing area, thereby generating more accurate restoration results.

[0022] Furthermore, the cross attention layer in the encoder module includes:

[0023] A linear transformation unit, configured to perform linear transformation on the feature vector of the dentition to be completed and the feature vector of the opposing dentition to obtain a cross-query matrix, a cross-key matrix, and a cross-value matrix;

[0024] an attention weight unit, configured to calculate an attention weight based on the cross-query matrix, the cross-key matrix, and the cross-value matrix to obtain a cross-attention weight result;

[0025] The cross attention unit is used to concatenate multiple cross attention weight results to obtain the cross attention result:

[0026] ;

[0027] in, represents the cross attention result, represents the splicing calculation, represents the feature vector of the dentition to be completed, represents the eigenvector of the opposing tooth dentition.

[0028] By introducing a curvature-aware query generator, the curvature geometric features of the area to be completed can be effectively captured, generating a more representative query embedding vector. Specifically, local curvature information is incorporated into the generation of the query embedding vector, enabling the model to better understand the geometric form of the area to be completed. The maximum pooling operation further enhances the model's perception of global features, while the coordinate projection operation ensures the consistency of the query embedding vector with the spatial position. This design ensures that the query embedding vector not only contains rich semantic information, but also accurately reflects the geometric characteristics of the area to be completed, thereby providing high-quality input for the subsequent decoder module.

[0029] Furthermore, the expression of the query embedding vector is:

[0030] ;

[0031] ;

[0032] in, represents the query embedding vector, represents a multilayer perceptron, represents sparse point coordinates, express The local curvature of represents the maximum pooling operation, represents a linear layer, represents the result of cross attention, Represents a coordinate projection.

[0033] By introducing a decoder module with a multi-layer multi-head attention mechanism, the missing prediction data can be gradually refined to improve the accuracy and naturalness of the restoration results. Specifically, the linear transformation unit converts the feature vector of the dentition to be completed and the cross-attention results into a decoding value matrix, a decoding key matrix, and a decoding query matrix respectively. The attention weight unit can dynamically adjust the correlation strength between features by calculating the decoding attention weight. The multi-head attention unit can integrate feature information from multiple angles by splicing multiple decoding attention weight results to generate a more expressive decoding splicing vector. The geometric perception unit further enhances the model's perception of geometric features through the maximum pooling operation, and connects the decoding splicing vector with the geometric features to generate high-quality missing prediction data. This enables the model to gradually optimize the restoration results and generate restoration data that is more in line with the anatomical structure.

[0034] Furthermore, the decoder module includes a plurality of multi-head attention layers, wherein the multi-head attention layers in the decoder module include:

[0035] A linear transformation unit is used to perform a linear transformation on the feature vector of the dentition to be completed to obtain a decoding value matrix and a decoding key matrix; and to perform a linear transformation on the cross attention result to obtain a first-layer decoding query matrix;

[0036] an attention weight unit, configured to calculate an attention weight based on the decoding query matrix, the decoding key matrix, and the decoding value matrix to obtain a decoding attention weight result;

[0037] Multi-head attention unit, used to splice multiple decoding attention weight results to obtain the decoding splicing vector;

[0038] The geometric perception unit is used to perform maximum pooling on the feature vector of the to-be-completed dentition and the cross-attention result, and connect them with the decoding splicing vector to obtain a connection result as the decoding query matrix of the next layer; the connection result of the last layer is used as the missing prediction data.

[0039] Furthermore, the expression of the weighted loss function is:

[0040] ;

[0041] ;

[0042] in, Represents the loss function result, represents the sparse point chamfer distance loss function, and Both represent weight factors, represents the dense point chamfer distance loss function, represents the earth moving distance loss function, represents the minimum value of the bijective mapping, represents the ground truth complete point cloud, represents a point in the predicted point cloud, represents a bijective mapping, represents the Euclidean distance; .

[0043] By introducing a weighted loss function, we can effectively balance inpainting errors across different regions and scales, improving model training and inpainting accuracy. Specifically, the weighted loss function adjusts the loss weight based on the geometric complexity and functional importance of the region to be completed, allowing the model to focus more on the inpainting effect in key areas during training. This design not only improves the overall quality of the inpainting results, but also avoids the problem of the model overfitting simple areas or neglecting complex ones.

[0044] By introducing noise convolution distribution modeling and denoising, we can effectively remove noise points from missing repair point cloud features, improving the smoothness and accuracy of the repair results. Specifically, noise convolution distribution modeling captures the noise distribution pattern in the point cloud features, while score calculation quantifies the degree of deviation of noise points. This design enables the model to generate smoother and more accurate repair point clouds, laying a solid foundation for subsequent mesh modeling and reconstruction.

[0045] Furthermore, the S3 includes:

[0046] Performing distribution modeling on the missing repair point cloud features to obtain noise convolution distribution;

[0047] Based on the noise convolution distribution, a score calculation is performed to obtain a noise convolution distribution score;

[0048] Based on the noise convolution distribution score, the missing repair point cloud feature is denoised, and the noise points are moved to a distribution pattern corresponding to the underlying clean surface to obtain a denoised missing repair point cloud feature.

[0049] Furthermore, the S4 includes:

[0050] Using the missing repair point cloud features after denoising, point cloud mesh modeling is performed to obtain a dense point cloud;

[0051] Predicting the dense point cloud to obtain a corresponding unsigned distance field;

[0052] By extracting the triangular mesh of the unsigned distance field and performing mesh reconstruction, the craniomaxillofacial hard tissue loss repair result is obtained, thereby completing the craniomaxillofacial hard tissue loss repair.

[0053] By introducing point cloud meshing modeling and unsigned distance field extraction technology, it is possible to convert missing repair point cloud features into high-quality triangular meshes, generating craniomaxillofacial hard tissue repair results that conform to the anatomical structure. Specifically, point cloud meshing modeling can convert sparse point cloud features into dense point clouds, while unsigned distance field extraction technology can accurately capture the geometric morphology of the point cloud surface. By extracting the triangular mesh of the unsigned distance field, a high-precision mesh model can be generated, thereby achieving accurate repair of craniomaxillofacial hard tissue. This ensures that the repair results not only have a high degree of geometric accuracy, but also meet the strict requirements of clinical applications. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] This specification will be further described in the form of exemplary embodiments, which will be described in detail with reference to the accompanying drawings. These embodiments are not limiting, and in these embodiments, like numbers represent like structures, wherein:

[0055] Figure 1 This is an exemplary flow chart of a method for repairing craniomaxillofacial hard tissue loss based on a universal repair model according to some embodiments of this specification. DETAILED DESCRIPTION

[0056] The specific embodiments of the present invention are described below to facilitate understanding of the present invention by those skilled in the art. However, it should be clear that the present invention is not limited to the scope of the specific embodiments. For those skilled in the art, as long as various changes are within the spirit and scope of the present invention as defined and determined by the appended claims, these changes are obvious, and all inventions and creations utilizing the concepts of the present invention are protected.

[0057] Example

[0058] Figure 1 This is an exemplary flow chart of a method for repairing craniofacial hard tissue loss based on a universal repair model according to some embodiments of this specification. Figure 1 As shown, the process includes the following steps. In some embodiments, the process can be executed by a processor.

[0059] S1: Build a universal craniomaxillofacial hard tissue repair model using a plug-and-play conditional attention mechanism and a curvature-aware query generator.

[0060] The universal craniomaxillofacial hard tissue restoration model is a neural network model used to acquire point cloud data of missing craniomaxillofacial hard tissue. The universal craniomaxillofacial hard tissue restoration model can be of various types. For example, the universal craniomaxillofacial hard tissue restoration model can include a transformer neural network model.

[0061] In some embodiments, the input of the universal craniofacial hard tissue restoration model can be the dentition to be completed and the opposing dentition of the teeth to be completed, and the output of the universal craniofacial hard tissue restoration model can be the missing restoration point cloud features.

[0062] In some embodiments, the structure of the universal craniomaxillofacial hard tissue repair model is as follows:

[0063] The universal craniomaxillofacial hard tissue repair model consists of a feature extraction module, an encoder module, a query generator module, and a decoder module. The output of the feature extraction module serves as the input to the encoder module, which in turn serves as the input to the query generator module. The outputs of the encoder and query generator modules serve as the input to the decoder module, which in turn serves as the final output of the universal craniomaxillofacial hard tissue repair model.

[0064] The feature extraction module is configured to extract features from the dentition to be completed and the opposing dentition of the teeth to be completed, respectively, to obtain corresponding features of the dentition to be completed and the opposing dentition. The input of the feature extraction module may include the dentition to be completed and the opposing dentition of the teeth to be completed, and the output may include features of the dentition to be completed and the opposing dentition.

[0065] The dentition to be completed is data reflecting the local information around the missing dentition.

[0066] The dentition of the opposing teeth of the tooth to be completed is data reflecting local information around the dentition of the opposing teeth of the tooth to be completed.

[0067] In some embodiments, the processor can convert the original tooth data of the teeth, jaws and skull, which is the three-dimensional grid data collected by the three-dimensional oral scanning device, into craniomaxillofacial hard tissue point cloud data with a fixed number of points to obtain the dentition to be completed and the dentition of the opposing teeth to be completed.

[0068] The characteristics of the dentition to be completed are data reflecting the local characteristic information around the missing dentition.

[0069] The dentition features of the opposing teeth are data reflecting local feature information around the dentition of the opposing teeth corresponding to the dentition to be completed.

[0070] In some embodiments, the processor can perform farthest point sampling on the dentition of the teeth to be completed and the dentition of the opposing teeth to be completed, and use a lightweight DGCNN with hierarchical downsampling to extract the features of the point center from the input point cloud to obtain the features of the dentition of the teeth to be completed and the features of the dentition of the opposing teeth to be completed.

[0071] The encoder module includes multiple multi-head attention layers and cross-attention layers; wherein the multi-head attention layer is used to encode the features of the dentition to be completed and the features of the opposing teeth to obtain a feature vector of the dentition to be completed and a feature vector of the opposing teeth; the cross-attention layer is used to perform cross-attention calculation on the feature vector of the dentition to be completed and the feature vector of the opposing teeth based on a plug-and-play conditional attention mechanism to obtain a cross-attention result. The input of the encoder module may include the features of the dentition to be completed and the features of the opposing teeth, and the output may include the feature vector of the dentition to be completed, the feature vector of the opposing teeth, and the cross-attention result.

[0072] The eigenvector of the dentition to be completed is a eigenvector that reflects the local information around the missing dentition.

[0073] The antagonistic tooth dentition feature vector is a feature vector reflecting the local information around the antagonistic tooth dentition corresponding to the tooth dentition to be completed.

[0074] In some embodiments, the multi-head attention layer in the encoder module includes: a linear transformation unit, which is used to perform linear transformation on the features of the dentition to be completed and the features of the opposing teeth to obtain a coding query matrix, a coding key matrix and a coding value matrix; an attention weight unit, which is used to perform attention weight calculation based on the coding query matrix, the coding key matrix and the coding value matrix to obtain a coding attention weight result; a multi-head attention unit, which is used to splice multiple coding attention weight results to obtain a coding splicing vector; a geometric perception unit, which is used to perform maximum pooling on the features of the dentition to be completed and the features of the opposing teeth, and connect them with the coding splicing vector to obtain a feature vector of the dentition to be completed and a feature vector of the opposing teeth, respectively.

[0075] The encoding query matrix is ​​the query matrix generated by the multi-head attention layer in the encoder. For example, the encoding query matrix can include a query matrix for the features of the dentition to be completed and a query matrix for the features of the opposing teeth.

[0076] The encoding bond matrix is ​​a bond matrix generated by the multi-head attention layer in the encoder. For example, the encoding bond matrix may include a bond matrix of the features of the dentition to be completed and a bond matrix of the features of the opposing teeth.

[0077] The encoding value matrix is ​​the value matrix generated by the multi-head attention layer in the encoder. For example, the encoding value matrix can include the value matrix of the features of the dentition to be completed and the value matrix of the features of the opposing teeth.

[0078] In some embodiments, the processor may perform linear transformation on the features of the dentition to be completed and the features of the opposing teeth to obtain a coding query matrix, a coding key matrix, and a coding value matrix:

[0079] ;

[0080] ;

[0081] ;

[0082] ;

[0083] ;

[0084] ;

[0085] in, The query matrix representing the characteristics of the dentition to be completed, Indicates the characteristics of the dentition to be completed, The bond matrix representing the characteristics of the dentition to be completed, A matrix of values ​​representing the characteristics of the dentition to be completed, The query matrix representing the dentition characteristics of the opposing teeth, Indicates the characteristics of the opposing teeth. The bond matrix representing the characteristics of the opposing tooth dentition, A matrix of values ​​representing the characteristics of the opposing tooth dentition, 、 、 、 、 and Both represent the corresponding learnable weight matrices.

[0086] The encoded attention weight result is data that reflects the correlation between different positions of the input sequence value matrix.

[0087] In some embodiments, the expression encoding the attention weight result can be:

[0088] ;

[0089] in, Represents the result of encoding attention weight, represents the normalized weight, Indicates the dimensions of the bond matrix.

[0090] The encoding concatenation vector is the concatenation result reflecting multiple sets of independent encoding attention weight results.

[0091] In some embodiments, the expression for encoding the splicing vector may be:

[0092] ;

[0093] in, represents the coded concatenated vector, represents the concatenation function, 、 and Both represent the corresponding encoding attention weight results, represents the output weight matrix.

[0094] In some embodiments, the processor may pass the encoded concatenated vector through a linear layer to obtain the linear layer output result:

[0095] ;

[0096] in, Represents the output of the linear layer, Represents the linear layer function.

[0097] In some embodiments, the processor may perform geometric perception on the features of the dentition to be completed and the features of the opposing dentition to obtain corresponding geometric perception results:

[0098] ;

[0099] in, Represents the geometric perception result, represents the maximum pooling operation, represents a feature set, express The coordinates of Indicates from Find the distance Recent The features corresponding to the points.

[0100] In some embodiments, the processor can connect the encoded splicing vector and the geometric perception result, map them through a linear layer, and perform residual splicing with the input of the next multi-head attention layer, and use the output of the last multi-head attention layer as the feature vector of the dentition to be completed and the feature vector of the opposing tooth dentition.

[0101] The cross-attention result is the result obtained by the feature vector of the dentition to be completed and the feature vector of the opposing dentition through the cross-attention layer.

[0102] In some embodiments, the cross-attention layer in the encoder module includes: a linear transformation unit, used to perform linear transformation on the feature vector of the dentition to be completed and the feature vector of the opposing dentition to obtain a cross-query matrix, a cross-key matrix and a cross-value matrix; an attention weight unit, used to perform attention weight calculation based on the cross-query matrix, the cross-key matrix and the cross-value matrix to obtain a cross-attention weight result; a cross-attention unit, used to splice multiple cross-attention weight results to obtain a cross-attention result.

[0103] The cross query matrix is ​​a query matrix obtained by linear transformation of the eigenvectors of the dentition to be completed.

[0104] The cross-bond matrix is ​​a bond matrix obtained based on the linear transformation of the eigenvectors of the opposing teeth.

[0105] The cross-value matrix is ​​a value matrix obtained based on the linear transformation of the eigenvectors of the opposing teeth.

[0106] In some embodiments, the expressions of the cross-query matrix, the cross-key matrix, and the cross-value matrix may be:

[0107] ;

[0108] ;

[0109] ;

[0110] in, represents the cross query matrix, represents the cross-bond matrix, represents the cross-value matrix, represents the feature vector of the dentition to be completed, represents the eigenvector of the opposing teeth, 、 and Both represent learnable weight matrices.

[0111] The cross-attention weight results reflect the data of the correlation at different positions in the cross-value matrix.

[0112] In some embodiments, the expression of the cross-attention result can be:

[0113] ;

[0114] in, represents the cross attention result, represents weighted splicing calculation, represents the feature vector of the dentition to be completed, represents the eigenvector of the opposing tooth dentition.

[0115] A query generator module includes a curvature-aware query generator configured to linearly project the cross-attention result to obtain a query embedding vector. The input of the query generator module may include the cross-attention result, and the output may include the query embedding vector.

[0116] The query embedding vector is the feature data used to adjust the query vector in the decoder.

[0117] In some embodiments, the expression for querying the embedding vector may be:

[0118] ;

[0119] ;

[0120] in, represents the query embedding vector, represents a multilayer perceptron, represents sparse point coordinates, express The local curvature of represents the maximum pooling operation, represents a linear layer, represents the result of cross attention, Represents a coordinate projection.

[0121] The decoder module is configured to calculate the feature vector of the dentition to be completed and the query embedding vector to obtain missing prediction data. The multi-scale point cloud generation module performs multi-scale point cloud generation calculations based on the missing prediction data to obtain missing repair point cloud features. The decoder module's input may include the feature vector of the dentition to be completed and the query embedding vector, and its output may include missing repair point cloud features.

[0122] The missing prediction data is the output of the multi-head attention layer in the decoder.

[0123] In some embodiments, the decoder module includes multiple multi-head attention layers, wherein the multi-head attention layers in the decoder module include: a linear transformation unit, used to perform linear transformation on the feature vector of the dentition to be completed to obtain a decoding value matrix and a decoding key matrix; perform linear transformation on the cross-attention result to obtain a first-layer decoding query matrix; an attention weight unit, used to perform attention weight calculation based on the decoding query matrix, the decoding key matrix and the decoding value matrix to obtain a decoding attention weight result; a multi-head attention unit, used to splice multiple decoding attention weight results to obtain a decoding splicing vector; a geometric perception unit, used to perform maximum pooling on the feature vector of the dentition to be completed and the cross-attention result, and connect them with the decoding splicing vector to obtain a connection result as the decoding query matrix of the next layer; the connection result of the last layer is used as the missing prediction data.

[0124] The decoding value matrix is ​​a value matrix obtained by linear transformation of the eigenvectors of the dentition to be completed.

[0125] The decoding key matrix is ​​a key matrix obtained based on the linear transformation of the eigenvectors of the dentition to be completed.

[0126] The decoding query matrix is ​​a query matrix obtained based on the cross attention result and the output of the previous multi-head attention layer.

[0127] In some embodiments, the processor may perform a linear transformation on the cross-attention result to obtain corresponding value parameters, key parameters, and query parameters:

[0128] ;

[0129] ;

[0130] ;

[0131] in, Indicates query parameters, Represents the key parameter, Represents a value parameter, represents the cross attention result, 、 and Both represent learnable weight matrices.

[0132] In some embodiments, the processor can input the feature vector of the dentition to be completed, the value parameter, the key parameter and the query parameter into the multi-head attention layer in the decoder, use the feature vector of the dentition to be completed to obtain the decoding value matrix and the decoding key matrix of each layer of the multi-head attention layer, and use the value parameter, the key parameter and the query parameter to obtain the decoding query matrix of the first layer; based on the output result of the first layer of the multi-head attention layer, splicing is performed to obtain the connection result as the decoding query matrix of the next layer, and the output result of the last layer of the multi-head attention layer is used as the missing prediction data.

[0133] The decoding attention weight result is data reflecting the correlation between different positions in the decoding value matrix.

[0134] The decoding concatenation vector is the concatenation result reflecting multiple sets of independent decoding attention weight results.

[0135] The join result is the result of joining the decoded value matrix, the decoded key matrix, and the decoded query matrix.

[0136] The missing repair point cloud feature is data that reflects the point cloud feature information of the craniomaxillofacial hard tissue missing area.

[0137] In some embodiments, the processor can perform multi-scale point cloud generation calculations based on the missing prediction data to obtain missing repair point cloud features. For example, the processor can use the multi-scale point cloud generation framework to recover the missing point cloud at full resolution from the missing prediction point proxy obtained by the decoder, reuse the M coordinates generated by the query generator as the local center of the missing point cloud, and use the FoldingNet reconstruction head to recover the detailed point cloud shape centered on the prediction point proxy to obtain the missing repair point cloud features.

[0138] In some embodiments, the expression for missing repair point cloud features may be:

[0139] ;

[0140] in, Indicates missing repair point cloud features, Represents the reconstruction header output result, Indicates missing prediction data, represents the local center of the missing point cloud, Indicates the Missing point cloud coordinates, Indicates the number of missing point cloud coordinates.

[0141] S2: Input the craniomaxillofacial hard tissue point cloud data into the universal craniomaxillofacial hard tissue repair model, and train it using a weighted loss function to obtain a trained universal craniomaxillofacial hard tissue repair model; wherein the trained universal craniomaxillofacial hard tissue repair model is used to process the craniomaxillofacial hard tissue point cloud data to obtain missing repair point cloud features.

[0142] In some embodiments, a universal craniofacial hard tissue repair model can be trained using multiple labeled training samples. For example, multiple labeled training samples can be input into an initial universal craniofacial hard tissue repair model. A loss function can be constructed using the labels and the results of the initial universal craniofacial hard tissue repair model. Based on the loss function, the parameters of the initial universal craniofacial hard tissue repair model are iteratively updated using gradient descent or other methods. Model training is completed when preset conditions are met, resulting in a trained universal craniofacial hard tissue repair model. The preset conditions may include convergence of the loss function, a threshold number of iterations, and the like.

[0143] In some embodiments, the training samples may include historical craniomaxillofacial hard tissue point cloud data. The labels may be corresponding missing and repaired point cloud features. The labels may be manually annotated.

[0144] In some embodiments, the loss function may be expressed as:

[0145] ;

[0146] ;

[0147] in, Represents the loss function result, represents the sparse point chamfer distance loss function, and Both represent weight factors, represents the dense point chamfer distance loss function, represents the earth moving distance loss function, represents the minimum value of the bijective mapping, represents the ground truth complete point cloud, represents a point in the predicted point cloud, represents a bijective mapping, represents the Euclidean distance; .

[0148] In some embodiments, and The expressions can be:

[0149] ;

[0150] ;

[0151] in, express The number of points in , represents a sparse point cloud, represents the ground truth point cloud, represents the nearest square distance, express A local center, express The number of points in .

[0152] S3: De-noising the missing repair point cloud features to obtain denoised missing repair point cloud features.

[0153] In some embodiments, the processor can implement S3 based on the following steps: performing distribution modeling on the missing repair point cloud features to obtain a noise volume distribution; performing score calculation based on the noise volume distribution to obtain a noise volume distribution score; and denoising the missing repair point cloud features based on the noise volume distribution score, moving the noise points to a distribution pattern corresponding to the underlying clean surface, and obtaining denoised missing repair point cloud features.

[0154] The noise convolution distribution is the data that reflects the distribution modeling of the noise point cloud. For example, the convolution noise distribution can be expressed as , Indicates the number of missing repair point cloud features, indicates that the noise follows the distribution.

[0155] In some embodiments, the processor can process the missing repair point cloud features using the optimized adapointr model to obtain a predicted point cloud, modeling the underlying noise-free point cloud as a 3D distribution supported by a 2D manifold. A set of samples, and assuming that the noise follows the distribution , and get the noise convolution distribution.

[0156] The noise convolution distribution score is the gradient of the logarithmic probability function, which is used to represent the corresponding score of the noise convolution distribution. For example, the noise convolution distribution score can be expressed as .

[0157] In some embodiments, the processor may denoise the missing repair point cloud features using the noise convolution distribution score through gradient ascent, move the noise points to a distribution pattern corresponding to the underlying clean surface, and obtain the denoised missing repair point cloud features.

[0158] S4: performing point cloud mesh modeling processing using the denoised missing repair point cloud features to obtain a craniomaxillofacial hard tissue missing repair result, thereby completing the craniomaxillofacial hard tissue missing repair.

[0159] The results of craniomaxillofacial hard tissue loss repair reflect the repair status of the craniomaxillofacial hard tissue loss area.

[0160] In some embodiments, the processor can implement S4 based on the following steps: using the denoised missing repair point cloud features to perform point cloud mesh modeling processing to obtain a dense point cloud; predicting the dense point cloud to obtain a corresponding unsigned distance field; extracting the triangular mesh of the unsigned distance field to perform mesh reconstruction to obtain a craniofacial hard tissue missing repair result, thereby completing the craniofacial hard tissue missing repair.

[0161] The dense point cloud is point cloud data associated with the non-directional normal vectors in the missing repair point cloud features after denoising.

[0162] In some embodiments, the processor may first model the local geometry of the denoised missing repair point cloud features through LGR to obtain a dense point cloud.

[0163] The unsigned distance field is the distance field in the predicted dense point cloud.

[0164] In some embodiments, the processor may use the GUE to predict the dense point cloud to obtain an unsigned distance field of the dense point cloud.

[0165] In some embodiments, the processor can use the E-MC module to extract the triangular mesh of the zero level set in the unsigned distance field, perform mesh reconstruction, obtain the craniomaxillofacial hard tissue loss repair result, and complete the craniomaxillofacial hard tissue loss repair.

[0166] In some embodiments of this specification, a universal restoration model is used to repair craniofacial hard tissue loss, resulting in restoration results. This model can uniformly address the reconstruction of missing and missing teeth in the field of implant restoration, the reconstruction of jaw defects in the field of craniofacial surgery, and the restoration of skull defects in the field of neurosurgery, solving the entire cranial hard tissue restoration and reconstruction challenge with a universal end-to-end deep learning model.

Claims

1. A method for repairing craniomaxillofacial hard tissue loss based on a universal repair model, characterized in that: include: S1: Build a universal craniomaxillofacial hard tissue repair model using a plug-and-play conditional attention mechanism and a curvature-aware query generator; The universal craniomaxillofacial hard tissue repair model includes: A feature extraction module is used to extract features of the dentition to be completed and the dentition of the opposing teeth to be completed, respectively, to obtain corresponding features of the dentition to be completed and the dentition of the opposing teeth; The encoder module includes multiple multi-head attention layers and cross-attention layers; wherein the multi-head attention layer is used to encode the features of the dentition to be completed and the features of the opposing teeth to obtain a feature vector of the dentition to be completed and a feature vector of the opposing teeth; the cross-attention layer is used to perform cross-attention calculation on the feature vector of the dentition to be completed and the feature vector of the opposing teeth based on a plug-and-play conditional attention mechanism to obtain a cross-attention result; A query generator module, comprising a curvature-aware query generator, configured to perform linear projection on the cross-attention result to obtain a query embedding vector; A decoder module is configured to calculate the feature vector of the to-be-completed dentition and the query embedding vector to obtain missing prediction data; a multi-scale point cloud generation module is configured to perform multi-scale point cloud generation calculation based on the missing prediction data to obtain missing repair point cloud features; S2: Inputting the craniomaxillofacial hard tissue point cloud data into the craniomaxillofacial hard tissue universal repair model, and training using a weighted loss function to obtain a trained craniomaxillofacial hard tissue universal repair model; wherein the trained craniomaxillofacial hard tissue universal repair model is used to process the craniomaxillofacial hard tissue point cloud data to obtain missing repair point cloud features; S3: performing denoising processing on the missing repair point cloud features to obtain denoised missing repair point cloud features; S4: performing point cloud mesh modeling processing using the denoised missing repair point cloud features to obtain a craniomaxillofacial hard tissue missing repair result, thereby completing the craniomaxillofacial hard tissue missing repair.

2. The method for repairing craniomaxillofacial hard tissue loss based on a universal repair model according to claim 1, characterized in that: The multi-head attention layer in the encoder module includes: A linear transformation unit, configured to perform linear transformation on the features of the dentition to be completed and the features of the opposing teeth to obtain a coding query matrix, a coding key matrix, and a coding value matrix; an attention weight unit, configured to calculate an attention weight based on the encoding query matrix, the encoding key matrix, and the encoding value matrix to obtain an encoding attention weight result; Multi-head attention unit, used to splice multiple encoding attention weight results to obtain the encoding splicing vector; The geometric perception unit is used to perform maximum pooling on the features of the dentition to be completed and the features of the opposing teeth, and connect them with the encoded splicing vector to obtain the feature vector of the dentition to be completed and the feature vector of the opposing teeth respectively.

3. The method for repairing craniofacial hard tissue loss based on a universal repair model according to claim 1, characterized in that: The crisscross attention layer in the encoder module includes: A linear transformation unit, configured to perform linear transformation on the feature vector of the dentition to be completed and the feature vector of the opposing dentition to obtain a cross-query matrix, a cross-key matrix, and a cross-value matrix; an attention weight unit, configured to calculate an attention weight based on the cross-query matrix, the cross-key matrix, and the cross-value matrix to obtain a cross-attention weight result; The cross attention unit is used to concatenate multiple cross attention weight results to obtain the cross attention result: ； in, represents the cross attention result, represents the splicing calculation, represents the feature vector of the dentition to be completed, represents the eigenvector of the opposing tooth dentition.

4. The method for repairing craniomaxillofacial hard tissue loss based on a universal repair model according to claim 1, characterized in that: The expression of the query embedding vector is: ； ； in, represents the query embedding vector, represents a multilayer perceptron, represents sparse point coordinates, express The local curvature of represents the maximum pooling operation, represents a linear layer, represents the result of cross attention, Represents a coordinate projection.

5. The method for repairing craniomaxillofacial hard tissue loss based on a universal repair model according to claim 1, characterized in that: The decoder module includes multiple multi-head attention layers, wherein the multi-head attention layers in the decoder module include: A linear transformation unit is used to perform a linear transformation on the feature vector of the dentition to be completed to obtain a decoding value matrix and a decoding key matrix; and to perform a linear transformation on the cross attention result to obtain a first-layer decoding query matrix; an attention weight unit, configured to calculate an attention weight based on the decoding query matrix, the decoding key matrix, and the decoding value matrix to obtain a decoding attention weight result; Multi-head attention unit, used to splice multiple decoding attention weight results to obtain the decoding splicing vector; The geometric perception unit is used to perform maximum pooling on the feature vector of the to-be-completed dentition and the cross-attention result, and connect them with the decoding splicing vector to obtain a connection result as the decoding query matrix of the next layer; the connection result of the last layer is used as the missing prediction data.

6. The method for repairing craniomaxillofacial hard tissue loss based on a universal repair model according to claim 1, characterized in that: The S3 includes: Performing distribution modeling on the missing repair point cloud features to obtain noise convolution distribution; Based on the noise convolution distribution, a score calculation is performed to obtain a noise convolution distribution score; Based on the noise convolution distribution score, the missing repair point cloud feature is denoised, and the noise points are moved to a distribution pattern corresponding to the underlying clean surface to obtain a denoised missing repair point cloud feature.

7. The method for repairing craniofacial hard tissue loss based on a universal repair model according to claim 1, characterized in that: The S4 includes: Using the missing repair point cloud features after denoising, point cloud mesh modeling is performed to obtain a dense point cloud; Predicting the dense point cloud to obtain a corresponding unsigned distance field; By extracting the triangular mesh of the unsigned distance field and performing mesh reconstruction, the craniomaxillofacial hard tissue loss repair result is obtained, thereby completing the craniomaxillofacial hard tissue loss repair.

Images