Craniomaxillofacial hard tissue deletion repair method based on general repair model

Abstract

The invention provides a cranio-maxillofacial hard tissue deletion repair method based on a general repair model, and relates to the technical field of jaw defect reconstruction, the method comprises the following steps: constructing a cranio-maxillofacial hard tissue general repair model by using a plug-and-play conditional attention mechanism and a query generator based on curvature perception; inputting the cranio-maxillofacial hard tissue point cloud data into a cranio-maxillofacial hard tissue general repair model, and training by using a weighted loss function to obtain a trained cranio-maxillofacial hard tissue general repair model; performing de-noising processing on the missing repair point cloud features to obtain de-noised missing repair point cloud features; and carrying out point cloud gridding modeling processing by using the denoised deletion repair point cloud features to obtain a cranio-maxillofacial hard tissue deletion repair result, and completing the deletion repair of the cranio-maxillofacial hard tissue. The problem that repair and reconstruction of hard tissues at the head part are difficult to solve uniformly is solved.

Description

Technical Field

[0001] This specification relates to the technical field of jaw defect reconstruction, and particularly relates to a method for repairing craniofacial hard tissue defects based on a general repair model. Background Art

[0002] Currently, hard tissue repair algorithms based on artificial intelligence are mainly divided into two branches. One is the repair algorithm for missing teeth, and the other is the repair algorithm for jaw or skull defects. The core difference between these two types of algorithms lies in the data sources they rely on. Specifically, tooth repair algorithms usually rely on 3D oral scan data, which consists of tens of thousands of triangular meshes or point clouds, while jaw and skull repair algorithms mostly use three-dimensional CT image data. Although the repair of tissues can be classified as hard tissue repair problems, there is currently no general model that can cover the repair of hard tissues in various cranial regions. Summary of the Invention

[0003] Aiming at the above deficiencies in the prior art, a method for repairing craniofacial hard tissue defects based on a general repair model provided by the present invention solves the problem that it is difficult to uniformly solve the repair and reconstruction of hard tissues in the cranial region.

[0004] To achieve the above invention objective, the technical solution adopted by the present invention is: A method for repairing craniofacial hard tissue defects based on a general repair model, comprising:

[0005] S1: Construct a general repair model for craniofacial hard tissues by using a plug-and-play conditional attention mechanism and a query generator based on curvature perception;

[0006] S2: Input the craniofacial hard tissue point cloud data into the general repair model for craniofacial hard tissues, and train it using a weighted loss function to obtain a trained general repair model for craniofacial hard tissues; wherein, the trained general repair model for craniofacial hard tissues is used to process the craniofacial hard tissue point cloud data to obtain missing repair point cloud features;

[0007] S3: Denoise the missing repair point cloud features to obtain denoised missing repair point cloud features;

[0008] S4: Use the denoised missing repair point cloud features for point cloud meshing modeling processing to obtain the craniofacial hard tissue defect repair result, and complete the repair of the craniofacial hard tissue defect.

[0009] The beneficial effects of the present invention are as follows: Using a general repair model to repair craniofacial hard tissue defects, a craniofacial hard tissue defect repair result is obtained. It can uniformly solve the problems of defect and missing tooth reconstruction in the field of implant restoration, jaw defect reconstruction in the field of craniofacial surgery, and skull defect repair in the field of neurosurgery, and solve the difficult problem of hard tissue repair and reconstruction of the entire head region with a general end-to-end deep learning model.

[0010] Furthermore, the general craniofacial hard tissue repair model includes:

[0011] A feature extraction module, which is used to respectively extract features from the dentition of the teeth to be filled and the dentition of the opposing teeth of the teeth to be filled, and obtain the corresponding dentition features to be filled and opposing dentition features;

[0012] An encoder module, including multiple multi-head attention layers and cross-attention layers; wherein, the multi-head attention layer is used to perform encoding processing on the dentition features to be filled and the opposing dentition features, and obtain a dentition feature vector to be filled and an opposing dentition feature vector; the cross-attention layer is used to perform cross-attention calculation on the dentition feature vector to be filled and the opposing dentition feature vector based on the plug-and-play conditional attention mechanism, and obtain a cross-attention result;

[0013] A query generator module, including a curvature-aware query generator, which is used to perform linear projection on the cross-attention result to obtain a query embedding vector;

[0014] A decoder module, which is used to calculate the dentition feature vector to be filled and the query embedding vector to obtain missing prediction data; a multi-scale point cloud generation module, based on the missing prediction data, performs multi-scale point cloud generation calculation to obtain missing repair point cloud features.

[0015] By introducing an encoder module with multi-head attention layers, it can effectively capture the complex geometric features of the dentition features to be filled, and improve the accuracy and efficiency of feature extraction. The linear transformation unit converts the input features into a query matrix, a key matrix, and a value matrix. The attention weight unit further enhances the model's attention to key features by calculating attention weights. The multi-head attention unit can comprehensively integrate feature information from different angles by splicing multiple attention weight results, and improve the expression ability of features. The geometric perception unit further enhances the model's perception ability of geometric features through max pooling operation, making the finally generated dentition feature vector to be filled more accurate and robust. When the model processes the craniofacial hard tissue defect repair task, it can better capture local and global features, and improve the accuracy and naturalness of the repair result.

[0016] Furthermore, the multi-head attention layer in the encoder module includes:

[0017] A linear transformation unit for linearly transforming the to-be-completed dentition features and the opposing dentition features to obtain an encoded query matrix, an encoded key matrix, and an encoded value matrix;

[0018] An attention weight unit for calculating attention weights based on the encoded query matrix, the encoded key matrix, and the encoded value matrix to obtain an encoded attention weight result;

[0019] A multi-head attention unit for concatenating multiple encoded attention weight results to obtain an encoded concatenated vector;

[0020] A geometric perception unit for performing max pooling on the to-be-completed dentition features and the opposing dentition features and connecting them with the encoded concatenated vector to respectively obtain a to-be-completed dentition feature vector and an opposing dentition feature vector.

[0021] By introducing a cross-attention layer, the interaction information between the to-be-completed dentition feature vector and the opposing dentition feature vector can be effectively captured, further improving the accuracy of feature fusion. Specifically, the linear transformation unit converts the to-be-completed dentition feature vector and the opposing dentition feature vector into a cross query matrix, a cross key matrix, and a cross value matrix respectively. The attention weight unit can dynamically adjust the correlation strength between features by calculating cross-attention weights. The cross-attention unit can comprehensively integrate feature information from multiple perspectives by concatenating multiple cross-attention weight results, generating a more expressive cross-attention result. This enables the model to better understand the geometric and functional relationships between the to-be-completed region and the opposing region, thereby generating more accurate restoration results.

[0022] Furthermore, the cross-attention layer in the encoder module includes:

[0023] A linear transformation unit for linearly transforming the to-be-completed dentition feature vector and the opposing dentition feature vector to obtain a cross query matrix, a cross key matrix, and a cross value matrix;

[0024] An attention weight unit for calculating attention weights based on the cross query matrix, the cross key matrix, and the cross value matrix to obtain a cross-attention weight result;

[0025] A cross-attention unit for concatenating multiple cross-attention weight results to obtain a cross-attention result:

[0026] V = M cross E (V up , V down );

[0027] where V represents the cross-attention result, and Mcross E Indicates splicing calculation, V up Indicates the dentition feature vector to be completed, V down Indicates the dentition feature vector of the opposing teeth.

[0028] By introducing a curvature-aware query generator, the curvature geometric features of the region to be completed can be effectively captured, generating more representative query embedding vectors. Specifically, local curvature information is incorporated into the generation process of the query embedding vectors, enabling the model to better understand the geometric shape of the region to be completed. The max pooling operation further enhances the model's perception ability of global features, while the coordinate projection operation ensures the consistency of the query embedding vectors with the spatial positions. This design makes the query embedding vectors not only contain rich semantic information but also accurately reflect the geometric characteristics of the region to be completed, thus providing high-quality inputs for the subsequent decoder module.

[0029] Furthermore, the expression of the query embedding vector is:

[0030] Q = MLP([C, Cur C , M(Linear(V)]);

[0031] C = P(M(Linear(V)));

[0032] where Q represents the query embedding vector, MLP represents the multi-layer perceptron, C represents the sparse point coordinates, Cur C represents the local curvature of C, M represents the max pooling operation, Linear represents the linear layer, V represents the result of cross-attention, and P represents the coordinate projection.

[0033] By introducing a decoder module with a multi-layer multi-head attention mechanism, the missing prediction data can be gradually refined, improving the accuracy and naturalness of the repair results. Specifically, the linear transformation unit converts the dentition feature vector to be completed and the cross-attention result into a decoded value matrix, a decoded key matrix, and a decoded query matrix respectively. The attention weight unit can dynamically adjust the association strength between features by calculating the decoded attention weights. The multi-head attention unit can comprehensively integrate feature information from multiple perspectives by splicing the results of multiple decoded attention weights, generating a more expressive decoded splicing vector. The geometric perception unit further enhances the model's perception ability of geometric features through the max pooling operation and connects the decoded splicing vector with the geometric features to generate high-quality missing prediction data. This enables the model to gradually optimize the repair results and generate repair data that better conforms to the anatomical structure.

[0034] Furthermore, the decoder module includes multiple multi-head attention layers, where the multi-head attention layers in the decoder module include:

[0035] A linear transformation unit for performing a linear transformation on the to-be-completed dental arch feature vector to obtain a decoded value matrix and a decoded key matrix; performing a linear transformation on the cross-attention result to obtain a first-layer decoded query matrix;

[0036] An attention weight unit for calculating attention weights based on the decoded query matrix, the decoded key matrix, and the decoded value matrix to obtain a decoded attention weight result;

[0037] A multi-head attention unit for splicing a plurality of decoded attention weight results to obtain a decoded spliced vector;

[0038] A geometric perception unit for performing max pooling on the to-be-completed dental arch feature vector and the cross-attention result, and connecting the result with the decoded spliced vector to obtain a connection result as the decoded query matrix of the next layer; the connection result of the last layer is used as the missing prediction data.

[0039] Furthermore, the expression of the weighted loss function is:

[0040] L all =β·J 0 +(1 - β - α)·J 1 +α·L emd ;

[0041]

[0042] where L all represents the result of the loss function, J 0 represents the sparse point chamfer distance loss function, both α and β represent weight factors, J 1 represents the dense point chamfer distance loss function, L emd represents the earth mover's distance loss function, represents the minimum value of the bijective mapping, G represents the ground truth complete point cloud, p represents the points of the complete point cloud, P represents the points in the predicted point cloud, φ:P→G represents the bijective mapping, ||·|| 2 represents the Euclidean distance; α + β = 1, α>0, β>0.

[0043] By introducing the weighted loss function, the repair errors in different regions and at different scales can be effectively balanced, and the training effect and repair accuracy of the model can be improved. Specifically, the weighted loss function can adjust the loss weights according to the geometric complexity and functional importance of the to-be-completed region, enabling the model to pay more attention to the repair effect of key regions during training. This design can not only improve the overall quality of the repair results but also avoid the problems of the model overfitting simple regions or ignoring complex regions.

[0044] By introducing noise convolution distribution modeling and denoising operations, it is possible to effectively remove the noise points in the missing repair point cloud features, improving the smoothness and accuracy of the repair results. Specifically, noise convolution distribution modeling can capture the noise distribution patterns in the point cloud features, and score calculation can quantify the deviation degree of the noise points. This design enables the model to generate a more smooth and accurate repaired point cloud, laying a solid foundation for subsequent meshing modeling and mesh reconstruction.

[0045] Further, the S3 includes:

[0046] Perform distribution modeling on the missing repair point cloud features to obtain a noise convolution distribution;

[0047] Based on the noise convolution distribution, perform score calculation to obtain a noise convolution distribution score;

[0048] Based on the noise convolution distribution score, denoise the missing repair point cloud features, move the noise points towards the distribution pattern corresponding to the underlying clean surface, and obtain the denoised missing repair point cloud features.

[0049] Further, the S4 includes:

[0050] Use the denoised missing repair point cloud features to perform point cloud meshing modeling processing to obtain a dense point cloud;

[0051] Perform prediction on the dense point cloud to obtain the corresponding unsigned distance field;

[0052] By extracting the triangular mesh of the unsigned distance field, perform mesh reconstruction to obtain the craniofacial hard tissue missing repair result, and complete the missing repair of the craniofacial hard tissue.

[0053] By introducing point cloud meshing modeling and unsigned distance field extraction technologies, it is possible to convert the missing repair point cloud features into high-quality triangular meshes and generate craniofacial hard tissue repair results that conform to the anatomical structure. Specifically, point cloud meshing modeling can convert sparse point cloud features into dense point clouds, and the unsigned distance field extraction technology can accurately capture the geometric shape of the point cloud surface. By extracting the triangular mesh of the unsigned distance field, a high-precision mesh model can be generated, thereby achieving precise repair of the craniofacial hard tissue. This makes the repair results not only have high geometric accuracy but also meet the strict requirements of clinical applications. Description of the Drawings

[0054] This specification will further illustrate in the form of exemplary embodiments, and these exemplary embodiments will be described in detail through the drawings. These embodiments are not restrictive. In these embodiments, the same numbers represent the same structures, where:

[0055] Figure 1 It is an exemplary flowchart of a method for repairing craniofacial hard tissue defects based on a general repair model shown in some embodiments of this specification. Detailed implementation manners

[0056] The following describes the detailed implementation manners of the present invention to facilitate those skilled in the art of this technology to understand the present invention. However, it should be clear that the present invention is not limited to the scope of the detailed implementation manners. For those of ordinary skill in the art of this technology, as long as various changes are within the spirit and scope of the present invention defined and determined by the appended claims, these changes are obvious, and all inventions created using the concept of the present invention are within the scope of protection.

[0057] Embodiment

[0058] Figure 1 It is an exemplary flowchart of a method for repairing craniofacial hard tissue defects based on a general repair model shown in some embodiments of this specification. As Figure 1 shown, the process includes the following steps. In some embodiments, the process can be executed by a processor.

[0059] S1: Construct a general craniofacial hard tissue repair model by using a plug-and-play conditional attention mechanism and a curvature-aware query generator.

[0060] The general craniofacial hard tissue repair model is a neural network model used to obtain the point cloud data of the missing part of the craniofacial hard tissue. The type of the general craniofacial hard tissue repair model can be various. For example, the type of the general craniofacial hard tissue repair model can include a transformer neural network model.

[0061] In some embodiments, the input of the general craniofacial hard tissue repair model can be the tooth dentition to be filled and the opposing tooth dentition of the tooth to be filled, and the output of the general craniofacial hard tissue repair model can be the missing repair point cloud feature.

[0062] In some embodiments, the structure of the general craniofacial hard tissue repair model is as follows:

[0063] The general craniofacial hard tissue repair model includes a feature extraction module, an encoder module, a query generator module, and a decoder module. The output of the feature extraction module serves as the input of the encoder module, the output of the encoder module serves as the input of the query generator module, the output of the encoder module and the output of the query generator module serve as the input of the decoder module, and the output of the decoder module serves as the final output of the general craniofacial hard tissue repair model.

[0064] A feature extraction module is used to respectively extract features from the dentition of the tooth to be restored and the dentition of the opposing tooth of the tooth to be restored, so as to obtain the corresponding dentition feature to be restored and the opposing dentition feature. The input of the feature extraction module may include the dentition of the tooth to be restored and the dentition of the opposing tooth of the tooth to be restored, and the output may include the dentition feature to be restored and the opposing dentition feature.

[0065] The dentition of the tooth to be restored is data reflecting the local information around the missing dentition.

[0066] The dentition of the opposing tooth of the tooth to be restored is data reflecting the local information around the dentition of the opposing tooth corresponding to the dentition of the tooth to be restored.

[0067] In some embodiments, the processor can convert the original tooth data, which includes teeth, jaws and skulls, from the three-dimensional mesh data collected by the three-dimensional oral scanner into the craniofacial hard tissue point cloud data with fixed-point numbers, so as to obtain the dentition of the tooth to be restored and the dentition of the opposing tooth of the tooth to be restored.

[0068] The dentition feature to be restored is data reflecting the local feature information around the missing dentition.

[0069] The opposing dentition feature is data reflecting the local feature information around the dentition of the opposing tooth corresponding to the dentition of the tooth to be restored.

[0070] In some embodiments, the processor can perform farthest point sampling on the dentition of the tooth to be restored and the dentition of the opposing tooth of the tooth to be restored, and use a lightweight DGCNN with hierarchical downsampling to extract the features of the point centers from the input point cloud, so as to obtain the dentition feature of the tooth to be restored and the dentition feature of the opposing tooth of the tooth to be restored.

[0071] An encoder module includes a plurality of multi-head attention layers and cross-attention layers; wherein, the multi-head attention layers are used to encode the dentition feature to be restored and the opposing dentition feature to obtain a dentition feature vector to be restored and an opposing dentition feature vector; the cross-attention layers are used to perform cross-attention calculation on the dentition feature vector to be restored and the opposing dentition feature vector based on the plug-and-play conditional attention mechanism to obtain a cross-attention result. The input of the encoder module may include the dentition feature to be restored and the opposing dentition feature, and the output may include the dentition feature vector to be restored, the opposing dentition feature vector and the cross-attention result.

[0072] The dentition feature vector to be restored is a feature vector reflecting the local information around the missing dentition.

[0073] The opposing dentition feature vector is a feature vector reflecting the local information around the dentition of the opposing tooth corresponding to the dentition of the tooth to be restored.

[0074] In some embodiments, the multi-head attention layer in the encoder module includes: a linear transformation unit for performing a linear transformation on the to-be-complemented dentition feature and the opposing dentition feature to obtain an encoded query matrix, an encoded key matrix, and an encoded value matrix; an attention weight unit for calculating attention weights based on the encoded query matrix, the encoded key matrix, and the encoded value matrix to obtain an encoded attention weight result; a multi-head attention unit for concatenating multiple encoded attention weight results to obtain an encoded concatenated vector; and a geometric perception unit for performing max pooling on the to-be-complemented dentition feature and the opposing dentition feature and connecting them with the encoded concatenated vector to obtain a to-be-complemented dentition feature vector and an opposing dentition feature vector respectively.

[0075] The encoded query matrix is the query matrix generated by the multi-head attention layer in the encoder. For example, the encoded query matrix may include the query matrix of the to-be-complemented dentition feature and the query matrix of the opposing dentition feature.

[0076] The encoded key matrix is the key matrix generated by the multi-head attention layer in the encoder. For example, the encoded key matrix may include the key matrix of the to-be-complemented dentition feature and the key matrix of the opposing dentition feature.

[0077] The encoded value matrix is the value matrix generated by the multi-head attention layer in the encoder. For example, the encoded value matrix may include the value matrix of the to-be-complemented dentition feature and the value matrix of the opposing dentition feature.

[0078] In some embodiments, the processor may perform a linear transformation on the to-be-complemented dentition feature and the opposing dentition feature to obtain an encoded query matrix, an encoded key matrix, and an encoded value matrix:

[0079]

[0080]

[0081] where represents the query matrix of the to-be-complemented dentition feature, represents the to-be-complemented dentition feature, represents the key matrix of the to-be-complemented dentition feature, represents the value matrix of the to-be-complemented dentition feature, represents the query matrix of the opposing dentition feature, represents the opposing dentition feature, represents the key matrix of the opposing dentition feature, represents the value matrix of the opposing dentition feature, and both represent the corresponding learnable weight matrices.

[0082] The encoded attention weight result is data reflecting the correlation of different positions in the input sequence value matrix.

[0083] In some embodiments, the expression of the encoded attention weight result can be:

[0084]

[0085] where Attention represents the encoded attention weight result, softmax represents the normalized weight, and d k represents the dimension of the key matrix.

[0086] The encoded concatenated vector is the concatenation result reflecting multiple groups of independent encoded attention weight results.

[0087] In some embodiments, the expression of the encoded concatenated vector can be:

[0088]

[0089] where MultiHead represents the encoded concatenated vector, Concat represents the concatenation function, and both represent the corresponding encoded attention weight results, represents the output weight matrix.

[0090] In some embodiments, the processor can pass the encoded concatenated vector through a linear layer to obtain the linear layer output result:

[0091] L mid = Linear(MultiHead(Query up , Key up , Value up ));

[0092] where L mid represents the linear layer output result, and Linear represents the linear layer function.

[0093] In some embodiments, the processor can perform geometric perception on the tooth arch feature to be completed and the tooth arch feature of the opposing teeth to obtain the corresponding geometric perception result:

[0094]

[0095] where φ represents the geometric perception result, M represents the max pooling operation, m represents the feature set, represents the coordinate of Key up , κ represents finding the feature corresponding to the κ points closest to from .

[0096] In some embodiments, the processor may concatenate the encoded stitching vector and the geometric perception result, map them through a linear layer, and perform residual stitching with the input of the next multi-head attention layer, and use the output of the last multi-head attention layer as the dentition feature vector to be completed and the dentition feature vector of the opposing teeth.

[0097] The cross-attention result is the result obtained by passing the dentition feature vector to be completed and the dentition feature vector of the opposing teeth through the cross-attention layer.

[0098] In some embodiments, the cross-attention layer in the encoder module includes: a linear transformation unit for performing linear transformation on the dentition feature vector to be completed and the dentition feature vector of the opposing teeth to obtain a cross-query matrix, a cross-key matrix, and a cross-value matrix; an attention weight unit for calculating attention weights based on the cross-query matrix, the cross-key matrix, and the cross-value matrix to obtain a cross-attention weight result; and a cross-attention unit for concatenating a plurality of cross-attention weight results to obtain a cross-attention result.

[0099] The cross-query matrix is a query matrix obtained by linear transformation based on the dentition feature vector to be completed.

[0100] The cross-key matrix is a key matrix obtained by linear transformation based on the dentition feature vector of the opposing teeth.

[0101] The cross-value matrix is a value matrix obtained by linear transformation based on the dentition feature vector of the opposing teeth.

[0102] In some embodiments, the expressions of the cross-query matrix, the cross-key matrix, and the cross-value matrix can be respectively:

[0103] Query i =V i up W Q ;

[0104] Key i =V i down W K ;

[0105] Value i =V i down W V ;

[0106] Wherein, Query i represents the cross-query matrix, Key i represents the cross-key matrix, Value i represents the cross-value matrix, V i upDenote the feature vector of the dentition to be completed, V i down Denote the feature vector of the opposing dentition, W Q , W K and W V All represent learnable weight matrices.

[0107] The cross-attention weight result reflects the data of the correlation at different positions of the cross-value matrix.

[0108] In some embodiments, the expression of the cross-attention result can be:

[0109] V = M cross E (V up , V down );

[0110] Among them, V represents the cross-attention result, and M cross E represents weighted splicing calculation, V up represents the feature vector of the dentition to be completed, V down represents the feature vector of the opposing dentition.

[0111] The query generator module includes a curvature-aware query generator for linearly projecting the cross-attention result to obtain a query embedding vector. The input of the query generator module can include the cross-attention result, and the output can include the query embedding vector.

[0112] The query embedding vector is the feature data used to adjust the query vector in the decoder.

[0113] In some embodiments, the expression of the query embedding vector can be:

[0114] Q = MLP([C, Cur C , M(Linear(V)]);

[0115] C = P(M(Linear(V)));

[0116] Among them, Q represents the query embedding vector, MLP represents the multi-layer perceptron, C represents the sparse point coordinates, Cur C represents the local curvature of C, M represents the max pooling operation, Linear represents the linear layer, V represents the result of the cross-attention, and P represents the coordinate projection.

[0117] A decoder module for calculating the to-be-completed dentition feature vector and the query embedding vector to obtain missing prediction data; a multi-scale point cloud generation module for performing multi-scale point cloud generation calculation based on the missing prediction data to obtain missing repair point cloud features. The input of the decoder module may include the to-be-completed dentition feature vector and the query embedding vector, and the output may include the missing repair point cloud features.

[0118] The missing prediction data is the data output by the multi-head attention layer in the decoder.

[0119] In some embodiments, the decoder module includes multiple multi-head attention layers. Among them, the multi-head attention layer in the decoder module includes: a linear transformation unit for linearly transforming the to-be-completed dentition feature vector to obtain a decoded value matrix and a decoded key matrix; linearly transforming the cross-attention result to obtain a first-layer decoded query matrix; an attention weight unit for calculating attention weights based on the decoded query matrix, the decoded key matrix, and the decoded value matrix to obtain a decoded attention weight result; a multi-head attention unit for splicing multiple decoded attention weight results to obtain a decoded spliced vector; a geometric perception unit for performing max pooling on the to-be-completed dentition feature vector and the cross-attention result and connecting them with the decoded spliced vector to obtain a connection result as the decoded query matrix for the next layer; the connection result of the last layer is used as the missing prediction data.

[0120] The decoded value matrix is a value matrix linearly transformed based on the to-be-completed dentition feature vector.

[0121] The decoded key matrix is a key matrix linearly transformed based on the to-be-completed dentition feature vector.

[0122] The decoded query matrix is a query matrix obtained based on the cross-attention result and the output result of the previous multi-head attention layer.

[0123] In some embodiments, the processor can linearly transform the cross-attention result to obtain corresponding value parameters, key parameters, and query parameters:

[0124] Query' 0_0 =QW' Q ;

[0125] Key' 0_0 =QW' K ;

[0126] Value' 0_0 =QW' V ;

[0127] Among them, Query' 0_0 represents the query parameter, Key'0_0 Represents the key parameter, Value' 0_0 Represents the value parameter, Q represents the cross-attention result, W' Q 、W' K and W' V All represent learnable weight matrices.

[0128] In some embodiments, the processor may input the to-be-completed dentition feature vector, value parameter, key parameter, and query parameter into the multi-head attention layer in the decoder, use the to-be-completed dentition feature vector to obtain the decoded value matrix and decoded key matrix of each layer of the multi-head attention layer, and use the value parameter, key parameter, and query parameter to obtain the decoded query matrix of the first layer; based on the output result of the first layer of the multi-head attention layer, perform splicing to obtain a connection result, which is used as the decoded query matrix of the next layer, and use the output result of the last layer of the multi-head attention layer as the missing prediction data.

[0129] The decoded attention weight result is data reflecting the correlation of different positions of the decoded value matrix.

[0130] The decoded splicing vector is the splicing result reflecting multiple groups of independent decoded attention weight results.

[0131] The connection result is the result of connecting the decoded value matrix, decoded key matrix, and decoded query matrix.

[0132] The missing repair point cloud feature is data reflecting the point cloud feature information of the craniofacial hard tissue missing area.

[0133] In some embodiments, the processor may perform multi-scale point cloud generation calculation based on the missing prediction data to obtain the missing repair point cloud feature. For example, the processor may use a multi-scale point cloud generation framework to recover the missing point cloud from the missing prediction point proxy obtained from the decoder at full resolution, reuse the M coordinates generated by the query generator as the local center of the missing point cloud, and use the FoldingNet reconstruction head to recover the detailed point cloud shape centered on the prediction point proxy to obtain the missing repair point cloud feature.

[0134] In some embodiments, the expression of the missing repair point cloud feature may be:

[0135] P i =f(H i )+c i ,i=1,2,...,M;

[0136] Wherein, P i represents the missing repair point cloud feature, f(H i ) represents the output result of the reconstruction head, H i represents the missing prediction data, c irepresents the local center of the missing point cloud, i represents the i-th missing point cloud coordinate, and M represents the number of missing point cloud coordinates.

[0137] S2: Input the cranio-maxillofacial hard tissue point cloud data into the general cranio-maxillofacial hard tissue repair model, and train it using a weighted loss function to obtain a trained general cranio-maxillofacial hard tissue repair model; wherein, the trained general cranio-maxillofacial hard tissue repair model is used to process the cranio-maxillofacial hard tissue point cloud data to obtain the missing repair point cloud features.

[0138] In some embodiments, the general cranio-maxillofacial hard tissue repair model can be trained with multiple labeled training samples. For example, multiple labeled training samples can be input into the initial general cranio-maxillofacial hard tissue repair model, and a loss function can be constructed based on the labels and the results of the initial general cranio-maxillofacial hard tissue repair model. The parameters of the initial general cranio-maxillofacial hard tissue repair model are iteratively updated based on the loss function through gradient descent or other methods. When the preset conditions are met, the model training is completed, and a trained general cranio-maxillofacial hard tissue repair model is obtained. Among them, the preset conditions can be the convergence of the loss function, the number of iterations reaching a threshold, etc.

[0139] In some embodiments, the training samples can include historical cranio-maxillofacial hard tissue point cloud data. The labels can be the corresponding missing repair point cloud features. The labels can be manually annotated.

[0140] In some embodiments, the expression of the loss function can be:

[0141] L all =β·J 0 +(1-β-α)·J 1 +α·L emd ;

[0142]

[0143] wherein, L all represents the result of the loss function, J 0 represents the sparse point chamfer distance loss function, α and β both represent weight factors, J 1 represents the dense point chamfer distance loss function, L emd represents the earth mover's distance loss function, represents the minimum value of the bijective mapping, G represents the ground truth complete point cloud, p represents the point of the complete point cloud, P represents the point in the predicted point cloud, φ: P → G represents the bijective mapping, ||·|| 2 represents the Euclidean distance; α + β = 1, α > 0, β > 0.

[0144] In some embodiments, the expressions of J 0 and J 1 can be respectively:

[0145]

[0146] Among them, n C represents the number of points in c, c represents the sparse point cloud, g represents the ground truth point cloud, represents the nearest squared distance, C represents n C local center, n G represents the number of points in G.

[0147] S3: Denoise the missing repair point cloud feature to obtain the denoised missing repair point cloud feature.

[0148] In some embodiments, the processor can implement S3 based on the following steps: perform distribution modeling on the missing repair point cloud feature to obtain a noise convolution distribution; based on the noise convolution distribution, calculate a score to obtain a noise convolution distribution score; based on the noise convolution distribution score, denoise the missing repair point cloud feature, and move the noise points to the distribution mode corresponding to the underlying clean surface to obtain the denoised missing repair point cloud feature.

[0149] The noise convolution distribution is data reflecting the distribution modeling of the noise point cloud. For example, the convolution noise distribution can be expressed as m*n, where m represents the number of missing repair point cloud features and n represents the distribution followed by the noise.

[0150] In some embodiments, the processor can use an optimized adapointr model to process the missing repair point cloud feature to obtain a predicted point cloud, model the underlying noise-free point cloud as a set of samples of a 3D distribution m supported by a 2D manifold, and assume that the noise follows a distribution n to obtain a noise convolution distribution.

[0151] The noise convolution distribution score is the gradient of the logarithmic probability function, used to represent the corresponding score of the noise convolution distribution. For example, the noise convolution distribution score can be expressed as

[0152] In some embodiments, the processor can denoise the missing repair point cloud feature by gradient ascent using the noise convolution distribution score, move the noise points to the distribution mode corresponding to the underlying clean surface, and obtain the denoised missing repair point cloud feature.

[0153] S4: Use the denoised missing repair point cloud feature to perform point cloud meshing modeling processing to obtain a craniofacial hard tissue missing repair result, and complete the missing repair of the craniofacial hard tissue.

[0154] The craniofacial hard tissue missing repair result is a result reflecting the repair situation of the craniofacial hard tissue missing area.

[0155] In some embodiments, the processor may implement S4 based on the following steps: using the denoised missing repair point cloud features, performing point cloud meshing modeling processing to obtain a dense point cloud; predicting the dense point cloud to obtain a corresponding unsigned distance field; by extracting the triangular mesh of the unsigned distance field, performing mesh reconstruction to obtain the craniofacial hard tissue missing repair result, and completing the missing repair of the craniofacial hard tissue.

[0156] The dense point cloud is point cloud data related to the undirected normal vector in the denoised missing repair point cloud features.

[0157] In some embodiments, the processor may first model the local geometry of the denoised missing repair point cloud features through LGR to obtain a dense point cloud.

[0158] The unsigned distance field is the distance field in the predicted dense point cloud.

[0159] In some embodiments, the processor may use GUE to predict the dense point cloud to obtain the unsigned distance field of the dense point cloud.

[0160] In some embodiments, the processor may use the E-MC module to extract the triangular mesh of the zero level set in the unsigned distance field, perform mesh reconstruction to obtain the craniofacial hard tissue missing repair result, and complete the missing repair of the craniofacial hard tissue.

[0161] In some embodiments of this specification, using a general repair model to perform craniofacial hard tissue missing repair to obtain the craniofacial hard tissue missing repair result. It can uniformly solve the problems of defect and missing tooth reconstruction in the field of implant restoration, jaw defect reconstruction in the field of craniofacial surgery, and skull defect repair in the field of neurosurgery, and solve the difficult problem of hard tissue repair and reconstruction of the entire head area with a general end-to-end deep learning model.

Claims

1. A method for repairing craniomaxillofacial hard tissue loss based on a universal repair model, characterized in that: include: S1: Using a plug-and-play conditional attention mechanism and a curvature-aware query generator to build a universal craniomaxillofacial hard tissue repair model; S2: inputting the craniomaxillofacial hard tissue point cloud data into the craniomaxillofacial hard tissue universal repair model, and training using a weighted loss function to obtain a trained craniomaxillofacial hard tissue universal repair model; wherein the trained craniomaxillofacial hard tissue universal repair model is used to process the craniomaxillofacial hard tissue point cloud data to obtain missing repair point cloud features; S3: performing denoising processing on the missing repair point cloud features to obtain denoised missing repair point cloud features; S4: performing point cloud mesh modeling processing using the denoised missing repair point cloud features to obtain craniomaxillofacial hard tissue missing repair results, thereby completing craniomaxillofacial hard tissue missing repair.

2. The method for repairing craniomaxillofacial hard tissue loss based on a universal repair model according to claim 1, characterized in that: The universal craniomaxillofacial hard tissue repair model comprises: A feature extraction module is used to extract features of the dentition to be completed and the dentition of the opposing teeth to be completed, respectively, to obtain corresponding features of the dentition to be completed and the dentition of the opposing teeth; The encoder module includes a plurality of multi-head attention layers and cross-attention layers; wherein the multi-head attention layer is used to encode the features of the dentition to be completed and the features of the dentition of the opposing teeth to obtain a feature vector of the dentition to be completed and a feature vector of the dentition of the opposing teeth; the cross-attention layer is used to perform cross-attention calculation on the feature vector of the dentition to be completed and the feature vector of the dentition of the opposing teeth based on a plug-and-play conditional attention mechanism to obtain a cross-attention result; A query generator module, comprising a curvature-aware query generator, configured to linearly project the cross-attention result to obtain a query embedding vector; The decoder module is used to calculate the feature vector of the to-be-completed dentition and the query embedding vector to obtain missing prediction data; the multi-scale point cloud generation module performs multi-scale point cloud generation calculation based on the missing prediction data to obtain missing repair point cloud features.

3. The method for repairing craniomaxillofacial hard tissue loss based on a universal repair model according to claim 2, characterized in that: The multi-head attention layer in the encoder module includes: A linear transformation unit, used for performing linear transformation on the features of the dentition to be completed and the features of the opposing teeth to obtain a coding query matrix, a coding key matrix and a coding value matrix; An attention weight unit, used to calculate the attention weight based on the encoding query matrix, the encoding key matrix and the encoding value matrix to obtain an encoding attention weight result; The multi-head attention unit is used to concatenate multiple encoding attention weight results to obtain the encoding concatenation vector; The geometric perception unit is used to perform maximum pooling on the features of the dentition to be completed and the features of the opposing teeth, and connect them with the encoded splicing vector to obtain the feature vector of the dentition to be completed and the feature vector of the opposing teeth, respectively.

4. The method for repairing craniomaxillofacial hard tissue loss based on a universal repair model according to claim 2, characterized in that: The cross-attention layer in the encoder module includes: A linear transformation unit, used for performing linear transformation on the feature vector of the dentition to be completed and the feature vector of the opposing dentition to obtain a cross-query matrix, a cross-key matrix and a cross-value matrix; An attention weight unit, configured to calculate an attention weight based on the cross query matrix, the cross key matrix and the cross value matrix to obtain a cross attention weight result; The cross attention unit is used to concatenate multiple cross attention weight results to obtain the cross attention result: V=M cross E (V up ,V down ); Among them, V represents the cross attention result, M cross E represents the splicing calculation, V up represents the feature vector of the dentition to be completed, V down Represents the eigenvector of the opposing teeth.

5. The method for repairing craniomaxillofacial hard tissue loss based on a universal repair model according to claim 2, characterized in that: The expression of the query embedding vector is: Q=MLP([C,Cur C ,M(Linear(V)]); C = P(M(Linear(V))); Among them, Q represents the query embedding vector, MLP represents the multi-layer perceptron, C represents the sparse point coordinates, Cur C Represents the local curvature of C, M represents the maximum pooling operation, Linear represents the linear layer, V represents the result of cross attention, and P represents coordinate projection.

6. The method for repairing craniomaxillofacial hard tissue loss based on a universal repair model according to claim 2, characterized in that: The decoder module includes a plurality of multi-head attention layers, wherein the multi-head attention layers in the decoder module include: A linear transformation unit is used to perform a linear transformation on the feature vector of the dentition to be completed to obtain a decoding value matrix and a decoding key matrix; and to perform a linear transformation on the cross attention result to obtain a first-layer decoding query matrix; An attention weight unit, used to calculate the attention weight based on the decoding query matrix, the decoding key matrix and the decoding value matrix to obtain a decoding attention weight result; Multi-head attention unit, used to concatenate multiple decoding attention weight results to obtain a decoding concatenation vector; The geometric perception unit is used to perform maximum pooling on the feature vector of the dentition to be completed and the cross-attention result, and connect them with the decoding splicing vector to obtain a connection result as the decoding query matrix of the next layer; the connection result of the last layer is used as the missing prediction data.

7. The method for repairing craniomaxillofacial hard tissue loss based on a universal repair model according to claim 1, characterized in that: The expression of the weighted loss function is: L all =β·J0+(1-β-α)·J1+α·L emd ; Among them, L all represents the loss function result, J0 represents the sparse point chamfer distance loss function, α and β represent weight factors, J1 represents the dense point chamfer distance loss function, L emd represents the earth moving distance loss function, represents the minimum value of the bijective mapping, G represents the ground truth complete point cloud, p represents the point of the complete point cloud, P represents the point in the predicted point cloud, φ:P→G represents the bijective mapping, ||·||2 represents the Euclidean distance; α+β=1, α>0, β>0.

8. The method for repairing craniomaxillofacial hard tissue loss based on a universal repair model according to claim 1, characterized in that: The S3 includes: Performing distribution modeling on the missing repair point cloud features to obtain noise convolution distribution; Based on the noise convolution distribution, score calculation is performed to obtain a noise convolution distribution score; Based on the noise convolution distribution score, the missing repair point cloud feature is denoised, and the noise points are moved to a distribution pattern corresponding to the underlying clean surface to obtain the denoised missing repair point cloud feature.

9. The method for repairing craniomaxillofacial hard tissue loss based on a universal repair model according to claim 1, characterized in that: The S4 includes: Using the missing repair point cloud features after denoising, point cloud mesh modeling is performed to obtain a dense point cloud; Predicting the dense point cloud to obtain a corresponding unsigned distance field; By extracting the triangular mesh of the unsigned distance field and performing mesh reconstruction, the repair result of the craniomaxillofacial hard tissue loss is obtained, and the repair of the craniomaxillofacial hard tissue loss is completed.

Images