Method for Detecting and Repairing Maxillofacial CT Fractures Guided by Geometric Structure Priors

Abstract

The present invention discloses a method for detecting and repairing maxillofacial CT fractures guided by geometric structure priors, which mainly solves the problems that existing methods are not applicable to the repair of fractured bones and cannot better restore the occlusal function of patients. The implementation scheme is as follows: Obtain the CT data of maxillofacial fracture patients, construct a fracture localization data set and a bone contour data set, and train a fracture detection model and a bone contour segmentation model respectively based on these; construct a simulated fractured defect bone contour training data set and a fractured bone test data set according to the trained models; construct a maxillofacial contour repair model based on geometric prior-guided self-attention learning, and use the simulated fractured defect bone contour training data set to train it using an adaptive learning rate optimization algorithm; input the fractured bone test data set into the trained maxillofacial contour repair model based on geometric prior-guided self-attention learning to simulate the normal facial bone contour. The present invention can effectively restore the occlusal function of facial fracture patients and can be used for the repair of the bones of facial fracture patients.

Description

Technical Field

[0001] The present invention belongs to the technical field of medical image processing, and particularly relates to a method for detecting and repairing maxillofacial CT fractures, which can be used to provide a reference for the reduction of maxillofacial fracture fragments and assist doctors in preoperative planning. Background Art

[0002] Patients with maxillofacial fractures usually have changed facial appearances and chewing difficulties, and suffer great psychological pain. Open reduction and internal fixation of the maxillofacial region can reduce the fracture fragments of the patients, thereby restoring the skeletal function and facial appearance of the patients to the greatest extent. Since the bones before the patients' fractures cannot be restored, this kind of fixation usually requires preoperative simulation planning. Doctors often rely on mirror image information and personal experience to segment and reduce the broken bones through digital surgical software, and simulate the reduction position to form a surgical navigation plan. However, this simulation plan mainly based on doctors' experience lacks the reference of normal bones, resulting in deviations in the reduction work, and it is necessary to repeatedly simulate the surgery to adjust the best surgical effect. This will not only cause a large amount of time consumption and delay the treatment of patients, but also lead to poor surgical effects and poor postoperative effects of patients. Therefore, there is an urgent need for an effective method to repair the CT images of fracture patients, accurately simulate the normal bone contours of fracture patients, provide a reference for the reduction of maxillofacial fracture fragments, and assist doctors in better preoperative planning.

[0003] At present, some scholars have conducted research on the preoperative planning of craniofacial deformity correction and the repair of maxillofacial tumors. Xiao Deqiang et al. proposed a supervised deformation network, Deformation Network (DefNet), in the journal IEEE JOURNAL OF BIOMEDICAL AND HEALTH INFORMATICS. It uses the PointNet network of point clouds to predict the displacement vectors of points, which are then superimposed on the original deformed bone to obtain the corrected normal bone. Since this method requires paired data and there is a lack of paired data in practice, the author's team then proposed an unsupervised surface deformation network, SDNet, in the journal MEDICAL PHYSICS. The deformed bones of patients and multiple normal bones are respectively input into the SDNet to predict multiple corrected normal bones, forming a sparse dictionary of normal bones for specific patients. The final corrected bone is obtained through sparse representation learning. However, the sparse representation fitting ability of this method is poor, and the database is too small, which is likely to lead to failure. In response to this, the author's team proposed a self-supervised learning method at the 24th International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI). By using a simulator, the real deformed mandible and the real midfacial bone are naturally fused to simulate the deformed bone, and then a corrector is used to simulate the corrected normal bone, greatly improving the sparse representation ability of the model. However, the above methods are only applicable to the situation where the occurrence site of craniofacial deformities is relatively fixed, the deformity types are relatively single, and there are fixed rules. In reality, the occurrence site of maxillofacial fractures may be located throughout the face, and the sizes and numbers of fracture fragments vary, that is, the occurrence of real fracture lines is random and there is no fixed rule to follow. Therefore, the planning directly using a single bone displacement vector prediction network before craniofacial deformity correction cannot be used for fracture reduction surgery.

[0004] In the aspect of maxillofacial tumor repair, Bimeng Jie et al. proposed an ICP iterative closest point registration method in the article published in the journal "Clinical Oral Investigations". They selected the sample closest to the defective bone from the normal facial bone database as the repair reference standard. Similarly, Boxuan Han et al. proposed a statistical shape model in the article published in the journal "International Journal of Computer Assisted Radiology and Surgery". They constructed a normal facial bone database, matched the defective bone with the normal bone through non-rigid registration, and completed the repair of the defective bone by combining marker point positioning. However, since these methods only used the normal facial bone database and failed to utilize the patient's own personalized information, they were only applicable to the repair of tumors with a single defective morphology and could not be used for fractured bones with diverse fracture line morphologies.

[0005] Due to the failure to consider the unique characteristics of diverse fractured bone morphologies, the above-mentioned existing methods are not applicable to the repair of fractured bones. In addition, since these methods only consider the restoration of the patient's facial bone structure and do not consider the actual situations such as the dislocation of mandibular fracture fragments and Lefort III fractures that can cause the loss of the patient's occlusal function, they cannot restore the patient's occlusal function well, and their practical application value is limited. Summary of the Invention

[0006] The purpose of the present invention is to address the deficiencies of the above-mentioned existing technologies and propose a method for maxillofacial CT fracture detection and repair guided by geometric structure prior, so as to make full use of the prior information of the curvature of facial bones, simulate and restore the normal bones of the patient before fracture, realize the repair of fractured bones, and better restore the patient's occlusal function.

[0007] To achieve the above purpose, the technical solution of the present invention includes the following steps:

[0008] (1) Obtain the computed tomography (CT) data of a maxillofacial fracture patient, and respectively obtain the two-dimensional CT slices and their manually annotated rectangular box xml files from the CT data to form a fracture location data set; manually outline the maxillofacial contour labels and form a bone contour data set together with the CT data; then downsample the CT data to a unified resolution of 128×128×96 and adjust it to the window level and window width of the bone to obtain the preprocessed three-dimensional CT data;

[0009] (2) Input the fracture localization dataset into the existing object detection Faster-RCNN network for iterative training. Use the error backpropagation mechanism to update the weights W and biases b of the Faster-RCNN network until the number of iterations reaches 50, then stop training to obtain the fracture detection model DET;

[0010] (3) Input the bone contour dataset into the existing 3D UNet network for iterative training. Use the error backpropagation mechanism to update the weights W' and biases b' of the 3D UNet network until the number of iterations reaches 200, then stop training to obtain the bone contour segmentation model SEG;

[0011] (4) Construct a normal bone CT training dataset, a simulated fracture defect bone contour training dataset, and a fractured bone test dataset:

[0012] (4a) Select at least 100 cases of maxillofacial CT data of normal people and patients with small lesion cysts to form a normal bone CT training dataset;

[0013] (4b) Input the normal bone CT training dataset in (4a) into the trained bone contour segmentation model SEG in (3) to obtain the facial bone contour. Make a random blank mask mask according to the fracture-prone positions and regional sizes to cover this normal facial bone contour, and jointly form a simulated fracture defect bone contour training dataset with the masked simulated fracture defect bone contour and the normal bone contour label;

[0014] (4c) Input the CT data of maxillofacial fracture patients in (1) into the trained fracture detection model DET in (2) to detect the coordinates of the bounding box of the fracture occurrence position, and use a blank mask mask to cover the fracture area formed by this coordinate to form a fractured bone test dataset;

[0015] (5) Respectively establish an encoder composed of 16 feature extraction Transformer blocks and a decoder composed of 4 layers of convolutional neural network CNN. Connect the encoder and the decoder, and add a geometric prior guidance module based on the 3D Sobel convolutional edge map to the last layer of the decoder to form a maxillofacial contour repair model REC based on geometric prior-guided self-attention learning;

[0016] (6) Iteratively train the maxillofacial contour repair model REC based on geometric prior-guided self-attention learning with the simulated fracture defect bone contour training dataset:

[0017] (6a) Shuffle the order of the images in the simulated fracture defect bone contour training dataset in (4b), and sequentially select a single simulated defect bone and its normal bone label from it, denoted as a training batch {I, G, R, Z}, where I is the simulated defect bone in a training batch, G is the normal bone label corresponding to I, R is the three-dimensional region of interest (ROI) coordinate information of the mask, and Z is the slice coordinate information of the mandible;

[0018] (6b) Input the simulated defect bone I into the maxillofacial contour repair model REC based on geometric prior-guided self-attention learning. Through the forward propagation of the maxillofacial contour repair model REC based on geometric prior-guided self-attention learning, obtain the output simulated normal bone O, simulated normal bone edge map OE, and normal bone label edge map GE;

[0019] (6c) Map the mask coordinate information R onto the simulated normal bone O and the normal bone label G to obtain the simulated normal bone OL in the defect area and the normal bone label GL in the defect area; Map the mandible coordinate information Z onto the normal bone edge map OE and the normal bone label edge map GE to obtain the simulated normal mandible edge map OE and the normal mandible label edge map GE;

[0020] (6d) Calculate the loss L between the simulated normal bone O and the normal bone label G global , the loss L between the simulated normal bone OL in the defect area and the normal bone label GL in the defect area local , and the edge loss L between the simulated normal mandible edge map OE and the normal mandible label edge map GE edge , and obtain the loss function L of the maxillofacial contour repair model REC based on geometric prior-guided self-attention learning as L = L global + L local + L edge ;

[0021] (6e) Set the initial learning rate to 0.001, use the adaptive learning rate optimization algorithm Adam to optimize the loss function L of the maxillofacial contour repair model REC based on geometric prior-guided self-attention learning, and use the error backpropagation mechanism to update the weights W and biases b of the maxillofacial contour repair model REC based on geometric prior-guided self-attention learning;

[0022] (6f) Repeat (5a) to (5e) until the loss function L does not decrease, then stop training to obtain the trained maxillofacial contour repair model REC based on geometric prior-guided self-attention learning;

[0023] (7) Input the fractured bone test dataset in (4c) into the trained maxillofacial contour repair model REC based on geometric prior-guided self-attention learning to obtain the repaired simulated normal facial bone contour.

[0024] The present invention has the following advantages compared with the prior art:

[0025] 1. The present invention integrates the facial fracture detection model into the maxillofacial contour repair model, constructs a simulated fracture defect sample training set that conforms to the diverse defect morphologies of fractured bones for the maxillofacial contour repair model, and uses the facial fracture detection model to automatically locate the real fracture position to construct a real fracture defect sample test set. Through the two-stage neural network structure of first locating and then repairing, a repair effect more suitable for real fractured bones can be achieved.

[0026] 2. The present invention constructs a maxillofacial contour repair model based on geometric prior-guided self-attention learning, which can make full use of the arc prior information of the facial bone contour, can better restore the occlusal function of patients, and is more in line with the actual clinical needs. Brief Description of the Drawings

[0027] Figure 1 is the implementation flowchart of the present invention;

[0028] Figure 2 is a maxillofacial CT image obtained clinically in the present invention;

[0029] Figure 3 is the fracture detection box label and facial bone contour label map hand-drawn in the present invention;

[0030] Figure 4 is the result map of fracture detection of the CT image using the present invention;

[0031] Figure 5 is the result map of facial contour segmentation of the CT image using the present invention;

[0032] Figure 6 is the result map of repairing the fractured bone contour using the present invention. Detailed Embodiments

[0033] The following further explains and illustrates the specific implementation schemes and effects of the present invention with reference to the drawings.

[0034] Refer to Figure 1 , the implementation steps of this example are as follows:

[0035] Step 1, CT data acquisition.

[0036] Obtain the computed tomography (CT) data of maxillofacial fracture patients from the hospital, such as Figure 2As shown, two-dimensional CT slices and their manually annotated rectangular box xml files are respectively obtained from the CT data, such as Figure 3 As shown, where 3(a) constitutes the fracture location dataset A; 3(b) is the manually outlined jaw contour label map, which together with the CT data constitutes the bone contour dataset B;

[0037] Then, the CT data is downsampled to a unified resolution of 128×128×96 and adjusted to the window level and window width of the bone to obtain the preprocessed three-dimensional CT data.

[0038] Step 2: Use the fracture location dataset A to train the existing object detection Faster-RCNN network to obtain the trained fracture detection model DET.

[0039] (2.1) Set the total number of iterations to 50;

[0040] (2.2) Set the loss function L of the fracture detection network as follows:

[0041]

[0042] where L cls is the classification loss between the prediction result and the true fracture label, L reg is the regression loss between the predicted fracture bounding box and the true fracture bounding box, λ is the balance weight, p i is the probability that the candidate box is predicted as a fracture, is the true fracture label, t i ={t x ,t y ,t w ,t h} are the four parametric coordinates of the predicted fracture bounding box, are the four parametric coordinates of the true fracture bounding box label;

[0043] (2.3) Input the fracture location dataset A into the existing Faster-RCNN network to obtain the detection result;

[0044] (2.4) Substitute the true label of the fracture location dataset A and the predicted label of the detection result into the loss function L in (2.2) to calculate the loss value of the network;

[0045] (2.5) Backpropagate the loss value calculated in (2.4);

[0046] (2.6) Repeat (2.1)-(2.5) until the number of iterations reaches 50, stop training, and obtain the trained fracture detection model.

[0047] Step 3: Use the bone contour dataset B to train the existing 3D UNet network to obtain a trained bone contour segmentation model SEG.

[0048] (3.1) Set the total number of iterations to 200;

[0049] (3.2) Set the Dice loss as the loss function L of the segmentation network dice :

[0050]

[0051] where \(O_{bone}\) i and \(G_{bone}\) i respectively represent the \(i\)-th channel of the predicted bone contour \(O_{bone}\) and the bone contour label \(G_{bone}\), \(T\), \(U\), and \(H\) respectively represent the length, width, and height of the input image, and \(s\), \(w\), and \(h\) respectively represent the position indices of the length, width, and height in the input image;

[0052] (3.3) Input the bone contour dataset B into the existing 3D UNet network to obtain a segmentation result;

[0053] (3.4) Substitute the ground truth label of the bone contour dataset B and the predicted label of the segmentation result into the loss function L in (3.2) dice , and calculate the loss value of the network;

[0054] (3.5) Backpropagate the loss value calculated in (3.4);

[0055] (3.6) Repeat (3.1)-(3.5) until the number of iterations reaches 200, stop training, and obtain a trained bone contour segmentation model.

[0056] Step 4: Construct a normal bone CT training dataset C, a simulated fracture defect bone contour training dataset D, and a fractured bone test dataset E.

[0057] (4.1) Select at least 100 cases of maxillofacial CT data of normal people and patients with small lesion cysts to form a normal bone CT training dataset C;

[0058] (4.2) Construct a simulated fracture defect bone contour training dataset D;

[0059] 4.2.1) Input the normal bone CT training dataset C into the trained bone contour segmentation model SEG in Step 3 to obtain a normal facial bone contour;

[0060] 4.2.2) On the normal facial bone contour, according to the multiple fracture positions, it is roughly divided into the mandibular body, mandibular angle, condylar process top, zygomatic arch, maxillary Lefort I, II, III type fracture regions, and fracture-prone parts such as the orbit. Combining the size of the regions with multiple fractures in specific parts, a random blank mask mask is made to cover the fracture lines with diverse shapes in the fracture region, simulating the fracture defect situation;

[0061] 4.2.3) The simulated fracture defect bone contour after masking and the normal bone contour label together constitute the simulated fracture defect bone contour training dataset D;

[0062] (4.3) Input the computed tomography (CT) data of the maxillofacial fracture patient shown in Figure 2 into the fracture detection model DET trained in step 2, detect the coordinates of the boundary box of the fracture occurrence position, and use the blank mask mask to cover the fracture region formed by this coordinate, constituting the fracture bone test dataset E.

[0063] Step 5, construct the maxillofacial contour repair model REC based on geometric prior-guided self-attention learning.

[0064] (5.1) Establish the encoder of the maxillofacial contour repair model REC based on geometric prior-guided self-attention learning, which consists of 16 basic Transformer blocks: Each basic Transformer block of the encoder is composed of two layer normalization layers, a multi-head attention layer, and a multi-layer perceptron layer. Among them, the input of the first layer normalization layer is skip-connected to the output of the multi-head attention layer, and the input of the second layer normalization layer is skip-connected to the output of the multi-layer perceptron layer;

[0065] (5.2) Construct a four-layer decoder:

[0066] Establish the first decoding layer composed of three deconvolution layers with a convolution kernel size of 3×3×3 and a stride of 2×2×2 and two residual layers with a convolution kernel size of 3×3×3 and a stride of 1×1×1 connected in series;

[0067] Establish the second decoding layer composed of two deconvolution layers with a convolution kernel size of 3×3×3 and a stride of 2×2×2 and one residual layer with a convolution kernel size of 3×3×3 and a stride of 1×1×1 connected in series;

[0068] Establish the third decoding layer composed of one deconvolution layer with a convolution kernel size of 3×3×3 and a stride of 2×2×2;

[0069] Build the fourth decoding layer, which consists of a transposed convolution layer with a convolution kernel size of 3×3×3 and a stride of 2×2×2, and three 3D Sobel convolution layers with a convolution kernel size of 3×3×3 and a convolution stride of 1×1×1, a Sigmoid activation layer, and a normalization layer connected serially.

[0070] Connect the first decoding layer, the second decoding layer, the third decoding layer, and the fourth decoding layer in parallel to form the REC decoder of the maxillofacial contour repair model based on geometric prior-guided self-attention learning.

[0071] Step 6: Iteratively train the maxillofacial contour repair model REC based on geometric prior-guided self-attention learning using the simulated fracture defect bone contour training dataset.

[0072] (6.1) Shuffle the order of the images in the simulated fracture defect bone contour training dataset D, and sequentially select a single simulated defect bone and its normal bone label from it, denoted as a training batch {I, G, R, Z}, where I is the simulated defect bone in a training batch, G is the normal bone label corresponding to I, R is the three-dimensional region of interest (ROI) coordinate information of the mask, and Z is the slice coordinate information of the mandible.

[0073] (6.2) Input the simulated defect bone I into the maxillofacial contour repair model REC based on geometric prior-guided self-attention learning constructed in step (5), and obtain the output simulated normal bone O, simulated normal bone edge map OE, and normal bone label edge map GE through the forward propagation of the network.

[0074] (6.3) Obtain the simulated normal bone OL in the defect area, the normal bone label GL in the defect area, the simulated normal mandible edge map OE, and the normal mandible label edge map GE:

[0075] 6.3.1) Map the mask coordinate information R to the simulated normal bone O and the normal bone label G to obtain the simulated normal bone OL in the defect area and the normal bone label GL in the defect area:

[0076] GL = G(s_b:s_e, h_b:h_e, w_b:w_e)

[0077] where s_b and s_e respectively represent the starting and ending coordinates of the fracture ROI area in the length dimension, w_b and w_e respectively represent the starting and ending coordinates of the fracture ROI area in the width dimension, and h_b and h_e respectively represent the starting and ending coordinates of the fracture ROI area in the height dimension.

[0078] 6.3.2) Map the mandible coordinate information Z onto the normal bone edge map OE and the normal bone label edge map GE to obtain the simulated normal mandible edge map OE and the normal mandible label edge map GE:

[0079] GE_m = GE(:,:,z_b:z_e)

[0080] OE_m = OE(:,:,z_b:z_e)

[0081] Among them, z_b and z_e respectively represent the starting and ending coordinates of the mandible slice coordinate range.

[0082] (6.4) Construct the loss function L of the maxillofacial contour repair model REC based on geometric prior-guided self-attention learning:

[0083] 6.4.1) Calculate the loss L between the simulated normal bone O and the normal bone label G global :

[0084]

[0085] Among them, O i and G i respectively represent the i-th channels of the simulated normal bone result O and the normal bone label G, T, U, and H respectively represent the length, width, and height of the input image, and s, w, and h respectively represent the position indices of the length, width, and height in the input image.

[0086] 6.4.2) Calculate the loss L between the simulated normal bone OL in the defect area and the normal bone label GL in the defect area local :

[0087]

[0088] Among them, OL i and GL i respectively represent the i-th channels of the simulated normal bone OL in the defect area and the normal bone label GL in the defect area.

[0089] 6.4.3) Calculate the edge loss L between the simulated normal mandible edge map OE and the normal mandible label edge map GE edge :

[0090]

[0091] Among them, OE i and GE i respectively represent the i-th channels of the simulated normal mandible edge map OE and the normal mandible label edge map GE.

[0092] 6.4.4) According to Lglobal and L local and L edge , the loss function L of the maxillofacial contour repair model REC based on geometric prior-guided self-attention learning is obtained as follows:

[0093] L = L global + L local + L edge ;

[0094] (6.5) Suppose the initial learning rate is 0.001. Use the Adam optimization algorithm with adaptive learning rate to optimize the loss function L of the maxillofacial contour repair model REC based on geometric prior-guided self-attention learning, and use the error backpropagation mechanism to update the weights W' and biases b' of the maxillofacial contour repair model REC based on geometric prior-guided self-attention learning;

[0095]

[0096]

[0097] where i ∈ {1, 2, 3, 4, 5, 6}, denotes the gradient of W i , denotes the gradient of b i , and · denotes the dot product.

[0098] (6.6) Repeat (6.1) to (6.5) until the loss function L no longer decreases, then stop the training to obtain the trained maxillofacial contour repair model REC based on geometric prior-guided self-attention learning.

[0099] Step 7: Input the fractured bone test data set into the trained maxillofacial contour repair model REC based on geometric prior-guided self-attention learning to obtain the repaired simulated normal facial bone contour.

[0100] The effects of the present invention can be obtained through the following simulation experiments:

[0101] I. Simulation conditions

[0102] The simulation experiment platform of the present invention is the Win10 operating system, configured with an Intel Core i7-6900K CPU at 3.2 GHz and an NVIDIA GeForce GTX 1080 Ti GPU, using the pytorch deep learning framework, and the development language is python.

[0103] II. Simulation content

[0104] Simulation 1: Using the fracture detection model DET in the present invention to locate a fractured bone test sample in the fracture location dataset A. The results are as follows Figure 4 .

[0105] As can be seen Figure 4 from this, the sample had an "M"-shaped fracture at the position of the right zygomatic bone and zygomatic arch, indicating that the fracture detection model DET of the present invention can accurately locate the occurrence position of the fracture and display the corresponding rectangular box.

[0106] Simulation 2: Using the bone contour segmentation model SEG in the present invention to segment the facial bones of a normal bone sample in the normal bone CT training dataset C. The results are as follows Figure 5 .

[0107] As can be seen Figure 5 from this, the bone contour segmentation model SEG of the present invention can accurately segment the maxilla, upper teeth, lower teeth, mandible, and condyle of the sample, indicating that the bone contour segmentation model SEG of the present invention can completely segment the bone contour of the sample.

[0108] Simulation 3: Using the maxillofacial contour repair model REC based on geometric prior-guided self-attention learning in the present invention to repair a fractured bone sample in the fractured bone test dataset E. The results are as follows Figure 6 , where Figure 6(a) is the contour of the fractured bone and Figure 6(b) is the contour of the repaired bone.

[0109] As can be seen Figure 6 (a) from this, due to the severe dislocation of the fractured bone blocks caused by the mandibular fracture, the occlusal function of the sample patient was lost. The maxillofacial contour repair model REC based on geometric prior-guided self-attention learning in the present invention can simulate the normal bone of the patient before the fracture. As can be seen Figure 6 (b) from this, the maxillofacial contour repair model REC based on geometric prior-guided self-attention learning in the present invention can simulate and restore the normal mandibular curvature of the patient, thereby better restoring the occlusal function of the patient.

[0110] The above description is only a specific example of the present invention and does not constitute any limitation to the present invention. Obviously, for professionals in the field, after understanding the content and principle of the present invention, various corrections and changes in form and details may be made without departing from the principle and structure of the present invention. However, these corrections and changes based on the idea of the present invention are still within the protection scope of the claims of the present invention.

Claims

1. A method for maxillofacial CT fracture detection and repair guided by geometric structure prior, including the following: (1) Obtain the computed tomography (CT) data of maxillofacial fracture patients, and respectively obtain two-dimensional CT slices and their manually labeled rectangular box xml files from the CT data to form a fracture localization data set; manually outline the maxillofacial contour labels and form a bone contour data set together with the CT data; then downsample the CT data to a unified resolution of 128×128×96 and adjust it to the window level and window width of the bone to obtain the preprocessed three-dimensional CT data; (2) Input the fracture localization data set into the existing object detection Faster-RCNN network for iterative training, and use the error backpropagation mechanism to update the weights W and biases b of the Faster-RCNN network until the number of iterations reaches 50, then stop training to obtain the fracture detection model DET; (3) Input the bone contour data set into the existing 3D UNet network for iterative training, and use the error backpropagation mechanism to update the weights W' and biases b' of the 3D UNet network until the number of iterations reaches 200, then stop training to obtain the bone contour segmentation model SEG; (4) Construct a normal bone CT training data set, a simulated fracture defect bone contour training data set, and a fracture bone test data set: (4a) Select at least 100 cases of maxillofacial CT data of normal people and patients with small lesion cysts to form a normal bone CT training data set; (4b) Input the normal bone CT training data set in (4a) into the trained bone contour segmentation model SEG in (3) to obtain the facial bone contour. Make a random blank mask mask according to the fracture-prone positions and regional sizes to cover the normal facial bone contour, and form a simulated fracture defect bone contour training data set together with the covered simulated fracture defect bone contour and the normal bone contour labels; (4c) Input the CT data of maxillofacial fracture patients in (1) into the trained fracture detection model DET in (2) to detect the coordinates of the fracture occurrence position bounding box, and use the blank mask mask to cover the fracture area formed by this coordinate to form a fracture bone test data set; (5) Respectively establish an encoder composed of 16 feature extraction Transformer blocks, a decoder composed of 4 layers of convolutional neural network (CNN), connect the encoder and the decoder, and add a geometric prior guidance module based on the 3D Sobel convolutional edge map to the last layer of the decoder to form a maxillofacial contour repair model REC based on geometric prior-guided self-attention learning; (6) Iteratively train the maxillofacial contour repair model REC based on geometric prior-guided self-attention learning with the simulated fracture defect bone contour training data set; (6a) Shuffle the order of the images in the simulated fracture defect bone contour training dataset in (4b), and sequentially select a single simulated defect bone and its normal bone label therefrom, denoted as a training batch {I, G, R, Z}, where I is the simulated defect bone in a training batch, G is the normal bone label corresponding to I, R is the three-dimensional region of interest ROI coordinate information of the mask, and Z is the slice coordinate information of the mandible; (6b) Input the simulated defect bone I into the maxillofacial contour repair model REC based on geometric prior-guided self-attention learning. Through the forward propagation of the normal bone contour simulation network, obtain the output simulated normal bone O, simulated normal bone edge map OE, and normal bone label edge map GE; (6c) Map the mask coordinate information R to the simulated normal bone O and the normal bone label G to obtain the simulated normal bone OL in the defect area and the normal bone label GL in the defect area; Map the mandible coordinate information Z to the normal bone edge map OE and the normal bone label edge map GE to obtain the simulated normal mandible edge map OE and the normal mandible label edge map GE; (6d) Calculate the loss L between the simulated normal bone O and the normal bone label G global , the loss L between the simulated normal bone OL in the defect area and the normal bone label GL in the defect area local , and the edge loss L between the simulated normal mandibular edge map OE and the normal mandibular label edge map GE edge , to obtain the loss function L = L of the maxillofacial contour repair model REC guided by geometric prior-guided self-attention learning global + L local + L edge ; (6e) Set the initial learning rate to 0.001, use the adaptive learning rate optimization algorithm Adam to optimize the loss function L of the maxillofacial contour repair model REC based on geometric prior-guided self-attention learning, and use the error backpropagation mechanism to update the weights W and biases b of the maxillofacial contour repair model REC based on geometric prior-guided self-attention learning; (6f) Repeat (5a) to (5e) until the loss function L does not decrease, then stop training to obtain the trained maxillofacial contour repair model REC based on geometric prior-guided self-attention learning; (7) Input the fracture bone test dataset in (4c) into the trained maxillofacial contour repair model REC based on geometric prior-guided self-attention learning to obtain the repaired simulated normal facial bone contour.

2. The method according to claim 1, wherein step (2) uses the fracture location dataset to train the existing object detection Faster-RCNN network, and the implementation is as follows: (2.1) Set the total number of iterations to 50; (2.2) Set the loss function L of this fracture detection network as follows: Among them, L cls is the classification loss between the predicted result and the true fracture label, reg is the regression loss between the predicted fracture bounding box and the true fracture bounding box, λ is the balancing weight, and p i is the probability that the candidate box is predicted to be a fracture, is the true fracture label, and t i is the coordinate vector of the predicted fracture bounding box, is the coordinate vector of the true fracture bounding box label; (2.3) Input the fracture location dataset into the existing Faster-RCNN network to obtain the detection result; (2.4) Substitute the true label of the fracture location dataset and the predicted label of the detection result into the loss function L in (2.2) to calculate the loss value of the network; (2.5) Backpropagate the loss value calculated in (2.4); (2.6) Repeat steps (2.1)-(2.5) until the number of iterations reaches 50, then stop training to obtain the trained fracture detection model DET.

3. The method according to claim 1, wherein step (3) uses the bone contour dataset to train the existing 3D UNet network, and the implementation is as follows: (3.1) Set the total number of iterations to 200; (3.2) Set the Dice loss as the loss function L of this segmentation network dice : Among them, O _bone i and G_bone i respectively represent the i-th channel of the predicted bone contour O_bone and the bone contour label G_bone. T, U, and H respectively represent the length, width, and height of the input image, and s, w, and h respectively represent the position indices of the length, width, and height in the input image; (3.3) Input the bone contour dataset into the existing 3D UNet network to obtain the segmentation result; (3.4) Substitute the ground truth labels of the bone contour dataset and the predicted labels of the segmentation results into the loss function \(L\) in (3.2). dice Calculate the loss value of the network. (3.5) Backpropagate the loss value calculated in (3.4); (3.6) Repeat (3.1)-(3.5) until the number of iterations reaches 200, stop training, and obtain the trained bone contour segmentation model SEG.

4. The method according to claim 1, wherein each basic Transformer block constituting the encoder in step (5) is composed of two layer normalization layers, a multi-head attention layer, and a multi-layer perceptron layer, wherein the input of the first layer normalization layer is skip-connected to the output of the multi-head attention layer, and the input of the second layer normalization layer is skip-connected to the output of the multi-layer perceptron layer.

5. The method according to claim 1, wherein the 4-layer convolutional neural network CNN structure constituting the decoder in step (5) is as follows: The first layer is serially connected by three transposed convolution layers with a kernel size of 3×3×3 and a stride of 2×2×2 and two residual layers with a kernel size of 3×3×3 and a stride of 1×1×1; The second layer is serially connected by two transposed convolution layers with a kernel size of 3×3×3 and a stride of 2×2×2 and one residual layer with a kernel size of 3×3×3 and a stride of 1×1×1; The third layer is composed of one transposed convolution layer with a kernel size of 3×3×3 and a stride of 2×2×2; The fourth layer is serially connected by one transposed convolution layer with a kernel size of 3×3×3 and a stride of 2×2×2 and three 3D Sobel convolution layers, a Sigmoid activation layer, and a normalization layer with a kernel size of 3×3×3 and a convolution stride of 1×1×1.

6. The method according to claim 1, wherein the loss L between the simulated normal bone O and the normal bone label G in step (6d) is expressed as follows: global , is as follows: Among them, O i and G i respectively represent the i-th channel of the simulated normal bone result O and the normal bone label G. T, U, and H respectively represent the length, width, and height of the input image, and s, w, and h respectively represent the position indices of the length, width, and height in the input image.

7. The method according to claim 1, wherein the loss L between the normal bone OL simulated in the defect area and the normal bone label GL in the defect area in step (6d) is expressed as follows: local , which is expressed as follows: Among them, OL i and GL i respectively represent the i-th channel of the normal bone OL in the defect area and the normal bone label GL in the defect area.

8. The method according to claim 1, wherein the edge loss L between the simulated normal mandibular edge map OE and the normal mandibular label edge map GE in step (6d) is expressed as follows: edge , as follows: Among them, O E i and GE i respectively represent the i-th channel of the simulated normal mandibular edge map OE and the normal mandibular label edge map GE.

Images