Facial form prediction method after orthognathic operation

Abstract

The invention discloses a facial form prediction method after orthognathic operation. The method comprises the following steps: firstly, acquiring preoperative three-dimensional CBCT data and a bone mark point displacement vector of a patient, and converting the three-dimensional data into a two-dimensional depth image through spatial registration, ROI cutting and projection transformation; an MCA-UNet model is provided, depth image features are extracted by using a UNet network, cross-modal fusion is performed on the image features and mark point embedded features through a multi-modal cross attention (MCA) module, and the nonlinear driving effect of skeleton displacement on soft tissue is modeled. And performing model prediction to generate postoperative and preoperative difference images, superposing the difference images to the preoperative image and reversely mapping the difference images to a three-dimensional space, and performing reconstruction to obtain a predicted postoperative three-dimensional facial grid. The method effectively solves the problems that in the prior art, the calculated amount is large, and nonlinear deformation prediction is not accurate, high-precision prediction with the whole face average error smaller than 2 mm is achieved, and a visual and reliable auxiliary tool is provided for orthognathic surgery planning.

Description

Technical Field

[0001] This invention relates to the field of medical image processing and computer-aided surgery, and in particular to the application of deep learning technology in oral and maxillofacial surgery. Specifically, this invention relates to a method for predicting facial morphology after orthognathic surgery based on depth images and a multimodal cross-attention mechanism. Background Technology

[0002] Modern orthognathic surgery aims to improve facial aesthetics and oral function in patients with severe dentofacial deformities. Due to the complexity of facial anatomy, surgical planning increasingly relies on computer-aided surgical simulation (CASS) systems to improve accuracy, with virtual osteotomy being a typical application. This process typically involves: (1) reconstructing a three-dimensional skeletal model from cone-beam computed tomography (CBCT) data; (2) virtually segmenting the model using osteotomy; and (3) repositioning bone segments to correct dentofacial deformities. By simulating these movements, CASS can generate predicted postoperative skeletal morphology.

[0003] However, although CASS aims to infer postoperative facial appearance through skeletal movement, the relationship between bone and soft tissue movement is extremely complex and influenced by nonlinear biomechanical interactions. These interactions do not present a direct proportional correspondence, and individual differences in soft tissue characteristics further limit the accuracy of appearance prediction based solely on skeletal movement. Accurately simulating postoperative facial soft tissue morphology is crucial for orthognathic surgery planning in patients with dentofacial deformities, directly impacting patient satisfaction and the surgeon's surgical approach.

[0004] The existing technologies mainly include the following two types of simulation methods: 1. Traditional Biomechanical Simulation Methods: For a long time, the classic approach to orthognathic surgery has relied on biomechanical simulation. Finite Element Models (FEMs) use patient-specific meshes to simulate deformations caused by skeletal movement; although FEMs possess physical accuracy, their computational demands are enormous and they are limited by strict mesh generation requirements, resulting in low feasibility for real-time clinical application. Mass Tensor Models (MTMs) approximate deformations through empirical tensor mappings, providing a more efficient alternative; however, due to the highly nonlinear nature of bone-soft tissue relationships, MTMs perform poorly in dynamic facial regions (such as perioral muscles). While these methods are effective in principle, they often require significant manual input and complex calculations, limiting their clinical scalability and practicality.

[0005] 2. Existing Prediction Methods Based on Deep Learning: With the rise of deep learning, data-driven methods have been proposed. For example, Park et al. used two-dimensional cephalometric landmarks to predict changes in lateral appearance, but lacked the ability to generalize to three-dimensional morphological prediction. Tanikawa et al. combined geometric morphometry (GMMs) with regressive neural networks to predict three-dimensional facial changes from a sparse set of bony landmarks; this hybrid method is suitable for predicting minor surgical adjustments, but struggles to capture complex large-scale deformations. Furthermore, existing three-dimensional prediction methods (such as ACMT-Net and P2P-Conv) typically suffer from the following drawbacks: First, they primarily use point cloud data to represent facial contours, limiting spatial continuity and visual intuitiveness; second, they overly rely on postoperative bones as input, requiring a perfect postoperative bone model to predict soft tissue, failing to achieve end-to-end prediction from surgical planning (displacement vector) to postoperative soft tissue; and third, they ignore the deep semantic relationships between bony landmarks and facial pixels, leading to loss of prediction details or artifacts in key areas (such as the lips and jaw). Summary of the Invention

[0006] This invention primarily addresses the technical problems of existing technologies, such as high computational load, low accuracy in predicting nonlinear soft tissue deformation, and difficulty in achieving end-to-end three-dimensional prediction. It provides a method for predicting facial morphology after orthognathic surgery. This method introduces the MCA-UNet architecture, converting a three-dimensional facial soft tissue mesh into a two-dimensional depth image to reduce computational complexity. Furthermore, it utilizes a cross-attention mechanism to deeply fuse preoperative depth images with bony landmarks and their displacement information, achieving accurate end-to-end prediction from preoperative to postoperative stages.

[0007] The present invention addresses the aforementioned technical problems primarily through the following technical solution: a method for predicting facial morphology after orthognathic surgery, the method comprising the following steps: S1: Acquire the patient's preoperative three-dimensional cone-beam computed tomography (CBCT) data, and obtain the coordinates L of the preset N preoperative bony landmarks. pre And the displacement vector ΔL of N bony landmarks determined based on the surgical plan; CBCT data forms the basis of three-dimensional medical imaging data, containing complete anatomical information of the patient's craniofacial region. The bony landmarks are anatomically significant features on the facial bones (e.g., the anterior nasal spine (ANS), the premental point (Pog), etc.), and the displacement vector ΔL represents the surgeon's designed surgical plan, i.e., the direction and distance of bone segment movement. By introducing the displacement vector, the model can learn the correlation between surgical manipulation and soft tissue deformation. In this scheme, N is set to 21.

[0008] S2: Convert CBCT data into preoperative two-dimensional depth images X PreOpThis step specifically includes: spatial registration and region of interest (ROI) cropping of CBCT data; extracting facial soft tissue contours from the cropped CBCT data using a threshold segmentation algorithm; defining a coronal plane passing through the anterior edge of the cervical spine as a reference projection plane; calculating the vertical distance from points on the soft tissue contour surface to the reference projection plane; and using the vertical distance as pixel values ​​to generate the preoperative two-dimensional depth image X. PreOp ; Given the enormous computational complexity of directly processing 3D voxels or point clouds, this invention compresses the 3D surface into a 2D depth image. Compared to processing 3D point clouds or voxels with tens of thousands of vertices, 2D images can be efficiently processed using mature image processing networks (such as UNet), significantly reducing computational complexity and memory usage. To address the issue of ambiguous geometric definitions, this invention explicitly uses the anatomically stable coronal plane of the anterior edge of the cervical spine as the projection reference (i.e., a plane with a depth of 0). This means that in the generated depth image, the larger the pixel value, the farther the location is from the cervical spine (i.e., the more prominent the face). Before projection, a threshold segmentation algorithm is used to remove bone and atmospheric background, retaining only the maximum connected component of the facial soft tissue, ensuring that the depth map only reflects the soft tissue morphology.

[0009] S3: Utilize the facial feature extractor (UNet network) of the deep learning model to process the preoperative two-dimensional depth image X. PreOp Feature extraction was performed to obtain the preoperative image feature map F. PreOp ; The deep learning model mentioned is the MCA-UNet architecture proposed in this solution; S4: Use the landmark feature extractor of the deep learning model to extract the coordinates L of the preoperative bony landmarks. pre The displacement vector ΔL is processed to extract the marker embedding feature F. landmark ; This step aims to transform discrete geometric coordinate information into high-dimensional semantic features that can be fused with image features.

[0010] S5: Utilize the multimodal cross-attention (MCA) module to fuse the preoperative image feature map F PreOp and the marker embedding feature F landmark The specific process is as follows: The preoperative image feature map F PreOp The marker points are embedded in feature F as the query vector Q. landmark As the key vector K and the value vector V; According to formula F attn =softmax(Q×K T The attention-weighted feature F is calculated by √(d / V). attn Where d is the feature dimension, KT Let K denote the transpose of matrix K, and softmax be the normalized exponential function. Given that image features are spatially dense while landmark features are spatially sparse, direct stitching can lead to information asymmetry. This invention employs a cross-attention mechanism to address this issue. By calculating the correlation between each image pixel and all landmark points, bone displacement information is dynamically injected into the corresponding soft tissue image region, achieving nonlinear modeling of bone-driven soft tissue.

[0011] S6: Apply the attention-weighted feature F attn Input a multilayer perceptron (MLP) to predict and generate postoperative and preoperative difference images X. diff The difference image X diff It characterizes the amount of change in facial soft tissue in the depth direction caused by bone displacement; This invention employs a differential learning strategy, whereby the network does not directly predict the complete postoperative face, but rather predicts the amount of change. Since most areas of the face (such as the forehead and eyes) undergo relatively small changes during orthognathic surgery, predicting the differential map allows the network to focus on areas with large changes, such as the mandible and perioral region, significantly improving prediction accuracy and accelerating convergence.

[0012] S7: According to formula X post =X PreOp +X diff The predicted postoperative two-dimensional depth image X was calculated. post ; S8: The predicted postoperative two-dimensional depth image X post Each pixel in the image is inversely mapped back to three-dimensional space by combining the reference projection plane and clipping parameters defined in step S2, and the predicted postoperative three-dimensional facial soft tissue mesh is reconstructed.

[0013] Preferably, in step S2, the specific range of the region of interest (ROI) cropping is: extending from the submental point to the supraorbital margin in the vertical direction, and extending from the tip of the nose to the anterior margin of the cervical spine in the anteroposterior direction; Furthermore, before generating the preoperative two-dimensional depth image, an outlier correction step is included: calculating the mean μ and standard deviation σ of all pixel values ​​in the image, and cropping pixel values ​​with an absolute difference |x-μ| greater than 3σ to a preset effective range, where x is the pixel value.

[0014] This operation can effectively remove noise or artifacts caused by the patient's slight movements during CBCT scanning.

[0015] Preferably, in step S5, before performing the formula calculation, a step of aligning the feature dimensions is also included: The preoperative image feature map F with dimensions B×C×H×WPreOp Flatten it in the spatial dimension and convert it into a sequence with dimensions B×(H×W)×C, where B is the batch size, C is the number of channels, and H and W are the height and width of the feature map; If the number of channels C of the preoperative image feature map is equal to the number of marker embedding features F... landmark If the feature dimension d is inconsistent, the number of channels C of the preoperative image feature map is mapped to d through a linear projection layer or a 1x1 convolutional layer to meet the dimension requirement for matrix multiplication of Q and K.

[0016] Preferably, in step S4, the marker embedding feature F is extracted using the marker feature extractor. landmark The steps include: S41: L pre Concatenating ΔL with ΔL yields the combined vector L. concat This vector contains both anatomical location and surgical changes. S42: Capturing L using a self-attention mechanism concat The spatial dependencies between various landmarks (e.g., how the movement of the maxilla affects the relative position of the mandible) are used to obtain contextual features; S43: Combine the context features with the original L concat Perform residual join, which is to add elements one by one, in order to preserve the high-frequency position information of the original coordinates; S44: Input the summed result into a multilayer perceptron (MLP), and project it onto an embedding space of dimension d through linear transformation, layer normalization (LayerNorm), and LeakyReLU activation function to obtain the marker embedding feature F. landmark .

[0017] Preferably, in step S6, the multilayer perceptron (MLP) generates a difference image X. diff The specific calculation formula is as follows: X diff =Sigmoid(FC2(ReLU(FC1(Fattn)))); Wherein, FC1 and FC2 represent fully connected layers, and ReLU and Sigmoid are activation functions; the Sigmoid function restricts the output to between 0 and 1 (or within the normalized space) to represent the normalized depth change.

[0018] Preferably, in step S8, the specific steps of the reverse mapping back to three-dimensional space include: S81: For X post For each pixel (u, v) in the dataset, obtain its pixel value d. depth ; S82: Determine the projection coordinates (y, y) of the pixel point (u, v) on the reference projection plane based on the spatial transformation parameters saved in step S2. proj ,z proj ); S83: Taking the reference projection plane as the origin, along the normal direction perpendicular to the plane, according to the pixel value d depth Determine the coordinates x of the point in the forward and backward directions. proj ; S84: Combine coordinates (x) proj, y proj ,z proj The generated point cloud is converted to the patient's original three-dimensional coordinate system and then meshed (e.g., through triangulation).

[0019] This step is the reverse of the previous conversion from 3D CBCT to 2D depth image, ensuring that the prediction results can be directly imported back into medical software for 3D visualization.

[0020] Preferably, the deep learning model is trained using a training dataset, and the training process employs a semantic grouping normalization strategy. The semantic grouping normalization strategy includes: dividing the input data into a first group {X}. PreOp ,L pre} and the second group {X diff ,ΔL}; For the first set of data, calculate the global minimum and maximum values ​​within the set and normalize them; For the second set of data, calculate the global minimum and maximum values ​​of the data within the set independently and then normalize them.

[0021] This grouping strategy enables the model to distinguish between absolute anatomical structural information and relative change information, avoiding the situation where minute surgical displacement signals are overwhelmed by huge anatomical coordinate values ​​due to differences in numerical magnitude.

[0022] Preferably, the deep learning model is trained using a training dataset, and the training process employs the Masked Mean Square Error (MSE) loss function, the formula for which the loss function is calculated is: Among them, y i and These represent the true depth value and the predicted depth value, respectively; mask i A binary mask is used when a pixel is located in the foreground region, i.e., the union of the pre- and post-operative depth images. i The value is 1 if it is not 0 otherwise; n is the total number of pixels. The loss function is used to mask the error contribution of the background region during backpropagation training, allowing only the foreground region of facial soft tissue to participate in error calculation.

[0023] Because CBCT scans contain a lot of air noise in the background and outliers are prone to appear near the jawline, standard MSE calculates the overall image error, causing background noise and outliers to interfere with the gradient descent. The Masked MSE loss function can simultaneously shield the influence of background noise and outliers near the jawline, forcing the network to focus on the core area of ​​facial soft tissue, ensuring that the model learns more accurate facial depth variation patterns.

[0024] Preferably, the multimodal cross-attention (MCA) module is used to fuse facial features and landmark features; the facial feature extractor includes an encoder and a decoder, and a skip connection is provided between the encoder and the decoder; The encoder comprises several cascaded downsampling modules, each containing a convolutional layer (Conv) and a pooling layer (Max Pooling) to progressively increase the number of feature channels (e.g., doubling from the initial number of channels C to 16C) and reduce the spatial resolution of the feature map (e.g., shrinking it to 1 / 16 of the input size), thereby achieving hierarchical feature extraction of facial depth images. The decoder comprises several cascaded upsampling modules to progressively reduce the number of feature channels and restore the spatial resolution of the feature map. This structural design can extract details from shallow layers (such as changes in soft tissue depth values) and extract high-dimensional semantic information from deep layers (such as the overall facial contour structure). Simultaneously, by combining the encoder's corresponding hierarchical features introduced by skip connections, it optimizes the feature representation of local facial details (such as the chin area). The multimodal cross-attention (MCA) module receives the output from the last upsampling layer of the facial feature extractor UNet as the preoperative image feature map F. PreOp The output from the marker feature extractor is used as the marker embedding feature, and the attention-weighted feature F of the output is added. attn The image is transmitted to a multilayer perceptron (MLP) for prediction of differences.

[0025] The substantial effects of this invention are: 1. High accuracy and clinical applicability: By introducing the MCA-UNet architecture and differential image prediction strategy, this invention achieves an average facial morphology prediction error of less than 2 mm (specifically, surface deviation error of 1.63±0.50 mm and marker error of 1.91±0.41 mm) on a dataset of 102 real patients. Especially in key surgical areas such as the mandible and perioral region, it significantly outperforms the StyleGAN and UNet-concat methods, meeting the standards for clinical application.

[0026] 2. High computational efficiency and low resource consumption: This invention innovatively converts 3D CBCT data into 2D depth images for processing. Compared to directly processing 3D point clouds or voxel meshes, this method significantly reduces data dimensionality and computational complexity, resulting in a substantial increase in model training and inference speed. It can quickly complete predictions on ordinary GPUs, facilitating deployment in hospitals.

[0027] 3. Solves the problem of nonlinear deformation: Through a multimodal cross-attention module, this invention can explicitly utilize surgical landmarks and their displacements to drive image feature updates. This mechanism simulates real anatomical logic (bones driving muscles), effectively solving the highly nonlinear mapping problem between bones and soft tissues, and avoiding the prediction failure of traditional methods in major plastic surgeries.

[0028] 4. End-to-end visualization: This invention provides a complete closed-loop process from 3D CBCT input to 3D mesh output (including specific projection and inverse projection algorithms), without relying on unknown postoperative bone models as intermediate input, and can be directly used in the preoperative planning stage, providing doctors and patients with an intuitive preview of postoperative effects.

[0029] 5. Noise robustness: By introducing the Masked MSE loss function and outlier correction preprocessing, this invention effectively suppresses background noise and scanning artifacts commonly found in medical images, ensuring that the generated facial contours are smooth and natural, and reducing the generation of artifact patches. Attached Figure Description

[0030] Figure 1 This is a schematic diagram of a method for predicting facial morphology after orthognathic surgery according to the present invention; Figure 2 This invention provides a facial soft tissue point for marker error assessment and a schematic diagram of six facial sub-regions for quantitative assessment, where regions AF correspond to the nose, upper lip, lower lip, chin, right cheek, and left cheek, respectively. Figure 3 A comparison chart showing the differences between the predicted grids generated by the three visualized models and the actual postoperative grids; Figure 4 A comparison chart of the surface deviation errors in the front and back directions of the prediction results of the three models; Figure 5 The output image shows a comparison of the differences between the three different models. Detailed Implementation

[0031] The technical solution of the present invention will be further described in detail below through embodiments and in conjunction with the accompanying drawings.

[0032] Example: This example provides a method for predicting facial morphology after orthognathic surgery, based on the multimodal cross-attention mechanism (MCA-UNet). The method mainly includes four stages: data preprocessing, feature extraction and fusion, difference prediction, and three-dimensional reconstruction. Figure 1 The diagram shown is a flowchart of the entire method.

[0033] 1. Data Acquisition and Preprocessing First, preoperative three-dimensional CBCT data of the patients were acquired. In this embodiment, the dataset came from 102 adult patients who underwent comprehensive orthognathic surgery at the Affiliated Stomatological Hospital of Zhejiang University School of Medicine. All CBCT data were acquired using a NEW TOM VG device (Aperio Service, Italy), with scanning parameters of a voxel spacing of 0.4286 mm, covering the entire skull.

[0034] Meanwhile, professional doctors marked 21 bony anatomical landmarks related to orthognathic surgery on the CBCT data. pre These include the anterior nasal spine (ANS), the premental point (Pog), the maxillary canine point (U3), and the mandibular angle point, as detailed in Table 1. Among these, the piriform aperture lateral point, the maxillary canine point (U3), the maxillary first molar point (U6), the mandibular canine point (L3), the mandibular first molar point (L6), the mental foramen, and the mandibular angle point are located on each of the left and right sides of the face.

[0035] Table 1 Based on surgical planning (virtual osteotomy and bone segment movement), the displacement vector ΔL of these landmark points is determined. To convert the 3D data into a 2D format suitable for deep learning network processing, the following steps are performed: (1) Spatial registration: Using Dolphin Imaging software (v11.95) with preoperative data as a reference, the bilateral zygomaticofrontal suture and zygomatic face were used as reference points to perform rigid registration of the postoperative CBCT, with the error controlled within 0.5 mm.

[0036] (2) Standardized cropping and projection: The region of interest (ROI) was cropped using 3D Slicer software. The ROI range was set as follows: vertically from the submental point to the supraorbital margin, and anteroposteriorly from the tip of the nose to the anterior margin of the cervical spine.

[0037] (3) Depth Map Generation: The coronal plane passing through the anterior edge of the cervical spine is defined as the reference projection plane. A threshold segmentation algorithm is used to extract the facial soft tissue contour, and the vertical distance from each point on the soft tissue surface to this reference plane is calculated. This distance value is used as the pixel value to generate a two-dimensional depth image X. PreOp .

[0038] (4) Image scaling: Scaling the generated depth image to a uniform resolution of 256×256 pixels.

[0039] (5) Outlier correction: Statistically calculate the mean μ and standard deviation σ of the image pixels, and truncate the pixel values ​​|x-μ| greater than 3σ to eliminate noise or artifacts caused by the patient’s micro-movement during CBCT scanning.

[0040] 2. Network architecture setup This embodiment constructs an MCA-UNet model.

[0041] Facial Feature Extractor: Based on the UNet architecture, it includes an encoder and a decoder. The encoder contains four cascaded downsampling modules. Each module consists of two 3×3 convolutional layers, a batch normalization layer, a ReLU activation function, and a 2×2 max-pooling layer. As the network depth increases, the spatial resolution of the feature maps is halved step-by-step (from 256×256 to 16×16), while the number of channels is doubled step-by-step (from the initial number of channels C to 16C). The decoder gradually restores the spatial resolution through upsampling operations and fuses features from corresponding layers of the encoder through skip connections.

[0042] Landmark feature extractor: Receives a 21×3 coordinate matrix L pre The 21×3 displacement matrix ΔL is concatenated and then used to capture the spatial dependencies between marker points through a self-attention module. The original coordinate information is then superimposed via residual connections. Finally, a multilayer perceptron (containing linear layers, LayerNorm, and LeakyReLU activation functions) projects the features into a 128-dimensional embedding space to obtain F. landmark ∈R 128×21 .

[0043] Multimodal Cross-Attention (MCA) module: First, dimensional alignment is performed: the image feature map output by the facial feature extractor is fused through 1×1 convolution with skip connections and mapped to 128 channels, then flattened into a sequence form as the query vector Q. The landmark feature F... landmark K is the key vector and V is the value vector.

[0044] Calculate attention: F attn =softmax(Q×K T / sqrt(d))×V.

[0045] This step calculates the association strength between each image pixel location and 21 anatomical landmarks, and injects the displacement semantics of the landmarks into the image features.

[0046] Multilayer Perceptron and Differential Prediction: A multilayer perceptron (MLP) consisting of two fully connected (FC) layers and a sigmoid activation function is used to predict the output differential image X. diff .

[0047] 3. Training Strategies The model is implemented based on the PyTorch framework and trained on a single NVIDIA RTX A6000 GPU.

[0048] Optimizer: Adam, learning rate set to 10 -5 The batch size is 4.

[0049] Training sessions: 200 epochs, taking approximately 4 hours.

[0050] Normalization strategy: Semantic grouping normalization is used. The input is divided into two groups: {X} PreOp ,L pre} (absolute coordinate system) and {X diff ,ΔL} (relative change groups). The min-max normalization parameter is calculated independently for each group.

[0051] Loss function: Masked mean squared error (MSE). Only the pixel error of the foreground region (i.e., the facial region with a depth value greater than 0) is calculated; the background region is not included in gradient backpropagation.

[0052] 4. Three-dimensional reconstruction During the inference phase, the model outputs X. diff Calculate X post =X PreOp +X diff Using the projection parameters saved in the preprocessing stage, X is... post The depth values ​​are back-mapped back to three-dimensional spatial coordinates (x, y, z), and reconstructed into a three-dimensional mesh model (STL or OBJ format) through triangulation for clinical visualization.

[0053] Experimental Results and Comparative Analysis To verify the effectiveness of the invention, an evaluation was conducted on a test set (13 randomly selected patients). Evaluation metrics included mean surface deviation error and marker error, both in millimeters (mm). Both quantified the difference between the predicted and actual postoperative facial mesh. Surface deviation error measured the average absolute distance between the predicted and actual facial surface along the anteroposterior direction (y-axis, corresponding to the coronal plane normal vector). Specifically, for each point (x, z) on the depth image, the absolute value of |y1-y2| was calculated, where y1 and y2 represent the predicted and actual depth values, respectively. Facial surface error was obtained by averaging these differences over all facial points.

[0054] like Figure 2 As shown, the face was divided into six sub-regions using four key landmarks (subnasal point, left and right corner points of the mouth, and soft tissue point B). The regions were divided as follows: the left cheek included the area to the left of the left corner point of the mouth; the right cheek included the area to the right of the right corner point of the mouth; between the two corner points of the mouth, the area above the subnasal point was defined as the nose, and the area below the soft tissue point B was defined as the chin; the remaining area between the subnasal point and the soft tissue point B was further divided into the upper and lower lips by a horizontal midline passing through the two corner points of the mouth. The mean surface error for each sub-region was calculated using the same method as the full-face assessment.

[0055] Furthermore, this invention utilizes soft tissue landmarks for a more detailed and precise evaluation. Specifically, 16 key landmarks in the mandibular face (the region most relevant to orthognathic surgery) were selected for analysis. For each landmark, a professional clinician manually labeled the coordinates (x1, y1, z1) on a real postoperative grid and (x2, y2, z2) on a predicted postoperative grid. The spatial difference between the two was quantified using Euclidean distance. This indicator directly measures the accuracy of point-by-point prediction of key locations in orthognathic surgery.

[0056] 1. Comparison with the benchmark model This embodiment compares MCA-UNet with two mainstream benchmark methods: (1) StyleGAN: Based on the pre-trained StyleGAN model, it is adjusted for the orthognathic surgery task, the single-channel depth image is copied into a three-channel, the encoder is frozen, the markers are fused into the decoder and fine-tuned.

[0057] (2) UNet-concat: Based on the UNet network, the marker information is incorporated only through simple channel concatenation, without using cross attention.

[0058] Table 2 compares the prediction accuracy of different models (unit: millimeters).

[0059] Table 2 Results Analysis: The present invention (MCA-UNet) achieved an average error of only 1.63 mm across the entire face, significantly outperforming UNet-concat (2.28 mm) and StyleGAN (1.72 mm). In the crucial lower lip region of orthognathic surgery, the present invention reduced the error to 1.28 mm, while other models exceeded 1.7 mm. Regarding landmark localization accuracy, the present invention achieved an error of only 1.91 mm, a reduction of approximately 44% compared to other models.

[0060] like Figure 3The image shows the differences between the predicted meshes generated by the three models and the actual postoperative meshes after visualization. Specifically, it compares the predicted postoperative facial meshes (green) and the actual postoperative facial meshes (red) generated by the MCA-UNet, UNet-concat, and StyleGAN models in six cases, illustrating the differences from multiple angles, including front and side views. Dashed boxes highlight magnified areas. From the side view, it can be seen that the predictions from MCA-UNet in this approach better match the actual postoperative contours, especially in the jaw and perioral regions. Notably, the lip contours generated by MCA-UNet are smoother and more anatomically realistic.

[0061] Figure 4 The results show a visualization of the 3D mesh error. Overall, MCA-UNet has lower prediction errors, and the error magnitude is consistently closer to zero.

[0062] at the same time, Figure 5 In the images (b) and (c), the limitations of StyleGAN and UNet-concat are revealed: the differential images predicted by StyleGAN lack clear facial features, while the output of UNet-concat contains obvious patch artifacts.

[0063] Combined with the quantitative results in Table 2, these visual comparisons highlight the superior performance of MCA-UNet. It achieved the lowest surface deviation error in almost all sub-regions (nose, lips, chin, cheeks) and across the entire face. Regarding landmark accuracy, MCA-UNet's mean error was 1.91 ± 0.41 mm, significantly better than UNet-concat (3.40 ± 0.88 mm) and StyleGAN (3.45 ± 0.93 mm). These results underscore the clinical feasibility of MCA-UNet, demonstrating its ability to produce more accurate and reliable predictions in key surgical areas such as the jaw and perioral region.

[0064] 2. Ablation Studies To demonstrate the necessity of each technical feature, multiple ablation experiments were conducted. The specific results are shown in Table 3, which are the evaluation results obtained based on the marker conditions, output data content, and normalization method (the AF region and the whole face are evaluated based on surface deviation error, in mm).

[0065] Table 3 Impact of marker inputs: To evaluate the contribution of marker conditions to prediction performance, this approach employs ablation studies by removing marker inputs from the model. For example... Figure 5As shown in (d), the difference images generated by the unmarker-guided model are relatively flat and fail to reflect clinically anticipated changes—especially in the mandibular region, where soft tissue changes are typically most pronounced. The quantitative results in Table 3 further highlight this effect. The mean surface deviation of the entire face decreased from 4.00 ± 0.75 mm without markers to 1.63 ± 0.51 mm with markers, a reduction of over 59%. Similarly, the marker error decreased from 4.80 ± 0.88 mm to 1.91 ± 0.41 mm. These results confirm that introducing marker information, crucial for orthognathic surgery, helps the model focus on key areas such as the mandible and significantly improves predictive accuracy.

[0066] Impact of Attention-Based Fusion Mechanism: To evaluate the effectiveness of the attention mechanism in modeling spatial correspondences, this approach compares two marker fusion strategies: simple concatenation (UNet-concat) and cross-attention (MCA-UNet). Figure 5 As shown in (b), UNet-concat produces patch artifacts and exhibits early overfitting, indicating its inability to capture complex landmark-depth image relationships. In contrast, the attention-based fusion mechanism in MCA-UNet dynamically weights the contribution of landmarks, resulting in more accurate and anatomically consistent soft tissue predictions. This demonstrates the advantage of cross-attention mechanisms in capturing the nonlinear relationship between images and landmarks.

[0067] Impact of prediction targets: As shown in Table 3, difference-based prediction significantly reduced errors—marker errors were reduced by approximately 55.6%, and errors in key surgical areas (such as the mandible) were improved by 30.1%. Unlike direct prediction, which requires learning both basic anatomical structures and surgical changes simultaneously to predict postoperative depth images, difference-based prediction focuses specifically on surgically induced deformations, making the learning task more targeted and effective.

[0068] Impact of Normalization Strategies: This paper compares two normalization strategies: global normalization and semantic grouping normalization. In global normalization, all input modalities (preoperative images (X...)... PreOp ), preoperative landmarks (L) pre ), difference image (X) diff The input and difference markers (ΔL) are normalized together using a shared global minimum and maximum. In contrast, semantic grouping normalization divides the input into two distinct groups: {X} PreOp ,L pre} and {X diffWithin each group, min-max normalization is applied independently. As shown in Table 3, semantic grouping normalization achieves superior performance across all metrics. It reduces the full-face surface deviation error to 1.63 ± 0.51 mm (a 79% improvement compared to the global normalization prediction error of 8.07 ± 1.98 mm) and the marker error to 1.91 ± 0.41 mm (a 75% reduction compared to the global normalization prediction error of 7.75 ± 1.78 mm).

[0069] In summary, this invention, by combining 3D-to-2D depth map representation, multimodal cross-attention fusion mechanism, and differential image prediction strategy, successfully solves the problems of large computational load and difficulty in modeling nonlinear deformation in soft tissue prediction for orthognathic surgery, and achieves high-precision, end-to-end clinical auxiliary prediction.

[0070] The specific embodiments described herein are merely illustrative of the spirit of the invention. Those skilled in the art to which this invention pertains may make various modifications or additions to the described specific embodiments or use similar methods to substitute them, without departing from the spirit of the invention or exceeding the scope defined by the appended claims.

[0071] Although this document uses a variety of terms, the possibility of using other terms is not excluded. These terms are used merely for the convenience of describing and explaining the essence of the invention; interpreting them as any additional limitation would contradict the spirit of the invention.

Claims

1. A method for predicting facial morphology after orthognathic surgery, characterized in that, The method includes the following steps: S1: Acquire the patient's preoperative three-dimensional cone-beam computed tomography (CBCT) data, and obtain the coordinates L of the preset N preoperative bony landmarks. pre And the displacement vector ΔL of N bony landmarks determined based on the surgical plan; S2: Convert CBCT data into preoperative two-dimensional depth images X PreOp This step specifically includes: spatial registration and region of interest (ROI) cropping of CBCT data; extracting facial soft tissue contours from the cropped CBCT data using a threshold segmentation algorithm; defining a coronal plane passing through the anterior edge of the cervical spine as a reference projection plane; calculating the vertical distance from points on the soft tissue contour surface to the reference projection plane; and using the vertical distance as pixel values ​​to generate the preoperative two-dimensional depth image X. PreOp ; S3: Utilize the facial feature extractor of the deep learning model to process the preoperative two-dimensional depth image X. PreOp Feature extraction was performed to obtain the preoperative image feature map F. PreOp ; S4: Use the landmark feature extractor of the deep learning model to extract the coordinates L of the preoperative bony landmarks. pre The displacement vector ΔL is processed to extract the marker embedding feature F. landmark ; S5: Utilize the multimodal cross-attention (MCA) module to fuse the preoperative image feature map F PreOp and the marker embedding feature F landmark The specific process is as follows: The preoperative image feature map F PreOp The marker points are embedded in feature F as the query vector Q. landmark As the key vector K and the value vector V; According to formula F attn =softmax(Q×K T The attention-weighted feature F is calculated by √(d / V). attn Where d is the feature dimension, K T Let K denote the transpose of matrix K, and softmax be the normalized exponential function. S6: Apply the attention-weighted feature F attn Input a multilayer perceptron (MLP) to predict and generate postoperative and preoperative difference images X. diff ; S7: According to formula X post =X PreOp +X diff The predicted postoperative two-dimensional depth image X was calculated. post ; S8: The predicted postoperative two-dimensional depth image X post Each pixel in the image is inversely mapped back to three-dimensional space by combining the reference projection plane and clipping parameters defined in step S2, and the predicted postoperative three-dimensional facial soft tissue mesh is reconstructed.

2. The method for predicting facial morphology after orthognathic surgery according to claim 1, characterized in that, In step S2, the specific range of the region of interest (ROI) cropping is: extending from the submental point to the supraorbital margin in the vertical direction, and extending from the tip of the nose to the anterior margin of the cervical spine in the anteroposterior direction; Furthermore, before generating the preoperative two-dimensional depth image, an outlier correction step is included: calculating the mean μ and standard deviation σ of all pixel values ​​in the image, and cropping pixel values ​​with an absolute difference |x-μ| greater than 3σ to a preset effective range, where x is the pixel value.

3. The method for predicting facial morphology after orthognathic surgery according to claim 1, characterized in that, In step S5, before performing the formula calculation, a step of aligning the feature dimensions is also included: The preoperative image feature map F with dimensions B×C×H×W PreOp Flatten it in the spatial dimension and convert it into a sequence with dimensions B×(H×W)×C, where B is the batch size, C is the number of channels, and H and W are the height and width of the feature map; If the number of channels C of the preoperative image feature map is equal to the number of marker embedding features F... landmark If the feature dimension d is inconsistent, the number of channels C of the preoperative image feature map is mapped to d through a linear projection layer or a 1x1 convolutional layer to meet the dimension requirement for matrix multiplication of Q and K.

4. The method for predicting facial morphology after orthognathic surgery according to claim 1, characterized in that, In step S4, the marker embedding feature F is extracted using the marker feature extractor. landmark The steps include: S41: L pre Concatenating ΔL with ΔL yields the combined vector L. concat ; S42: Capturing L using a self-attention mechanism concat The spatial dependencies between various markers are used to obtain contextual features; S43: Combine the context features with the original L concat Perform residual join, i.e., add elements one by one; S44: Input the summed result into a multilayer perceptron, and project it onto an embedding space of dimension d through linear transformation, layer normalization (LayerNorm), and LeakyReLU activation function to obtain the marker embedding feature F. landmark .

5. The method for predicting facial morphology after orthognathic surgery according to claim 1, characterized in that, In step S6, the multilayer perceptron (MLP) generates a difference image X. diff The specific calculation formula is as follows: X diff =Sigmoid(FC2(ReLU(FC1(Fattn)))); In this context, FC1 and FC2 represent fully connected layers, and ReLU and Sigmoid are activation functions.

6. The method for predicting facial morphology after orthognathic surgery according to claim 1, characterized in that, In step S8, the specific steps of the reverse mapping back to three-dimensional space include: S81: For X post For each pixel (u, v) in the dataset, obtain its pixel value d. depth ; S82: Determine the projection coordinates (y, y) of the pixel point (u, v) on the reference projection plane based on the spatial transformation parameters saved in step S2. proj ,z proj ); S83: Taking the reference projection plane as the origin, along the normal direction perpendicular to the plane, according to the pixel value d depth Determine the coordinates x of the point in the forward and backward directions. proj ; S84: Combine coordinates (x) proj, y proj ,z proj The coordinates are converted to the patient's original three-dimensional coordinate system, and the generated point cloud is then meshed.

7. The method for predicting facial morphology after orthognathic surgery according to claim 1, characterized in that, The deep learning model is trained using a training dataset, and the training process employs a semantic grouping normalization strategy. The semantic grouping normalization strategy includes: dividing the input data into a first group {X}. PreOp ,L pre } and the second group {X diff ,ΔL}; For the first set of data, calculate the global minimum and maximum values ​​within the set and normalize them; For the second set of data, calculate the global minimum and maximum values ​​of the data within the set independently and then normalize them.

8. A method for predicting facial morphology after orthognathic surgery according to any one of claims 1 to 7, characterized in that, The deep learning model is trained using a training dataset. The training process employs the Masked Mean Squared Error (MSE) loss function, and the formula for calculating the loss function is as follows: Among them, y i and points Don't represent the true depth value and the predicted depth value; mask i A binary mask is used when a pixel is located in the foreground region, i.e., the union of the pre- and post-operative depth images. i The value is 1 if it is not 0 otherwise; n is the total number of pixels.

9. The method for predicting facial morphology after orthognathic surgery according to claim 1, characterized in that, The facial feature extractor includes an encoder and a decoder, and a skip connection is provided between the encoder and the decoder; The encoder contains several cascaded downsampling modules, each containing a convolutional layer and a pooling layer, used to progressively increase the number of feature channels and reduce the spatial resolution of the feature map; the decoder contains several cascaded upsampling modules, used to progressively reduce the number of feature channels and restore the spatial resolution of the feature map. The multimodal cross-attention (MCA) module receives the output from the last upsampling layer of the facial feature extractor as the preoperative image feature map, and simultaneously receives the landmark embedding features from the landmark feature extractor, and applies the output attention-weighted features F. attn The image is transmitted to a multilayer perceptron (MLP) for prediction of differences.

Images