A landmark estimation-guided orthodontic facial prediction method

Abstract

This invention discloses a landmark estimation-guided method for predicting orthodontic facial features, comprising: collecting and detecting landmarks in pre- and post-orthodontic facial image pairs to construct a landmark image dataset; training a landmark prediction network using the landmark image dataset; training an image generation network using the landmark coordinates predicted by the landmark prediction network and the pre- and post-orthodontic facial images; and predicting and outputting the corresponding post-orthodontic facial image using the landmark prediction network and the image generation network sequentially for the pre-orthodontic facial image to be predicted. This invention's method can generate more accurate images while preserving the patient's identity characteristics, providing a reference for doctors and patients during treatment.

Description

Technical Field

[0001] This invention belongs to the field of image processing technology, specifically relating to a landmark estimation-guided method for predicting orthodontic faces. Background Technology

[0002] Malocclusion, a common oral and maxillofacial disorder, not only affects patients' physiological functions such as chewing and speech, but also directly relates to facial aesthetics and mental health, making it a core focus of orthodontic clinical treatment. Orthodontic treatment, especially in cases involving significant retraction of the anterior teeth, can effectively improve the aesthetics of the soft tissue profile. Therefore, accurately predicting changes in facial profile is crucial for orthodontic patients in designing their orthodontic treatment plans.

[0003] Existing cGAN-based methods for predicting orthodontic soft tissue contours utilize the adversarial nature of cGANs, taking input conditional information and leveraging the adversarial interaction between the generator and discriminator to generate post-orthodontic images. A conditional vector input module is incorporated, vectorizing changes in tooth parameters as input to aid in image generation. This model employs a U-Net-based generator module and a PatchGAN-based discriminator module. A soft loss mechanism is designed to enhance soft tissue contour generation, while a multi-scale feature pyramid is used to improve image quality. However, training with GANs suffers from training difficulties and pattern collapse. Furthermore, the generated images often lack clarity, are prone to detail loss, and lack reference value. Summary of the Invention

[0004] To address the aforementioned shortcomings in existing technologies, the landmark estimation-guided orthodontic face prediction method provided by this invention solves the problems of poor image quality and difficulty in preserving identity features in existing methods.

[0005] To achieve the above-mentioned objectives, the technical solution adopted by this invention is as follows: a landmark estimation-guided orthodontic facial prediction method, comprising the following steps: S100. Collect and detect landmark points in pre-orthodontic-post-orthodontic facial image pairs, and construct a landmark point image dataset; S200. A landmark prediction network is trained using a landmark image dataset; S300. Use the landmark coordinates predicted by the landmark prediction network and the pre- and post-orthodontic facial images to train the image generation network. S400: For the pre-orthodontic facial image to be predicted, the marker prediction network and the image generation network are used sequentially to predict and output the corresponding post-orthodontic facial image.

[0006] Further, S200 includes the following steps: S201. For facial images in the landmark image dataset, register the pre-orthodontic and post-orthodontic facial images using landmarks that do not change before and after orthodontic treatment; wherein, the landmarks include the midpoint of the sella turcica and the nasal root point; S202. Use a pre-trained image feature extractor to extract features from the registered pre- and post-orthodontic facial images to obtain a composite feature field that preserves global features and local texture details around the landmark points. S203. Construct a 5-dimensional initial node feature vector for each marker point. This is transformed into a high-dimensional feature representation; where, These are the normalized coordinates of the current node in the image. As a priori for displacement, For node type; S204. Extract the corresponding local image feature vector from the composite feature field based on the coordinates of the marker points, and concatenate it with the high-dimensional feature representation in the channel dimension to construct the node features as a graph neural network. S205. Based on node characteristics and the anatomical relationships of the craniofacial structure, construct a topological graph of bidirectional constraint edges between landmarks; wherein, the bidirectional constraint edges include anatomical structure edges and soft tissue-hard tissue association edges; S206. Based on the topological relationship graph, the GraphSAGE graph neural network model is used for recursive message passing. The anatomical constraint information of the neighborhood information is gradually aggregated through three layers of graph convolution to gradually expand the receptive field and simulate the traction effect of hard tissue on soft tissue in order to predict the coordinates of anatomically consistent landmarks in the facial images after orthodontics. The landmark prediction network is then trained.

[0007] Furthermore, in step S201, the registration method is as follows: Using the coordinates of the pre-orthodontic facial image as a reference, the spatial coordinate system of the post-orthodontic facial image is aligned using the affine transformation formula. The affine transformation formula is as follows: In the formula, Let be any pixel coordinate in the post-orthodontic facial image. for The registered coordinates, For isotropic scaling operators, Let be a rotation matrix. It is a translation vector. The vector formed by the midpoint of the sella turcica and the nasal root point on the pre-orthodontic facial image. The vector formed by the midpoint of the sella turcica and the nasal root point on the facial image after orthodontic treatment. , , Let be the coordinates of the midpoint of the sella turcica before orthodontic treatment. These are the coordinates of the midpoint of the sella turcica after orthodontic treatment. These are the coordinates of the nasal root point before orthodontic treatment. These are the coordinates of the nasal root point after orthodontic treatment.

[0008] Furthermore, in S202, the image feature extractor is a ResNet50 model that retains only convolutional and pooling layers in the classification head and does not contain fully connected layers.

[0009] Furthermore, in S200, the training loss function of the marker prediction network... for: In the formula, For mixed numerical regression loss, For scale loss, For directional loss, For soft-hard tissue consistency loss, For local geometric smoothing loss, , , , and They are respectively , , , and The weighting coefficients, For balance coefficient, To predict movement, This represents the actual movement volume. For smoothing loss, for loss, It is a set of soft tissue landmarks. This is the set of hard tissue landmarks that are connected to soft tissue landmarks. For soft tissue adjacent node connection edges defined based on anatomical contours, the subscript is... and Each is a different node index. For the first The node to the first Edges between nodes.

[0010] Furthermore, step S300 includes the following sub-steps: S301. Convert the predicted marker coordinates into a spatially continuous Gaussian heatmap; S302. Construct the I2SB model, using pre- and post-orthodontic facial image pairs as model inputs and Gaussian heatmaps as constraints for image generation, to train the I2SB model and obtain the image generation network.

[0011] Furthermore, in step S301, during the construction of the Gaussian heatmap, for any marker point... Its coordinates on the Gaussian heatmap The local response intensity generated at the location for: In the formula, Predict the output of the markers for the marker prediction network. Normalized floating-point coordinates, denoted as the standard deviation of the Gaussian kernel.

[0012] Furthermore, in step S302, the training method for the I2SB model is as follows: During the noise addition process, the I2SB model is used to perform linear interpolation on the post-orthodontic facial image and the corresponding pre-orthodontic facial image and gradually add noise to obtain the noisy image in the current state. In the denoising process, the I2SB model is used to start denoising from the pre-orthodontic facial image as the initial state. The predicted estimate of the post-orthodontic facial image is obtained through the denoising network, and the corresponding post-orthodontic facial image is generated step by step by iterative sampling. The denoising network is determined according to the noisy image in the current state. The noise addition and denoising processes are repeated until the I2SB model predicts and outputs the post-orthodontic facial image based on the pre-orthodontic facial image, thus obtaining the image generation network.

[0013] Furthermore, in S300, the training loss function of the image generation network for: In the formula, It is a composite weight matrix. The weight of the marker point, The weight of the mandibular region, The predicted output of the denoising network. This is the current state. This is a facial image after orthodontic treatment. Standard deviation For expectation operators.

[0014] Furthermore, S400 specifically includes: Detect landmarks in the pre-orthodontic facial image, input the pre-orthodontic facial image, landmarks, and preset tooth movement into the landmark prediction network to predict the landmarks in the post-orthodontic facial image; The predicted landmarks are combined with the pre-orthodontic facial image to be predicted and input into the image generation network to predict and output the corresponding post-orthodontic facial image.

[0015] The beneficial effects of this invention are as follows: (1) This invention uses the fusion of image features and node features to capture landmark information and uses facial anatomical relationships to accurately predict changes in soft tissue landmarks caused by tooth movement, thus achieving accurate prediction of landmarks.

[0016] (2) This invention generates lateral cephalometric radiographs after orthodontic treatment by progressively adding and removing noise. Compared with the GAN method, this method has the advantages of stable training and clear, high-quality generated images. This method outperforms other methods in terms of PSNR, SSIM, and LPIPS.

[0017] (3) The method of the present invention differs from Diffusion, which generates directly from pure Gaussian noise, thus wasting the rich structural information in the input image. The present invention generates the image by denoising from the pre-orthodontic image, thus effectively utilizing some of the unchanging anatomical structures in the pre-orthodontic image to directly establish the changes in the distribution between the pre-orthodontic image and the post-orthodontic image, thereby maintaining identity features, especially the shape of the chin, nose, forehead, etc.

[0018] (4) This invention introduces heat maps into the generation model as a condition constraint during generation, establishes a loss function based on the importance of landmarks and regions, focuses on landmarks and the mandibular region, and achieves accurate generation of post-orthodontic images. In particular, the generation of details such as the shape of the upper and lower incisors and the mandibular shape after orthodontics has high accuracy and high reference value, with a generation error of 1.1mm, ensuring the accuracy of post-orthodontic facial image generation. Attached Figure Description

[0019] Figure 1 The flowchart of the orthodontic facial prediction method guided by landmark estimation provided by the present invention is shown.

[0020] Figure 2 This is a diagram illustrating the predicted landmark points in a post-orthodontic facial image provided by the present invention.

[0021] Figure 3 The image shows the effect of facial image generation after orthodontic treatment provided by this invention. Detailed Implementation

[0022] The specific embodiments of the present invention are described below to enable those skilled in the art to understand the present invention. However, it should be understood that the present invention is not limited to the scope of the specific embodiments. For those skilled in the art, various changes are obvious as long as they are within the spirit and scope of the present invention as defined and determined by the appended claims. All inventions utilizing the concept of the present invention are protected.

[0023] This invention provides a landmark estimation-guided orthodontic facial prediction method. It generates facial images starting from the pre-orthodontic image using the IS2B model, and uses the Gaussian heatmap corresponding to the coordinates of the landmarks after orthodontic treatment, predicted by the GraphSAGE graph neural network model, as a conditional input. It predicts the displacement of soft tissue points through the displacement of hard tissue points, and uses this as a guiding condition for image generation, thereby predicting the facial image after orthodontic treatment. This process can generate more accurate prediction results while maintaining the patient's identity characteristics, providing a reference for doctors and patients.

[0024] refer to Figure 1 A landmark estimation-guided orthodontic facial prediction method includes the following steps: S100. Collect and detect landmark points in pre-orthodontic-post-orthodontic facial image pairs, and construct a landmark point image dataset; S200. A landmark prediction network is trained using a landmark image dataset; S300. Use the landmark coordinates predicted by the landmark prediction network and the pre- and post-orthodontic facial images to train the image generation network. S400: For the pre-orthodontic facial image to be predicted, the marker prediction network and the image generation network are used sequentially to predict and output the corresponding post-orthodontic facial image.

[0025] In S100 of this embodiment of the invention, X-ray images of the patient are taken before and after orthodontic treatment to obtain corresponding lateral cephalometric radiographs. These radiographs are then processed to obtain pre- and post-orthodontic facial image pairs for the same patient. The images are aligned and pre-processed, cropped to the same size, and then input into a landmark detection network to obtain the corresponding landmark coordinates. This results in a landmark image dataset constructed from several pre- and post-orthodontic facial image pairs with landmarks.

[0026] S200 of this embodiment of the invention includes the following steps: S201. For the facial images in the landmark image dataset, register the facial images before and after orthodontic treatment using landmarks that do not change before and after orthodontic treatment; where the landmarks include the midpoint of the sella turcica (S) and the root of the nose (N). S202. Use a pre-trained image feature extractor to extract features from the registered pre- and post-orthodontic facial images to obtain a composite feature field that preserves global features and local texture details around the landmark points. S203. Construct a 5-dimensional initial node feature vector for each marker point. This is transformed into a high-dimensional feature representation; where, These are the normalized coordinates of the current node in the image. As a priori for displacement, For node type; S204. Extract the corresponding local image feature vector from the composite feature field based on the coordinates of the marker points, and concatenate it with the high-dimensional feature representation in the channel dimension to construct the node features as a graph neural network. S205. Based on node characteristics and the anatomical relationships of the craniofacial structure, construct a topological graph of bidirectional constraint edges between landmarks; wherein, the bidirectional constraint edges include anatomical structure edges and soft tissue-hard tissue association edges; S206. Based on the topological relationship graph, the GraphSAGE graph neural network model is used for recursive message passing. The anatomical constraint information of the neighborhood information is gradually aggregated through three layers of graph convolution to gradually expand the receptive field and simulate the traction effect of hard tissue on soft tissue in order to predict the coordinates of anatomically consistent landmarks in the facial images after orthodontics. The landmark prediction network is then trained.

[0027] In S201, during the registration process, landmarks that remain unchanged before and after orthodontic treatment, including the midpoint of the sella turcica (Sella, S) and the nasal root point (Nasion, N) on the skull, are used. The skull base overlap method is employed to register facial images before and after orthodontic treatment, minimizing the influence of posture, equipment, and other factors. Among these, the SN plane formed by the two unchanged landmarks, i.e., the anterior skull base plane, exhibits extremely high stability during late adolescence and adult orthodontic treatment, and is minimally affected by tooth movement and jaw surgery.

[0028] In the specific registration process, using the coordinates of the pre-orthodontic facial image as a reference, the affine transformation formula is used to perform consistent spatial coordinate system alignment on the post-orthodontic facial image, eliminating interference from pose and shooting equipment. The coordinates of any pixel after orthodontic treatment are defined as follows: [equation missing], and their registered coordinates are [equation missing]. The registered coordinates are Therefore, the affine transformation formula is constructed as follows: In the formula, Let be any pixel coordinate in the post-orthodontic facial image. for The registered coordinates, An isotropic scaling operator is used to eliminate differences in shooting ratios. This is a rotation matrix used to correct the pose angles between images. This is a translation vector used to completely align the anatomical origin (saddle point) in space. The vector formed by the midpoint of the sella turcica and the nasal root point on the pre-orthodontic facial image. The vector formed by the midpoint of the sella turcica and the nasal root point on the facial image after orthodontic treatment. , , Let be the coordinates of the midpoint of the sella turcica before orthodontic treatment. These are the coordinates of the midpoint of the sella turcica after orthodontic treatment. These are the coordinates of the nasal root point before orthodontic treatment. Let be the coordinates of the nasal root point after orthodontic treatment. The posture deviation angle satisfies... , At the same time, the corresponding markers in the image must also be registered synchronously and unified in the same coordinate system.

[0029] In S202, the image feature extractor is a ResNet50 model that retains only convolutional and pooling layers in the classification head and does not contain fully connected layers. It extracts and fuses features at different scales, enabling the model to perceive both local details and global structural information simultaneously.

[0030] Specifically, considering the high overlap and low contrast of anatomical structures in lateral cephalometric radiographs, the original fully connected layers of the ResNet50 model's classification head were removed, retaining only its convolutional and pooling layers. Feature maps with different receptive fields and resolutions were output from multiple layers of the ResNet50 model. This allows the model to simultaneously capture global facial anatomical structures and local soft tissue details. Then, a 1×1 convolution operator is used... Mapping multi-size feature maps to a unified dimension , To address the issue of low resolution in high-level semantic features, a bilinear upsampling operator is utilized. (in, These are the feature values ​​of adjacent pixels. It uses interpolation weights based on spatial distance distribution to fuse features of different resolutions using pixel-wise addition, constructing a composite feature field that combines global features with local texture details. It preserves the spatial resolution around key landmarks.

[0031] In S203, a 5-dimensional initial feature vector is constructed for each marker point. .in It is the coordinate dimension, representing the current node. Normalized coordinates in the image; As a priori displacement, this value is a known displacement for hard tissue points, and is initially set to a value for soft tissue points. ; The node type is indicated by 0 for hard tissue and 1 for soft tissue. After constructing the initial 5-dimensional node features, an MLP is used to map the features, transforming the low-dimensional node features into high-dimensional feature representations. .

[0032] In S204, the spatial bilinear sampling operator is used. According to the coordinates of the marker point Extract the corresponding local image feature vector from the composite feature field. The extracted local image features are concatenated with the high-dimensional feature representations of the nodes along the channel dimension to form the node features of the graph neural network. This approach allows each node to carry structural context information about its specific anatomical location, laying the foundation for accurate displacement prediction.

[0033] In S205, a topological relationship graph of bidirectional constraint edges between landmarks is constructed based on the anatomical relationship of the craniofacial structure, so that the network learning process conforms to clinical rules. Among them, the anatomical structure edge represents the connection relationship of landmarks defined by craniofacial anatomy to simulate the continuity of facial structure; the soft tissue-hard tissue association edge represents the connection relationship between each soft tissue point and its corresponding hard tissue reference point to simulate the linkage between soft and hard tissues.

[0034] In S206, during the inference process of the GraphSAGE graph neural network model, at the... In layer graph convolution, nodes The feature information of all related nodes in its neighborhood is aggregated and represented as follows: In the formula, Indicates the first The neighboring node is at the _th ... The hidden state feature vector of the layer Represents nodes The set of connected neighbor nodes, This represents the mean aggregation operator, which captures local anatomical constraints by calculating the average value of neighborhood features.

[0035] When a node is updated, it fuses its own features with the aggregated features of its neighborhood to generate a higher-order feature representation, which is expressed as follows: In the formula, Represents a node In the Hidden states after convolutional layers Indicates the first The learnable weight matrices of each layer are used to extract self-information and neighborhood constraint information, respectively. This represents the ReLU activation function.

[0036] Finally, the two-dimensional residual displacement of each node is output through a fully connected network (MLP). and apply it to the initial coordinates. The coordinates of anatomically consistent landmarks were predicted. .

[0037] In this embodiment of the invention, to address the complex problem of predicting soft tissue deformation after orthodontic treatment, a multi-dimensional, adaptive training loss function is proposed for the marker prediction network. It is represented as: In the formula, For mixed numerical regression loss, For scale loss, For directional loss, For soft-hard tissue consistency loss, For local geometric smoothing loss, , , , and They are respectively , , , and The weighting coefficients, For balance coefficient, To predict movement, This represents the actual movement volume. For smoothing loss, for loss, It is a set of soft tissue landmarks. This is the set of hard tissue landmarks that are connected to soft tissue landmarks. For soft tissue adjacent node connection edges defined based on anatomical contours, the subscript is... and Each is a different node index. For the first The node to the first Edges between nodes.

[0038] Among them, the mixed numerical regression loss Combination Loss and Loss, utilization The sensitivity of the loss to large errors accelerates the initial convergence, and utilizes Smoothness within a small error range improves fine-tuning accuracy and prevents prediction oscillations.

[0039] Scale loss and directional loss The predicted length and direction of the displacement vector are consistent with the actual clinical changes, thereby enhancing the model's perception of spatial displacement trends.

[0040] soft-hard tissue consistency loss Simulate the traction effect of hard tissue on soft tissue.

[0041] Local geometric smoothing loss Constraining the displacement differences between adjacent soft tissue points ensures that the predicted facial contour curve is natural and continuous.

[0042] Furthermore, during the training process, the weights of the loss values ​​for each directional term are dynamically adjusted according to the number of training rounds, so that the model focuses on coarse localization of points in the early stage of training and on refined alignment of deep anatomical structures in the later stage of training.

[0043] like Figure 2 As shown, based on the above process, the registered pre-orthodontic facial image and its landmark coordinates are input into the landmark prediction network, which then predicts the landmark coordinates in the post-orthodontic facial image through the pre-orthodontic facial image, landmark coordinates, and the movement of hard tissue.

[0044] S300 of this embodiment of the invention includes the following sub-steps: S301. Convert the predicted marker coordinates into a spatially continuous Gaussian heatmap; S302. Construct the I2SB model, using pre- and post-orthodontic facial image pairs as model inputs and Gaussian heatmaps as constraints for image generation, to train the I2SB model and obtain the image generation network.

[0045] In S301, the predicted landmark coordinates are converted into a spatially continuous Gaussian heatmap, aiming to transform discrete anatomical location information into a probability distribution field that is easy for the model to learn.

[0046] Specifically, with each marker point as the center, a two-dimensional Gaussian kernel function is used to assign response intensity values ​​to the surrounding pixels, transforming them into a continuous spatial distribution. This continuous spatial distribution can effectively compensate for the abrupt changes in discrete coordinates on pixels, enhancing the ability of the subsequent image generation network to perceive anatomical locations.

[0047] Therefore, in the process of constructing a Gaussian heatmap, for any marker point Its coordinates on the Gaussian heatmap The local response intensity generated at the location for: In the formula, Predict the output of the markers for the marker prediction network. Normalized floating-point coordinates, The standard deviation of the Gaussian kernel is used to control the diffusion degree of the response points in the Gaussian heatmap, while the effective radius of the Gaussian distribution is defined as the surrounding area.

[0048] In S302, the post-orthodontic facial image is used as the target image. The original image before orthodontics serves as During training, both are used as inputs, and in order to use the predicted landmarks as constraints for image generation, the Gaussian heatmap is used as a conditional constraint for image generation.

[0049] Specifically, the training method for the I2SB model is as follows: During the noise addition process, the I2SB model is used to perform linear interpolation on the post-orthodontic facial image and the corresponding pre-orthodontic facial image and gradually add noise to obtain the noisy image in the current state. Among them, At time t, the noisy image in the current state is In the formula, This represents the mean, which is and linear interpolation, , Indicates the noise coefficient. For variance; In the denoising process, the I2SB model is used to start denoising from the pre-orthodontic facial image as the initial state. The denoising network obtains the predicted estimate of the post-orthodontic facial image, and iteratively samples it to generate the corresponding post-orthodontic facial image. The denoising network is determined based on the noisy image in the current state. Among them, the denoising network 𝜖( The prediction yields an estimate of the current noisy post-orthodontic facial image. , and then with Sampled to obtain the corresponding post-orthodontic facial image ; The noise addition and denoising processes are repeated until the I2SB model predicts and outputs the post-orthodontic facial image based on the pre-orthodontic facial image, thus obtaining the image generation network.

[0050] In the training process described above, the model starts denoising directly from the patient's pre-orthodontic facial image, rather than from Gaussian noise. Therefore, it can utilize the rich structural information in the pre-orthodontic facial image to preserve the patient's identity features as much as possible. Furthermore, in the image generation process, the introduction of Gaussian heatmaps as constraints can ensure more controllable and accurate generation. At the same time, starting the generation directly from the pre-orthodontic facial image can achieve better identity preservation.

[0051] In this embodiment of the invention, the original loss function of the I2SB model maintains the same weight on all pixels. Therefore, the skull information, which accounts for a large proportion of the total pixels but has almost no change, dilutes the information of the mandible, teeth, etc., which account for a small proportion of pixels but have greater changes. Ultimately, the model does not pay enough attention to the areas with greater changes and cannot accurately generate the changing details such as the upper and lower incisors.

[0052] Therefore, it is necessary to introduce a Gaussian heatmap containing post-orthodontic landmark changes as an anatomical prior, and to establish a loss function based on landmark and region importance. This loss function divides the pixels in the image into three parts: the first part is the pixels corresponding to the landmark and its surrounding properties. The second part is the rectangular area corresponding to pixels formed by landmarks on the mandible. The third part consists of other pixels. This invention assigns different weights to the pixels in these three regions, first focusing on the weight of the marker points. Then focus on the mandible, which has undergone significant changes, and its weight. The rest are all 1.

[0053] The loss function based on the importance of landmarks and regions can achieve high-precision local denoising by giving greater attention to key regions, thus achieving accurate generation results. This leads to the construction of the training loss function for the image generation network. for: In the formula, This is a composite weight matrix that determines the contribution of each pixel in the image to the total loss. The weight of the marker point, The weight of the mandibular region, The predicted output of the denoising network. This is the current state. This is a facial image after orthodontic treatment. Standard deviation Let be the expectation operator, representing the expectation over the entire training dataset and all random time steps. Calculate the average.

[0054] S400 of this embodiment of the invention is specifically as follows: Detect landmarks in the pre-orthodontic facial image, input the pre-orthodontic facial image, landmarks, and preset tooth movement into the landmark prediction network to predict the landmarks in the post-orthodontic facial image; The predicted landmarks are combined with the pre-orthodontic facial image to be predicted and input into the image generation network to predict and output the corresponding post-orthodontic facial image.

[0055] like Figure 3 The image shown is a facial image generated after orthodontic treatment based on the method described above.

[0056] Specific embodiments have been used to illustrate the principles and implementation methods of this invention. The descriptions of the embodiments above are only for the purpose of helping to understand the method and core ideas of this invention. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this invention. Therefore, the content of this specification should not be construed as a limitation of this invention.

[0057] Those skilled in the art will recognize that the embodiments described herein are intended to help the reader understand the principles of the invention, and should be understood that the scope of protection of the invention is not limited to such specific statements and embodiments. Those skilled in the art can make various other specific modifications and combinations based on the technical teachings disclosed in this invention without departing from the spirit of the invention, and these modifications and combinations are still within the scope of protection of this invention.

Claims

1. A landmark estimation-guided orthodontic facial prediction method, characterized in that, Includes the following steps: S100. Collect and detect landmark points in pre-orthodontic-post-orthodontic facial image pairs, and construct a landmark point image dataset; S200. A landmark prediction network is trained using a landmark image dataset; S300. Use the landmark coordinates predicted by the landmark prediction network and the pre- and post-orthodontic facial images to train the image generation network. S400: For the pre-orthodontic facial image to be predicted, the marker prediction network and the image generation network are used sequentially to predict and output the corresponding post-orthodontic facial image.

2. The orthodontic facial prediction method guided by landmark estimation according to claim 1, characterized in that, S200 includes the following steps: S201. For facial images in the landmark image dataset, register the pre-orthodontic and post-orthodontic facial images using landmarks that do not change before and after orthodontic treatment; wherein, the landmarks include the midpoint of the sella turcica and the nasal root point; S202. Use a pre-trained image feature extractor to extract features from the registered pre- and post-orthodontic facial images to obtain a composite feature field that preserves global features and local texture details around the landmark points. S203. Construct a 5-dimensional initial node feature vector for each marker point. This is transformed into a high-dimensional feature representation; where, These are the normalized coordinates of the current node in the image. As a priori for displacement, For node type; S204. Extract the corresponding local image feature vector from the composite feature field based on the coordinates of the marker points, and concatenate it with the high-dimensional feature representation in the channel dimension to construct the node features as a graph neural network. S205. Based on node characteristics and the anatomical relationships of the craniofacial structure, construct a topological graph of bidirectional constraint edges between landmarks; wherein, the bidirectional constraint edges include anatomical structure edges and soft tissue-hard tissue association edges; S206. Based on the topological relationship graph, the GraphSAGE graph neural network model is used for recursive message passing. The anatomical constraint information of the neighborhood information is gradually aggregated through three layers of graph convolution to gradually expand the receptive field and simulate the traction effect of hard tissue on soft tissue in order to predict the coordinates of anatomically consistent landmarks in the facial images after orthodontics. The landmark prediction network is then trained.

3. The orthodontic facial prediction method guided by landmark estimation according to claim 2, characterized in that, In step S201, the registration method is as follows: Using the coordinates of the pre-orthodontic facial image as a reference, the spatial coordinate system of the post-orthodontic facial image is aligned using the affine transformation formula. The affine transformation formula is as follows: In the formula, Let be any pixel coordinate in the post-orthodontic facial image. for The registered coordinates, For isotropic scaling operators, Let be a rotation matrix. It is a translation vector. The vector formed by the midpoint of the sella turcica and the nasal root point on the pre-orthodontic facial image. The vector formed by the midpoint of the sella turcica and the nasal root point on the facial image after orthodontic treatment. , , Let be the coordinates of the midpoint of the sella turcica before orthodontic treatment. These are the coordinates of the midpoint of the sella turcica after orthodontic treatment. These are the coordinates of the nasal root point before orthodontic treatment. These are the coordinates of the nasal root point after orthodontic treatment.

4. The orthodontic facial prediction method guided by landmark estimation according to claim 2, characterized in that, In step S202, the image feature extractor is a ResNet50 model that retains only convolutional and pooling layers in the classification head and does not contain fully connected layers.

5. The orthodontic facial prediction method guided by landmark estimation according to claim 1, characterized in that, In step S200, the training loss function of the marker prediction network for: In the formula, For mixed numerical regression loss, For scale loss, For directional loss, For soft-hard tissue consistency loss, For local geometric smoothing loss, , , , and They are respectively , , , and The weighting coefficients, For balance coefficient, To predict movement, This represents the actual movement volume. For smoothing loss, for loss, It is a set of soft tissue landmarks. This is the set of hard tissue landmarks that are connected to soft tissue landmarks. For soft tissue adjacent node connection edges defined based on anatomical contours, the subscript is... and Each is a different node index. For the first The node to the first Edges between nodes.

6. The orthodontic facial prediction method guided by landmark estimation according to claim 1, characterized in that, S300 includes the following steps: S301. Convert the predicted marker coordinates into a spatially continuous Gaussian heatmap; S302. Construct the I2SB model, using pre- and post-orthodontic facial image pairs as model inputs and Gaussian heatmaps as constraints for image generation, to train the I2SB model and obtain the image generation network.

7. The orthodontic facial prediction method guided by landmark estimation according to claim 6, characterized in that, In step S301, during the construction of the Gaussian heatmap, for any marker point... Its coordinates on the Gaussian heatmap The local response intensity generated at the location for: In the formula, Predict the output of the markers for the marker prediction network. Normalized floating-point coordinates, denoted as the standard deviation of the Gaussian kernel.

8. The orthodontic facial prediction method guided by landmark estimation according to claim 6, characterized in that, In S302, the training method for the I2SB model is as follows: During the noise addition process, the I2SB model is used to perform linear interpolation on the post-orthodontic facial image and the corresponding pre-orthodontic facial image and gradually add noise to obtain the noisy image in the current state. In the denoising process, the I2SB model is used to start denoising from the pre-orthodontic facial image as the initial state. The predicted estimate of the post-orthodontic facial image is obtained through the denoising network, and the corresponding post-orthodontic facial image is generated step by step by iterative sampling. The denoising network is determined according to the noisy image in the current state. The noise addition and denoising processes are repeated until the I2SB model predicts and outputs the post-orthodontic facial image based on the pre-orthodontic facial image, thus obtaining the image generation network.

9. The orthodontic facial prediction method guided by landmark estimation according to claim 1, characterized in that, In step S300, the training loss function of the image generation network for: In the formula, It is a composite weight matrix. The weight of the marker point, The weight of the mandibular region, The predicted output of the denoising network. This is the current state. This is a facial image after orthodontic treatment. Standard deviation For expectation operators.

10. The orthodontic facial prediction method guided by landmark estimation according to claim 1, characterized in that, Specifically, S400 is: Detect landmarks in the pre-orthodontic facial image, input the pre-orthodontic facial image, landmarks, and preset tooth movement into the landmark prediction network to predict the landmarks in the post-orthodontic facial image; The predicted landmarks are combined with the pre-orthodontic facial image to be predicted and input into the image generation network to predict and output the corresponding post-orthodontic facial image.

Images