Apparatus, method, device, storage medium, and program product for recognizing abnormal development of a child's maxillofacial region

Abstract

This disclosure relates to a device, method, equipment, storage medium, and program product for identifying abnormal development of the maxillofacial region in children. The device includes: a three-dimensional image registration module for performing global rigid registration and local non-rigid registration of three-dimensional images of facial soft tissue and jawbone; a geometric and visual feature extraction module for inputting spatially aligned three-dimensional images of facial soft tissue and jawbone into a hybrid neural network to obtain geometric and visual features; a geometric and visual feature fusion module for inputting geometric and visual features into an attention-based feature fusion network to obtain maxillofacial fusion features; and a child maxillofacial abnormal development identification module for inputting the maxillofacial fusion features into a child maxillofacial abnormal development identification model to obtain the identification result of child maxillofacial abnormal development. This reduces the user experience requirement, avoids the influence of subjective factors, and identifies early and subtle abnormalities.

Description

Technical Field

[0001] This disclosure relates to the field of digital orthodontic technology, and in particular to a device, method, equipment, storage medium, and program product for identifying abnormal development of the maxillofacial region in children. Background Technology

[0002] This section is intended to provide background or context for the embodiments of this disclosure as set forth in the claims. The description herein is not intended to be a prior art simply because it is included in this section.

[0003] Medical imaging is an important tool for identifying abnormal development of the maxillofacial region in children.

[0004] In related technologies, the use of medical imaging in identifying abnormal development of the maxillofacial region in children usually relies on the doctor's experience-based analysis and judgment.

[0005] However, manual analysis and judgment require a high level of experience from doctors, are greatly influenced by subjective factors, and are difficult to identify early and subtle abnormalities. Summary of the Invention

[0006] In order to at least partially solve one of the technical problems in the related art, this disclosure provides a device, method, equipment, storage medium and program product for identifying abnormal development of the maxillofacial region in children.

[0007] To achieve the above objectives, an exemplary embodiment of this disclosure provides, in a first aspect, a device for identifying abnormal development of the maxillofacial region in children, comprising: The three-dimensional image information registration module is configured to determine the three-dimensional image information of facial soft tissue and the three-dimensional image information of jawbone, and to perform global rigid registration and local non-rigid registration on the three-dimensional image information of facial soft tissue and the three-dimensional image information of jawbone to obtain spatially aligned three-dimensional image information of facial soft tissue and three-dimensional image information of jawbone. The geometric and visual feature extraction module is configured to input the spatially aligned three-dimensional image information of facial soft tissue and the three-dimensional image information of jawbone into a hybrid neural network to obtain the geometric and visual features output by the hybrid neural network, wherein the hybrid neural network includes a graph neural network and a convolutional neural network. The geometric and visual feature fusion module is configured to input the geometric and visual features into an attention-based feature fusion network for cross-modal feature interaction and fusion to obtain maxillofacial fusion features; The pediatric maxillofacial abnormal development identification module is configured to input the maxillofacial fusion features into the pediatric maxillofacial abnormal development identification model for identification based on the maxillofacial fusion features, and obtain the pediatric maxillofacial abnormal development identification result.

[0008] In some exemplary embodiments, the three-dimensional image information registration module is specifically configured as follows: Based on the iterative nearest point algorithm and landmark constraints, global rigid registration is performed on the three-dimensional image information of the facial soft tissue and the three-dimensional image information of the jawbone to obtain the three-dimensional image information of the facial soft tissue and the three-dimensional image information of the jawbone after global rigid registration. Based on the thin-plate spline algorithm and statistical shape model constraints, local non-rigid registration is performed on the three-dimensional images of facial soft tissue and jawbone after global rigid registration to obtain the spatially aligned three-dimensional images of facial soft tissue and jawbone.

[0009] In some exemplary embodiments, the geometric feature and visual feature extraction module is specifically configured as follows: Based on the graph neural network, spatially aligned 3D facial soft tissue image information is processed to extract the geometric features. The visual features are extracted based on the spatially aligned three-dimensional image information of the jawbone processed by the convolutional neural network.

[0010] In some exemplary embodiments, the geometric feature and visual feature fusion module is specifically configured as follows: The geometric features and the visual features are concatenated to obtain a fused input sequence; The fused input sequence is input into a feature fusion network based on deep learning and multi-head self-attention mechanisms, so that the geometric features and the visual features interact and are weighted to generate a joint feature vector representing cross-modal association as the maxillofacial fusion feature.

[0011] In some exemplary embodiments, the child maxillofacial abnormality development recognition module is further configured to: A first sample set is constructed, comprising several first samples; wherein, the first sample includes: first sample data and first label data; the first sample data includes training maxillofacial fusion features; the first label data includes training child maxillofacial abnormal development identification results corresponding to the training maxillofacial fusion features; Based on a predetermined machine learning algorithm, a model for identifying abnormal development of the maxillofacial region in children is constructed for training. The predetermined machine learning algorithm can be selected from one or more of the following: Naive Bayes algorithm, decision tree algorithm, support vector machine algorithm, kNN algorithm, neural network algorithm, deep learning algorithm, and logistic regression algorithm. The first sample set is input into the child maxillofacial abnormal development recognition model to be trained for iteration to obtain the performance index of the current model training. In the iteration process, a loss function combining the dice loss function and the focus loss function is introduced. If the performance index does not improve continuously during the iteration process, the model parameters of the current child maxillofacial abnormal development recognition model are saved to obtain the child maxillofacial abnormal development recognition model.

[0012] In some exemplary embodiments, a child maxillofacial abnormality development prediction module is also included, configured to: The maxillofacial fusion features are input into the child maxillofacial abnormal development prediction model to make predictions based on the maxillofacial fusion features, and the prediction results of child maxillofacial abnormal development are obtained. The module for predicting abnormal development of the maxillofacial region in children is also configured to: A second sample set is constructed, comprising several second samples; wherein, the second sample includes: second sample data and second label data; the second sample data includes training maxillofacial fusion features; the second label data includes training prediction results of abnormal maxillofacial development in children corresponding to the training maxillofacial fusion features; Based on a predetermined machine learning algorithm, a predictive model for abnormal development of the maxillofacial region in children is constructed for training. The predetermined machine learning algorithm can be selected from one or more of the following: Naive Bayes algorithm, decision tree algorithm, support vector machine algorithm, kNN algorithm, neural network algorithm, deep learning algorithm, and logistic regression algorithm. The second sample set is input into the child maxillofacial abnormal development prediction model to be trained for iteration to obtain the performance index of the current model training. In the iteration process, Bayesian optimization algorithm and transfer learning mechanism are introduced. If the performance index does not improve continuously during the iteration process, the model parameters of the current child maxillofacial abnormal development prediction model are saved to obtain the child maxillofacial abnormal development prediction model.

[0013] Based on the same inventive concept, a second aspect of the exemplary embodiments of this disclosure provides a method for identifying abnormal development of the maxillofacial region in children, including: Determine the three-dimensional image information of facial soft tissue and the three-dimensional image information of jawbone, and perform global rigid registration and local non-rigid registration on the three-dimensional image information of facial soft tissue and the three-dimensional image information of jawbone to obtain spatially aligned three-dimensional image information of facial soft tissue and three-dimensional image information of jawbone. The spatially aligned three-dimensional images of facial soft tissue and jawbone are input into a hybrid neural network to obtain the geometric and visual features output by the hybrid neural network. The hybrid neural network includes a graph neural network and a convolutional neural network. The geometric features and visual features are input into an attention-based feature fusion network for cross-modal feature interaction and fusion to obtain maxillofacial fusion features; The maxillofacial fusion features are input into the child maxillofacial abnormal development identification model to identify the abnormal development of the child's maxillofacial region.

[0014] Based on the same inventive concept, a third aspect of the exemplary embodiments of this disclosure provides a computer device including a memory, a processor, and a computer program stored in the memory, characterized in that the processor executes the computer program to implement the steps of the method as described in the second aspect.

[0015] Based on the same inventive concept, a fourth aspect of the exemplary embodiments of this disclosure provides a computer-readable storage medium having a computer program / instructions stored thereon, characterized in that the computer program / instructions, when executed by a processor, implement the steps of the method as described in the second aspect.

[0016] Based on the same inventive concept, a fifth aspect of the present disclosure provides a computer program product, including a computer program / instructions, characterized in that, when executed by a processor, the computer program / instructions implement the steps of the method as described in the second aspect. The technical solution provided in this disclosure has the following advantages compared with the prior art: The automated registration of multimodal data eliminates the need for user space integration experience, the data-driven deep neural network feature extraction and fusion eliminates subjective bias of human judgment, and the intelligent and sensitive identification of early and subtle anomalies is achieved through multi-source information complementarity and attention enhancement mechanisms.

[0017] In summary, it reduces the user experience requirements, avoids the influence of subjective factors, and identifies early, subtle anomalies. Attached Figure Description

[0018] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure.

[0019] To more clearly illustrate the technical solutions in the embodiments of this disclosure or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 A schematic diagram of a structure for a device for identifying abnormal development of the maxillofacial region in children, provided as an exemplary embodiment of the present disclosure; Figure 2 A flowchart illustrating a method for identifying abnormal development of the maxillofacial region in children, provided as an exemplary embodiment of this disclosure; Figure 3 A schematic diagram of the structure of an electronic device provided for an exemplary embodiment of this disclosure. Detailed Implementation

[0021] It is understood that before using the technical solutions disclosed in the various embodiments of this application, users should be informed of the types, scope of use, and usage scenarios of the personal information involved in this application in an appropriate manner in accordance with relevant laws and regulations, and user authorization should be obtained.

[0022] It is understood that the data involved in this technical solution (including but not limited to the data itself, the acquisition or use of the data) shall comply with the requirements of relevant laws, regulations and related provisions.

[0023] To better understand the above-mentioned objectives, features, and advantages of this disclosure, the solutions disclosed herein will be further described below. It should be noted that, unless otherwise specified, the embodiments and features described herein can be combined with each other.

[0024] Numerous specific details are set forth in the following description in order to provide a full understanding of this disclosure, but this disclosure may also be implemented in other ways different from those described herein; obviously, the embodiments in the specification are only some, and not all, of the embodiments of this disclosure.

[0025] As described in the background section, medical imaging is an important tool for assisting in the identification of abnormal development of the maxillofacial region in children.

[0026] In related technologies, the use of medical imaging in identifying abnormal development of the maxillofacial region in children usually relies on the doctor's experience-based analysis and judgment.

[0027] However, the inventors of this disclosure have found that manual analysis and judgment methods require a high level of experience from doctors, are greatly influenced by subjective factors, and are difficult to identify early and subtle abnormalities.

[0028] Specifically, skeletal Class II malocclusion is one of the most common maxillofacial developmental abnormalities in children. Its prevention has received widespread attention from various disciplines, including dentistry, otolaryngology, and child psychology. Current techniques often involve intraoral occlusion examination, X-ray or cone-beam computed tomography (CBCT), and analysis of lateral cephalometric radiographs to assess jaw development. However, this approach carries risks of radiation exposure and lacks sufficient analysis of the correlation between orofacial muscle function and soft tissue morphology, making it difficult to objectively identify early, subtle deformities. Furthermore, there is a widespread problem of non-orthodontic specialists misjudging the maxillofacial developmental abnormalities in children, leading to missed treatment opportunities. Determining the need for early treatment requires consideration of many factors, including occlusion, jaw development, airway condition, soft tissue muscle function, and the presence of bad habits, placing a high workload and demanding requirements on physicians. However, the analysis methods in these techniques suffer from problems such as reliance on a single imaging modality, significant subjective influence, lack of soft tissue function assessment, and insufficient identification of subtle abnormalities.

[0029] To address the aforementioned problems, this disclosure provides a scheme for identifying abnormal development of the maxillofacial region in children, specifically including: A device for identifying abnormal development of the maxillofacial region in children includes: a three-dimensional image registration module configured to determine three-dimensional image information of facial soft tissue and three-dimensional image information of the jawbone, and to perform global rigid registration and local non-rigid registration on the three-dimensional image information of the facial soft tissue and the three-dimensional image information of the jawbone to obtain spatially aligned three-dimensional image information of the facial soft tissue and the jawbone; a geometric and visual feature extraction module configured to input the spatially aligned three-dimensional image information of the facial soft tissue and the three-dimensional image information of the jawbone into a hybrid neural network to obtain geometric and visual features output by the hybrid neural network, wherein the hybrid neural network includes a graph neural network and a convolutional neural network; a geometric and visual feature fusion module configured to input the geometric and visual features into a feature fusion network based on an attention mechanism for cross-modal feature interaction and fusion to obtain maxillofacial fusion features; and a child maxillofacial abnormal development identification module configured to input the maxillofacial fusion features into a child maxillofacial abnormal development identification model for identification based on the maxillofacial fusion features to obtain the child maxillofacial abnormal development identification result.

[0030] Among them, the automatic registration of multimodal data eliminates the requirement for user space integration experience, the data-driven deep neural network feature extraction and fusion eliminates the subjective bias of human judgment, and the intelligent and sensitive identification of early and subtle anomalies is achieved through multi-source information complementarity and attention enhancement mechanisms.

[0031] In summary, it reduces the user experience requirements, avoids the influence of subjective factors, and identifies early, subtle anomalies.

[0032] In some exemplary embodiments, the identification scheme for abnormal development of the maxillofacial region in children provided in this disclosure can be applied to the identification of skeletal Class II malocclusion.

[0033] After introducing the basic principles of this disclosure, various non-limiting embodiments of this disclosure will be described in detail below.

[0034] refer to Figure 1 This is a schematic diagram of a structure of a device for identifying abnormal development of the maxillofacial region in children, provided in an exemplary embodiment of this disclosure.

[0035] A device 100 for identifying abnormal development of the maxillofacial region in children includes the following modules: The three-dimensional image information registration module 110 is configured to determine the three-dimensional image information of facial soft tissue and the three-dimensional image information of jawbone, and to perform global rigid registration and local non-rigid registration on the three-dimensional image information of facial soft tissue and the three-dimensional image information of jawbone to obtain spatially aligned three-dimensional image information of facial soft tissue and three-dimensional image information of jawbone.

[0036] In practice, facial 3D structured light scanning technology is used to acquire 3D images of facial soft tissue, obtaining high-precision 3D geometric morphology of the facial skin and soft tissue surface. By projecting a grating and calculating deformation, the (x, y, z) coordinates of each point on the surface are directly measured to form a point cloud. Pure geometric information is obtained, accurately recording nose height, lip protrusion, facial contour curvature, etc.

[0037] Three-dimensional images of the jawbone are obtained using CBCT (cone-beam computed tomography) technology, visualizing the three-dimensional structure of the hard tissues (bones and teeth) of the maxillofacial region. X-ray surround imaging is used, and three-dimensional voxel data is reconstructed by computer. Imaging of high-density tissues (calcified bone and teeth) is extremely clear. The resulting three-dimensional voxel images clearly show the morphology of the upper and lower jaws, bone wall thickness, tooth root location, and the bony structure of the temporomandibular joint.

[0038] As an example, after acquiring three-dimensional images of facial soft tissue using facial three-dimensional structured light scanning technology, MRI (Magnetic Resonance Imaging) technology is used as a supplement to acquire three-dimensional images of facial soft tissue, visualizing the internal structures of soft tissue (muscles, nerves, blood vessels, joint discs, etc.) and their physiological state. Magnetic fields and radio frequency pulses are used to excite hydrogen atoms in the body to generate signals. It has extremely high resolution for tissues with different water contents and molecular environments. The resulting three-dimensional voxel images can clearly show the morphology and signal intensity of the masticatory muscles (to determine whether there is hyperplasia or atrophy), the location of the temporomandibular joint disc, the course of neurovascular bundles, etc.

[0039] In some exemplary embodiments, the three-dimensional image information registration module 110 is specifically configured as follows: Based on the iterative nearest point algorithm and landmark constraints, global rigid registration is performed on the three-dimensional image information of the facial soft tissue and the three-dimensional image information of the jawbone to obtain the three-dimensional image information of the facial soft tissue and the three-dimensional image information of the jawbone after global rigid registration. Based on the thin-plate spline algorithm and statistical shape model constraints, local non-rigid registration is performed on the three-dimensional images of facial soft tissue and jawbone after global rigid registration to obtain the spatially aligned three-dimensional images of facial soft tissue and jawbone.

[0040] In practice, rigid registration means that during the registration process, only the source data is allowed to be rotated and translated as a whole, without any changes in its shape.

[0041] As an example, rigid registration is achieved based on the Iterative Closest Point (ICP) algorithm and landmark constraints.

[0042] ICP solves for the optimal rigid transformation (R, T) by iteratively performing the following steps: Step (1) Corresponding point search: For each point in the point cloud, find the nearest neighbor point on the surface of the target 3D voxel as the current hypothetical corresponding point pair.

[0043] Step (2) Transformation solution and update: Based on this set of corresponding point pairs, calculate a rigid transformation (R, T) that minimizes the sum of squared distances between all corresponding point pairs, and apply this transformation to the point cloud.

[0044] Step (3) Repeat steps (1) and (2) until the transformation converges.

[0045] In the process of achieving rigid registration based on the iterative nearest point algorithm, geometric constraints based on landmark points are introduced. Specifically, in the error function of ICP, in addition to the "nearest point distance" term, a "landmark point pair distance" term is added. This is equivalent to telling the algorithm some anatomical key points that must be aligned (such as the tip of the nose and the corner of the eye) at the beginning of the algorithm, thereby guiding ICP to converge to the correct global solution faster and more stably, and avoiding getting trapped in local optima due to poor initial position.

[0046] In practice, non-rigid registration refers to the registration process that not only allows overall rotation and translation, but also allows, and even requires, local elastic deformations such as stretching, compression, and bending of the data.

[0047] As an example, non-rigid registration is achieved based on the Thin-Plate Spline (TPS) algorithm and statistical shape model constraints.

[0048] TPS is used to find a smooth deformation field with minimum bending energy that can accurately pass through (or fit) a given set of control points (i.e., corresponding anatomical landmarks on the point cloud and 3D voxels) and perform natural interpolation deformation on the region between the control points.

[0049] Input: A set of corresponding control point pairs (e.g., the same set of landmarks labeled on the point cloud and 3D voxels); Modeling: TPS establishes a mathematical function that ensures all control point pairs are perfectly aligned while minimizing the overall “bending” or “stretching” of the entire deformation field (i.e., bending energy). Output: A continuous deformation field. Applying this deformation field to the entire point cloud causes the point cloud to deform smoothly, as if it were being moved by a smooth, extremely thin metal plate, precisely conforming to the surface of the target 3D voxel.

[0050] In the process of achieving non-rigid registration based on thin-plate spline algorithms, the introduction of a statistical shape model (SSM) for accuracy control is an important biophysical constraint on TPS deformation. Pure TPS may produce anatomically unreasonable or even impossible deformations in pursuit of mathematically "perfect fit" (e.g., pulling skin into the bone). SSM defines a "reasonable" range of facial shape variation through statistical analysis of a large number of normal maxillofacial morphologies. During optimization, the algorithm seeks a balance: to satisfy the smooth fit of TPS while ensuring that the deformed shape conforms to the statistical laws of SSM. This ensures that the final deformation result is both accurate and anatomically reliable.

[0051] In related technologies, precise registration analysis between structure (such as the jawbone) and functional appearance (facial soft tissue) has not yet been achieved. However, through the exemplary embodiments described above, in multimodal data registration, both the correct alignment of the overall structure and the fine correction of local soft tissue deformation caused by patient posture, expression, etc., are ensured to achieve high-precision spatial fusion. A two-stage strategy of "global rigid registration (ICP + landmark)" and "local non-rigid registration (TPS + SSM)" is adopted. ICP quickly achieves overall alignment under the constraint of landmark, while TPS performs fine elastic deformation correction under the anatomical rationality constraint of SSM. This scheme ensures that the registration results have both overall spatial accuracy and conform to the local deformation law of facial soft tissue biomechanics, providing a highly reliable and anatomically significant spatial alignment data foundation for subsequent feature extraction.

[0052] The geometric and visual feature extraction module 120 is configured to input the spatially aligned three-dimensional image information of facial soft tissue and the three-dimensional image information of jawbone into a hybrid neural network to obtain the geometric and visual features output by the hybrid neural network, wherein the hybrid neural network includes a graph neural network and a convolutional neural network.

[0053] In some exemplary embodiments, the geometric and visual feature extraction module 120 is specifically configured as follows: Based on the graph neural network, spatially aligned 3D facial soft tissue image information is processed to extract the geometric features. The visual features are extracted based on the spatially aligned three-dimensional image information of the jawbone processed by the convolutional neural network.

[0054] As an example, hybrid neural networks, which combine convolutional neural networks (such as the convolutional modules of ResNet variants) and graph neural networks (such as ViT, Vision Transformer) structures, can achieve multi-channel, multi-scale feature extraction from point cloud data and 3D voxel data.

[0055] Through the above exemplary embodiments, an effective network architecture is employed to extract the most essential features from two data types with vastly different structures: 3D point clouds (unordered, geometric) and voxel images (regular mesh, visual). A graph neural network is assigned specifically to process facial soft tissue point clouds to extract geometric features such as curvature, normal vectors, and topological relationships; a convolutional neural network is assigned specifically to process chin bone images to extract visual features such as bone density, texture, and internal structural patterns. This specialized division of labor in feature extraction ensures that the features extracted from each modality are the most discriminative, providing high-quality, complementary input for subsequent cross-modal fusion.

[0056] The geometric and visual feature fusion module 130 is configured to input the geometric and visual features into an attention-based feature fusion network for cross-modal feature interaction and fusion to obtain maxillofacial fusion features.

[0057] In some exemplary embodiments, the geometric feature and visual feature fusion module 130 is specifically configured as follows: The geometric features and the visual features are concatenated to obtain a fused input sequence; The fused input sequence is input into a feature fusion network based on deep learning and multi-head self-attention mechanisms, so that the geometric features and the visual features interact and are weighted to generate a joint feature vector representing cross-modal association as the maxillofacial fusion feature.

[0058] In practice, to improve the ability to identify minor deformities in children's maxillofacial development, attention mechanisms such as SE-block (deep learning mechanism) and multi-head self-attention are designed to enhance the weight of key regional features, so as to optimize the identification bias caused by class imbalance in small samples and improve the sensitivity and accuracy of early lesion features.

[0059] Through the exemplary embodiments described above, geometric and visual features from different modalities are deeply fused to capture the complex, non-linear relationships between them. Heterogeneous features are serialized through concatenation and then input into a feature fusion network based on a multi-head self-attention mechanism. This mechanism enables the model to automatically calculate and learn the correlation weights between different features (e.g., automatically discovering strong associations between "specific facial depressions" and "specific jawbone morphology"). The resulting "joint feature vector" is no longer an independent set of features, but a unified representation encoding cross-modal semantic associations, greatly enhancing the feature representation capabilities and directly improving the accuracy and robustness of the final recognition.

[0060] The pediatric maxillofacial abnormal development identification module 140 is configured to input the maxillofacial fusion features into the pediatric maxillofacial abnormal development identification model for identification based on the maxillofacial fusion features, and obtain the pediatric maxillofacial abnormal development identification result.

[0061] In some exemplary embodiments, the child maxillofacial abnormality development recognition module 140 is further configured to: A first sample set is constructed, comprising several first samples; wherein, the first sample includes: first sample data and first label data; the first sample data includes training maxillofacial fusion features; the first label data includes training child maxillofacial abnormal development identification results corresponding to the training maxillofacial fusion features; Based on a predetermined machine learning algorithm, a model for identifying abnormal development of the maxillofacial region in children is constructed for training. The predetermined machine learning algorithm can be selected from one or more of the following: Naive Bayes algorithm, decision tree algorithm, support vector machine algorithm, kNN algorithm, neural network algorithm, deep learning algorithm, and logistic regression algorithm. The first sample set is input into the child maxillofacial abnormal development recognition model to be trained for iteration to obtain the performance index of the current model training. In the iteration process, a loss function combining the dice loss function and the focus loss function is introduced. If the performance index does not improve continuously during the iteration process, the model parameters of the current child maxillofacial abnormal development recognition model are saved to obtain the child maxillofacial abnormal development recognition model.

[0062] In practice, the first sample set covers developmental data of normal children aged 6 to 11 (e.g., 1,000 cases) and a database of abnormal children including skeletal type III and congenital maxillofacial deformities (e.g., 500 cases).

[0063] In practice, to improve the ability to identify minor deformities in the development of children's maxillofacial abnormalities, a combined loss function of Dice Loss and Focal Loss is introduced during the training process to optimize the identification bias caused by class imbalance in small samples and improve the sensitivity and accuracy of early lesion features.

[0064] Through the above exemplary embodiments, the training of a model for recognizing abnormal maxillofacial development in children overcomes the class imbalance problem caused by a small number of early positive samples and subtle lesion features, and prevents the model training process from getting stuck in local optima or overfitting. Specifically, a joint loss function of Dice Loss and Focal Loss is introduced for model training. Focal Loss effectively focuses on samples that are difficult to classify (usually early, subtle abnormalities), reducing the dominant influence of a large number of easily classified samples (normal regions); Dice Loss optimizes the overall matching degree of the model to the target region. The combined use significantly improves the model's sensitivity (recall) and recognition accuracy (precision) for early, mild maxillofacial developmental abnormalities in children, enabling the device to truly possess the ability of "early diagnosis". At the same time, the iteration stopping condition (no improvement in performance indicators) described in the claims is a conventional but effective training strategy to prevent overfitting.

[0065] In some exemplary embodiments, the device for identifying abnormal development of the maxillofacial region in children further includes a prediction module for abnormal development of the maxillofacial region in children (not shown in the figure), which is configured to: The maxillofacial fusion features are input into the child maxillofacial abnormal development prediction model to make predictions based on the maxillofacial fusion features, and the prediction results of child maxillofacial abnormal development are obtained. The module for predicting abnormal development of the maxillofacial region in children is also configured to: A second sample set is constructed, comprising several second samples; wherein, the second sample includes: second sample data and second label data; the second sample data includes training maxillofacial fusion features; the second label data includes training prediction results of abnormal maxillofacial development in children corresponding to the training maxillofacial fusion features; Based on a predetermined machine learning algorithm, a predictive model for abnormal development of the maxillofacial region in children is constructed for training. The predetermined machine learning algorithm can be selected from one or more of the following: Naive Bayes algorithm, decision tree algorithm, support vector machine algorithm, kNN algorithm, neural network algorithm, deep learning algorithm, and logistic regression algorithm. The second sample set is input into the child maxillofacial abnormal development prediction model to be trained for iteration to obtain the performance index of the current model training. In the iteration process, Bayesian optimization algorithm and transfer learning mechanism are introduced. If the performance index does not improve continuously during the iteration process, the model parameters of the current child maxillofacial abnormal development prediction model are saved to obtain the child maxillofacial abnormal development prediction model.

[0066] In practice, the second sample set covers developmental data of normal children aged 6 to 11 (e.g., 1000 cases) and a database of abnormal children including skeletal type III and congenital maxillofacial deformities (e.g., 500 cases).

[0067] In practice, the pediatric maxillofacial abnormal development prediction model includes a digital twin device for maxillofacial development built based on XGBoost and LSTM time-series prediction models. This device can simulate the structural development trend of specific children under different intervention strategies and provide automatic early warnings for deviations from the standard trajectory. It can also visualize and quantitatively score future developmental risks, assisting doctors in making refined decisions on intervention plans.

[0068] Through the above exemplary embodiments, it is not only sufficient to diagnose the current state, but also to predict the future maxillofacial development trajectory and abnormal risks of an individual based on the current multimodal fusion features, thus achieving a leap from "identification" to "early warning".

[0069] Specifically, the device incorporates a module for predicting abnormal maxillofacial development in children, utilizing a trained prediction model to analyze fusion features. This expands the device's functional boundaries, upgrading its output from static "identification results" to "predictive results" that include dynamic trends, such as developmental deviation warnings and risk scores. This provides clinicians with proactive decision support, enabling a shift in treatment models from passive therapy to proactive intervention and dynamic management. Its training method incorporates Bayesian optimization for efficient hyperparameter tuning and utilizes transfer learning mechanisms to continuously improve the model using new data, ensuring high performance and strong adaptability of the prediction model in real clinical environments.

[0070] In some exemplary embodiments, stratified sampling is used to construct an independent validation set (e.g., 300 cases) to compare the consistency between the output of the pediatric maxillofacial abnormal development recognition model and the pediatric maxillofacial abnormal development prediction model and expert human interpretation. Performance is evaluated using indicators such as area under the ROC curve (AUC), sensitivity, specificity, positive predictive value, and negative predictive value. The algorithm optimization iteration adopts Bayesian parameter tuning and transfer learning mechanisms to continuously fine-tune the model after new cases are input, so as to ensure that the platform has stable and scalable adaptability in real clinical environments.

[0071] The technical solution provided in this disclosure has the following advantages compared with the prior art: The automated registration of multimodal data eliminates the need for user space integration experience, the data-driven deep neural network feature extraction and fusion eliminates subjective bias of human judgment, and the intelligent and sensitive identification of early and subtle anomalies is achieved through multi-source information complementarity and attention enhancement mechanisms.

[0072] In summary, it reduces the user experience requirements, avoids the influence of subjective factors, and identifies early, subtle anomalies.

[0073] Furthermore, non-invasive, low-radiation multimodal imaging methods effectively improve the sensitivity and specificity of identification, reducing radiation risks (especially suitable for children); by establishing an intelligent identification platform, the identification model is shifted from experience-driven to data-driven, enabling early identification and intervention and improving identification efficiency. It also provides auxiliary tools for non-orthodontic specialists, improving the accuracy of condition identification.

[0074] For ease of description, the above apparatus is described in terms of its functions, divided into various modules. Of course, in implementing this disclosure, the functions of each module can be implemented in one or more software and / or hardware.

[0075] Based on the same inventive concept, corresponding to the identification device for abnormal development of the maxillofacial region in children provided in any of the above embodiments, this disclosure also provides a method for identifying abnormal development of the maxillofacial region in children.

[0076] refer to Figure 2 This is a flowchart illustrating a method for identifying abnormal development of the maxillofacial region in children, provided by an exemplary embodiment of this disclosure.

[0077] The identification method for abnormal development of the maxillofacial region in children includes the following steps: Step S210: Determine the three-dimensional image information of facial soft tissue and the three-dimensional image information of jawbone, and perform global rigid registration and local non-rigid registration on the three-dimensional image information of facial soft tissue and the three-dimensional image information of jawbone to obtain spatially aligned three-dimensional image information of facial soft tissue and jawbone.

[0078] In some exemplary embodiments, the step of performing global rigid registration and local non-rigid registration on the three-dimensional image information of the facial soft tissue and the three-dimensional image information of the jawbone to obtain spatially aligned three-dimensional image information of the facial soft tissue and the jawbone includes: Based on the iterative nearest point algorithm and landmark constraints, global rigid registration is performed on the three-dimensional image information of the facial soft tissue and the three-dimensional image information of the jawbone to obtain the three-dimensional image information of the facial soft tissue and the three-dimensional image information of the jawbone after global rigid registration. Based on the thin-plate spline algorithm and statistical shape model constraints, local non-rigid registration is performed on the three-dimensional images of facial soft tissue and jawbone after global rigid registration to obtain the spatially aligned three-dimensional images of facial soft tissue and jawbone.

[0079] Step S220: Input the spatially aligned three-dimensional image information of facial soft tissue and the three-dimensional image information of jawbone into a hybrid neural network to obtain the geometric features and visual features output by the hybrid neural network, wherein the hybrid neural network includes a graph neural network and a convolutional neural network.

[0080] In some exemplary embodiments, the spatially aligned three-dimensional image information of facial soft tissue and three-dimensional image information of the jawbone are input into a hybrid neural network to obtain geometric and visual features output by the hybrid neural network. The hybrid neural network includes a graph neural network and a convolutional neural network, comprising: Based on the graph neural network, spatially aligned 3D facial soft tissue image information is processed to extract the geometric features. The visual features are extracted based on the spatially aligned three-dimensional image information of the jawbone processed by the convolutional neural network.

[0081] Step S230: Input the geometric features and the visual features into an attention-based feature fusion network for cross-modal feature interaction and fusion to obtain maxillofacial fusion features.

[0082] In some exemplary embodiments, the step of inputting the geometric features and the visual features into an attention-based feature fusion network for cross-modal feature interaction and fusion to obtain maxillofacial fusion features includes: The geometric features and the visual features are concatenated to obtain a fused input sequence; The fused input sequence is input into a feature fusion network based on deep learning and multi-head self-attention mechanisms, so that the geometric features and the visual features interact and are weighted to generate a joint feature vector representing cross-modal association as the maxillofacial fusion feature.

[0083] Step S240: Input the maxillofacial fusion features into the child maxillofacial abnormal development recognition model and perform recognition based on the maxillofacial fusion features to obtain the child maxillofacial abnormal development recognition result.

[0084] In some exemplary embodiments, the training method of the child maxillofacial abnormal development recognition model includes: A first sample set is constructed, comprising several first samples; wherein, the first sample includes: first sample data and first label data; the first sample data includes training maxillofacial fusion features; the first label data includes training child maxillofacial abnormal development identification results corresponding to the training maxillofacial fusion features; Based on a predetermined machine learning algorithm, a model for identifying abnormal development of the maxillofacial region in children is constructed for training. The predetermined machine learning algorithm can be selected from one or more of the following: Naive Bayes algorithm, decision tree algorithm, support vector machine algorithm, kNN algorithm, neural network algorithm, deep learning algorithm, and logistic regression algorithm. The first sample set is input into the child maxillofacial abnormal development recognition model to be trained for iteration to obtain the performance index of the current model training. In the iteration process, a loss function combining the dice loss function and the focus loss function is introduced. If the performance index does not improve continuously during the iteration process, the model parameters of the current child maxillofacial abnormal development recognition model are saved to obtain the child maxillofacial abnormal development recognition model.

[0085] In some exemplary embodiments, the method further includes: The maxillofacial fusion features are input into the child maxillofacial abnormal development prediction model to make predictions based on the maxillofacial fusion features, and the prediction results of child maxillofacial abnormal development are obtained. The training methods for the prediction model of abnormal development of the maxillofacial region in children include: A second sample set is constructed, comprising several second samples; wherein, the second sample includes: second sample data and second label data; the second sample data includes training maxillofacial fusion features; the second label data includes training prediction results of abnormal maxillofacial development in children corresponding to the training maxillofacial fusion features; Based on a predetermined machine learning algorithm, a predictive model for abnormal development of the maxillofacial region in children is constructed for training. The predetermined machine learning algorithm can be selected from one or more of the following: Naive Bayes algorithm, decision tree algorithm, support vector machine algorithm, kNN algorithm, neural network algorithm, deep learning algorithm, and logistic regression algorithm. The second sample set is input into the child maxillofacial abnormal development prediction model to be trained for iteration to obtain the performance index of the current model training. In the iteration process, Bayesian optimization algorithm and transfer learning mechanism are introduced. If the performance index does not improve continuously during the iteration process, the model parameters of the current child maxillofacial abnormal development prediction model are saved to obtain the child maxillofacial abnormal development prediction model.

[0086] The methods described above are implemented using the corresponding identification device for abnormal development of the maxillofacial region in children in any of the foregoing embodiments, and have the beneficial effects of the corresponding device embodiments, which will not be repeated here.

[0087] It should be noted that the method of this disclosure embodiment can be executed by a single device, such as a computer or server. The method of this embodiment can also be applied to a distributed scenario, where multiple devices cooperate to complete the task. In such a distributed scenario, one of these devices may execute only one or more steps of the method of this disclosure embodiment, and the multiple devices will interact with each other to complete the method described.

[0088] It should be noted that the above description describes some embodiments of this disclosure. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in a different order than that shown in the above embodiments and still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require a specific or sequential order to achieve the desired result. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0089] Based on the same inventive concept, corresponding to any of the above embodiments, this disclosure also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the method for identifying abnormal development of the maxillofacial region in children as described in any of the above embodiments.

[0090] Figure 3 This embodiment illustrates a more specific hardware structure of an electronic device, which may include a processor 310, a memory 320, an input / output interface 330, a communication interface 340, and a bus 350. The processor 310, memory 320, input / output interface 330, and communication interface 340 are interconnected internally via the bus 350.

[0091] The processor 310 can be implemented using a general-purpose CPU (Central Processing Unit), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this specification.

[0092] The memory 320 can be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory), static storage device, dynamic storage device, etc. The memory 320 can store the operating system and other applications. When the technical solutions provided in the embodiments of this specification are implemented by software or firmware, the relevant program code is stored in the memory 320 and is called and executed by the processor 310.

[0093] Input / output interface 330 is used to connect input / output modules to realize information input and output. Input / output modules can be configured as components in the device (not shown in the figure) or externally connected to the device to provide corresponding functions. Input devices may include keyboards, mice, touch screens, microphones, various sensors, etc., and output devices may include displays, speakers, vibrators, indicator lights, etc.

[0094] The communication interface 340 is used to connect a communication module (not shown in the figure) to enable communication between this device and other devices. The communication module can communicate via wired means (such as USB, Ethernet cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.).

[0095] Bus 350 includes a pathway for transmitting information between various components of the device, such as processor 310, memory 320, input / output interface 330, and communication interface 340.

[0096] It should be noted that although the above-described device only shows the processor 310, memory 320, input / output interface 330, communication interface 340, and bus 350, in specific implementations, the device may also include other components necessary for normal operation. Furthermore, those skilled in the art will understand that the above-described device may only include the components necessary for implementing the embodiments of this specification, and not necessarily all the components shown in the figures.

[0097] The electronic devices described above are used to implement the corresponding methods for identifying abnormal development of the maxillofacial region in children in any of the foregoing embodiments, and have the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0098] The memory 320 stores machine-readable instructions executable by the processor 310. When the electronic device is running, the processor 310 communicates with the memory 320 via the bus 330, causing the processor 310 to execute the following instructions during operation: Determine the three-dimensional image information of facial soft tissue and the three-dimensional image information of jawbone, and perform global rigid registration and local non-rigid registration on the three-dimensional image information of facial soft tissue and the three-dimensional image information of jawbone to obtain spatially aligned three-dimensional image information of facial soft tissue and three-dimensional image information of jawbone. The spatially aligned three-dimensional images of facial soft tissue and jawbone are input into a hybrid neural network to obtain the geometric and visual features output by the hybrid neural network. The hybrid neural network includes a graph neural network and a convolutional neural network. The geometric features and visual features are input into an attention-based feature fusion network for cross-modal feature interaction and fusion to obtain maxillofacial fusion features; The maxillofacial fusion features are input into the child maxillofacial abnormal development identification model to identify the abnormal development of the child's maxillofacial region.

[0099] In one possible implementation, the instructions executed by processor 310, which involve performing global rigid registration and local non-rigid registration on the three-dimensional image information of the facial soft tissue and the three-dimensional image information of the jawbone to obtain spatially aligned three-dimensional image information of the facial soft tissue and the jawbone, include: Based on the iterative nearest point algorithm and landmark constraints, global rigid registration is performed on the three-dimensional image information of the facial soft tissue and the three-dimensional image information of the jawbone to obtain the three-dimensional image information of the facial soft tissue and the three-dimensional image information of the jawbone after global rigid registration. Based on the thin-plate spline algorithm and statistical shape model constraints, local non-rigid registration is performed on the three-dimensional images of facial soft tissue and jawbone after global rigid registration to obtain the spatially aligned three-dimensional images of facial soft tissue and jawbone.

[0100] In one possible implementation, the instructions executed by processor 310 include inputting the spatially aligned three-dimensional facial soft tissue image information and jawbone three-dimensional image information into a hybrid neural network to obtain the geometric and visual features output by the hybrid neural network. The hybrid neural network includes a graph neural network and a convolutional neural network, comprising: Based on the graph neural network, spatially aligned 3D facial soft tissue image information is processed to extract the geometric features. The visual features are extracted based on the spatially aligned three-dimensional image information of the jawbone processed by the convolutional neural network.

[0101] In one possible implementation, the instructions executed by processor 310, which involve inputting the geometric features and the visual features into an attention-based feature fusion network for cross-modal feature interaction and fusion to obtain maxillofacial fusion features, include: The geometric features and the visual features are concatenated to obtain a fused input sequence; The fused input sequence is input into a feature fusion network based on deep learning and multi-head self-attention mechanisms, so that the geometric features and the visual features interact and are weighted to generate a joint feature vector representing cross-modal association as the maxillofacial fusion feature.

[0102] In one possible implementation, the training method of the child maxillofacial abnormal development recognition model in the instructions executed by the processor 310 includes: A first sample set is constructed, comprising several first samples; wherein, the first sample includes: first sample data and first label data; the first sample data includes training maxillofacial fusion features; the first label data includes training child maxillofacial abnormal development identification results corresponding to the training maxillofacial fusion features; Based on a predetermined machine learning algorithm, a model for identifying abnormal development of the maxillofacial region in children is constructed for training. The predetermined machine learning algorithm can be selected from one or more of the following: Naive Bayes algorithm, decision tree algorithm, support vector machine algorithm, kNN algorithm, neural network algorithm, deep learning algorithm, and logistic regression algorithm. The first sample set is input into the child maxillofacial abnormal development recognition model to be trained for iteration to obtain the performance index of the current model training. In the iteration process, a loss function combining the dice loss function and the focus loss function is introduced. If the performance index does not improve continuously during the iteration process, the model parameters of the current child maxillofacial abnormal development recognition model are saved to obtain the child maxillofacial abnormal development recognition model.

[0103] In one possible implementation, the method further includes the following instructions executed by the processor 310: The maxillofacial fusion features are input into the child maxillofacial abnormal development prediction model to make predictions based on the maxillofacial fusion features, and the prediction results of child maxillofacial abnormal development are obtained. The training methods for the prediction model of abnormal development of the maxillofacial region in children include: A second sample set is constructed, comprising several second samples; wherein, the second sample includes: second sample data and second label data; the second sample data includes training maxillofacial fusion features; the second label data includes training prediction results of abnormal maxillofacial development in children corresponding to the training maxillofacial fusion features; Based on a predetermined machine learning algorithm, a predictive model for abnormal development of the maxillofacial region in children is constructed for training. The predetermined machine learning algorithm can be selected from one or more of the following: Naive Bayes algorithm, decision tree algorithm, support vector machine algorithm, kNN algorithm, neural network algorithm, deep learning algorithm, and logistic regression algorithm. The second sample set is input into the child maxillofacial abnormal development prediction model to be trained for iteration to obtain the performance index of the current model training. In the iteration process, Bayesian optimization algorithm and transfer learning mechanism are introduced. If the performance index does not improve continuously during the iteration process, the model parameters of the current child maxillofacial abnormal development prediction model are saved to obtain the child maxillofacial abnormal development prediction model.

[0104] Based on the same inventive concept, corresponding to the methods of any of the above embodiments, this disclosure also provides a non-transitory computer-readable storage medium storing computer instructions for causing the computer to execute the method for identifying abnormal development of the maxillofacial region in children as described in any of the above embodiments.

[0105] The computer-readable medium of this embodiment includes permanent and non-permanent, removable and non-removable media, and information storage can be implemented by any method or technology. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transfer medium that can be used to store information accessible by a computing device.

[0106] The aforementioned non-transitory computer-readable storage media can be any available medium or data storage device that a computer can access, including but not limited to magnetic storage (e.g., floppy disks, hard disks, magnetic tapes, magneto-optical disks (MOs), etc.), optical storage (e.g., CDs, DVDs, BDs, HVDs, etc.), and semiconductor storage (e.g., ROMs, EPROMs, EEPROMs, non-volatile memory (NAND flash), solid-state drives (SSDs)).

[0107] The computer instructions stored in the storage medium of the above embodiments are used to cause the computer to execute the method for identifying abnormal development of the maxillofacial region in children as described in any of the above exemplary method sections, and have the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0108] Based on the same inventive concept, corresponding to the method for identifying abnormal development of the maxillofacial region in children described in any of the above embodiments, this disclosure also provides a computer program product, which includes computer program instructions. In some embodiments, the computer program instructions can be executed by one or more processors of a computer to cause the computer and / or the processor to perform the method for identifying abnormal development of the maxillofacial region in children. Corresponding to the execution entity for each step in each embodiment of the method for identifying abnormal development of the maxillofacial region in children, the processor executing the corresponding step can belong to the corresponding execution entity.

[0109] The computer program product of the above embodiments is used to enable the computer and / or the processor to execute the method for identifying abnormal development of the maxillofacial region in children as described in any of the above embodiments, and has the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0110] Those skilled in the art will recognize that embodiments of this disclosure can be implemented as a system, method, or computer program product. Therefore, this disclosure can be implemented as entirely hardware, entirely software (including firmware, resident software, microcode, etc.), or a combination of hardware and software, generally referred to herein as a "circuit," "module," or "system." Furthermore, in some embodiments, this disclosure can also be implemented as a computer program product contained in one or more computer-readable media, which includes computer-readable program code.

[0111] Any combination of one or more computer-readable media may be used. A computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. A computer-readable storage medium can be, for example,, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples (not exhaustive) of a computer-readable storage medium may include: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this document, a computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in connection with an instruction execution system, apparatus, or device.

[0112] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, capable of sending, propagating, or transmitting programs for use by or in connection with an instruction execution system, apparatus, or device.

[0113] Program code contained on a computer-readable medium may be transmitted using any suitable medium, including but not limited to wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.

[0114] Computer program code for performing the operations of this disclosure can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0115] It should be understood that each block of a flowchart and / or block diagram, as well as combinations of blocks in a flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device to produce a machine that, when executed by a computer or other programmable data processing device, creates means for implementing the functions / operations specified in the blocks of the flowchart and / or block diagram.

[0116] These computer program instructions may also be stored in a computer-readable medium that enables a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable medium produce a product comprising an instruction apparatus that implements the functions / operations specified in the boxes of a flowchart and / or block diagram.

[0117] Computer program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, such that the instructions that execute on the computer or other programmable apparatus can provide a process for implementing the functions / operations specified in the boxes of a flowchart and / or block diagram.

[0118] Furthermore, although the operations of the methods of this disclosure are described in a specific order in the accompanying drawings, this does not require or imply that these operations must be performed in that specific order, or that all of the operations shown must be performed to achieve the desired result. Rather, the steps depicted in the flowcharts may be executed in a different order. Additionally or alternatively, certain steps may be omitted, multiple steps may be combined into one step, and / or one step may be broken down into multiple steps.

[0119] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. Each block in a flowchart or block diagram may represent a module, segment, or portion of code, which contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0120] It should be noted that although several modules or units for the device used to perform actions have been mentioned in the detailed description above, this division is not mandatory. In fact, according to the embodiments of this application, the features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided and embodied by multiple modules or units.

[0121] Those skilled in the art should understand that the discussion of any of the above embodiments is merely exemplary and is not intended to imply that the scope of this application (including the claims) is limited to these examples; within the framework of this application, the technical features of the above embodiments or different embodiments can also be combined, the steps can be implemented in any order, and there are many other variations of different aspects of the embodiments of this application as described above, which are not provided in the details for the sake of brevity.

[0122] Additionally, to simplify the description and discussion, and to avoid obscuring the embodiments of this application, the well-known power / ground connections to integrated circuit (IC) chips and other components may or may not be shown in the provided drawings. Furthermore, the apparatus may be shown in block diagram form to avoid obscuring the embodiments of this application, and this also takes into account the fact that the details of the implementation of these block diagram apparatuses are highly dependent on the platform on which the embodiments of this application will be implemented (i.e., these details should be fully understood by those skilled in the art). While specific details (e.g., circuits) have been set forth to describe exemplary embodiments of this application, it will be apparent to those skilled in the art that the embodiments of this application can be implemented without these specific details or with variations thereof. Therefore, these descriptions should be considered illustrative rather than restrictive.

[0123] Although this application has been described in conjunction with specific embodiments thereof, many substitutions, modifications, and variations of these embodiments will be apparent to those skilled in the art from the foregoing description. For example, other memory architectures (e.g., dynamic RAM (DRAM)) may be used with the embodiments discussed.

[0124] The embodiments of this application are intended to cover all such substitutions, modifications, and variations that fall within the broad scope of the appended claims. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the embodiments of this application should be included within the protection scope of this application.

[0125] While the spirit and principles of this disclosure have been described with reference to several specific embodiments, it should be understood that this disclosure is not limited to the disclosed specific embodiments, and the division of aspects does not imply that features in these aspects cannot be combined for benefit; such division is merely for convenience of expression. This disclosure is intended to cover various modifications and equivalent arrangements included within the spirit and scope of the appended claims. The scope of the appended claims is to be interpreted in the broadest sense, thereby encompassing all such modifications and equivalent structures and functions.

[0126] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0127] The above description is merely a specific embodiment of this disclosure, enabling those skilled in the art to understand or implement it. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this disclosure. Therefore, this disclosure is not to be limited to the embodiments described herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A device for identifying abnormal development of the maxillofacial region in children, characterized in that, include: The three-dimensional image information registration module is configured to determine the three-dimensional image information of facial soft tissue and the three-dimensional image information of jawbone, and to perform global rigid registration and local non-rigid registration on the three-dimensional image information of facial soft tissue and the three-dimensional image information of jawbone to obtain spatially aligned three-dimensional image information of facial soft tissue and three-dimensional image information of jawbone. The geometric and visual feature extraction module is configured to input the spatially aligned three-dimensional image information of facial soft tissue and the three-dimensional image information of jawbone into a hybrid neural network to obtain the geometric and visual features output by the hybrid neural network, wherein the hybrid neural network includes a graph neural network and a convolutional neural network. The geometric and visual feature fusion module is configured to input the geometric and visual features into an attention-based feature fusion network for cross-modal feature interaction and fusion to obtain maxillofacial fusion features; The pediatric maxillofacial abnormal development identification module is configured to input the maxillofacial fusion features into the pediatric maxillofacial abnormal development identification model for identification based on the maxillofacial fusion features, and obtain the pediatric maxillofacial abnormal development identification result.

2. The apparatus according to claim 1, characterized in that, The three-dimensional image information registration module is specifically configured as follows: Based on the iterative nearest point algorithm and landmark constraints, global rigid registration is performed on the three-dimensional image information of the facial soft tissue and the three-dimensional image information of the jawbone to obtain the three-dimensional image information of the facial soft tissue and the three-dimensional image information of the jawbone after global rigid registration. Based on the thin-plate spline algorithm and statistical shape model constraints, local non-rigid registration is performed on the three-dimensional images of facial soft tissue and jawbone after global rigid registration to obtain the spatially aligned three-dimensional images of facial soft tissue and jawbone.

3. The apparatus according to claim 1, characterized in that, The geometric and visual feature extraction module is specifically configured as follows: Based on the graph neural network, spatially aligned 3D facial soft tissue image information is processed to extract the geometric features. The visual features are extracted based on the spatially aligned three-dimensional image information of the jawbone processed by the convolutional neural network.

4. The apparatus according to claim 1, characterized in that, The geometric feature and visual feature fusion module is specifically configured as follows: The geometric features and the visual features are concatenated to obtain a fused input sequence; The fused input sequence is input into a feature fusion network based on deep learning and multi-head self-attention mechanisms, so that the geometric features and the visual features interact and are weighted to generate a joint feature vector representing cross-modal association as the maxillofacial fusion feature.

5. The apparatus according to claim 1, characterized in that, The module for identifying abnormal development of the child's maxillofacial region is also configured to: A first sample set is constructed, comprising several first samples; wherein, the first sample includes: first sample data and first label data; the first sample data includes training maxillofacial fusion features; the first label data includes training child maxillofacial abnormal development identification results corresponding to the training maxillofacial fusion features; Based on a predetermined machine learning algorithm, a model for identifying abnormal development of the maxillofacial region in children is constructed for training. The predetermined machine learning algorithm can be selected from one or more of the following: Naive Bayes algorithm, decision tree algorithm, support vector machine algorithm, kNN algorithm, neural network algorithm, deep learning algorithm, and logistic regression algorithm. The first sample set is input into the child maxillofacial abnormal development recognition model to be trained for iteration to obtain the performance index of the current model training. In the iteration process, a loss function combining the dice loss function and the focus loss function is introduced. If the performance index does not improve continuously during the iteration process, the model parameters of the current child maxillofacial abnormal development recognition model are saved to obtain the child maxillofacial abnormal development recognition model.

6. The apparatus according to claim 1, characterized in that, It also includes a module for predicting abnormal development of the maxillofacial region in children, which is configured as follows: The maxillofacial fusion features are input into the child maxillofacial abnormal development prediction model to make predictions based on the maxillofacial fusion features, and the prediction results of child maxillofacial abnormal development are obtained. The module for predicting abnormal development of the maxillofacial region in children is also configured to: A second sample set is constructed, comprising several second samples; wherein, the second sample includes: second sample data and second label data; the second sample data includes training maxillofacial fusion features; the second label data includes training prediction results of abnormal maxillofacial development in children corresponding to the training maxillofacial fusion features; Based on a predetermined machine learning algorithm, a predictive model for abnormal development of the maxillofacial region in children is constructed for training. The predetermined machine learning algorithm can be selected from one or more of the following: Naive Bayes algorithm, decision tree algorithm, support vector machine algorithm, kNN algorithm, neural network algorithm, deep learning algorithm, and logistic regression algorithm. The second sample set is input into the child maxillofacial abnormal development prediction model to be trained for iteration to obtain the performance index of the current model training. In the iteration process, Bayesian optimization algorithm and transfer learning mechanism are introduced. If the performance index does not improve continuously during the iteration process, the model parameters of the current child maxillofacial abnormal development prediction model are saved to obtain the child maxillofacial abnormal development prediction model.

7. A method for identifying abnormal development of the maxillofacial region in children, characterized in that, include: Determine the three-dimensional image information of facial soft tissue and the three-dimensional image information of jawbone, and perform global rigid registration and local non-rigid registration on the three-dimensional image information of facial soft tissue and the three-dimensional image information of jawbone to obtain spatially aligned three-dimensional image information of facial soft tissue and three-dimensional image information of jawbone. The spatially aligned three-dimensional images of facial soft tissue and jawbone are input into a hybrid neural network to obtain the geometric and visual features output by the hybrid neural network. The hybrid neural network includes a graph neural network and a convolutional neural network. The geometric features and visual features are input into an attention-based feature fusion network for cross-modal feature interaction and fusion to obtain maxillofacial fusion features; The maxillofacial fusion features are input into the child maxillofacial abnormal development identification model to identify the abnormal development of the child's maxillofacial region.

8. A computer device, comprising a memory, a processor, and a computer program stored in the memory, characterized in that, The processor executes the computer program to implement the steps of the method of claim 7.

9. A computer-readable storage medium having a computer program / instructions stored thereon, characterized in that, When the computer program / instructions are executed by the processor, they implement the steps of the method of claim 7.

10. A computer program product comprising a computer program / instructions, characterized in that, When the computer program / instructions are executed by the processor, they implement the steps of the method of claim 7.

Images