Method and device for generating orthodontic effect preview image

Abstract

The invention discloses an orthodontic effect preview image generation method and device, and relates to the technical field of orthodontics, and the method comprises the following steps: 1, data collection; 2, data preprocessing; step 3, semantic association construction; 4, model prediction; step 5, carrying out difference visualization; according to the method, the problem of format and dimension unification in multi-modal data integration is solved, semantic association is constructed to enable the model to understand internal logic of data, difference visualization means are perfected to assist doctors in positioning reasons and making decisions, orthodontic effect prediction accuracy is effectively improved, making of personalized treatment schemes is assisted, and the method is suitable for popularization and application. And many challenges confronted by a traditional method in the field of orthodontics are overcome.

Description

Technical Field

[0001] The present invention relates to the technical field of oral orthodontics, and in particular to a method and device for generating an oral orthodontic effect preview image. Background Art

[0002] In the field of orthodontics, accurate prediction of orthodontic outcomes is crucial for developing personalized treatment plans.

[0003] Traditional methods face many challenges when integrating multimodal data such as oral cone beam CT data, oral scan data, facial image data, muscle strength data, fat thickness data, and basic patient information. On the one hand, it is difficult to unify data formats and physical dimensions. On the other hand, more importantly, it is difficult for the model to understand the inherent logical relationship between data from different sources at a semantic level. In addition, when the model prediction results differ from the doctor's clinical experience or intuitive judgment, there is a lack of effective visualization methods to assist doctors in quickly locating the cause of the difference and providing decision support.

[0004] In view of this, a method and device for generating an orthodontic effect preview image are provided to overcome the above problems. Summary of the invention

[0005] The purpose of the present invention is to provide a method and device for generating an orthodontic effect preview image to solve the problems raised in the above-mentioned background technology.

[0006] In order to solve the above technical problems, the present invention provides a method and device for generating an orthodontic effect preview image, comprising the following steps: Step 1: Data collection: Comprehensively obtain the patient's oral cone beam CT data, oral scan data, facial image data, muscle strength data, fat thickness data and basic patient information to ensure the integrity of multimodal data; Step 2: Data preprocessing: The collected multimodal data are converted into different formats and processed in a unified dimension. A dynamic range adjustment normalization algorithm is used for muscle strength data, and a proportional normalization algorithm based on anatomical structure characteristics is used for fat thickness data to improve the stability and comparability of the data and lay the foundation for subsequent accurate analysis. Step 3: Semantic association construction: Construct a semantic mapping framework and logical reasoning mechanism between multimodal data. Through modular semantic construction, construct different types of data semantic associations into independent sub-modules, including but not limited to constructing independent facial muscle strength-tooth movement sub-modules and facial fat distribution-tooth movement influence sub-modules. The model is used to deeply understand the intrinsic connection between various data elements in the orthodontic process, including the precise relationship between changes in facial muscle strength and the direction and distance of specific tooth movement, as well as the complex regulatory effect of facial fat distribution on it, so that when the knowledge in the field of orthodontics is updated, the relevant modules can be adjusted and retrained with the smallest impact range; Step 4: Model prediction: Based on the constructed semantic association, the trained model is used to predict the orthodontic effect and generate a preview image. During the training phase, the model uses an adversarial training method combined with L1 and L2 regularization. While learning individual difference characteristics, it effectively maintains the grasp of the overall patient group characteristic distribution to avoid overfitting. The training data set is highly diverse, covering a large number of different types of oral orthodontic case data with different individual characteristics. Step 5: Difference visualization: Compare the model prediction results with the patient's original multimodal data images, color-code the difference areas and mark the specific deviation values ​​according to the gradient color scheme of the deviation degree, and establish an automated testing and adaptation process. After each semantic update, use a massive test data set to verify the visualization function, and automatically detect and repair possible display anomalies and interaction failures.

[0007] Furthermore, in the semantic association construction step, the independent sub-modules interact with each other through interfaces based on semantic rules to ensure the accuracy and efficiency of data transmission and integration between different modules.

[0008] Furthermore, the automated testing and adaptation process includes independent testing modules for different types of visualization elements, including but not limited to images, charts, and numerical annotations, which can accurately locate and fix problems.

[0009] Furthermore, the adversarial training method introduces a virtual adversarial sample generation mechanism during the model training process to further enhance the model's sensitivity and adaptability to slight changes in the data.

[0010] Furthermore, ways to expand the diversity of training data sets include, but are not limited to, widely collecting case data from different regions, different age groups, and different ethnic groups, and weighting special case data to increase its influence in training.

[0011] Furthermore, in the data preprocessing step, an intelligent correction algorithm based on data distribution characteristics is used for outlier data rather than simple elimination to retain more valid information.

[0012] Furthermore, in the difference visualization step, the color identification of the difference area can also be classified and displayed according to the data type, which includes but is not limited to tooth position difference and facial contour difference, so as to facilitate the doctor to quickly distinguish different types of differences.

[0013] Furthermore, before the model prediction step, a multi-branch racial feature recognition module is constructed according to the differences in facial aesthetic standards of different races. The module performs deep neural network training based on massive facial image big data of different races to identify the racial category to which the input data belongs. The pre-trained aesthetic standard model of the corresponding race is called according to the recognition result. The model is based on statistical analysis and machine learning training of aesthetic feature data such as facial contours, facial feature proportions, and soft tissue fullness of large populations of various races. The race-specific aesthetic standard parameters it outputs are integrated into the logical reasoning mechanism constructed in the semantic association construction step, and are used together with other multimodal data in the model prediction step, so as to integrate the cross-racial aesthetic standard differences into the prediction process of the oral orthodontic effect preview image generation model.

[0014] A device for generating an oral orthodontic effect preview image comprises a data acquisition module, a data preprocessing module, a semantic association building module, a model training and prediction module, and a difference visualization module.

[0015] Compared with the prior art, the present invention has the following beneficial effects: Through comprehensive and highly correlated multimodal data collection, targeted preprocessing, and deep semantic association construction, multiple factors such as tooth structure, facial soft tissue, and racial aesthetic standards are fully considered, so that the orthodontic plan can be accurately customized from different dimensions based on the actual situation of the individual patient, avoiding the problems of being general and lacking in specificity due to one-sided data or insufficient correlation analysis in the past, meeting the diverse aesthetic and treatment needs of patients of different races, and improving the personalization of orthodontic treatment.

[0016] The construction of a multi-branch ethnic feature recognition module and a race-specific aesthetic standard model, coupled with an advanced semantic association construction method, enables the model to accurately incorporate cross-ethnic aesthetic standard differences, deeply understand the inherent logical connections between various data, and quantify the impact of multiple factors on orthodontic effects. Combined with a diverse and reasonably processed training data set and advanced training strategies, it effectively overcomes problems such as model overfitting and insufficient generalization ability, making the model more accurate and reliable in predicting orthodontic effects, and the output orthodontic effect preview image is more in line with the actual situation, providing doctors and patients with more valuable reference treatment expectations.

[0017] The automated data preprocessing process and targeted algorithms solve the problems of data format and dimensional unification, reduce the tediousness and errors of manual data processing, and facilitate the efficient operation of subsequent models. At the same time, the accurate and fully functional difference visualization module can help doctors quickly locate the differences and causes between model predictions and actual conditions, and ensure the accuracy and stability of visualization through automated testing and adaptation processes, so that doctors can optimize and adjust treatment plans in a timely manner based on visualization results, improve the convenience and overall efficiency of clinical applications, and reduce the time cost caused by repeated evaluation and adjustment of plans.

[0018] The intuitive three-dimensional preview images of orthodontic effects and the clear visualization of differences allow patients to more intuitively understand the orthodontic treatment process and expected results. In particular, seeing the treatment expectations based on their own ethnic aesthetic standards can enhance patients' recognition of the treatment plan. At the same time, the effective explanation of the causes of the differences in the visualization of differences can further eliminate patients' doubts, increase their trust and understanding of the treatment process, thereby improving their cooperation in orthodontic treatment and helping to better achieve treatment goals.

[0019] By constructing a multi-branch ethnic feature recognition module and introducing an aesthetic standard model of the corresponding race, the adaptability of the model to multi-ethnic patient groups is effectively solved, so that the model can accurately identify patients of different races and predict orthodontic effects based on their racially specific facial aesthetic standards; by incorporating racially specific aesthetic standard parameters into the logical reasoning mechanism of semantic association construction, the problem of how to effectively integrate racial difference data and optimize the model prediction logic in the process of multimodal data integration is solved, avoiding model prediction bias caused by racial factors; because it can accurately identify race and apply corresponding aesthetic standard parameters, the model can better meet the actual needs and aesthetic expectations of patients of different races when predicting orthodontic effects; after seeing the preview images of orthodontic effects that meet their own racial aesthetic standards, patients can more intuitively understand the treatment expectations, enhance their confidence in orthodontic treatment, and thus improve compliance during the treatment process; it breaks through the racial limitations of traditional models, enabling oral orthodontic technology to better serve patients of different races around the world, and promotes the development of personalized treatment technology in the field of oral orthodontics in a multi-ethnic context. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 The present invention is a schematic diagram of a method and device for generating a preview image of an oral orthodontic effect. DETAILED DESCRIPTION

[0021] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0022] Embodiment 1: See also Figure 1 As shown, the present invention provides a technical solution: a method and device for generating a preview image of an oral orthodontic effect: Device: Data acquisition module: It is configured to fully acquire the patient's oral cone beam CT data, oral scan data, facial image data, muscle strength data, fat thickness data and basic patient information to ensure the integrity of multimodal data. This module uses professional acquisition equipment. The oral cone beam CT scanner acquires oral internal structure images at a specific resolution (such as 0.2-0.5mm voxel resolution), the intraoral scanner accurately scans the oral surface to obtain point cloud data (the number of points can reach hundreds of thousands to millions), the high-definition facial imaging equipment captures clear facial images (such as 1920×1080 pixels and above), the professional muscle strength measuring instrument measures facial muscle strength (range 0-500N, etc.), the ultrasonic fat thickness measuring instrument acquires facial fat thickness data, and at the same time completely collects the patient's basic information to provide a comprehensive data basis for subsequent orthodontic analysis.

[0023] Data preprocessing module: used to convert the format and unify the dimensions of the collected multimodal data.

[0024] Semantic association construction module: constructs a semantic mapping framework and logical reasoning mechanism between multimodal data, adopting a modular semantic construction method.

[0025] Model training and prediction module: Based on the constructed semantic associations, the trained model is used to predict the orthodontic effect and generate a preview image.

[0026] Difference visualization module: Compare the model prediction results with the patient's original multimodal data images, color-code the difference areas and mark the specific deviation values ​​using a gradient color scheme based on the degree of deviation.

[0027] Step 1: Data collection: Steps: Using a professional oral cone beam CT scanner, the patient's oral cone beam CT data is acquired with a voxel resolution of 0.3mm. The image grayscale value ranges from -1000 to 3000HU. The size of a single patient data file is approximately 300MB, accurately presenting structural details such as teeth and alveolar bones.

[0028] The patient's mouth is scanned with an intraoral scanner to obtain oral scan data of approximately 1.5 million points with an accuracy of 0.08mm, which can carefully depict the surface morphology and position information of the teeth. Each point has three-dimensional coordinates and color information.

[0029] The patient's facial image is captured by high-definition facial imaging equipment, with an image size of 1920×1080 pixels and 24-bit true color, clearly capturing facial features and contour details.

[0030] The muscle strength measuring instrument was used to measure the facial muscle strength of the patient, with a measurement range of 0-500N, an accuracy of 1N, and a sampling frequency of 50Hz. For example, the strength of the patient's masseter muscle at maximum biting force was 350N, and the strength of the temporalis muscle under a specific action was 180N. 15 data points were collected in one chewing cycle to record the muscle strength change curve.

[0031] The patient's facial fat thickness is measured using an ultrasonic fat thickness meter with an accuracy of 0.1mm. For example, the patient's cheek fat thickness is 12mm below the zygomatic bone, 8mm above the mandibular angle, and 6mm under the chin.

[0032] Collect basic information of patients, such as a 15-year-old male patient with no history of oral disease but a history of mild malocclusion in the family.

[0033] Corresponding beneficial effects: The comprehensive collection of multimodal data provides a rich source of information for building accurate orthodontic effect preview images. High-resolution oral cone beam CT and oral scan data can accurately outline the teeth and internal oral structures. Facial image data helps analyze the coordination between the overall facial appearance and teeth. Muscle strength data and fat thickness data provide key basis for orthodontic effect prediction from the perspective of physiological function and facial soft tissue. The patient's basic information can assist in considering factors such as individual growth and development, thereby laying a solid foundation for the formulation of personalized treatment plans and preliminarily solving the problem of single and incomplete data sources in traditional methods.

[0034] Step 2: Data preprocessing: Steps: Convert oral cone beam CT data and oral scan data into a unified STL format; For muscle strength data, let the original muscle strength data be , its minimum value , the maximum value , with masseter muscle strength For example, the normalized muscle strength data: ; For fat thickness data, set the vertical distance from the zygomatic bone to the mandibular bone , cheek fat thickness, normalized fat thickness data.

[0035] Format conversion achieves standardized integration of different types of oral data, facilitating subsequent unified processing. Specific normalization algorithms for muscle strength data and fat thickness data effectively solve the problem of difficulty in unifying data formats and physical dimensions, and improve the stability and comparability of data. Normalized data can be more accurately involved in model construction and analysis, enhancing the model's ability to understand and process data and reducing model errors caused by data differences.

[0036] Step 3: Semantic association construction: Steps: Constructing the facial muscle strength-tooth movement submodule: Through multiple linear regression analysis, assuming that the teeth are Direction moving distance , Masseter muscle strength Temporalis muscle strength is the independent variable, and the regression equation is (These are example coefficients only). Assume ,but (Plus the error term calculation), the regression coefficients are adjusted through a large amount of data training, so that the model can accurately predict the effect of muscle strength on tooth movement.

[0037] Construct facial fat distribution-tooth movement influence submodule: extract fat distribution features in facial images, such as fat area , Fat center of gravity location etc., using these features as input and tooth movement related parameters as output to train a support vector machine (SVM) model. For example, using the Gaussian kernel function , determined by cross validation The model accuracy is the highest when the SVM model is used, which enables the SVM model to predict the degree of facial soft tissue deformation during tooth movement based on facial fat distribution. Each submodule interacts through a semantic rule-based interface to optimize the tooth movement plan.

[0038] The modular semantic construction method successfully overcomes the difficulty of the model to understand the internal logical relationship of data from different sources. The facial muscle strength-tooth movement submodule quantifies the mechanism of muscle strength on tooth movement through regression analysis, and the facial fat distribution-tooth movement influence submodule reveals the relationship between fat distribution and tooth movement and facial soft tissue changes with the help of the SVM model. The two work together and interact through the semantic rule interface, enabling the model to fully and deeply understand the internal connection of multimodal data in the orthodontic process, thereby providing a reliable semantic framework support for accurate orthodontic effect prediction, greatly improving the prediction accuracy and reliability of the model.

[0039] Step 4: Model prediction: Steps: Model training phase: 8,000 orthodontic case data were collected as training sets, of which the ratio of Asian, European, and African patients was 4:3:3, the ratio of children, adolescents, and adults was 2:5:3, and special cases accounted for 30%. Special cases were weighted, and common cases were weighted , special case weight .

[0040] The GAN model training is combined with adversarial training method and L1 and L2 regularization. Generator loss function , train the initial generator The value is about 8, the discriminator The value is about 0.6. As the training progresses, the orthodontic effect prediction data generated by the generator is gradually optimized, and the values ​​of both tend to be stable, such as , .

[0041] Use the trained model to predict the orthodontic effect and generate a preview image: input the processed patient data into the model, the model calculates the tooth movement path, final position and facial soft tissue changes, and generates a three-dimensional preview image of the oral orthodontic effect.

[0042] The diverse and large-scale training data set and weighted processing of special cases enable the model to fully learn the orthodontic characteristics and rules of different regions, ages, races and case types, effectively avoiding the problem of insufficient generalization ability of the model due to data limitations, and improving the adaptability of the model to various patient groups. The training strategy that combines adversarial training methods with L1 and L2 regularization effectively prevents overfitting while ensuring that the model learns individual difference characteristics, further improving the accuracy and stability of the model. The final generated three-dimensional preview image can intuitively present the orthodontic effect, provide doctors and patients with clear treatment expectations, help doctors develop more accurate personalized treatment plans and improve patient treatment compliance.

[0043] Step 5: Visualize the differences: Steps: Compare the model prediction results with the original multimodal data image of the patient, extract the feature points and calculate the deviation. Suppose the coordinates of the tooth vertices in the model prediction image are , the corresponding coordinates in the original image are , the deviation vector , deviation size For example, if a tooth is misaligned .

[0044] According to the degree of deviation, the gradient color scheme identifies the difference area and marks the deviation value. Set the deviation threshold , , ,because , mark the difference area in yellow and mark the deviation value on the image .

[0045] Establish an automated testing and adaptation process: Build independent testing modules for visualization elements such as images, charts, and numerical annotations. The test data set contains 500 samples, covering a variety of cases and ethnic data. If the image is found to be blurred, adjust the image rendering parameters, such as increasing the resolution from 1920×1080 to 3840×2160, adjusting the contrast from 50 to 60, etc. After repair, verify the accuracy and stability of the visualization function again.

[0046] The difference visualization step effectively solves the problem of lack of effective analysis methods when there is a difference between the model prediction results and the doctor's clinical experience or intuitive judgment. By accurately calculating the deviation and using intuitive color identification and numerical annotation, doctors can quickly locate the difference and degree between the model prediction and the actual situation, and deeply analyze the reasons for the difference, such as data errors, model limitations or special individual factors. The automated testing and adaptation process ensures the continued accuracy and stability of the visualization function, providing doctors with reliable decision-making assistance tools, enabling them to optimize and adjust the treatment plan based on the visualization results, improve the scientificity and effectiveness of the treatment plan, and also enhance patients' trust and understanding of the treatment process.

[0047] Through the synergy of the above five key steps, many problems faced by the traditional orthodontic field in terms of multimodal data integration, semantic association construction, model prediction accuracy and visual analysis have been comprehensively and effectively solved. The overall accuracy and reliability of orthodontic effect prediction have been significantly improved, providing strong technical support for doctors to formulate personalized treatment plans. At the same time, intuitive preview images and effective difference visualization methods have greatly improved patients' understanding and cooperation with orthodontic treatment, which helps to improve treatment effects and increase patient satisfaction.

[0048] Embodiment 2: See also Figure 1 As shown, the present invention provides a technical solution: a method and device for generating a preview image of an oral orthodontic effect: Device: Data acquisition module: configured to comprehensively acquire the patient's oral cone beam CT data, oral scan data, facial image data, muscle strength data, fat thickness data, and basic patient information to ensure the integrity of multimodal data; Data preprocessing module: used to convert the format and unify the dimensions of the collected multimodal data. A dynamic range adjustment normalization algorithm is used for muscle strength data, and a proportional normalization algorithm based on anatomical structure characteristics is used for fat thickness data, so as to improve the stability and comparability of the data and lay the foundation for subsequent precise analysis; Semantic association construction module: Construct a semantic mapping framework and logical reasoning mechanism between multimodal data. Through modular semantic construction, different types of data semantic associations are constructed into independent sub-modules, including but not limited to constructing independent facial muscle strength-tooth movement sub-modules and facial fat distribution-tooth movement influence sub-modules. The model is used to deeply understand the intrinsic connection between various data elements in the orthodontic process, including the precise relationship between changes in facial muscle strength and the direction and distance of specific tooth movement and the complex regulatory effect of facial fat distribution on them, so that when the knowledge in the field of orthodontics is updated, the relevant modules can be adjusted and retrained with the smallest impact range. The independent sub-modules interact with each other through interfaces based on semantic rules to ensure the accuracy and efficiency of data transmission and integration between different modules. Model training and prediction module: Based on the constructed semantic association, the trained model is used to predict the orthodontic effect and generate a preview image. The model adopts adversarial training method combined with L1 and L2 regularization in the training stage. While learning individual difference characteristics, it effectively maintains the grasp of the overall patient group characteristic distribution to avoid overfitting. The training data set is highly diverse, covering a large number of oral orthodontic case data of different types and individual characteristics. In addition, a virtual adversarial sample generation mechanism is introduced in the model training process to further enhance the model's sensitivity and adaptability to slight changes in data. Before the model prediction step, a multi-branch ethnic feature recognition module is constructed for the differences in facial aesthetic standards of different races. The module performs deep neural network training based on massive facial image big data of different races, identifies the ethnic category to which the input data belongs, and calls the pre-trained aesthetic standard model of the corresponding race based on the recognition result. The output of the race-specific aesthetic standard parameter is integrated into the logical reasoning mechanism constructed by the semantic association construction module, and acts together with other multimodal data in the model prediction step; Difference visualization module: compares the model prediction results with the patient's original multimodal data images, colors the difference areas according to the gradient color scheme of the deviation degree and marks the specific deviation values, and establishes an automated testing and adaptation process. After each semantic update, the visualization function is verified using a massive test data set to automatically detect and repair possible display anomalies and interaction failures. The automated testing and adaptation process includes independent testing modules for different types of visualization elements, including but not limited to images, charts, and numerical annotations, which can accurately locate and repair problems. In the difference visualization step, the color identification of the difference area can also be classified and displayed according to the data type, including but not limited to tooth position differences and facial contour differences, so that doctors can quickly distinguish different types of differences.

[0049] Data collection: Using a professional oral cone beam CT scanner, the patient's oral cone beam CT data is acquired with a voxel resolution of 0.3mm. The image grayscale value ranges from -1000 to 3000HU. The size of a single patient data file is approximately 300MB, accurately presenting structural details such as teeth and alveolar bones.

[0050] The patient's mouth is scanned with an intraoral scanner to obtain oral scan data of approximately 1.5 million points with an accuracy of 0.08mm, which can carefully depict the surface morphology and position information of the teeth. Each point has three-dimensional coordinates and color information.

[0051] The patient's facial image is captured by a high-definition facial imaging device with an image size of 1920×1080 pixels and 24-bit true color, clearly capturing the facial features and contour details. This facial image is used in the multi-branch ethnic feature recognition module to determine the ethnic category in preparation for the subsequent application of ethnic-specific aesthetic standards.

[0052] The muscle strength measuring instrument was used to measure the facial muscle strength of the patient, with a measurement range of 0-500N, an accuracy of 1N, and a sampling frequency of 50Hz. For example, the strength of the patient's masseter muscle at maximum biting force was 350N, and the strength of the temporalis muscle under a specific action was 180N. 15 data points were collected in one chewing cycle to record the muscle strength change curve.

[0053] The patient's facial fat thickness is measured using an ultrasonic fat thickness meter with an accuracy of 0.1mm. For example, the patient's cheek fat thickness is 12mm below the zygomatic bone, 8mm above the mandibular angle, and 6mm under the chin.

[0054] Collect basic information of patients, such as an 18-year-old male patient with no history of oral disease but a history of mild malocclusion in the family.

[0055] Comprehensive data collection solves the problem of single data source of traditional models. Oral cone beam CT and oral scanning data provide the basis for tooth structure analysis; facial image data not only assists in the analysis of facial and tooth coordination, but is also the key to racial feature identification, creating conditions for the integration of cross-racial aesthetic standards; muscle strength and fat thickness data enrich the prediction basis from the perspective of soft tissue and function; basic patient information takes into account individual differences and provides a multi-dimensional information source for the formulation of personalized orthodontic plans, so that the model can adapt to the diverse needs of patients of different races.

[0056] Data preprocessing: Convert oral cone beam CT data and oral scan data into a unified STL format.

[0057] For muscle strength data, let the original muscle strength data be , its minimum value , the maximum value , taking masseter muscle strength as an example , normalized muscle strength data: .

[0058] For fat thickness data, set the vertical distance from the zygomatic bone to the mandibular bone , cheek fat thickness , normalized fat thickness data .

[0059] Format conversion and normalization processing effectively unify the data format and dimension. Different types of oral data can be easily processed uniformly after conversion to STL format. The normalization algorithm of muscle strength and fat thickness data solves the problem of difficulty in unifying data format and physical dimension, and improves the stability and comparability of data. This enables data to participate in model construction and analysis more accurately, enhances the model's understanding and processing capabilities of multi-source data, reduces model errors caused by data differences, and provides a reliable data foundation for accurately integrating cross-racial aesthetic standard data, ensuring the stable operation of the model under multi-racial data.

[0060] Construction and training of multi-branch ethnic feature recognition module: Collect millions of facial image big data of different races (including Asian, European, African, American, etc.) and pre-process the images, such as cropping, normalizing brightness and contrast, and marking key facial features (such as eyes, nose, mouth, cheekbones, etc.).

[0061] A deep neural network is constructed, using a convolutional neural network (CNN) architecture based on an attention mechanism. The network focuses on key areas related to racial characteristics in facial images through the attention mechanism to improve the accuracy of feature extraction. It contains multiple convolutional layers, pooling layers, and fully connected layers. The convolutional layer extracts local features, the pooling layer reduces the data dimension, and the fully connected layer outputs the racial category prediction results.

[0062] The preprocessed facial image data was divided into training set, validation set and test set in a ratio of 7:2:1. The deep neural network was trained using the training set, and the stochastic gradient descent (SGD) optimization algorithm with an adaptive learning rate adjustment strategy was adopted. The initial learning rate was 0.001, and as the training progressed, the learning rate gradually decreased according to the set rules. The training cycle was 100 rounds, and the model performance was evaluated on the validation set every 10 rounds of training. The network parameters were adjusted according to the accuracy and loss value. When the accuracy of the validation set did not increase for 5 consecutive rounds or the loss value did not decrease for 5 consecutive rounds, the training was stopped.

[0063] After training, the model performance was evaluated on the test set, and the model's recognition accuracy for facial images of different races reached over 94%.

[0064] The multi-branch ethnic feature recognition module based on massive data and advanced neural network architecture effectively solves the problem that traditional models are difficult to accurately identify different ethnic features. The attention mechanism focuses on key areas to improve feature extraction accuracy and accurately identify the ethnic category to which the input data belongs. High-accuracy ethnic recognition provides a reliable basis for the subsequent call of the aesthetic standard model of the corresponding ethnicity, ensuring that cross-ethnic aesthetic standards can be accurately integrated into the orthodontic effect preview image generation model, making the model adaptable to multi-ethnic patients and overcoming the limitation of traditional models that are only based on single or limited ethnic sample data.

[0065] Model construction and training of ethnic-specific aesthetic standards: For different races, we collected data on facial contours, facial features proportions, soft tissue fullness and other aesthetic features of large-scale populations (no less than 60,000 samples for each race). For example, for Asians, we measured the ratio of facial width to length, the ratio of eye distance to facial width, and the thickness of cheek soft tissue; for Europeans, we focused on the ratio of brow bone height to eye socket depth, the ratio of nose bridge height to length, and the angle and clarity of the jaw line; for Africans, we analyzed parameters such as the ratio of lip fullness to facial width, the relationship between cheekbone prominence and facial contour, and the coordination of forehead inclination and facial features.

[0066] These aesthetic feature data are sorted and labeled, and divided into training set and test set with a ratio of 8:2.

[0067] The gradient boosted decision tree (GBDT) algorithm was selected to construct a model of ethnic-specific aesthetic standards. The GBDT algorithm improves the performance of weak learners by gradual iteration, and can better fit complex aesthetic feature data relationships. During the training process, by adjusting hyperparameters such as the number of trees, depth, and learning rate, the cross-validation method was used to find the optimal model configuration, so that the model has the highest accuracy and the strongest generalization ability on the training set.

[0068] The trained race-specific aesthetic standard model was evaluated using the test set. The prediction accuracy of the model was evaluated by calculating indicators such as the mean square error (MSE) and the mean absolute error (MAE) to ensure that the model could accurately predict the facial aesthetic standard parameters that meet the race based on the input aesthetic feature data, such as the ideal facial proportion range, the appropriate fullness range of soft tissue, etc. The MSE was less than 0.07 and the MAE was less than 0.03.

[0069] The race-specific aesthetic standard model successfully quantified the facial aesthetic standards of different races based on in-depth analysis and training of aesthetic feature data of large-scale populations of various races. The GBDT algorithm effectively fits complex data relationships and accurately predicts race-specific aesthetic standard parameters. After these parameters are incorporated into the semantic association construction step, the orthodontic effect prediction model can fully consider the differences in cross-racial aesthetic standards and generate orthodontic effect preview images that are more in line with the aesthetic expectations of different races for patients of different races. This solves the problem that traditional models cannot adapt to the differences in multi-racial aesthetic standards and improves the practicality and effectiveness of the model in multi-racial orthodontic treatment.

[0070] Semantic association construction: Constructing the facial muscle strength-tooth movement submodule: Through multiple linear regression analysis, assuming that the teeth are Direction moving distance , Masseter muscle strength Temporalis muscle strength is the independent variable, and the regression equation is (These are example coefficients only). Assume ,but (Plus the error term calculation), the regression coefficients are adjusted through a large amount of data training, so that the model can accurately predict the effect of muscle strength on tooth movement.

[0071] Construct facial fat distribution-tooth movement influence submodule: extract fat distribution features in facial images, such as fat area , Fat center of gravity location etc., using these features as input and tooth movement related parameters as output to train a support vector machine (SVM) model. For example, using the Gaussian kernel function , determined by cross validation The model accuracy is the highest when , which enables the SVM model to predict the degree of facial soft tissue deformation during tooth movement based on facial fat distribution.

[0072] When the multi-branch ethnic feature recognition module identifies the patient's ethnicity, it calls the pre-trained aesthetic standard model of the corresponding ethnicity and integrates the ethnic-specific aesthetic standard parameters it outputs into the logical reasoning mechanism constructed by the above semantic association. For example, if the patient is identified as European, the facial aesthetic standard parameters of the European population (such as higher brow bone and nose bridge height requirements, relatively narrow jaw line preference, etc.) are combined with muscle strength, fat thickness and other data as comprehensive factors affecting tooth movement and orthodontic effects, and participate in the calculation and reasoning of the facial muscle strength-tooth movement submodule and the facial fat distribution-tooth movement influence submodule. The submodules interact with each other through a semantic rule-based interface to optimize the tooth movement plan.

[0073] The semantic association construction module successfully integrates multimodal data and race-specific aesthetic standards through specific sub-module construction and logical reasoning mechanisms. The facial muscle strength-tooth movement sub-module and the facial fat distribution-tooth movement influence sub-module quantify the effects of muscle strength and fat distribution on tooth movement, respectively. The interface interaction based on semantic rules enables each sub-module to work together. After incorporating race-specific aesthetic standard parameters, the model can fully consider racial differences in orthodontic effect prediction, optimize tooth movement plans not only from the perspective of tooth structure and facial soft tissue physiology, but also from the perspective of racial aesthetics, improve the accuracy and reliability of model prediction, and lay a solid foundation for generating orthodontic effect preview images that meet racial aesthetics, solving the problem that traditional models are difficult to integrate multi-source data and cross-racial aesthetic standards.

[0074] Model predictions: Model training phase: 10,000 oral orthodontic case data were collected as training sets, of which the ratio of Asian, European, and African patients was 3:4:3, the ratio of children, adolescents, and adults was 3:4:3, and special cases accounted for 35%. Special cases were weighted, and common cases were weighted , special case weight .

[0075] The GAN model training is combined with adversarial training method and L1 and L2 regularization. Generator loss function , train the initial generator The value is about 7.8, the discriminator The value is about 0.58. As the training progresses, the orthodontic effect prediction data generated by the generator is gradually optimized, and the values ​​of both tend to be stable, such as , .

[0076] Use the trained model to predict the orthodontic effect and generate a preview image: input the processed patient data (including multimodal data after integrating racial aesthetic standard parameters) into the model, and the model calculates the tooth movement path, final position and changes in facial soft tissue as the teeth move, and generates a three-dimensional preview image of the oral orthodontic effect. This image will reflect the expected orthodontic effect based on the patient's racial aesthetic standards.

[0077] Diverse training data sets and advanced training strategies enable the model to fully learn the orthodontic characteristics and rules of different races, ages, regions and case types. Special case weighting avoids the problem of insufficient generalization ability of the model due to data limitations and improves its adaptability to various patient groups. The adversarial training method combined with L1 and L2 regularization effectively prevents overfitting, ensuring that the model learns individual difference characteristics while improving accuracy and stability. The final generated three-dimensional preview image intuitively presents the orthodontic effect based on racial aesthetic standards, providing a clear basis for doctors to formulate personalized treatment plans, improving patient treatment compliance, and solving the problem of weak generalization ability and low accuracy of traditional models when processing multi-racial patient data, making orthodontic treatment more accurate, personalized and in line with racial aesthetic requirements.

[0078] Visualizing the differences: Compare the model prediction results with the original multimodal data image of the patient, extract the feature points and calculate the deviation. Suppose the coordinates of the tooth vertices in the model prediction image are , the corresponding coordinates in the original image are , the deviation vector , deviation size For example, if a tooth is misaligned .

[0079] According to the degree of deviation, the gradient color scheme identifies the difference area and marks the deviation value. Set the deviation threshold , , ,because , mark the difference area in yellow and mark the deviation value on the image .

[0080] Establish an automated testing and adaptation process: Build independent testing modules for visualization elements such as images, charts, and numerical annotations. The test data set contains 500 samples, covering a variety of cases and ethnic data. If the image is found to be blurred, adjust the image rendering parameters, such as increasing the resolution from 1920×1080 to 3840×2160, adjusting the contrast from 49 to 59, etc. After repair, verify the accuracy and stability of the visualization function again.

[0081] The difference visualization step effectively solves the problem of lack of effective analysis methods when there is a difference between the model prediction results and the doctor's clinical experience or intuitive judgment. By accurately calculating the deviation and using intuitive color identification and numerical annotation, doctors can quickly locate the difference and degree between the model prediction and the actual situation, and deeply analyze the causes of the difference, such as data errors, model limitations or special individual factors. The automated testing and adaptation process ensures the continued accuracy and stability of the visualization function, providing doctors with reliable decision-making assistance tools, enabling them to optimize and adjust treatment plans based on visualization results, improve the scientificity and effectiveness of treatment plans, and enhance patients' trust and understanding of the treatment process. Especially in cross-racial orthodontic treatment, it helps doctors better explain the impact of differences in racial aesthetic standards on orthodontic results, improve patient satisfaction, and make up for the shortcomings of traditional models in visualization analysis and difference explanation.

[0082] Through the synergy of the above-mentioned key steps, many problems faced by the traditional orthodontic field in terms of multimodal data integration, semantic association construction, model prediction accuracy and visual analysis have been comprehensively and effectively solved, especially the successful integration of cross-racial aesthetic standard differences into the prediction process of the orthodontic effect preview image generation model. The overall accuracy and reliability of orthodontic effect prediction have been significantly improved, providing strong technical support for doctors to formulate personalized treatment plans. At the same time, intuitive preview images and effective difference visualization methods have greatly improved patients' understanding and cooperation with orthodontic treatment, which helps to improve treatment effects and increase patient satisfaction.

[0083] Summarize: Through the specially configured data acquisition module, oral cone beam CT data, oral scan data, facial image data, muscle strength data, fat thickness data and basic patient information are fully acquired. There is a close correlation between these multimodal data. Oral cone beam CT and oral scan data provide accurate morphological information for teeth and internal oral structures, which is the basis for tooth movement planning in orthodontic treatment; facial image data can not only assist in analyzing the coordination between face and teeth, but more importantly, it is used as a key input in the multi-branch ethnic feature recognition module to determine the ethnic category, thereby providing a basis for the subsequent introduction of ethnic-specific aesthetic standards; muscle strength data and fat thickness data interact with tooth movement from the perspective of the physiological function and morphology of facial soft tissue. For example, the imbalance of muscle strength may affect the force and movement direction of teeth, and the distribution of fat thickness will affect the deformation of soft tissue during tooth movement; basic patient information helps to consider factors such as individual growth and development. For example, the orthodontic plan for adolescent patients may need to be adjusted in combination with their growth potential. This comprehensive collection and integration of multimodal data provides a rich source of information for orthodontic effect prediction from multiple dimensions, forming an organic whole, and effectively solving the problem of single and incomplete data sources in traditional methods.

[0084] The data preprocessing module uses specific algorithms for different types of data. Oral cone beam CT data and oral scan data are converted into a unified STL format for subsequent unified processing; a dynamic range adjustment normalization algorithm is used for muscle strength data, and a proportional normalization algorithm based on anatomical structure characteristics is used for fat thickness data. These algorithms are designed based on a deep understanding of the physical characteristics of data and their role in orthodontic analysis. For example, the normalization of muscle strength data can map the muscle strength values ​​of different individuals to a unified comparable range, so that when it participates in model calculation together with other data, it can accurately reflect the relative degree of its influence on tooth movement; the normalization of fat thickness data takes into account the differences in facial anatomical structure, so that fat thickness data can be better correlated with other facial feature data and tooth data. This targeted data preprocessing effectively unifies the data format and dimension, improves the stability and comparability of the data, enhances the model's ability to understand and process multi-source data, reduces model errors caused by data differences, and provides a reliable data foundation for the accurate integration of cross-ethnic aesthetic standard data and the stable operation of the entire model.

[0085] Construct a multi-branch ethnic feature recognition module and perform deep neural network training based on massive facial image big data of different races. By adopting a convolutional neural network architecture based on an attention mechanism, focusing on key areas related to ethnic features in facial images, the accuracy of ethnic feature extraction is improved, and the ethnic category to which the input data belongs can be accurately identified. Based on the recognition results, the pre-trained aesthetic standard model of the corresponding race is called. This model is based on in-depth statistical analysis and machine learning training of aesthetic feature data such as facial contours, facial features proportions, and soft tissue fullness of large populations of various races. For example, the gradient boosting decision tree algorithm is used to fit complex aesthetic feature data relationships and accurately predict ethnic-specific aesthetic standard parameters. These parameters are integrated into the semantic association construction module, so that the orthodontic effect prediction model can fully consider racial differences, optimize tooth movement plans from the perspective of racial aesthetics, and generate orthodontic effect preview images that meet the aesthetic expectations of patients of different races.

[0086] The semantic association construction module adopts a modular semantic construction method to construct independent facial muscle strength-tooth movement submodules, facial fat distribution-tooth movement influence submodules, etc. The facial muscle strength-tooth movement submodule quantifies the mechanism of muscle strength on tooth movement through multivariate linear regression analysis, and the facial fat distribution-tooth movement influence submodule uses the support vector machine model to reveal the relationship between fat distribution and tooth movement and facial soft tissue changes. The submodules interact through interfaces based on semantic rules. And after identifying the race, the race-specific aesthetic standard parameters are incorporated, so that the model can optimize the tooth movement plan in the prediction of orthodontic effects not only from the perspective of tooth structure and facial soft tissue physiology, but also from the perspective of racial aesthetics. This deep semantic association construction method successfully overcomes the difficulty of the model in understanding the internal logical association of data from different sources, provides a reliable semantic framework support for accurate orthodontic effect prediction, and greatly improves the prediction accuracy and reliability of the model.

[0087] The model training and prediction module uses a GAN model training that combines adversarial training methods with L1 and L2 regularization. While learning individual difference characteristics, it effectively maintains the grasp of the overall patient group characteristic distribution and avoids overfitting. The training data set is highly diverse, covering a large number of oral orthodontic case data of different types and individual characteristics, including different races, ages, regions, and special cases. Special cases are weighted so that the model can fully learn various orthodontic characteristics and laws, effectively avoiding the problem of insufficient generalization ability caused by data limitations, and improving the adaptability of the model to various patient groups. At the same time, a virtual adversarial sample generation mechanism is introduced in the model training process to further enhance the model's sensitivity and adaptability to slight changes in data. The final generated three-dimensional preview image can intuitively present the orthodontic effect based on racial aesthetic standards, provide a clear basis for doctors to formulate personalized treatment plans, and improve patient treatment compliance.

[0088] The difference visualization module compares the model prediction results with the original multimodal data images of the patient, accurately extracts feature points to calculate deviations, and colors the difference areas according to the color scheme with a gradient of deviation degree and marks the specific deviation values. An automated testing and adaptation process is established, and independent testing modules are built for visualization elements such as images, charts, and numerical annotations. The visualization display function is verified using massive test data sets, and possible display anomalies and interaction failures are automatically detected and repaired. The color identification of the difference areas can be classified and displayed according to the data type (such as tooth position differences, facial contour differences), so that doctors can quickly distinguish different types of differences. This accurate and powerful difference visualization step effectively solves the problem of lack of effective analysis methods when there is a difference between the model prediction results and the doctor's clinical experience or intuitive judgment. It provides doctors with reliable decision-making assistance tools, enhances patients' trust and understanding of the treatment process, and especially in cross-racial orthodontic treatment, helps doctors better explain the impact of racial aesthetic standard differences on orthodontic effects and improves patient satisfaction.

Claims

1. A method for generating a preview image of an orthodontic effect, characterized in that: The following steps are involved: Step 1: Data collection: comprehensively obtain the patient's oral cone beam CT data, oral scan data, facial image data, muscle strength data, fat thickness data and basic patient information; Step 2: Data preprocessing: The collected multimodal data are converted into a unified format and dimensionalized. A dynamic range adjustment normalization algorithm is used for muscle strength data, and a proportional normalization algorithm based on anatomical structure characteristics is used for fat thickness data. Step 3: Semantic association construction: Construct a semantic mapping framework and logical reasoning mechanism between multimodal data. Through modular semantic construction, construct different types of data semantic associations into independent sub-modules, including but not limited to constructing independent facial muscle strength-tooth movement sub-modules and facial fat distribution-tooth movement influence sub-modules; Step 4: Model prediction: Based on the constructed semantic association, the trained model is used to predict the orthodontic effect and generate a preview image. During the training phase, the model uses adversarial training combined with L1 and L2 regularization. Step 5: Difference visualization: Compare the model prediction results with the patient's original multimodal data images, color-code the difference areas and mark the specific deviation values ​​according to the gradient color scheme of the deviation degree, and establish an automated testing and adaptation process. After each semantic update, use a massive test data set to verify the visualization function, and automatically detect and repair possible display anomalies and interaction failures.

2. The method for generating an orthodontic effect preview image according to claim 1, characterized in that: In the semantic association construction step, independent sub-modules interact with each other through interfaces based on semantic rules.

3. The method for generating an orthodontic effect preview image according to claim 1, characterized in that: The automated testing and adaptation process includes independent testing modules for different types of visualization elements, including but not limited to images, charts, and numerical annotations.

4. The method for generating an orthodontic effect preview image according to claim 1, characterized in that: The adversarial training method introduces a virtual adversarial sample generation mechanism during the model training process.

5. The method for generating an orthodontic effect preview image according to claim 1, characterized in that: Ways to expand the diversity of training data sets include, but are not limited to, collecting case data extensively from different regions, age groups, and ethnic groups, and weighting special case data.

6. The method for generating an orthodontic effect preview image according to claim 1, characterized in that: In the data preprocessing step, an intelligent correction algorithm based on data distribution characteristics is used for outlier data.

7. The method for generating an orthodontic effect preview image according to claim 1, characterized in that: In the difference visualization step, the color identification of the difference area can also be classified and displayed according to the data type, and the data type includes but is not limited to the difference in tooth position and the difference in facial contour.

8. The method for generating an orthodontic effect preview image according to claim 1, characterized in that: Before the model prediction step, a multi-branch racial feature recognition module is constructed based on the differences in facial aesthetic standards of different races. The module performs deep neural network training based on massive facial image big data of different races to identify the racial category to which the input data belongs. The pre-trained aesthetic standard model of the corresponding race is called based on the recognition result. The model is based on statistical analysis and machine learning training of aesthetic feature data such as facial contours, facial features proportions, and soft tissue fullness of large populations of various races. The race-specific aesthetic standard parameters it outputs are integrated into the logical reasoning mechanism constructed in the semantic association construction step, and work together with other multimodal data in the model prediction step.

9. A device for generating an orthodontic effect preview image, characterized in that: It includes data acquisition module, data preprocessing module, semantic association construction module, model training and prediction module, and difference visualization module.

Images