A visualization device for orthodontic treatment process based on artificial intelligence

Abstract

The present invention relates to the field of image data processing technology, and discloses an artificial intelligence-based visualization device for an orthodontic treatment process, comprising a 3D intraoral scanner, a multi-angle intraoral camera array group, a facial scanner, a CBCT scanner, a bracket, a controller, a visualization image data processing system; a visualization interactive screen, a chair, a footrest, a column, a face support, a chair base, and armrests. The visualization image data processing system can intelligently convert and accumulate experience in annotating orthodontic points, and analyze a user's oral 3D model and a user's oral 3D annotated model according to the user's oral characteristics, thereby quickly obtaining a high-precision user's oral 3D model and a user's oral 3D annotated model, and quickly helping the user understand the oral condition so as to analyze the oral cavity.

Description

Technical Field

[0001] The present invention relates to the technical field of image data processing, and in particular to an artificial intelligence-based visualization device for orthodontic treatment processes. Background Art

[0002] Traditional orthodontic treatment has long relied on physician experience and manual manipulation, resulting in core pain points such as unreplicable experience and unpredictable treatment. Although digital technology has been gradually applied in clinical practice, existing visualization systems still face three major bottlenecks: First, insufficient multi-source data fusion makes it difficult to integrate data on crowns, roots, jaws, and soft tissues. This results in tooth arrangement design focusing solely on crown morphology, ignoring the safe boundary between the root and cortical bone, which can easily lead to complications such as bone fenestration and bone cracking. Second, the level of intelligence is limited. Most systems only implement static tooth arrangement simulations, lack dynamic predictions of biomechanical mechanisms, and are unable to continuously optimize treatment plans through machine learning. Third, clinical experience is difficult to quantify and pass on. Traditional VTO relies on manual measurements by physicians, which are complex and highly subjective. Young physicians take years to master, resulting in inconsistent diagnosis and treatment quality. In recent years, although invisible orthodontic technology has improved efficiency through digital tooth arrangement, its biomechanical design is still limited to single crown data, making it impossible to accurately control the movement trajectory of the root, resulting in a rebound rate of treatment efficacy as high as 30%. Some companies have attempted to introduce AI-assisted design, but these algorithms are often based on two-dimensional images or limited case libraries, making them difficult to adapt to the personalized needs of patients with complex maxillofacial deformities. Furthermore, existing visualization devices generally lack real-time feedback mechanisms, making it impossible to dynamically modify the model during treatment. This leads to cumulative deviations from the "plan-execution" process, compromising the ultimate therapeutic outcome.

[0003] Chinese Patent Publication No.: CN119693552A discloses a method and device for generating a preview image of an orthodontic effect, which relates to the field of orthodontic technology and includes the following steps: Step 1: Data acquisition; Step 2: Data preprocessing; Step 3: Semantic association construction; Step 4: Model prediction; Step 5: Difference visualization; In the present invention, by solving the problem of format and dimension unification in multimodal data integration, building semantic associations to allow the model to understand the internal logic of the data, and improving the difference visualization means to assist doctors in locating causes and making decisions. However, this solution still cannot solve the problem of intelligently transforming and accumulating the experience of past orthodontic point annotations to achieve the effect of orthodontic point recommendation annotations, and the intelligent visualization accuracy is low, and it is impossible to combine the user's oral characteristics for visual image analysis, resulting in low efficiency of real-time visual analysis of the orthodontic analysis process. Summary of the Invention

[0004] To this end, the present invention provides an artificial intelligence-based visualization device for the orthodontic treatment process, which is used to overcome the problems in the existing technology of low efficiency of real-time visualization analysis of the orthodontic analysis process due to the inability to intelligently convert and accumulate experience in orthodontic point marking, low accuracy of intelligent visualization, and inability to perform visualization image analysis in combination with the user's oral characteristics.

[0005] To achieve the above objectives, the present invention provides an artificial intelligence-based orthodontic treatment process visualization device, the device comprising:

[0006] A 3D intraoral scanner is mounted on a bracket and electrically connected to a visual interactive screen and a controller;

[0007] A multi-angle intraoral camera array group is arranged on a bracket and electrically connected to a visual interactive screen and a controller;

[0008] A facial scanner is mounted on a bracket and electrically connected to a visual interactive screen and a controller;

[0009] a CBCT scanner electrically connected to a visual interactive screen and a controller;

[0010] a bracket, which is provided on the right side of the seat;

[0011] a controller, which is arranged on the bracket;

[0012] A visual image data processing system electrically connected to the visual interactive screen;

[0013] a seat connected to a footrest;

[0014] a footrest, which is arranged on the lower left side of the seat;

[0015] A column, which is arranged on the right side of the bracket;

[0016] A face support, which is mounted on the column and located on the left side of the CBCT scanner;

[0017] a seat base connected to the seat;

[0018] An armrest is provided on the seat.

[0019] Furthermore, the visual image data processing system includes:

[0020] Orthodontic image acquisition module, used to acquire multimodal image data of the user's oral cavity;

[0021] An image data processing module is used to perform cross-fusion processing on the multimodal image data of the user's oral cavity to obtain optimal fusion data, construct a 3D model of the user's oral cavity in real time based on the optimal fusion data to obtain the 3D model of the user's oral cavity, and send the 3D model of the user's oral cavity to the image visualization module. It is also used to perform real-time accuracy monitoring of the 3D model of the user's oral cavity to obtain a model accuracy compliance result, optimize the cross-fusion processing process based on the model accuracy compliance result, perform real-time matching monitoring of the 3D model of the user's oral cavity based on user information to obtain a model matching result, and correct the multimodal image data acquisition process of the user's oral cavity based on the model matching result, and adjust the optimization method of the cross-fusion processing process;

[0022] An image annotation processing module is used to annotate the user's oral cavity 3D model to obtain a user's oral cavity annotated 3D model, update the user's oral cavity 3D annotated model to obtain a final user's oral cavity annotated 3D model, and send the final user's oral cavity 3D annotated model to the image visualization module. It is also used to fine-tune the update process of the user's oral cavity annotated model and calibrate the fine-tune process of the update of the user's oral cavity annotated model;

[0023] The image visualization module is used to push the user's oral cavity 3D model and the user's oral cavity 3D annotation model to the visualization interaction screen.

[0024] Furthermore, the image data processing module preprocesses the user's oral multimodal image data to obtain preprocessed user's oral multimodal image data, performs cross-fusion processing on the preprocessed user's oral multimodal image data according to each preset cross-fusion processing path to obtain fused data of each preset cross-fusion processing path, and obtains the anatomical consistency coefficient S of the fused data of each preset cross-fusion processing path. anat , biomechanical reasonable coefficient S biomech , aesthetic matching coefficient S aes and calculation efficiency coefficient S speed , according to the anatomical consistency coefficient S of the fused data after each preset cross-fusion processing path anat , biomechanical reasonable coefficient S biomech , aesthetic matching coefficient S aes and calculation efficiency coefficient S speed Calculate the scoring coefficient S of the fused data of each preset cross-fusion processing path, and set S=a1×S anat +a2×S biomech +a3×S aes +a4×S speed, set a1=0.4, a2=0.3, a3=0.2, a4=0.1, a1, a2, a3 and a4 are data scoring weight parameters, where a1 is the first data scoring weight parameter, a2 is the second data scoring weight parameter, a3 is the third data scoring weight parameter, and a4 is the fourth data scoring weight parameter. Obtain the maximum value of the data scoring coefficients after fusion of each preset cross-fusion processing path, and use it as the optimal fusion scoring coefficient Smax. Compare the optimal fusion scoring coefficient Smax with the first preset fusion scoring coefficient S1 and the second preset fusion scoring coefficient S2, set S1=0.8, S2=0.6, and identify the optimal fusion data according to the comparison results, where:

[0025] When S≥S1, the image data processing module identifies the data fused by the preset cross-fusion processing path corresponding to the optimal fusion scoring coefficient Smax as the optimal fusion data;

[0026] When S2≤S<S1, the image data processing module pushes the fused data after the preset cross-fusion processing path to the visual interactive screen, obtains the fused data after the preset cross-fusion processing path selected by the user on the visual interactive screen, and identifies it as the optimal fused data;

[0027] When S<S2, the image data processing module cannot identify the optimal fusion data, and pushes a prompt indicating that the optimal fusion data cannot be identified to the visual interaction screen.

[0028] Furthermore, the image data processing module converts the optimal fusion data into a triangular mesh according to a triangulation algorithm to obtain a 3D model of the user's oral cavity, and sends the 3D model of the user's oral cavity to the image visualization module.

[0029] Furthermore, the image data processing module calculates a model fluctuation difference RP based on the user's oral cavity 3D model and the suboptimal fused 3D model, and compares the model fluctuation difference RP with a preset model fluctuation difference RP0. Based on the comparison result, the user's oral cavity 3D model is subjected to real-time accuracy monitoring to obtain a model accuracy compliance result, wherein:

[0030] When RP≤RP0, the image data processing module determines that the model accuracy meets the standard and the result is that the user's oral 3D model has met the standard;

[0031] When RP>RP0, the image data processing module determines that the model accuracy meets the standard and the result is that the user's oral 3D model does not meet the standard, and optimizes the cross-fusion processing process.

[0032] Furthermore, when the image data processing module performs real-time matching monitoring on the user's oral cavity 3D model according to the user information, obtains the model matching result, and corrects the user's oral multimodal image data acquisition process according to the model matching result, the age stage data and oral cavity material distribution data in the user information are converted into an age stage data vector and an oral cavity material distribution data vector, and the age stage data vector and the oral cavity material distribution data vector are input into the oral cavity 3D prediction model to obtain the predicted oral cavity 3D model output by the oral cavity 3D prediction model, and the matching mean square error RMSE is calculated according to the three-dimensional coordinates ai of the i-th point of the predicted oral cavity 3D model, the three-dimensional coordinates bi of the i-th point of the user's oral cavity 3D model and the number N of corresponding points, and the setting is performed. , compare the matching mean square error RMSE with the preset matching mean square error RMSE0, and judge the model matching result based on the comparison result, and correct the user's oral multimodal image data acquisition process based on the judgment result, where:

[0033] When RMSE≤RMSE0, the image data processing module determines that the model matching result is a match and does not correct the user's oral multimodal image data acquisition process;

[0034] When RMSE>RMSE0, the image data processing module determines that the model matching result is mismatched, corrects the user's oral multimodal image data acquisition process, increases the acquisition frequency of the multi-angle intraoral camera array in the image acquisition control scheme, obtains the corrected image acquisition control scheme, replaces the image acquisition control scheme with the corrected image acquisition control scheme, and controls the acquisition of the 3D intraoral scanner, multi-angle intraoral camera array group, facial scanner and CBCT scanner according to the corrected image acquisition control scheme.

[0035] Furthermore, when the image data processing module determines that the model matching result is a match, the user age stage data Rx is compared with the tooth development age stage data RA={R1, R2, R3, R4}, and the user's tooth development status is judged based on the comparison result. The process of real-time accuracy monitoring of the user's oral 3D model is adjusted based on the judgment result, wherein:

[0036] When Rx During RA, the image data processing module determines that the user's tooth development is agenesis, and does not adjust the process of real-time accuracy monitoring of the user's oral 3D model;

[0037] When Rx∈RA, the image data processing module determines that the user's tooth development status is developing, adjusts the process of real-time accuracy monitoring of the user's oral 3D model, adjusts the preset model fluctuation difference RP0 according to the adjustment coefficient α, obtains the adjusted preset model fluctuation difference RP0t, sets RP0t=RPO×α, replaces the preset model fluctuation difference RP0 with the adjusted preset model fluctuation difference RP0t, and re-compares the model fluctuation difference RP with the adjusted preset model fluctuation difference RP0t.

[0038] Furthermore, when the image annotation processing module annotates the user's oral 3D model, the user's oral 3D model and age stage data are input into the oral 3D analysis model to obtain the user's oral 3D annotated model output by the oral 3D analysis model, and the user's oral 3D annotated model includes the first recommended target coordinate vector annotated on the oral 3D annotated model. and recommended support points.

[0039] Furthermore, when the image annotation processing module updates the user's oral 3D annotation model, it updates the user's habit intensity index HIIk, tooth position coefficient Ctooth, direction vector F k and habit weight coefficient w k Single habit compensation vector Calculate and set , and according to the number of user habits t and the single habit compensation vector Compensation vector for user's total habit Calculate and set , also based on the first recommended target coordinate vector and the user's total habit compensation vector For the second recommended target coordinate vector Calculate and set = + , the first recommended target coordinate vector Replaced with the second recommended target coordinate vector , and the second recommended target coordinate vector The user's oral cavity 3D annotation model is annotated to obtain a final user's oral cavity 3D annotation model, and the image annotation processing module sends the final user's oral cavity 3D annotation model to the image visualization module.

[0040] Furthermore, when the image annotation processing module performs fine calibration on the update process of the user's oral 3D annotation model, the second recommended target vector (x2, y2, z2) and the initial tooth coordinate vector (x0, y0, z0) calculates the recommended tooth movement distance L and sets The recommended tooth movement distance L is compared with the preset recommended movement distance L0, L0 = 0.5mm. The habit compensation situation is judged based on the comparison results. The update process of the user's oral 3D annotation model is fine-tuned based on the judgment results, where:

[0041] When L≤L0, the image annotation processing module determines that the habit compensation is not over-compensated, and does not perform fine-tuning on the updating process of the user's oral cavity 3D annotation model;

[0042] When L>L0, the image annotation processing module determines that the habit compensation is over-compensation, and fine-tunes the updating process of the user's oral 3D annotation model. The user's total habit compensation vector is fine-tuned by the fine-tuning coefficient γ, and γ is set to 0.75+0.2× e-(L-L0) , e is the base of natural logarithm to obtain the total user habit compensation vector after fine-tuning ,set up, =γ× , the user's total habit compensation vector Replaced with the user's total habit compensation vector after fine-tuning , and compensate the vector according to the total habit of the user after fine-tuning For the second recommended target coordinate vector Recalculate until L≤L0.

[0043] Compared with the existing technology, the beneficial effect of the present invention lies in that by analyzing 3D intraoral scanning data, multi-angle intraoral camera array data, facial 3D scanning data and CBCT scanning data, a user's oral 3D model and a user's oral 3D annotation model are obtained, which is convenient for users to analyze the user's oral cavity through a visual interactive screen. At the same time, since the visual image data processing system can intelligently convert and accumulate the experience of annotating orthodontic points, and analyze the user's oral 3D model and the user's oral 3D annotation model according to the characteristics of the user's oral cavity, it can quickly obtain a high-precision user's oral 3D model and a user's oral 3D annotation model, which can quickly help users understand the oral condition in order to analyze the oral cavity.

[0044] In particular, the system acquires the multimodal image data of the user's oral cavity through the orthodontic image acquisition module in order to subsequently construct a 3D model of the user's oral cavity. The system also constructs the 3D model through the image data processing module, and further improves the accuracy of the user's oral cavity 3D model by performing real-time accuracy monitoring of the user's oral cavity 3D model, performing real-time matching monitoring of the user's oral cavity 3D model, and correcting the multimodal image data acquisition process of the user's oral cavity, thereby achieving the purpose of improving the accuracy of intelligent visualization. At the same time, the system also annotates the user's oral cavity 3D model through the image annotation processing module, and annotates the user's oral cavity 3D annotation model. The system also updates the user's oral 3D model, fine-tunes the updating process of the user's oral 3D annotation model, and calibrates the updating process of the user's oral 3D annotation model. It intelligently converts and accumulates the experience of orthodontic point annotation, and annotates it on the user's oral 3D annotation model in the form of recommendations, so as to facilitate users to perform visual image analysis according to oral characteristics, and further improve the efficiency of real-time visual analysis of the orthodontic treatment process. The system also pushes the user's oral 3D model and the user's oral 3D annotation model to the visual interactive screen through the image visualization module, so that the user can perform real-time visual analysis of the orthodontic treatment process, thereby achieving the purpose of efficiently analyzing the orthodontic treatment process. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 Schematic diagram of the structure of the device for visualizing the orthodontic treatment process based on artificial intelligence in this embodiment;

[0046] Figure 2 Schematic diagram of the structure of the visual image data processing system of this embodiment. DETAILED DESCRIPTION

[0047] In order to make the objects and advantages of the present invention more clearly understood, the present invention is further described below in conjunction with embodiments; it should be understood that the specific embodiments described herein are merely used to explain the present invention and are not intended to limit the present invention.

[0048] The preferred embodiments of the present invention are described below with reference to the accompanying drawings. It should be understood by those skilled in the art that these embodiments are only used to explain the technical principles of the present invention and are not intended to limit the scope of protection of the present invention.

[0049] It should be noted that, in the description of the present invention, terms such as "up", "down", "left", "right", "inside", and "outside" indicating directions or positional relationships are based on the directions or positional relationships shown in the accompanying drawings. This is only for the convenience of description and does not indicate or imply that the device or element must have a specific orientation, be constructed and operated in a specific orientation. Therefore, it cannot be understood as a limitation on the present invention.

[0050] Furthermore, it should be noted that, in the description of the present invention, unless otherwise expressly specified or limited, the terms "mounted," "connected," and "connected" should be understood in a broad sense. For example, they may refer to fixed connections, detachable connections, or integral connections; mechanical connections or electrical connections; direct connections or indirect connections through an intermediate medium; and internal communication between two components. Those skilled in the art will understand the specific meanings of the above terms in the present invention based on specific circumstances.

[0051] See also Figure 1 , which is a schematic diagram of the structure of the device for visualizing the orthodontic treatment process based on artificial intelligence in this embodiment, the device includes:

[0052] A 3D intraoral scanner 1 is mounted on a support 5 and electrically connected to a visual interactive screen 8 and a controller 6 for collecting 3D intraoral scan data;

[0053] A multi-angle intraoral camera array group 2 is provided on a bracket 5 and is electrically connected to a visual interactive screen 8 and a controller 6 for collecting multi-angle intraoral camera array data;

[0054] A facial scanner 3 is mounted on a support 5 and electrically connected to a visual interactive screen 8 and a controller 6 for collecting 3D facial scan data.

[0055] a CBCT scanner 4 , which is electrically connected to the visual interactive screen 8 and the controller 6 and is used to collect CBCT scan data;

[0056] Bracket 5, which is arranged on the right side of seat 9;

[0057] a controller 6 , which is disposed on the bracket 5 and is used to control the 3D intraoral scanner 1 , the multi-angle intraoral camera array group 2 , the facial scanner 3 , and the CBCT scanner 4 according to an image acquisition control scheme;

[0058] A visual image data processing system 7 is electrically connected to the visual interactive screen 8 and is used to process 3D intraoral scan data, multi-angle intraoral camera array data, facial 3D scan data, and CBCT scan data to obtain a 3D model of the user's oral cavity and a 3D annotated model of the user's oral cavity, and push the 3D model of the user's oral cavity and the 3D annotated model of the user's oral cavity to the visual interactive screen;

[0059] a seat 9 connected to a footrest 10 for providing a seat for a user;

[0060] A footrest 10 is provided on the lower left side of the seat 9 and is used to provide support for the user's feet;

[0061] A column 11 is provided on the right side of the bracket 5 and is used for mounting the face support 12 and the CBCT scanner 4;

[0062] Face support 12, which is mounted on the column 11 and disposed on the left side of the CBCT scanner 4, for limiting the movement of the user's face, so that the CBCT scanner 4 is scanned;

[0063] a seat base 13 connected to the seat 9 and used for mounting the seat 9;

[0064] An armrest 14 is provided on the seat 9 for the user to rest his hands on.

[0065] Specifically, the present invention is used to assist users in visually analyzing oral conditions. By analyzing 3D intraoral scanning data, multi-angle intraoral camera array data, facial 3D scanning data and CBCT scanning data, a 3D model of the user's oral cavity and a 3D annotated model of the user's oral cavity are obtained, which facilitates the user to analyze the user's oral cavity through a visual interactive screen. At the same time, since the visual image data processing system can intelligently convert and accumulate the experience of annotating orthodontic points, and analyze the user's oral 3D model and the user's oral 3D annotated model according to the characteristics of the user's oral cavity, it can quickly obtain a high-precision 3D model of the user's oral cavity and the user's oral 3D annotated model, which can quickly help the user understand the oral condition in order to analyze the oral cavity.

[0066] See also Figure 2 FIG. 1 is a schematic diagram of the structure of the visual image data processing system of this embodiment, wherein the system includes:

[0067] Orthodontic image acquisition module, used to acquire multimodal image data of the user's oral cavity;

[0068] An image data processing module is used to perform cross-fusion processing on the multimodal image data of the user's oral cavity to obtain optimal fusion data, to construct a 3D model of the user's oral cavity in real time based on the optimal fusion data to obtain the 3D model of the user's oral cavity, and to send the 3D model of the user's oral cavity to the image visualization module. It is also used to perform real-time accuracy monitoring on the 3D model of the user's oral cavity to obtain a model accuracy compliance result, to optimize the cross-fusion processing process based on the model accuracy compliance result, and to perform real-time matching monitoring on the 3D model of the user's oral cavity based on user information to obtain a model matching result, and to correct the multimodal image data acquisition process of the user's oral cavity based on the model matching result, and to adjust the optimization method of the cross-fusion processing process. The image data processing module is connected to the orthodontic image acquisition module;

[0069] An image annotation processing module is used to annotate the user's oral cavity 3D model to obtain a user's oral cavity annotated 3D model, update the user's oral cavity 3D annotated model to obtain a final user's oral cavity annotated 3D model, and send the final user's oral cavity 3D annotated model to the image visualization module. It is also used to fine-tune the update process of the user's oral cavity annotated model and calibrate the fine-tune process of the update of the user's oral cavity annotated model. The image annotation processing module is connected to the image data processing module;

[0070] The image visualization module is used to push the user's oral cavity 3D model and the user's oral cavity 3D annotation model to the visualization interaction screen. The image visualization module is connected to the image annotation processing module.

[0071] Specifically, the system is applied to a visualization device of the orthodontic treatment process based on artificial intelligence, and processes and annotates the multimodal image data of the user's oral cavity acquired by a 3D intraoral scanner, a multi-angle intraoral camera array group, a facial scanner and a CBCT scanner, so as to realize the construction of a 3D model of the user's oral cavity and a 3D annotated model of the user's oral cavity, and intelligently convert and accumulate the experience of orthodontic point annotation to improve the accuracy of intelligent visualization, and perform visual image analysis in combination with the characteristics of the user's oral cavity, so as to further improve the efficiency of real-time visual analysis of the orthodontic treatment process, wherein the system acquires the multimodal image data of the user's oral cavity through an orthodontic image acquisition module for subsequent construction of a 3D model of the user's oral cavity, and the system also constructs the 3D model through an image data processing module, and performs real-time accuracy monitoring of the user's oral cavity 3D model, real-time matching monitoring of the user's oral cavity 3D model and multimodal visualization of the user's oral cavity. The modal image data acquisition process is corrected to further improve the accuracy of the user's oral 3D model, thereby achieving the purpose of improving the accuracy of intelligent visualization. At the same time, the system also annotates the user's oral 3D model through the image annotation processing module, and updates the user's oral 3D annotation model, fine-tunes the update process of the user's oral 3D annotation model, and calibrates the fine-tuning process of the update of the user's oral 3D annotation model. The experience of orthodontic point annotation is intelligently converted and accumulated, and annotated in the form of recommendations on the user's oral 3D annotation model, so that users can perform visual image analysis according to oral characteristics, and further improve the efficiency of real-time visual analysis of the orthodontic treatment process. The system also pushes the user's oral 3D model and the user's oral 3D annotation model to the visualization interactive screen through the image visualization module, so that users can perform real-time visual analysis of the orthodontic treatment process, so as to achieve the purpose of efficiently analyzing the orthodontic treatment process.

[0072] Specifically, the orthodontic image acquisition module acquires user information and inputs the user information into the image acquisition control model. The image acquisition control model outputs an image acquisition control scheme based on the user information and sends the image acquisition control scheme to the controller. The controller controls and acquires the 3D intraoral scanner, multi-angle intraoral camera array group, facial scanner and CBCT scanner according to the image acquisition control scheme to obtain multimodal image data of the user's oral cavity.

[0073] Specifically, the user information refers to data representing the user's identity characteristics, physical characteristics and oral characteristics, including user identification code, gender data, medical history data, age stage data and oral material distribution data. The user identification code refers to a combination of letters and numbers used to distinguish the user's identity, such as XN001. The user refers to an object that requires a visual display of the orthodontic treatment process. The method for obtaining user information is that the gender data refers to the user's gender characteristic data. In this embodiment, when the user's gender is female, the gender data is set to 1. When the user's gender is male, the gender data is set to 0. The medical history data refers to the user's historical medical history data. In this embodiment, the medical history data is set to the medical history data. Birth time - medical condition code, the age stage data refers to the code data of the user's age stage, the age stages include infancy 0-2 years old, toddler age 2-6 years old, childhood 6-12 years old, adolescence 12-18 years old, early adulthood 18-40 years old, middle adulthood 40-65 years old and old age 65 years old and above, set the infant age stage data as R1, set the toddler age stage data as R2, set the childhood age stage data as R3, set the adolescence age stage data as R4, set the early adult age stage data as R5, set the middle adult age stage data as R6, set the old age stage data as R7, the oral material distribution data refers to the tooth coordinates and The tooth material type corresponding to the tooth coordinates. This embodiment does not limit the setting method of the oral material distribution data. Those skilled in the art can freely set it according to actual conditions. For example, this embodiment takes the middle of the root of the tongue in the user's mouth as the origin, establishes a spatial rectangular coordinate system, obtains the coordinates of the midpoints of each tooth as the tooth coordinates in the user's mouth, and takes the tooth material type where the coordinates are located as the tooth material type corresponding to the tooth coordinates. The tooth material type refers to the code data of the tooth generation material, including native tooth material Q1, metal restoration Q2, ceramic Q3 and glass ionomer material Q4. This embodiment does not limit the method of obtaining the user information. Those skilled in the art can freely set it according to actual conditions, such as It is set to obtain user information in a way that the user inputs the user information through a visual interactive screen. The image acquisition control model refers to a convolutional neural network model that takes the image format data of the user information as input and the image acquisition control scheme as output. This embodiment does not limit the construction process of the image acquisition control model. Those skilled in the art can freely set it according to actual conditions, and only need to meet the output requirements of the image acquisition control scheme. For example, the image format data of historical user information and its corresponding image acquisition control scheme can be set as the image acquisition control model data set, and the convolutional neural network model is trained and verified according to the image acquisition control model data set to obtain the image acquisition control model.The image acquisition control scheme refers to a control instruction for controlling the 3D intraoral scanner, the multi-angle intraoral camera array group, the facial scanner and the CBCT scanner through the controller to acquire the multimodal image data of the user's oral cavity. This embodiment does not limit the method of sending the image acquisition control scheme to the controller. Those skilled in the art can freely set it according to actual conditions, and only need to meet the data transmission requirements of the image acquisition control scheme. For example, the method of sending the image acquisition control scheme to the controller can be set to industrial Ethernet. The controller controls the 3D intraoral scanner, the multi-angle intraoral camera array group, the facial scanner and the CBCT scanner according to the control instruction content of the image acquisition control scheme. The multimodal image data of the user's oral cavity refers to the user's oral picture data acquired according to the image acquisition control scheme, including 3D intraoral scanning data, multi-angle intraoral camera array group, facial scanner and CBCT scanner. Camera array data, facial 3D scan data, and CBCT scan data. The 3D intraoral scan data refers to color texture map data that records the geometric morphology of the tooth crown surface in the form of triangular mesh model data and captures soft tissue details such as tooth enamel color, caries, and gingival margin inflammation in RGB information. The multi-angle intraoral camera array data refers to multi-view 2D image sequence data collected from the buccal, lingual, and occlusal angles after eliminating local highlights and shadows and enhancing details such as enamel microcracks and dental calculus. The 3D facial scan data refers to facial expression mesh model data with the nose tip, mouth corners, and submental points as facial landmark coordinates, including facial expression capture such as smiling and natural relaxation. The CBCT scan data refers to multi-planar reconstruction view data with axial, coronal, and sagittal plane tomographic images as content, showing tooth root morphology and cortical bone thickness, and a three-dimensional voxel matrix containing bone density information.

[0074] Specifically, the orthodontic image acquisition module acquires the multimodal image data of the user's oral cavity so that the subsequent system can create a 3D oral simulation model of the user based on the multimodal image data of the user's oral cavity and annotate it.

[0075] Specifically, the image data processing module preprocesses the user's oral multimodal image data to obtain preprocessed user's oral multimodal image data, performs cross-fusion processing on the preprocessed user's oral multimodal image data according to each preset cross-fusion processing path, obtains the fused data of each preset cross-fusion processing path, and obtains the anatomical consistency coefficient S of the fused data of each preset cross-fusion processing path. anat , biomechanical reasonable coefficient S biomech , aesthetic matching coefficient S aes and calculation efficiency coefficient S speed , according to the anatomical consistency coefficient S of the fused data after each preset cross-fusion processing path anat , biomechanical reasonable coefficient S biomech, aesthetic matching coefficient S aes and calculation efficiency coefficient S speed Calculate the scoring coefficient S of the fused data of each preset cross-fusion processing path, and set S=a1×S anat +a2×S biomech +a3×S aes +a4×S speed , set a1=0.4, a2=0.3, a3=0.2, a4=0.1, a1, a2, a3 and a4 are data scoring weight parameters, where a1 is the first data scoring weight parameter, a2 is the second data scoring weight parameter, a3 is the third data scoring weight parameter, and a4 is the fourth data scoring weight parameter. Obtain the maximum value of the data scoring coefficients after fusion of each preset cross-fusion processing path, and use it as the optimal fusion scoring coefficient Smax. Compare the optimal fusion scoring coefficient Smax with the first preset fusion scoring coefficient S1 and the second preset fusion scoring coefficient S2, set S1=0.8, S2=0.6, and identify the optimal fusion data according to the comparison results, where:

[0076] When S≥S1, the image data processing module identifies the data fused by the preset cross-fusion processing path corresponding to the optimal fusion scoring coefficient Smax as the optimal fusion data;

[0077] When S2≤S<S1, the image data processing module pushes the fused data after the preset cross-fusion processing path to the visual interactive screen, obtains the fused data after the preset cross-fusion processing path selected by the user on the visual interactive screen, and identifies it as the optimal fused data;

[0078] When S<S2, the image data processing module cannot identify the optimal fusion data, and pushes a prompt indicating that the optimal fusion data cannot be identified to the visual interaction screen.

[0079] Specifically, this embodiment does not limit the specific method of preprocessing the user's oral multimodal image data. Those skilled in the art can set it by themselves according to actual needs, such as performing noise reduction, resampling and registration on the user's oral multimodal image data to obtain a point cloud. The noise reduction, resampling and registration are common preprocessing steps for point cloud generation. The preset cross-fusion processing paths refer to a combination of user oral multimodal image data fusion algorithms preset by the system. This embodiment does not limit the setting method of each preset cross-fusion processing path. Those skilled in the art can set it by themselves according to actual conditions, such as setting each preset cross-fusion processing path by registering point clouds through the ICP algorithm. The ICP algorithm registering point clouds refers to registering point clouds through iterative The nearest point is used to achieve accurate spatial alignment of point clouds from different sources. Cross-fusion processing refers to the intelligent processing process of spatially registering multimodal data from 3D intraoral scanners, multi-angle intraoral camera arrays, facial scanners and CBCT scanners to obtain fused data. The anatomical consistency coefficient refers to the coefficient that quantifies the matching degree between the fusion model and the real anatomical structure. The biomechanical rationality coefficient refers to the coefficient that evaluates the mechanical safety of the tooth movement plan. The aesthetic matching coefficient refers to the coefficient that measures the conformity of the smile design result with the golden ratio. The computational efficiency coefficient refers to the coefficient of the standardized score of the algorithm processing speed. The anatomical consistency coefficient, the biomechanical rationality coefficient, the aesthetic matching coefficient and the computational efficiency coefficient are the coefficients of the standardized score of the algorithm processing speed. The coefficients are obtained by putting the multimodal image data of the user's oral cavity into the coefficient calculation model to obtain the coefficient distribution diagram output by the coefficient calculation model, the coefficient distribution diagram includes the anatomical consistency coefficient, the biomechanical rationality coefficient, the aesthetic matching coefficient and the computational efficiency coefficient. The coefficient calculation model refers to a convolutional neural network model that takes the multimodal image data of the user's oral cavity as input and outputs the coefficient distribution diagram. The coefficient distribution diagram refers to a coefficient distribution diagram in which the anatomical consistency coefficient, the biomechanical rationality coefficient, the aesthetic matching coefficient and the computational efficiency coefficient are marked in the form of a bar graph. This embodiment does not limit the specific construction method of the coefficient calculation model. Those skilled in the art can set it according to actual conditions, such as calculating the convolutional neural network through the coefficient calculation training set. After training the network model, a coefficient calculation model is obtained. The coefficient calculation training set refers to a learning data set of the coefficient calculation model obtained by training the convolutional neural network model. The coefficient calculation training set includes historical user oral multimodal image data and a historical coefficient distribution map corresponding to the historical user oral multimodal image data. The maximum value of the scoring coefficient of the data after fusion of each preset cross-fusion processing path refers to the maximum value of all calculated values ​​of the scoring coefficient S of the data after fusion of each preset cross-fusion processing path. The first preset fusion scoring coefficient S1 refers to the first preset value for identifying the optimal fusion data. The second preset fusion scoring coefficient S2 refers to the second preset value for identifying the optimal fusion data.The optimal fusion data refers to the preset cross-fusion processing path fusion data corresponding to the maximum value of the score coefficient of the fusion data of each preset cross-fusion processing path. This embodiment does not limit the push method of the preset cross-fusion processing path fusion data, such as the preset cross-fusion processing path fusion data can be pushed through the form of wireless Wi-Fi. This embodiment does not limit the method of the preset cross-fusion processing path fusion data selected by the user on the visual interactive screen, such as the screen touch selection. This embodiment does not limit the method of pushing the optimal fusion data prompt to the visual interactive screen if it cannot be identified, such as the wireless Wi-Fi. The optimal fusion data cannot be identified and pushed to the visual interactive screen in the form of a prompt, wherein a1=0.4 is to ensure absolute anatomical safety, and the weight of the anatomical consistency coefficient is set to the highest weight of 0.4. a2=0.3 is to ensure the functional stability of the teeth after orthodontic use, and the weight of the biomechanical rationality coefficient is set to the secondary weight of 0.3. a3=0.2 is to ensure patient satisfaction after orthodontic treatment, and the weight of the aesthetic matching coefficient is set to the third weight of 0.2. a4=0.1 is for clinical feasibility, avoiding excessive sacrifice of accuracy in pursuit of speed, and its weight is set to the lowest weight of 0.1.

[0080] Specifically, the image data processing module calculates the scoring coefficient S of the data after fusion of each preset cross-fusion processing path to obtain the optimal fusion scoring coefficient, so as to subsequently construct the user's 3D oral model according to the optimal fusion data corresponding to the optimal fusion scoring coefficient to obtain the user's 3D oral model with the highest accuracy.

[0081] Specifically, the image data processing module converts the optimal fusion data into a triangular mesh according to a triangulation algorithm to obtain a 3D model of the user's oral cavity, and sends the 3D model of the user's oral cavity to the image visualization module.

[0082] Specifically, the triangulation algorithm refers to an algorithm that divides the optimal fusion data into several non-overlapping triangles for generating a triangular mesh. This embodiment does not limit the specific method of sending the user's oral 3D model to the image visualization module. Those skilled in the art can set it according to actual needs, such as sending the user's oral 3D model to the image visualization module via wireless transmission. The wireless transmission refers to a technology that uses electromagnetic waves or magnetic fields as carriers to transmit data, energy or signals in space without physical connection.

[0083] Specifically, the image data processing module converts the optimal fusion data into a triangular mesh to obtain a 3D model of the user's oral cavity, and sends the 3D model of the user's oral cavity to the image visualization module so that the user can subsequently observe the oral condition directly through the visual interactive screen.

[0084] Specifically, the image data processing module calculates the model fluctuation difference RP based on the user's oral cavity 3D model and the suboptimal fused 3D model, and compares the model fluctuation difference RP with the preset model fluctuation difference RP0. Based on the comparison result, the user's oral cavity 3D model is subjected to real-time accuracy monitoring to obtain the model accuracy compliance result, wherein:

[0085] When RP≤RP0, the image data processing module determines that the model accuracy meets the standard and the result is that the user's oral 3D model has met the standard;

[0086] When RP>RP0, the image data processing module determines that the model accuracy meets the standard and the result is that the user's oral 3D model does not meet the standard, and optimizes the cross-fusion processing process.

[0087] Specifically, the real-time accuracy detection refers to comparing the model fluctuation difference RP with the preset model fluctuation difference RP0 in real time to achieve the accuracy monitoring process of real-time monitoring of the user's oral 3D model. The suboptimal fusion 3D model refers to the user's oral 3D model generated by overlapping the suboptimal fusion data and the predicted optimal fusion data. The suboptimal fusion data refers to the fusion data with the second largest optimal fusion score coefficient. The predicted optimal fusion data refers to the predicted optimal fusion data obtained by predicting the data after the fusion of each preset cross-fusion processing path. This embodiment does not limit the specific method of predicting the data after the fusion of each preset cross-fusion processing path. This embodiment does not limit the specific method of predicting the data after the fusion of each preset cross-fusion processing path. Those skilled in the art can set it up by themselves according to actual needs, such as using the optimal fusion data prediction model to predict the fused data of each preset cross-fusion processing path. The optimal fusion data prediction model refers to a recurrent neural network model for predicting the fused data of each preset cross-fusion processing path. This embodiment does not limit the specific calculation method of the model fluctuation difference RP calculated based on the user's oral 3D model and the suboptimal fusion 3D model. Those skilled in the art can set it up by themselves according to actual needs, such as using the point-by-point Euclidean distance fluctuation method to calculate the spatial distance between the corresponding points of the user's oral 3D model and the suboptimal fusion 3D model to obtain the local fluctuation dm, and set dm= , where m is the order of the corresponding points between the user's oral cavity 3D model and the suboptimal fused 3D model, m is a positive integer, the maximum value of m is N, and N is the total number of corresponding points between the user's oral cavity 3D model and the suboptimal fused 3D model. Refers to the vector of the mth corresponding point in the user's oral 3D model, Refers to the vector of the mth corresponding point in the suboptimal fusion 3D model, and calculates the average fluctuation value MeanRP based on the local fluctuation, setting , and outputs the average fluctuation value MeanRP as the model fluctuation difference RP. The preset model fluctuation difference RP0 refers to a preset value for real-time accuracy monitoring of the user's oral 3D model. This embodiment does not limit the specific numerical value of the preset model fluctuation difference. Those skilled in the art can set it according to actual needs, such as limiting the specific numerical value of the preset model fluctuation difference based on expert experience. The expert experience refers to the expert's experience in judging the fluctuation difference of the user's oral 3D model. The model accuracy result refers to the accuracy expression of the user's oral 3D model judged based on the model fluctuation difference RP and the preset model fluctuation difference RP0. The model accuracy result includes a model accuracy result indicating that the user's oral 3D model has met the standard and a model accuracy result indicating that the user's oral 3D model has not met the standard. This embodiment does not limit the specific optimization method for optimizing the cross-fusion processing process. Those skilled in the art can set it according to actual needs, as long as the requirement of improving the calculation accuracy of the optimal fusion data is met. For example, the weights of the data collected by the 3D intraoral scanner, the multi-angle intraoral camera array group, the facial scanner, and the CBCT scanner in the user's oral multimodal image data can be adjusted.

[0088] Specifically, the image data processing module performs real-time accuracy monitoring on the user's oral 3D model to obtain a model accuracy result. When the model accuracy result is that the user's oral 3D model does not meet the standard, the accuracy of the user's oral 3D model is inaccurate, which may easily lead to inaccurate annotation results when the user's oral 3D model is subsequently annotated. By optimizing the cross-fusion processing process, the calculation accuracy of the optimal fusion data is improved, thereby improving the accuracy of the user's oral 3D model.

[0089] Specifically, when the image data processing module performs real-time matching monitoring on the user's oral cavity 3D model according to the user information, obtains the model matching result, and corrects the user's oral multimodal image data acquisition process according to the model matching result, the age stage data and oral cavity material distribution data in the user information are converted into an age stage data vector and an oral cavity material distribution data vector, and the age stage data vector and the oral cavity material distribution data vector are input into the oral cavity 3D prediction model to obtain the predicted oral cavity 3D model output by the oral cavity 3D prediction model, and the matching mean square error RMSE is calculated according to the three-dimensional coordinates ai of the i-th point of the predicted oral cavity 3D model, the three-dimensional coordinates bi of the i-th point of the user's oral cavity 3D model and the number N of corresponding points, and the setting is performed. , compare the matching mean square error RMSE with the preset matching mean square error RMSE0, and judge the model matching result based on the comparison result, and correct the user's oral multimodal image data acquisition process based on the judgment result, where:

[0090] When RMSE≤RMSE0, the image data processing module determines that the model matching result is a match and does not correct the user's oral multimodal image data acquisition process;

[0091] When RMSE>RMSE0, the image data processing module determines that the model matching result is mismatched, corrects the user's oral multimodal image data acquisition process, increases the acquisition frequency of the multi-angle intraoral camera array group in the image acquisition control scheme, obtains the corrected image acquisition control scheme, replaces the image acquisition control scheme with the corrected image acquisition control scheme, and re-controls the 3D intraoral scanner, multi-angle intraoral camera array group, facial scanner and CBCT scanner according to the corrected image acquisition control scheme.

[0092] Specifically, the predicted oral 3D model refers to a theoretical oral 3D model of the user corresponding to the age stage data and oral material distribution data predicted based on the age stage data and oral material distribution data. The three-dimensional coordinates ai of the i-th point in the predicted oral 3D model refer to the three-dimensional coordinates of the i-th point in all the points in the predicted oral 3D model, including the X-axis coordinate Xai of the i-th point, the Y-axis coordinate Yai of the i-th point and the Z-axis coordinate Zai of the i-th point. This embodiment does not limit the setting method of the three-dimensional coordinates ai of the i-th point in the predicted oral 3D model. Those skilled in the art can freely set it according to actual conditions. For example, in this embodiment, the middle of the root of the tongue of the user's oral cavity is used as the origin to establish a spatial rectangular coordinate system, and the X-axis coordinate Xai of the i-th point, the Y-axis coordinate Yai of the i-th point and the Z-axis coordinate Zai of the i-th point are obtained according to the spatial rectangular coordinate system. The three-dimensional coordinates bi of the i-th point in the user's oral 3D model refers to the point corresponding to the i-th point in the predicted oral 3D model. The three-dimensional coordinates include the X-axis coordinate Xbi of the point corresponding to the i-th point, the Y-axis coordinate Ybi of the point corresponding to the i-th point, and the Z-axis coordinate Zbi of the point corresponding to the i-th point. This embodiment does not limit the setting method of the three-dimensional coordinate bi of the i-th point in the user's oral 3D model. The X-axis coordinate Xbi of the point corresponding to the i-th point, the Y-axis coordinate Ybi of the point corresponding to the i-th point, and the Z-axis coordinate Zbi of the point corresponding to the i-th point. The number N of corresponding points refers to the number of corresponding points between the user's oral 3D model and the predicted oral 3D model. The matching mean square error refers to the calculated mean square error between the user's oral 3D model and the predicted oral 3D model, which is used to determine the matching between the user's oral 3D model and the predicted oral 3D model. The preset matching mean square error refers to a preset value for determining the model matching result. This embodiment does not limit the specific value of the preset matching mean square error. Those skilled in the art can set it according to actual needs, such as setting RMSE0≥0.3mm, the model matching result refers to the matching between the predicted oral 3D model and the user's oral 3D model judged according to the matching mean square error and the preset matching mean square error, and the model matching result includes the model matching result being a match and the model matching result being a mismatch. This embodiment does not limit the specific method of increasing the acquisition frequency of the multi-angle intraoral camera array in the image acquisition control scheme. Those skilled in the art can set it according to actual conditions. For example, when the acquisition frequency of the initial multi-angle intraoral camera array is 15FkPS, the acquisition frequency of the initial multi-angle intraoral camera array is increased to 25FkPS. The oral 3D prediction model refers to a convolutional neural network model with age stage data vector and oral material distribution data vector as input and with predicted oral 3D model as output. This embodiment does not limit the specific construction method of the oral 3D prediction model. Those skilled in the art can set it according to actual conditions. The convolutional neural network model is trained using a 3D prediction model training dataset to obtain an oral 3D prediction model. The 3D prediction model training dataset includes a historical age stage data vector, a historical oral texture distribution data vector, and a predicted oral 3D model corresponding to the historical age stage data vector and the historical oral texture distribution data vector. This embodiment does not limit the specific method for converting the age stage data and oral texture distribution data in the user information into the age stage data vector and the oral texture distribution data vector. Those skilled in the art may customize the method based on actual needs. For example, the age stage data and oral texture distribution data in the user information may be converted into the age stage data vector and the oral texture distribution data vector using one-hot encoding. One-hot encoding is a common technique for converting categorical variables into a numerical format that is easily processed by machine learning models. Here, the categorical variables refer to the age stage data and oral texture distribution data.

[0093] Specifically, the image data processing module judges the model matching result and corrects the multimodal image data acquisition process of the user's oral cavity. When the image data processing module determines that the model matching result is not matched, the single-frame image may be blurred due to the patient's slight swallowing or tongue movement at low frequency. By increasing the acquisition frequency of the multi-angle intraoral camera array in the image acquisition control scheme, the multi-angle intraoral camera array "averages" the motion error through continuous multiple frames of images, thereby improving the model accuracy of the user's oral 3D model.

[0094] Specifically, when the image data processing module determines that the model matching result is a match, the user age stage data Rx is compared with the tooth development age stage data RA={R1, R2, R3, R4}, and the user's tooth development status is judged based on the comparison result. The process of real-time accuracy monitoring of the user's oral 3D model is adjusted based on the judgment result, wherein:

[0095] When Rx During RA, the image data processing module determines that the user's tooth development is agenesis, and does not adjust the process of real-time accuracy monitoring of the user's oral 3D model;

[0096] When Rx∈RA, the image data processing module determines that the user's tooth development status is developing, adjusts the process of real-time accuracy monitoring of the user's oral 3D model, adjusts the preset model fluctuation difference RP0 according to the adjustment coefficient α, obtains the adjusted preset model fluctuation difference RP0t, sets RP0t=RPO×α, replaces the preset model fluctuation difference RP0 with the adjusted preset model fluctuation difference RP0t, and re-compares the model fluctuation difference RP with the adjusted preset model fluctuation difference RP0t.

[0097] Specifically, the tooth development age stage data refers to the age stage of tooth development determined according to the physiological age method and the dental age assessment method. The stage includes infant age stage data R1, toddler age stage data R2, childhood age stage data R3 and adolescent age stage data R4. The user's tooth development status refers to the comparison of the user's age stage data Rx with the tooth development age stage data RA={R1, R2, R3, R4} to determine whether the user's teeth are in development. The user's tooth development status includes the user's tooth development status being undeveloped and the user's tooth development status being developed. The adjustment coefficient α refers to the coefficient used to adjust the preset model fluctuation difference RP0. This embodiment does not limit the specific value of the adjustment coefficient α. Those skilled in the art can set it according to actual needs. It only needs to meet the need to reduce the preset model fluctuation difference RP0. For example, α can be set to 0.84.

[0098] Specifically, the image data processing module judges the user's tooth development status and adjusts the process of real-time accuracy monitoring of the user's oral 3D model based on the judgment result. When the user's tooth development status is developing, the accuracy of the user's oral 3D model should be improved to enable more accurate monitoring of users with developing tooth development status. By reducing the preset model fluctuation difference, the monitoring requirements for the user's oral 3D model that fails to meet the model accuracy standard are improved, thereby making the judgment of the user's oral 3D model that fails to meet the model accuracy standard more stringent, thereby improving the model accuracy of the user's oral 3D model.

[0099] Specifically, when the image annotation processing module annotates the user's oral 3D model, it inputs the user's oral 3D model and age stage data into the oral 3D analysis model to obtain the user's oral 3D annotation model output by the oral 3D analysis model, and the user's oral 3D annotation model includes the first recommended target coordinate vector annotated on the oral 3D annotation model. and recommended support points.

[0100] Specifically, the oral 3D analysis model refers to a convolutional neural network model that takes the user's oral 3D model and age stage data as input and outputs the user's oral 3D annotation model. This embodiment does not limit the specific construction method of the oral 3D analysis model. Those skilled in the art can set it by themselves according to actual needs. For example, the convolutional neural network model can be trained by analyzing the training data set to obtain the oral 3D analysis model. The analysis training data set refers to the training data set used to construct the oral 3D analysis model. The analysis training data set includes historical user 3D oral models, historical age stage data and historical user 3D oral models, and historical user oral 3D annotation models corresponding to the historical age stage data. The first recommended target coordinate vector refers to the coordinate vector of the target position of orthodontic teeth recommended by the oral 3D analysis model based on historical data analysis. The target position of orthodontic teeth refers to the position to be achieved after orthodontic teeth. The recommended support point refers to the specific teeth and bones recommended by the oral 3D analysis model based on historical data analysis to resist the reaction force of the orthodontic force.

[0101] Specifically, the image annotation processing module obtains the user's oral cavity 3D annotation model through the oral cavity 3D analysis model, which can facilitate the user to intuitively see the first recommended target coordinate vector on the user's oral cavity 3D annotation model. and recommended anchorage points for subsequent oral analysis by users.

[0102] Specifically, when the image annotation processing module updates the user's oral 3D annotation model, it updates the user's habit intensity index HIIk, tooth position coefficient Ctooth, direction vector F k and habit weight coefficient w k Single habit compensation vector Calculate and set , and according to the number of user habits t and the single habit compensation vector Compensation vector for user's total habit Calculate and set , also based on the first recommended target coordinate vector and the user's total habit compensation vector For the second recommended target coordinate vector Calculate and set = + , the first recommended target coordinate vector Replaced with the second recommended target coordinate vector , and the second recommended target coordinate vector The user's oral cavity 3D annotation model is annotated to obtain a final user's oral cavity 3D annotation model, and the image annotation processing module sends the final user's oral cavity 3D annotation model to the image visualization module.

[0103] Specifically, the user habit strength index refers to the HII grade evaluated based on the user habit type and habit frequency. The user habit type refers to the type of user's dental habits, including unilateral chewing, anterior teeth biting, night bruxism, biting foreign objects and tongue thrust swallowing. The habit frequency refers to the frequency of occurrence of the user habit type. This embodiment does not limit the specific method of obtaining the user habit strength index. For example, the user habit type and habit frequency can be compared with the HII grade comparison table to obtain the user habit strength index. The HII grade comparison table is an HII grade table based on habit frequency. The HII grade includes mild 1, moderate 2 and severe 3. The tooth position coefficient Ctooth refers to the single habit compensation vector calculated according to the tooth serial number ToothID. The coefficient for calculation is set as Ctooth=0.2+0.05×ToothID. The tooth number refers to the serial number identification obtained by marking the teeth in the oral cavity. This embodiment does not limit the specific marking method of the tooth number. Those skilled in the art can set it according to actual needs. For example, the front teeth can be marked from left to right as 1-8, and the back teeth can be marked from left to right as 9-16. The habit weight coefficient refers to the weight coefficient corresponding to the user's habit type. The habit weight coefficient includes the front teeth biting weight coefficient w1=0.35, the tongue thrust swallowing weight coefficient w2=0.30, and the night grinding weight coefficient w3= 0.20, biting foreign objects weight coefficient w4=0.10 and unilateral chewing weight coefficient w5=0.05, the habit weight coefficient is designed according to the recurrence ratio of the user habit type, k is the category code of the user habit type, k=1, 2, 3, 4, 5, t is the total number of user habit types, and the maximum value of t is 5. The direction vector refers to the direction vector corresponding to the user habit type. This embodiment does not limit the specific setting method of the direction vector. For example, a three-dimensional coordinate system can be established with the root of the tongue as the coordinate origin, and the direction vectors corresponding to different user habit types can be obtained according to the force direction corresponding to the user habit type.

[0104] Specifically, the image annotation processing module calculates the user's total habit compensation vector and adjusts the recommended annotated target coordinate vector so that the recommended target coordinate vector can be individually recommended according to the user's personal habits, and the recommendation is more in line with the user's needs.

[0105] Specifically, when the image annotation processing module performs fine calibration on the update process of the user's oral 3D annotation model, the second recommended target vector (x2, y2, z2) and the initial tooth coordinate vector (x0, y0, z0) calculates the recommended tooth movement distance L and sets The recommended tooth movement distance L is compared with the preset recommended movement distance L0, L0 = 0.5mm. The habit compensation situation is judged based on the comparison results. The update process of the user's oral 3D annotation model is fine-tuned based on the judgment results, where:

[0106] When L≤L0, the image annotation processing module determines that the habit compensation is not over-compensated, and does not perform fine-tuning on the updating process of the user's oral cavity 3D annotation model;

[0107] When L>L0, the image annotation processing module determines that the habit compensation is over-compensation, and fine-tunes the updating process of the user's oral 3D annotation model. The user's total habit compensation vector is fine-tuned by the fine-tuning coefficient γ, and γ is set to 0.75+0.2×e -(L-L0) , e is the base of natural logarithm to obtain the total user habit compensation vector after fine-tuning ,set up, =γ× , the user's total habit compensation vector Replaced with the user's total habit compensation vector after fine-tuning , and compensate the vector according to the total habit of the user after fine-tuning For the second recommended target coordinate vector Recalculate until L≤L0.

[0108] Specifically, the x2 refers to the coordinate of the position to be reached after orthodontic treatment on the x-axis, the y2 refers to the coordinate of the position to be reached after orthodontic treatment on the y-axis, the z2 refers to the coordinate of the position to be reached after orthodontic treatment on the z-axis, the x0 refers to the coordinate of the initial position of the tooth on the x-axis, the y0 refers to the coordinate of the initial position of the tooth on the y-axis, and the z0 refers to the coordinate of the initial position of the tooth on the z-axis. The initial position of the tooth refers to the position of the tooth before orthodontic treatment. In this embodiment, the middle of the root of the tongue in the user's mouth is used as the origin to establish a spatial rectangular coordinate system, and the values ​​of x2, y2, z2, x0, y0 and z0 are obtained according to the spatial rectangular coordinate system. The preset recommended movement distance refers to a preset value for judging the habit compensation situation. The habit compensation situation refers to whether the habit compensation is excessive based on the recommended tooth movement distance and the preset recommended movement distance. The habit compensation situation includes the habit compensation situation being over-compensation and the habit compensation situation being not over-compensated. The L0=0.5mm is because when the movement of a single tooth exceeds 0.5mm Overcompensation will cause biomechanical imbalance and lead to irreversible periodontal tissue damage. -(L-L0) This is because when L is greater than L0, the movement of a single tooth will cause irreversible damage to the periodontal tissue. At this time, by multiplying y with the second recommended target coordinate vector, the value of the second recommended target coordinate vector decreases as L increases, thereby avoiding over-filling.

[0109] Specifically, the image annotation processing module judges the habit compensation situation and fine-tunes the updating process of the user's oral 3D annotation model according to the judgment result. When the habit compensation situation is judged to be over-compensation, since over-compensation may cause the tooth root to break through the bone cortex, the second recommended target coordinate vector is reduced. value to prevent overcompensation.

[0110] Specifically, when the image annotation processing module calibrates the updated fine calibration process of the user's oral 3D annotation model, it also calculates the habitual electromyographic value deviation △E according to the habitual behavior electromyographic value EMGhabit and the normal electromyographic value EMGbase, and sets , compare the habitual electromyographic value deviation △E with the first preset habitual electromyographic value deviation △E01, the second preset habitual electromyographic value deviation △E02, and the third preset habitual electromyographic value deviation △E03, △E01<△E02<△E03, judge the degree of habitual electromyographic deviation based on the comparison result, and calibrate the updated fine calibration process of the user's oral 3D annotation model based on the judgment result, wherein:

[0111] When ΔE≤ΔE01, the image annotation processing module determines that the degree of habitual electromyographic deviation is not a deviation, and does not calibrate the updating fine calibration process of the user's oral cavity 3D annotation model;

[0112] When △E01<△E≤△E02, the image annotation processing module determines that the degree of habitual electromyographic deviation is a mild deviation, and calibrates the updated fine calibration process of the user's oral 3D annotation model. According to the first calibration coefficient Jz1, Jz1=0.8+0.15×e -(△E02-△E) , e is the base of the natural logarithm, the preset recommended movement distance L0 is calibrated to obtain the first calibrated preset recommended movement distance L01, set L01=Jz1×L0, replace the preset recommended movement distance L0 with the first calibrated preset recommended movement distance L01, and re-compare the tooth recommended movement distance L with the first calibrated preset recommended movement distance L01;

[0113] When △E02<△E≤△E03, the image annotation processing module determines that the degree of habitual electromyographic deviation is moderate, and calibrates the updated fine calibration process of the user's oral 3D annotation model. According to the second calibration coefficient Jz2, Jz2=0.6+0.3×e -(△E03-△E) , e is the base of the natural logarithm, the preset recommended movement distance L0 is calibrated to obtain the second calibrated preset recommended movement distance L02, set L02=Jz2×L0, replace the preset recommended movement distance L0 with the second calibrated preset recommended movement distance L02, and re-compare the tooth recommended movement distance L with the second calibrated preset recommended movement distance L02;

[0114] When ΔE>ΔE03, the image annotation processing module determines that the degree of habitual electromyographic deviation is severe, does not calibrate the updating fine calibration process of the user's oral 3D annotation model, and sends an alarm signal to the visual interactive screen.

[0115] Specifically, the habitual behavior electromyographic value refers to the current value of the muscles around the user's mouth when the user is performing the user habit, and the normal electromyographic value refers to the average value of the current of the muscles around the user's mouth when the user is not performing the user habit. This embodiment does not limit the method of obtaining the habitual behavior electromyographic value and the normal electromyographic value. Those skilled in the art can set them according to actual needs. For example, the electrophysiological signals of the muscles around the user's mouth can be captured by surface electrodes. The first preset habitual electromyographic value deviation △E01, the second preset habitual electromyographic value deviation △E02 and the third preset habitual electromyographic value deviation △E03 refer to preset values ​​for judging the degree of habitual electromyographic deviation. In this embodiment, △E01=10 %, △E02=15%, △E03=25%, the habitual electromyographic deviation degree refers to the deviation degree of the habitual electromyographic value judged by the habitual electromyographic value deviation △E and the first preset habitual electromyographic value deviation △E01, the second preset habitual electromyographic value deviation △E02 and the third preset habitual electromyographic value deviation △E03, the habitual electromyographic deviation degree includes the habitual electromyographic deviation degree of no deviation, the habitual electromyographic deviation degree of mild deviation, the habitual electromyographic deviation degree of moderate deviation and the habitual electromyographic deviation degree of severe deviation. This embodiment does not limit the specific sending method of sending the alarm signal to the visual interactive screen, such as the alarm signal can be sent to the visual interactive screen in the form of wireless Wi-Fi.

[0116] Specifically, the image annotation processing module judges the degree of habitual electromyographic deviation and calibrates the updated precision calibration process of the user's oral 3D annotation model based on the judgment result. When the degree of habitual electromyographic deviation is mild and moderate, it proves that the user's personal habit is more serious than judged in the HII classification. This serious personal habit may cause the teeth to be unhealthy, and the teeth may be damaged due to personal habits. Excessive movement may easily cause tooth damage and the teeth cannot be maximized. At this time, the first calibration coefficient and the second calibration coefficient are multiplied by the preset recommended movement distance to reduce the value of the preset recommended movement distance, so that the judgment of the habit compensation as over-compensation is stricter, and the obtained second recommended target coordinate vector is more in line with the user's personal habits, thereby improving the personalization of the recommendation.

[0117] Specifically, when the image visualization module pushes the user oral 3D model and the user oral 3D annotation model to the visualization interaction screen, the final user oral 3D annotation model sent by the image annotation processing module and the user oral 3D model sent by the image data processing module are pushed to the visualization interaction screen as the user oral 3D annotation model and the user oral 3D model.

[0118] Specifically, this embodiment does not limit the specific pushing method of pushing the user's oral 3D annotation model and the user's oral 3D model to the visual interactive screen. For example, the user's oral 3D annotation model and the user's oral 3D model can be pushed to the visual interactive screen via wireless Wi-Fi.

[0119] Specifically, the image visualization module can quickly help users understand the oral condition by pushing the user's oral 3D model and the user's oral 3D annotation model to the visualization interactive screen, so as to analyze the oral cavity.

[0120] Specifically, the image visualization module also obtains user preferences and converts the user preferences into age stage data vectors and oral material distribution data vectors corresponding to the user preferences, inputs the age stage data vectors and oral material distribution data vectors corresponding to the user preferences into the oral 3D prediction model of the image data processing module, obtains the predicted oral 3D model output by the oral 3D prediction model, and pushes the predicted oral 3D model as the user's preferred oral 3D model to the visualization interactive screen.

[0121] Specifically, this embodiment does not limit the specific implementation method for obtaining user preferences. Those skilled in the art can set it themselves according to actual needs, such as obtaining user preferences by consulting users. This embodiment does not limit the specific implementation method for converting user preferences into age stage data vectors and oral material distribution data vectors corresponding to user preferences. Those skilled in the art can set it themselves according to actual conditions, such as converting user preferences into age stage data vectors and oral material distribution data vectors corresponding to user preferences through unique hot encoding. This embodiment does not limit the specific implementation method for pushing the predicted oral 3D model as the user's preferred oral 3D model to the visual interactive screen. Those skilled in the art can set it themselves according to actual conditions, such as pushing the user's preferred oral 3D model to the visual interactive screen through wireless Wi-Fi. The user's preferred oral 3D model refers to the oral model predicted according to the user's preferences.

[0122] Specifically, the image visualization module generates a 3D oral cavity model that the user prefers and pushes it to a visualization interactive screen for the user to select a recommended solution that the user prefers.

[0123] Thus far, the technical solutions of the present invention have been described in conjunction with the preferred embodiments shown in the accompanying drawings. However, it will be readily understood by those skilled in the art that the scope of protection of the present invention is obviously not limited to these specific embodiments. Without departing from the principles of the present invention, those skilled in the art may make equivalent changes or substitutions to the relevant technical features, and the technical solutions after such changes or substitutions will fall within the scope of protection of the present invention.

Claims

1. A device for visualizing orthodontic treatment process based on artificial intelligence, characterized in that: include: A 3D intraoral scanner is mounted on a bracket and electrically connected to a visual interactive screen and a controller; A multi-angle intraoral camera array group is arranged on a bracket and electrically connected to a visual interactive screen and a controller; A facial scanner is mounted on a bracket and electrically connected to a visual interactive screen and a controller; a CBCT scanner electrically connected to a visual interactive screen and a controller; a bracket, which is provided on the right side of the seat; a controller, which is arranged on the bracket; A visual image data processing system electrically connected to the visual interactive screen; The visual image data processing system includes: Orthodontic image acquisition module, used to acquire multimodal image data of the user's oral cavity; An image data processing module is used to perform cross-fusion processing on the multimodal image data of the user's oral cavity to obtain optimal fusion data, construct a 3D model of the user's oral cavity in real time based on the optimal fusion data to obtain the 3D model of the user's oral cavity, and send the 3D model of the user's oral cavity to the image visualization module. It is also used to perform real-time accuracy monitoring of the 3D model of the user's oral cavity to obtain a model accuracy compliance result, optimize the cross-fusion processing process based on the model accuracy compliance result, perform real-time matching monitoring of the 3D model of the user's oral cavity based on user information to obtain a model matching result, and correct the multimodal image data acquisition process of the user's oral cavity based on the model matching result, and adjust the optimization method of the cross-fusion processing process; An image annotation processing module is used to annotate the user's oral cavity 3D model to obtain a user's oral cavity annotated 3D model, update the user's oral cavity 3D annotated model to obtain a final user's oral cavity annotated 3D model, and send the final user's oral cavity 3D annotated model to the image visualization module. It is also used to fine-tune the update process of the user's oral cavity annotated model and calibrate the fine-tune process of the update of the user's oral cavity annotated model; An image visualization module, configured to push the user's oral cavity 3D model and the user's oral cavity 3D annotated model to a visualization interactive screen; The image data processing module preprocesses the user's oral multimodal image data to obtain preprocessed user's oral multimodal image data, performs cross-fusion processing on the preprocessed user's oral multimodal image data according to each preset cross-fusion processing path to obtain fused data of each preset cross-fusion processing path, and obtains the anatomical consistency coefficient S of the fused data of each preset cross-fusion processing path. anat , biomechanical reasonable coefficient S biomech , aesthetic matching coefficient S aes and calculation efficiency coefficient S speed , and calculate the scoring coefficient S of the fused data of each preset cross-fusion processing path, setting S = a1 × S anat +a2×S biomech +a3×S aes +a4×S speed , obtain the maximum value of the scoring coefficients of the fused data of each preset cross-fusion processing path, use it as the optimal fusion scoring coefficient Smax, compare the optimal fusion scoring coefficient Smax with the first preset fusion scoring coefficient S1 and the second preset fusion scoring coefficient S2, and identify the optimal fusion data according to the comparison results.

2. The artificial intelligence-based orthodontic treatment process visualization device according to claim 1, characterized in that: a seat connected to a footrest; a footrest, which is arranged on the lower left side of the seat; A column, which is arranged on the right side of the bracket; A face support, which is mounted on the column and located on the left side of the CBCT scanner; a seat base connected to the seat; An armrest is provided on the seat.

3. The artificial intelligence-based orthodontic treatment process visualization device according to claim 2, characterized in that: Set a1=0.4, a2=0.3, a3=0.2, a4=0.1, a1, a2, a3 and a4 are data scoring weight parameters, where a1 is the first data scoring weight parameter, a2 is the second data scoring weight parameter, a3 is the third data scoring weight parameter, and a4 is the fourth data scoring weight parameter. Set S1=0.8, S2=0.6, where: When S≥S1, the image data processing module identifies the data fused by the preset cross-fusion processing path corresponding to the optimal fusion scoring coefficient Smax as the optimal fusion data; When S2≤S<S1, the image data processing module pushes the fused data after the preset cross-fusion processing path to the visual interactive screen, obtains the fused data after the preset cross-fusion processing path selected by the user on the visual interactive screen, and identifies it as the optimal fused data; When S<S2, the image data processing module cannot identify the optimal fusion data, and pushes a prompt indicating that the optimal fusion data cannot be identified to the visual interaction screen.

4. The artificial intelligence-based orthodontic treatment process visualization device according to claim 3, characterized in that: The image data processing module converts the optimal fusion data into a triangular mesh according to a triangulation algorithm to obtain a 3D model of the user's oral cavity, and sends the 3D model of the user's oral cavity to the image visualization module.

5. The artificial intelligence-based orthodontic treatment process visualization device according to claim 4, characterized in that: The image data processing module calculates a model fluctuation difference RP based on the user's oral cavity 3D model and the suboptimal fused 3D model, compares the model fluctuation difference RP with a preset model fluctuation difference RP0, and performs real-time accuracy monitoring on the user's oral cavity 3D model based on the comparison result to obtain a model accuracy compliance result, wherein: When RP≤RP0, the image data processing module determines that the model accuracy meets the standard and the result is that the user's oral 3D model has met the standard; When RP>RP0, the image data processing module determines that the model accuracy meets the standard and the result is that the user's oral 3D model does not meet the standard, and optimizes the cross-fusion processing process.

6. The artificial intelligence-based orthodontic treatment process visualization device according to claim 5, characterized in that: The image data processing module performs real-time matching monitoring on the user's oral cavity 3D model according to the user information to obtain the model matching result, and corrects the user's oral multimodal image data acquisition process according to the model matching result. The age stage data and oral cavity material distribution data in the user information are converted into an age stage data vector and an oral cavity material distribution data vector, and the age stage data vector and the oral cavity material distribution data vector are input into the oral cavity 3D prediction model to obtain the predicted oral cavity 3D model output by the oral cavity 3D prediction model. The matching mean square error RMSE is calculated based on the three-dimensional coordinates ai of the i-th point of the predicted oral cavity 3D model, the three-dimensional coordinates bi of the i-th point of the user's oral cavity 3D model and the number N of corresponding points, and the setting is performed. The matching mean square error RMSE is compared with the preset matching mean square error RMSE0, and the model matching result is judged according to the comparison result. The user's oral multimodal image data acquisition process is corrected according to the judgment result, wherein: When RMSE≤RMSE0, the image data processing module determines that the model matching result is a match and does not correct the user's oral multimodal image data acquisition process; When RMSE>RMSE0, the image data processing module determines that the model matching result is mismatched, corrects the user's oral multimodal image data acquisition process, increases the acquisition frequency of the multi-angle intraoral camera array in the image acquisition control scheme, obtains the corrected image acquisition control scheme, replaces the image acquisition control scheme with the corrected image acquisition control scheme, and controls the acquisition of the 3D intraoral scanner, multi-angle intraoral camera array group, facial scanner and CBCT scanner according to the corrected image acquisition control scheme.

7. The artificial intelligence-based orthodontic treatment process visualization device according to claim 6, characterized in that: When the image data processing module determines that the model matching result is a match, the user age stage data Rx is compared with the tooth development age stage data RA={R1, R2, R3, R4}, and the user's tooth development status is judged based on the comparison result. The process of real-time accuracy monitoring of the user's oral 3D model is adjusted based on the judgment result, wherein: when When the image data processing module determines that the user's tooth development is not developed, the process of real-time accuracy monitoring of the user's oral 3D model is not adjusted; When Rx∈RA, the image data processing module determines that the user's tooth development status is developing, adjusts the process of real-time precision monitoring of the user's oral 3D model, adjusts the preset model fluctuation difference RP0 according to the adjustment coefficient α, obtains the adjusted preset model fluctuation difference RP0t, sets RP0t=RPO×α, replaces the preset model fluctuation difference RP0 with the adjusted preset model fluctuation difference RP0t, and re-compares the model fluctuation difference RP with the adjusted preset model fluctuation difference RP0t.

8. The artificial intelligence-based orthodontic treatment process visualization device according to claim 7, characterized in that: When the image annotation processing module annotates the user's oral 3D model, the user's oral 3D model and age stage data are input into the oral 3D analysis model to obtain the user's oral 3D annotated model output by the oral 3D analysis model. The user's oral 3D annotated model includes the first recommended target coordinate vector annotated on the oral 3D annotated model. and recommended support points.

9. The artificial intelligence-based orthodontic treatment process visualization device according to claim 8, characterized in that: The image annotation processing module updates the user's oral 3D annotation model according to the user's habit intensity index HIIk, tooth position coefficient Ctooth, direction vector F k and habit weight coefficient w k Single habit compensation vector Calculate and set And according to the number of user habits t and the single habit compensation vector Compensation vector for user's total habit Calculate and set Also based on the first recommended target coordinate vector and the user's total habit compensation vector For the second recommended target coordinate vector Calculate and set The first recommended target coordinate vector Replaced with the second recommended target coordinate vector And the second recommended target coordinate vector The user's oral cavity 3D annotation model is annotated to obtain a final user's oral cavity 3D annotation model, and the image annotation processing module sends the final user's oral cavity 3D annotation model to the image visualization module.

10. The artificial intelligence-based orthodontic treatment process visualization device according to claim 9, characterized in that: The image annotation processing module performs fine calibration on the update process of the user's oral 3D annotation model according to the second recommended target vector and the initial tooth coordinate vector Calculate the recommended tooth movement distance L and set The recommended tooth movement distance L is compared with the preset recommended movement distance L0, L0 = 0.5 mm. The habit compensation situation is judged based on the comparison result. The update process of the user's oral 3D annotation model is fine-tuned based on the judgment result, where: When L≤L0, the image annotation processing module determines that the habit compensation is not over-compensated, and does not perform fine-tuning on the updating process of the user's oral cavity 3D annotation model; When L>L0, the image annotation processing module determines that the habit compensation is over-compensation, and fine-tunes the updating process of the user's oral 3D annotation model. The user's total habit compensation vector is fine-tuned by the fine-tuning coefficient γ, and γ is set to 0.75+0.2× e-(L-L0) , e is the base of natural logarithm to obtain the total user habit compensation vector after fine-tuning set up, The user's total habit compensation vector Replaced with the user's total habit compensation vector after fine-tuning And according to the user's total habit compensation vector after fine-tuning For the second recommended target coordinate vector Recalculate until L≤L0.

Images