Tooth position monitoring method, system and terminal based on generative artificial intelligence

Abstract

The invention relates to a tooth position monitoring method, in particular to a system and a terminal capable of accurately generating a simulated medical image containing tooth information and marking tooth position change conditions based on an intraoral tooth picture of a patient in an orthodontic process. Based on a mapping relation model between an intraoral tooth picture and an oral medical image, a simulated medical image is generated by the intraoral tooth picture which is safe and easy to acquire by a person; by identifying and positioning tooth positions in real and simulated oral medical images, a change relation model of three-dimensional tooth positions at a plurality of time points is established, and dense monitoring of tooth conditions is realized. According to the invention, more clinical data can be tested, and under the expectation of better generalization, the method can be used as a monitoring means for tooth change conditions in the orthodontic treatment process, so that clinicians are helped to efficiently determine a treatment scheme and quantitatively evaluate the treatment effect.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and is a method for monitoring tooth positions, and in particular to a system and terminal that can accurately generate simulated medical images based on a tooth picture in a subject's mouth and based on generative artificial intelligence technology, and mark the positions of soft tissues such as teeth and gingival mucosa during oral orthodontic treatment. Background Art

[0002] According to the oral disease survey report, nearly 90% of the world's population has some degree of oral problems, and many of them need dental treatment. Currently, more and more patients choose to undergo oral orthodontics for oral diseases such as malocclusion.

[0003] However, during the orthodontic process, side effects such as bone fenestration and bone cracking are easily caused, resulting in tooth root exposure. CT is a powerful tool for detecting the position of tooth roots in the alveolar bone. However, CT scanning has limitations such as radiation, cannot be taken multiple times, requires special equipment and operators, is expensive, and is not convenient for taking pictures at any time.

[0004] On the other hand, during the orthodontic process, doctors need to judge the movement of teeth and monitor the position changes of crowns. Currently, they can obtain crown models of patients through oral plaster impression technology or digital oral scanner systems. However, this requires patients to go to the dental clinic for multiple impressions or scans, which increases the burden of visits and is not convenient for monitoring crown changes at any time.

[0005] Therefore, how to find a safe, convenient and effective tool to monitor the condition of teeth is a difficult problem in current technology. With the development of computer-aided technology, although digital dental systems have been used in oral disease diagnosis, clinical surgical restoration, treatment planning and postoperative treatment evaluation, these digital dental systems are based on sparse data such as X-ray images, CT images (including CBCT images), and oral scan images. How to generate simulated medical images based on intraoral pictures or videos that can be taken anytime and anywhere to accurately display the position of teeth, such as the position relationship of the tooth root in the alveolar bone, the occlusal relationship of the upper and lower teeth, etc., so as to monitor the condition of teeth safely, efficiently, conveniently and intensively, is the current technical bottleneck. Summary of the invention

[0006] In view of this, the purpose of the present invention is to provide a method, system and terminal device for monitoring tooth position using intraoral tooth images, so as to solve the deficiencies in the prior art and achieve the purpose of improving the treatment efficiency during tooth correction.

[0007] To achieve the above-mentioned purpose and other related purposes, the present invention provides a method for monitoring tooth position, the method comprising: obtaining an intraoral tooth image of a subject, wherein the intraoral tooth image is derived from a photo or a video; generating a simulated medical image according to the intraoral tooth image of the subject based on a mapping relationship model between the intraoral tooth image and an oral medical image, wherein the oral medical image is derived from CT image data or optical scanning image data; detecting tooth position based on the simulated medical image; constructing a progression relationship model between the oral medical image and the simulated medical image at multiple time points, and establishing a progression pattern of the subject's tooth position over time based on the results of data analysis of multiple real and simulated medical images of the subject, thereby realizing tooth position monitoring.

[0008] The present invention also provides a tooth position monitoring system, which includes a data acquisition module, a data preprocessing module, a data generation module, a data correction module, and a tooth position monitoring module. The data acquisition module collects the intraoral tooth images taken by the subject at different viewing angles and the corresponding oral medical images. The data preprocessing module preprocesses the collected data to obtain information such as the shape, position, three-dimensional geometric features, and texture features of the teeth. The data generation module establishes a mapping relationship between the preprocessed intraoral tooth images and the medical images, and takes the preprocessed intraoral tooth images of the subject as input to output a simulated medical image of the subject's oral cavity, such as a simulated CT image including the root, alveolar bone, and periodontal membrane, or a simulated oral scan image including the morphology of a single crown, the position relationship of the full mouth crowns, the crown inclination angle data, and the occlusal relationship of the upper and lower teeth. The data correction module compares the simulated medical image generated by the data generation module with the real oral medical image, and iteratively updates the algorithm and / or model parameters in the data preprocessing module and the data generation module. The tooth position monitoring module performs computational analysis on oral medical images and simulated medical images at multiple time points and displays the monitoring results of the subject's tooth position.

[0009] To achieve the above-mentioned purpose and other related purposes, the present invention provides a tooth position change monitoring terminal, including: a memory and a processor; the memory is used to store a computer program; the processor is connected to the memory and is used to run the computer program to implement the above-mentioned tooth position monitoring method.

[0010] As described above, the present invention is a tooth position monitoring method, system and terminal based on generative artificial intelligence. Compared with the prior art, it has at least the following obvious advantages and effects: the present invention utilizes widely used, easily accessible, simple, convenient, inexpensive, and non-radiative intraoral tooth images to generate simulated oral medical images, and automatically identifies teeth based on real and simulated medical images of the subjects at multiple time points, and calculates the tooth position information; it can monitor the changes in the root position, the crown status and its occlusal relationship at any time, and realize intensive detection of tooth conditions, which helps doctors and patients to more accurately and comprehensively grasp the tooth status and changes, and promotes the development of personalized precision diagnosis and treatment and telemedicine; and it greatly improves the patient's medical experience, reduces the pressure on the imaging department, saves costs, achieves the effect of reducing costs and increasing efficiency, and can also alleviate the current situation of severe shortage of oral medical resources to a certain extent.

[0011] In order to make the above features and advantages of the present invention more clearly understood, embodiments are given below with reference to the accompanying drawings for detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Figure 1 It is a schematic flow chart of the tooth root position monitoring method in the first embodiment of the present invention.

[0013] Figure 2 Shown is a schematic structural diagram of a tooth root position monitoring system in Embodiment 1 of the present invention.

[0014] Figure 3 It is a schematic flow chart of the intermediate processing process of the tooth root position monitoring system in the first embodiment of the present invention.

[0015] Figure 4 The diagram shows an example of clinical application of the tooth root position monitoring system in the first embodiment of the present invention.

[0016] Figure 5 It is a schematic flow chart of the method for monitoring the position of a tooth crown in the second embodiment of the present invention.

[0017] Figure 6 It is a schematic diagram of the structure of the tooth crown position monitoring system in the second embodiment of the present invention.

[0018] Figure 7 It is a schematic flow chart of the intermediate processing process of the tooth crown position monitoring system in the second embodiment of the present invention.

[0019] Figure 8 The diagram shows an example of clinical application of the tooth crown position monitoring system in the second embodiment of the present invention.

[0020] Fig. 9Shown is a structural schematic diagram of a tooth position monitoring system terminal in an embodiment of the present invention. DETAILED DESCRIPTION

[0021] The present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are only used to explain the present invention, rather than to limit the present invention. It should also be noted that, for ease of description, only the parts related to the present invention, rather than all structures, are shown in the accompanying drawings.

[0022] Before discussing the exemplary embodiments in more detail, it should be mentioned that some exemplary embodiments are described as processes or methods depicted as flow charts. Although the flow charts describe the operations (or steps) as sequential processes, many of the operations (or steps) therein can be implemented in parallel, concurrently or simultaneously. In addition, the order of the operations can be rearranged. The process can be terminated when its operation is completed, but can also have additional steps not included in the accompanying drawings. The process can correspond to a method, function, procedure, subroutine, subprogram, etc.

[0023] Furthermore, as used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless there is an indication to the contrary in the context. It should be further understood that the terms "comprise", "include" indicate the presence of the described features, operations, elements, components, items, kinds, and / or groups, but do not exclude the presence, occurrence or addition of one or more other features, operations, elements, components, items, kinds, and / or groups. The terms "or" and "and / or" used herein are interpreted as inclusive, or mean any one or any combination. Therefore, "A, B or C" or "A, B and / or C" means "any of the following: A; B; C; A and B; A and C; B and C; A, B and C". Exceptions to this definition will only occur when the combination of elements, functions or operations is inherently mutually exclusive in some way.

[0024] The present invention provides a tooth position monitoring method, system, terminal and medium based on generative artificial intelligence, which establishes a mapping relationship model between intraoral tooth images and oral medical images, generates oral medical images that are radioactive and / or require professional personnel and equipment to collect from safe and easily accessible intraoral tooth images; identifies and locates the tooth positions in real and simulated oral medical images, establishes a three-dimensional tooth position change relationship model at multiple time points, and realizes intensive monitoring of tooth conditions. The present invention can also be tested on more clinical data, and with the expectation of good generalization, it can be used as a means of monitoring tooth changes during orthodontic treatment, thereby helping clinicians to efficiently determine treatment plans and quantitatively evaluate treatment effects, thereby achieving the purpose of improving treatment efficiency.

[0025] The following is a detailed description of the embodiments of the present invention with reference to the accompanying drawings so that those skilled in the art can easily implement the present invention. The present invention can be embodied in many different forms and is not limited to the embodiments described herein.

[0026] Embodiment 1

[0027] Figure 1 The figure shows a flow chart of a tooth position monitoring method in an embodiment of the present invention, specifically, a flow chart of a tooth root position monitoring method. The tooth root position monitoring method comprises:

[0028] Step 101: Obtain a tooth image in the subject's mouth at the current time point.

[0029] Optionally, the intraoral teeth image may be taken from multiple viewing angles or extracted from an intraoral video.

[0030] Optionally, the intraoral tooth image may be preprocessed by noise reduction, normalization, data enhancement, etc.

[0031] Step 102: Based on deep learning technology, a mapping relationship model between intraoral tooth images and oral CT images is constructed.

[0032] Optionally, the construction of the mapping relationship model may embed auxiliary information such as the subject's age, expert prior knowledge, etc.

[0033] Optionally, the construction of the mapping relationship model can adopt a two-stage iterative learning strategy to establish the mapping relationship between the intraoral tooth images and the oral CT images, wherein the first stage is to use all the intraoral tooth images and oral CT images to learn the coarse mapping relationship between the two data domains, and the second stage is to optimize the model parameters of the first stage through an iterative method using the intraoral tooth images and oral CT image data of the subjects that are paired one by one, thereby learning the individualized fine mapping relationship between the two data domains.

[0034] Specifically, the mapping relationship model can be based on a generative artificial neural network learning model such as a VAE model, a generative adversarial network model (GAN), a diffusion model, etc. Preferably, a generative adversarial network model is used to establish an intraoral tooth image (I photo ) and oral CT images (I CT ) between: The loss function used by the model is as follows:

[0035] L GAN (G,D)=E[log D(I CT )]+E[log(1-D(G(Iphoto ,I aux ))]

[0036] L1(G)=E[||I CT -G(I photo ,I aux )||1]

[0037] Among them, I photo ,I CT and I aux In the first stage, the intraoral tooth images and oral CT images of all patients (not necessarily paired data, i.e., I images of different patients) are used to obtain the intraoral tooth images and oral CT images of all patients. photo or I CT ) learns the global features of the two data domains to ensure the rationality of the conversion between different data domains. The auxiliary information used can be the edge map, gradient map or expert prior knowledge of the image. It is added to the backbone learning network of the generative model through the attention mechanism at the image level or feature level, so that the auxiliary information is integrated with the intraoral tooth image information to generate higher quality simulated CT images. In the second stage, the model of the first stage is fine-tuned using paired intraoral tooth images and oral CT images of different patients (the intraoral tooth images and oral CT images are from the same patient) to learn a more detailed and personalized mapping relationship between the intraoral tooth images and oral CT images. In addition to expert prior knowledge and edge maps of the image, the auxiliary information used can also include individualized clinical information such as the patient's tooth morphology data, gender, age, and previous diagnostic data.

[0038] Step 103: According to the constructed mapping relationship model between the intraoral teeth image and the oral CT image, a simulated CT image of the subject is generated from the intraoral teeth image of the subject, that is, an oral CT image of the subject is simulated.

[0039] Step 104: Based on the real and simulated oral CT images of the subject at previous time points and the generated current time point, a progression relationship model of intraoral tooth changes is constructed, specifically focusing on changes in tooth root positions.

[0040] Optionally, the intraoral tooth CT image may be preprocessed. The preprocessing method is described in detail in the operation content of the data preprocessing module of the tooth root position monitoring system in the following embodiment.

[0041] Optionally, the progress relationship model may embed expert knowledge.

[0042] Optionally, the progression relationship model may use deep learning algorithms such as recurrent convolutional neural networks and graph neural networks to establish the spatiotemporal change relationship of the three-dimensional tooth root position at multiple time points.

[0043] Step 105: Based on the progression relationship model, monitor the changes in the tooth root positions of the subject at multiple time points.

[0044] Optionally, the change in the position of the tooth root may include indicator information such as the torsion angle, inclination angle, and movement distance of the tooth root.

[0045] Figure 2 The diagram is a schematic diagram of the structure of a tooth root position monitoring system in one embodiment of the present invention. The system comprises:

[0046] Data acquisition module 201: collects intraoral teeth pictures and oral CT images taken from different viewing angles of the subject, wherein the intraoral teeth pictures and oral CT images may be paired data or unpaired data.

[0047] Optionally, the data collection module 201 may also obtain individual information and / or clinical information of the subject, such as age, gender, blood type, and previous medical history.

[0048] Optionally, the data acquisition module 201 may also obtain expert knowledge base or prior information such as tooth root position information, tooth contour information, key point coordinate information, previous clinical decisions, etc. marked by practicing dentists.

[0049] Optionally, the data acquisition module 201 may also obtain data through the network, cloud, Bluetooth, external storage and other devices.

[0050] Optionally, the data acquisition module 201 can use a camera of a smart device such as a mobile phone, an electronic camera, or other photographing device to obtain multi-viewing intraoral dental images of the subject, such as Figure 3 As shown in the picture in step 301 in , or extracted from the intraoral video.

[0051] Optionally, the intraoral tooth images and oral CT images may be acquired at one or more time points.

[0052] Data preprocessing module 202: preprocesses the data collected by the data collection module 201 to obtain information such as the shape, position, three-dimensional geometric features, and texture features of the teeth.

[0053] Optionally, the preprocessing operation of the data preprocessing module 202 includes: performing crown recognition, tooth segmentation, three-dimensional tooth reconstruction and other processing on the intraoral tooth image, so as to calculate the torsion angle, tilt angle, movement distance and other index information of the crown. Some examples of processing results are as follows: Figure 3As shown in the picture of step 302 in .

[0054] Optionally, the preprocessing operation of the data preprocessing module 202 includes: identifying, segmenting, and positioning the crown, root, alveolar bone, and periodontal membrane of the oral CT image. Some examples of processing results are as follows: Figure 3 As shown in 304, the coordinate system of the three-dimensional tooth data can be polar coordinates or three-dimensional rectangular coordinates, such as Figure 3 The red dot shown as 304 in the figure indicates that the coordinate value of the tooth root is (x, y, z).

[0055] Optionally, the preprocessing operation of the data preprocessing module 202 includes: normalizing the intraoral tooth image and the oral CT image.

[0056] Optionally, the preprocessing operation of the data preprocessing module 202 includes: registering the three-dimensional tooth data constructed based on the intraoral tooth image with the oral CT image.

[0057] Optionally, the registration process can be based on the crown position, by establishing key points (landmarks, keypoints) at the edge of the crown, root, alveolar bone, and periodontal membrane to complete the registration of the three-dimensional tooth data and the oral CT image. The specific process of the registration based on key points or feature points includes:

[0058] First, build the key point detection network Detect p and Dtect CT , which is used to extract key points from the three-dimensional tooth data and oral CT image data. The key point sets of the intraoral tooth image and oral CT image can be expressed as or The output layer of the key point detection network uses a center-of-mass layer, which can calculate the center of mass of the activation map in each channel of the network output (that is, the center position of the feature estimated by calculating the weighted average position of the voxel intensity).

[0059] Then, affine registration is performed on the key points of the extracted 3D tooth data and oral CT image data, and K key points are aligned. Input into the affine calculation layer to calculate the affine matrix A to achieve global alignment of three-dimensional tooth data and oral CT image data.

[0060] Finally, it is input into the refined deformable registration network, and the local registration is dynamically adjusted through the attention mechanism to learn a dense deformation field and achieve accurate registration of 3D tooth data and oral CT image data.

[0061] By minimizing the loss function To drive the pre-training of the above key point detection network, where represents the spatial transformation operation, T represents the affine transformation, E T represents the expectation, Detect() represents the key point detection network, I i and P i denote the i-th image and its corresponding key points respectively.

[0062] The pre-trained key point detection network is used as the initialization of the registration network, and the normalized cross-correlation function (NCC) is used to train the registration network. The loss function NCC function used in affine registration is:

[0063]

[0064] In the formula, x i represents the local window centered at image pixel x, represents the template image after affine transformation, that is A represents the affine matrix calculated above, P p ,P CT Represents a set of key points extracted from 3D tooth data and CT image data.

[0065] The corresponding NCC function of deformation registration is:

[0066]

[0067] In the formula, x i represents the local window centered at image pixel x, Represents the template image after affine and deformable transformation, that is, The deformation field is the result of affine registration.

[0068] Optionally, the total loss function of the above model also includes a regularization term to control the smoothness of the deformation field.

[0069] Data generation module 203: establishes a mapping relationship between the preprocessed intraoral teeth image and the oral CT image, so as to output a simulated CT image of the oral cavity of the subject by taking the preprocessed intraoral teeth image of the subject as input.

[0070] Optionally, the simulated CT image includes CT data of tooth root, alveolar bone and periodontal membrane, for example Figure 3 As shown in the picture of step 303 in .

[0071] Optionally, the simulated CT image is CT image data after image enhancement processing, for example Figure 3 As shown in the picture of step 304 in .

[0072] Optionally, the data generation module 203 is embedded with a generative artificial neural network learning model such as an autoencoder model, a generative adversarial network model or a diffusion model, and the model parameters are determined by minimizing multiple loss functions.

[0073] The data correction module 204 compares the simulated CT image generated by the data generation module 203 with the real oral CT image, and updates the algorithms and / or model parameters in the data preprocessing module 202 and the data generation module 203 through iteration.

[0074] Optionally, the data correction module 204 can calculate and analyze the errors between the real and simulated oral CT images or the root position coordinates based on the newly collected data by the data acquisition module 201, and provide feedback to the data preprocessing module 202 and the data generation module 203 to further update the algorithm and / or model parameters in the optimization module.

[0075] The tooth root position monitoring module 205 performs computational analysis on real and simulated oral CT images at multiple time points, and quantifies and visualizes the monitoring results of the tooth root position of the subject.

[0076] Optionally, the root position monitoring results include one or more information such as four-dimensional visualization results of the root position at multiple time points, reminders of abnormal root position changes, recommendations for root position correction plans, calculation of the root position change curve, root position deflection angle, and distance to the alveolar bone wall.

[0077] Figure 4 Shown is a diagram showing an example of the clinical application of a tooth root position monitoring system in an embodiment of the present invention. At time point T1 shown in stage 401, a picture of the patient's teeth in the mouth and an oral CT image are obtained, and the data may be collected during the initial diagnosis. Based on the above data and the patient's individual information, the doctor makes a diagnosis and implements treatment plan I. At time point T2, a simulated CT image is generated based on the picture of the patient's teeth in the mouth, and the root position is monitored based on the simulated CT image. The interval between time point T1 and time point T2 may be any time interval selected by the patient. If the root position monitoring system has no abnormal alert, it means that treatment plan I is normal; if at T i-1At this time point, if the system prompts that the root position changes abnormally, the treatment plan needs to be re-formulated. i The intraoral dental images and oral CT images at the time point, as shown in stage 402, revise the treatment method, implement treatment plan II, and monitor the root position as described above until the ideal correction effect is achieved.

[0078] Embodiment 2

[0079] Figure 5 The figure shows a flow chart of a tooth position monitoring method in an embodiment of the present invention, specifically, a flow chart of a tooth crown position monitoring method. The tooth crown position monitoring method comprises:

[0080] Step 501: Obtain a tooth image in the subject's mouth at the current time point.

[0081] Optionally, the intraoral teeth image may be taken from multiple viewing angles or extracted from an intraoral video.

[0082] Optionally, the intraoral tooth image may be preprocessed by noise reduction, normalization, data enhancement, etc.

[0083] Step 502: Based on deep learning technology, a mapping relationship model between the intraoral tooth image and the three-dimensional crown image is constructed.

[0084] Optionally, the construction of the mapping relationship model may embed auxiliary information such as the subject's age, expert prior knowledge, etc.

[0085] Optionally, the three-dimensional crown image can be obtained through oral plaster impression technology or a digital oral scanner system and pre-processed by noise reduction, normalization, data enhancement, etc.

[0086] Optionally, the construction of the mapping relationship model can adopt a two-stage iterative learning strategy to establish the mapping relationship between the intraoral tooth images and the three-dimensional crown images, wherein the first stage is to use all the intraoral tooth images and three-dimensional crown images to learn the coarse mapping relationship between the two data domains, and the second stage is to optimize the model parameters of the first stage through an iterative method using the intraoral tooth images and three-dimensional crown image data of the subjects that are paired one by one, thereby learning the individualized fine mapping relationship between the two data domains.

[0087] Specifically, the mapping relationship model can be based on a generative artificial neural network learning model such as a VAE model, a generative adversarial network model (GAN), a diffusion model, etc. Preferably, a generative adversarial network model is used to establish an intraoral tooth image (I photo ) and 3D crown image (Icrown ) between: The loss function used by the model is as follows:

[0088] L GAN (G,D)=E[log D(I crown )]+E[log(1-D(G(I photo ,I aux ))]

[0089] L1(G)=E[||I crown -G(I photo ,I aux )||1]

[0090] Among them, I photo ,I crown and I aux In the first stage, the intraoral tooth images and 3D crown images of all patients (not necessarily paired data, i.e., images of different patients) are used to obtain the intraoral tooth images and 3D crown images. photo or I crown ) learns the global features of the two data domains to ensure the rationality of the conversion between different data domains. The auxiliary information used can be the edge map, gradient map or expert prior knowledge of the image. It is added to the backbone learning network of the generative model through the attention mechanism at the image level or feature level, so that the auxiliary information is integrated with the intraoral tooth image information to generate higher quality simulated CT images. In the second stage, the model of the first stage is fine-tuned using paired intraoral tooth images and 3D crown images of different patients (the intraoral tooth images and 3D crown images are from the same patient) to learn a more refined and personalized mapping relationship between the intraoral tooth images and 3D crown images. In addition to expert prior knowledge and edge maps of the image, the auxiliary information used can also include individualized clinical information such as the patient's tooth morphology data, gender, age, and previous diagnostic data.

[0091] Optionally, the loss function of the above model also includes a tooth biomechanical constraint term to make the generated three-dimensional crown image conform to the biomechanical properties and ensure the accuracy and robustness of the generated image.

[0092] Step 503: Generate a simulated CT image of the subject from the intraoral tooth image of the subject according to the constructed mapping relationship model between the intraoral tooth image and the three-dimensional crown image, that is, simulate a three-dimensional crown image of the subject.

[0093] Figure 6The diagram is a schematic diagram of the structure of a tooth crown position monitoring system in one embodiment of the present invention. The system comprises:

[0094] Data collection module 601: collects the teeth images taken by the subject at different viewing angles (such as Figure 7 ) and a three-dimensional crown image (as shown in the picture in step 701 in FIG. Figure 7 As shown in the picture in step 703 in the figure), the intraoral tooth picture and the three-dimensional crown image can be paired data or unpaired data.

[0095] Optionally, the three-dimensional crown image can be obtained by scanning the patient's oral cavity with a high-precision digital oral scanning device, or by taking a mold. For example, a plaster perfusion impression can be used to obtain a plaster model, and then the three-dimensional oral crown model data of the plaster model can be obtained using a warehouse scanning technology.

[0096] Optionally, the data collection module 601 may also obtain individual information and / or clinical information of the subject, such as age, gender, blood type, and previous medical history.

[0097] Optionally, the data acquisition module 601 may also obtain expert knowledge base or prior information such as crown position information, crown contour information, crown key point coordinate information, occlusal relationship information between crowns, previous clinical decisions, etc. marked by practicing dentists.

[0098] Optionally, the data acquisition module 601 may also obtain data through the network, cloud, Bluetooth, external storage and other devices.

[0099] Optionally, the data acquisition module 601 can use cameras of smart devices such as mobile phones, electronic cameras and other shooting devices to obtain multi-angle intraoral teeth images of the subject, or extract them from intraoral videos. The extraction principle is to describe the crown characteristics as comprehensively as possible.

[0100] Optionally, the intraoral tooth image and the three-dimensional crown image may be acquired at one or more time points.

[0101] Data preprocessing module 602: preprocesses the data collected by the data collection module 601 to obtain information such as the shape, position, three-dimensional geometric features, and texture features of the tooth crown.

[0102] Optionally, the preprocessing operations of the data preprocessing module 602 include: extracting regions of interest, normalizing data, aligning images, amplifying data, identifying crowns, segmenting teeth, detecting key points of crowns, and reconstructing three-dimensional teeth (e.g., Figure 7 702 in the figure).

[0103] In another embodiment, the preprocessing operation of the data preprocessing module 602 includes: quantifying the three-dimensional crown features, and constructing a crown analysis model to achieve quantification of the crown's morphology, position, torsion angle, tilt angle, movement angle, movement distance, occlusal relationship, torque, relative position relationship between crowns (tilt angle), etc. The crown analysis model can be constructed based on the crown key point detection results.

[0104] Data generation module 603: establishes a mapping relationship between the preprocessed intraoral tooth image and the three-dimensional crown image, so as to output a simulated crown image of the subject's oral cavity by taking the preprocessed intraoral tooth image of the subject as input.

[0105] Optionally, a generative model is used to learn the mapping relationship between the intraoral tooth photo and the three-dimensional crown image. The network structure of the generative model and the generated three-dimensional crown image are exemplified as follows: Figure 7 As shown in the picture of step 704 in .

[0106] Optionally, the data generation module 603 may be a generative artificial neural network learning model such as an autoencoder model, a generative adversarial network model or a diffusion model, and the model parameters are determined by minimizing multiple loss functions.

[0107] Optionally, the generative model can embed clinical information such as the subject's age, key dental features, and expert prior knowledge through an attention mechanism.

[0108] Optionally, the generative model can be trained using a two-stage training method. In the first stage, a general mapping between two data domains, the preprocessed intraoral tooth images and the real three-dimensional crown images, is learned. The training data are data pairs of different patients at different time points. In the second stage, the model parameters of the generative network are fine-tuned by learning the precise mapping between two data domains, the preprocessed intraoral tooth images and the real three-dimensional crown images. The training data used are data pairs of the same patient at different time points or at the same time point.

[0109] The data correction module 604 compares the simulated crown image generated by the data generation module 603 with the real three-dimensional crown image, and updates the algorithms and / or model parameters in the data preprocessing module 602 and the data generation module 603 through iteration.

[0110] Optionally, the data correction module 604 can calculate and analyze the error between the real and simulated three-dimensional crown images based on the newly collected data by the data acquisition module 601, and provide feedback to the data preprocessing module 602 and the data generation module 603 to further update the algorithm and / or model parameters in the optimization module.

[0111] The tooth crown position monitoring module 605 performs computational analysis on the real and simulated three-dimensional tooth crown images at multiple time points to quantify and visualize changes in the subject's tooth crown.

[0112] Optionally, the crown position monitoring results include one or more information such as four-dimensional visualization results of the crown position at multiple time points, reminders of abnormal crown position changes, recommendations for crown position correction plans, calculation of crown position change curves, crown deflection angles, and distances to the alveolar bone wall.

[0113] Optionally, the crown position monitoring module 605 includes a crown progression model, which may use deep learning algorithms such as recurrent convolutional neural networks and graph neural networks to establish the spatiotemporal variation relationship of crown features at multiple time points to monitor the crown changes of the subject at multiple time points.

[0114] Optionally, the dental crown progression model makes an individualized prediction of the longitudinal dental crown changes of the subject, that is, predicts the dental crown changes at one or more time points in the future.

[0115] Figure 8 Shown is a diagram showing an example of the clinical application of a crown position monitoring system in an embodiment of the present invention. At the T1 time point shown in stage 801, a photograph of the patient's teeth in the mouth and a three-dimensional crown image are obtained. The data can be collected by the patient during the initial diagnosis. Based on the above data and the patient's individual information, the doctor makes a diagnosis and executes treatment plan I. For example, for a patient with misplaced maxillary lateral incisors, the crown position before correction is shown in area 8011, and the occlusal relationship between the upper and lower jaws is shown in area 8012. At the T2 time point, a photograph of the patient's teeth in the mouth is taken to generate a corresponding simulated crown image of the patient, and the crown changes are monitored based on the generated simulated crown image. The interval between the T1 time point and the T2 time point can be any time interval selected by the patient. If the crown position monitoring system has no abnormal alert, it means that treatment plan I is normal. If at T i-1 At this time point, the system prompts that the crown position changes abnormally, and the treatment plan needs to be re-formulated. i The intraoral dental photographs at the time point and the corresponding three-dimensional crown images are used to revise the treatment method and implement treatment plan II, as shown in stage 802. For example, for patients with maxillary lateral incisor misalignment, the crown position during correction is shown in area 8021, and the occlusal relationship between the upper and lower jaws is shown in area 8022. The crown changes are continuously monitored as described above until the ideal correction effect is achieved, as shown in stage 803. For example, for patients with maxillary lateral incisor misalignment, the crown position after correction is shown in area 8031, and the occlusal relationship between the upper and lower jaws is shown in area 8032.

[0116] Fig. 9The structure diagram of the tooth position monitoring system terminal in one embodiment of the present invention is shown. The tooth position monitoring terminal based on intraoral tooth images includes a main control device 90, and the main control device 90 includes: one or more memories 901 and one or more processors 902. The memory 901 is used to store computer programs; the processor 902 is used to run the computer program to implement the tooth position monitoring method based on intraoral tooth images.

[0117] Optionally, the memory 901 in the main control device 90 of the tooth position monitoring terminal of the intraoral tooth image stores Figure 2 or Figure 6 The computer program instructions of the processes corresponding to the modules shown are called and loaded into the processor 902 according to the program process. The processor 902 executes the corresponding instructions to complete various functions in the tooth position monitoring method and system.

[0118] Optionally, the memory 901 includes at least one type of readable storage medium, which includes a flash memory, a hard disk, a multimedia card, a card-type memory (e.g., an SD or DX memory, etc.), a magnetic memory, a magnetic disk, an optical disk, etc. The memory 901 can be an internal storage unit of a computer device, such as a hard disk of a computer device, or an external storage device of a computer device, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the computer device. Further, the memory 901 can also include both an internal storage unit of a computer device and an external storage device. The memory 901 can be used not only to store application software and various types of data installed in the computer device, but also to temporarily store data that has been output or is to be output.

[0119] Optionally, the processor 902 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), a controller, a microcontroller, a microprocessor or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, digital processing chips.

[0120] Optionally, the tooth position change monitoring terminal includes a display, which provides a visual interface and can display the data stored in the memory, the intermediate process of the processor processing the data, or the processing results.

[0121] Optionally, the tooth position change monitoring terminal can be connected to a cloud server to obtain data such as the user's intraoral teeth images at any time.

[0122] In summary, the tooth position monitoring method, system and terminal based on intraoral tooth images of the present invention use a generative artificial intelligence algorithm to establish a mapping relationship model between intraoral tooth images and dental medical images, so as to realize the generation of radioactive medical images that require professional personnel and equipment to be collected by widely used, easily accessible, simple and convenient, low-cost, and non-radioactive intraoral tooth images, thereby improving the safety of patients, reducing medical costs, and reducing the pressure on the imaging department; based on real and generated medical images at multiple time points, an intraoral tooth change relationship model is established to achieve real-time monitoring and intensive monitoring of tooth positions, which helps doctors and patients to more accurately and comprehensively grasp the state and changes of teeth, thereby helping clinicians to efficiently determine treatment plans and quantitative evaluations of treatment effects, promoting the development of individualized precision diagnosis and treatment and telemedicine, and to a certain extent alleviating the current situation of severe shortage of oral medical resources. Therefore, the present invention effectively overcomes the various shortcomings of the prior art and has a high industrial utilization value.

[0123] The above embodiments are merely illustrative of the principles and effects of the present invention, and are not intended to limit the present invention. Anyone familiar with the technology may modify or change the above embodiments without violating the spirit and scope of the present invention. Therefore, all equivalent modifications or changes made by a person of ordinary skill in the art without departing from the spirit and technical ideas disclosed by the present invention shall still be covered by the claims of the present invention.

Claims

1. A tooth position monitoring method, characterized in that: include: Obtaining a dental image of the subject, wherein the dental image is from a photo or a video; Generate a simulated medical image based on the intraoral tooth image of the subject based on a mapping relationship model between the intraoral tooth image and the oral medical image, wherein the oral medical image is derived from CT image data or optical scanning image data; The tooth positions are detected based on the simulated medical image.

2. The tooth position monitoring method according to claim 1, characterized in that: The method comprises: Based on oral medical images and simulated medical images at multiple time points, a progression relationship model of the subjects' tooth positions was established to quantify and monitor changes in tooth positions over time.

3. The tooth position monitoring method according to claim 1, characterized in that: The mapping relationship model adopts a two-stage iterative learning strategy, in which the first stage uses all intraoral tooth images and oral medical images to learn the coarse mapping relationship between the two data domains, and the second stage uses the intraoral tooth images and oral medical images paired one by one with the subjects to iteratively optimize the model parameters of the first stage, thereby learning the individualized fine mapping relationship between the two data domains.

4. The tooth position monitoring method according to claim 1, characterized in that: The adversarial network model is used to establish the mapping relationship model, and the loss function of the model is: L GAN (G,D)=E[log D(I med )]+E[log(1-D(G(I photo ,I aux ))] L1(G)=E[||I med -G(I photo ,I aux )||1] Among them, I photo ,I med and I aux They respectively represent intraoral tooth pictures, oral medical images and preset auxiliary information.

5. The tooth position monitoring method according to claim 2, characterized in that: The indicator information of the change of the subject's tooth position includes the torsion angle, inclination angle and movement distance of the tooth.

6. A tooth position monitoring system, characterized in that: The tooth position monitoring system comprises: A data acquisition module, used to obtain the subject's intraoral teeth images at different viewing angles, wherein the intraoral teeth images are derived from photos or videos; A data preprocessing module, used for preprocessing the data collected by the data collection module; A data generation module generates simulated medical images based on the preprocessed intraoral dental images; The tooth position monitoring module locates the tooth position based on the simulated medical image and quantifies the change of the tooth position.

7. The tooth position monitoring system according to claim 6, characterized in that: The tooth position monitoring system further comprises: The data correction module is used to calculate and analyze the error between the simulated medical image and the real oral medical image data, and provide feedback to the data preprocessing module and the data generation module to update and optimize the algorithms and / or model parameters therein, wherein the real oral medical image can be obtained by the data acquisition module.

8. The tooth position monitoring system according to claim 6, characterized in that: The data in the data acquisition module can be obtained through one or more methods including the network, the cloud, Bluetooth, and an external memory.

9. The tooth position monitoring system according to claim 6, characterized in that: The preprocessing operation of the data preprocessing module includes obtaining the shape, position, three-dimensional geometric features, and texture feature information of the teeth.

10. The tooth position monitoring system according to claim 6, characterized in that: The tooth position monitoring module outputs abnormal reminders and diagnosis and treatment suggestions based on the quantified and visualized changes in tooth position.

Images