Automatic dental caries grading method and system based on panoramic radiographs

Abstract

The present invention belongs to the technical field of intelligent diagnosis of oral images, and specifically relates to a method and system for automatic grading of caries based on panoramic films. The method includes S1, segmenting and cropping tooth instances in the panoramic film to obtain instance segmentation samples of a single tooth in the panoramic film; S2, using a trained tooth rough classification model to segment the obtained tooth instance samples, and classify them into restoration state and non-restoration state; S3, subjecting the non-restoration tooth samples to a trained tooth tissue semantic segmentation model to obtain several different regional tissues of the segmented teeth; S4, according to the obtained tissue conditions of different regions of each tooth, using a caries grading model and according to the caries grade diagnosis standard, the four grades of health, shallow caries, medium caries, and deep caries are divided. The present invention has the characteristics of being able to realize the classification of tooth health status and automatic grading of caries and reducing the workload of clinicians.

Description

Technical Field

[0001] The present invention belongs to the technical field of intelligent diagnosis of oral images, and in particular relates to a method and system for automatic grading of caries based on panoramic films. Background Art

[0002] Oral health is an important part of overall health and an important indicator of residents' physical and mental health and civilization level. In addition to causing tooth pain and directly affecting physiological functions such as chewing food and pronunciation, oral diseases are also closely related to other systemic diseases, such as stroke, heart disease, diabetes, and digestive system. Digital medical imaging technologies in the oral field include periapical films, panoramic X-rays, optical coherence tomography, cone beam computed tomography, and oral 3D scanning. Caries is a progressive lesion caused by demineralization of hard tooth tissues. It is one of the three major oral diseases recognized by the World Health Organization (WHO) with a high incidence rate and a wide age coverage. The clinical diagnosis of caries mainly relies on visual examination and probe methods, which are highly subjective. The structure of teeth is mainly composed of enamel, dentin, and pulp cavity. According to the differences in the enamel, dentin, and pulp cavity of the demineralized lesion area invading the tooth tissue, it can be divided into four different levels: deep caries, medium caries, and shallow caries. Different treatment methods are required for caries at different stages. Therefore, the diagnosis of caries grade is an important clinical content.

[0003] Panoramic film is a simple and fast technology that uses the principle of narrow-slit and circular orbital tomography. Through one imaging, it can completely and clearly display the entire maxillary and mandibular bones, the upper and lower dentition, and the alveolar bone on one film. It is widely used in clinical practice due to its simple operation, wide inspection range and low radiation dose. Curved-surface tomography can display images of anatomical structures such as the full mouth of teeth, jaws, nasal cavity, maxillary sinus and temporomandibular joints in one exposure. It has a wide display range and is often used to symmetrically observe the morphological structure of the jaw, jaw lesions, the development of the jaw and teeth, and the degree of alveolar bone absorption in full-mouth periodontal disease. It is an important auxiliary examination method in clinical oral examination.

[0004] Since panoramic films are widely available in various grassroots community hospitals, the large-scale, wide-angle, and high-throughput imaging results of panoramic films can easily diagnose the health status of all teeth in the mouth, especially for different degrees of caries lesions, which can be judged based on the distribution of grayscale levels in the panoramic films, making it possible for the community to establish oral health records for all teeth. At the same time, due to the complex structure of oral tissues, including hard tissues such as teeth and jaws, as well as soft tissues such as muscles, and the interlaced spatial arrangement between various parts, the X-rays are superimposed on each other after transmission during the panoramic film imaging process, resulting in low contrast. Traditionally, the diagnosis of dental health status mainly relies on the visual examination of oral clinicians. With the deep integration and extensive development of artificial intelligence technology and medical imaging technology, it is expected to obtain automatic diagnosis results by using the self-learning ability of neural networks and high-quality image annotation results, assist in giving diagnosis results, and improve the intelligence of clinical decision-making.

[0005] Therefore, it is necessary to design an automatic dental caries grading method and system based on panoramic films.

[0006] For example, the Chinese patent document with application number CN202110159089.6 describes an artificial intelligence-based caries diagnosis system, which consists of an image acquisition module, a data transmission module, an intelligent diagnosis module, and a diagnosis result display module. The system uses the stored caries case big data, cooperates with artificial intelligence and cloud computers to train an artificial intelligence diagnosis model for caries, obtains the user's oral tooth image information by shooting with a mobile phone or tablet camera, uses the user client and flask as data transmission media, processes the image and diagnoses caries through the trained model, and displays the diagnosis results in the user interface in the form of images in the user client. Although it is easy to operate and has low cost, it can effectively help caries patients to detect and seek medical treatment early, but its disadvantage is that it cannot provide a visual diagnosis process for clinicians, cannot assist dentists in clinical diagnosis, and increases the workload of clinicians. Summary of the invention

[0007] The present invention aims to overcome the problem in the prior art that the existing clinical diagnosis of dental caries mainly relies on visual examination and probe, which is highly subjective and cannot realize the classification of dental health status and automatic grading of dental caries. The present invention provides a dental caries automatic grading method and system based on panoramic films, which can realize the classification of dental health status and automatic grading of dental caries and reduce the workload of clinical doctors.

[0008] In order to achieve the above-mentioned object of the invention, the present invention adopts the following technical solutions:

[0009] The method for automatic dental caries grading based on panoramic films comprises the following steps:

[0010] S1, segment and crop the tooth instances in the panoramic image to obtain instance segmentation samples of a single tooth in the panoramic image;

[0011] S2, using the trained tooth rough classification model, segmenting the tooth instance samples obtained in step S1 and classifying the restored state and the non-restored state;

[0012] S3, subjecting the unrestored tooth sample in step S2 to a trained tooth tissue semantic segmentation model to obtain a plurality of segmented tooth tissues in different regions;

[0013] S4, according to the tissue conditions of different areas of each tooth obtained in step S3, the dental caries grading model is used to classify the dental caries into four grades: healthy, shallow caries, moderate caries, and deep caries according to the dental caries grade diagnosis standard.

[0014] Preferably, step S1 comprises the following steps:

[0015] S11, manually annotating tooth instances in the collected panoramic data set;

[0016] S12, uses the Mask R-CNN instance segmentation model to train on a manually annotated panoramic dataset;

[0017] S13, using the trained Mask R-CNN instance segmentation model to perform tooth instance segmentation on the tooth instances in the panoramic image to obtain the external contour of each tooth;

[0018] S14, calculating the maximum rectangular bounding box for the obtained tooth outer contour;

[0019] S15, expanding the boundary of the largest rectangular bounding box outward along the major and minor axes, and cropping it to obtain an instance segmentation sample of a single tooth in the panoramic image.

[0020] Preferably, step S2 further comprises the following steps:

[0021] S21, design a lightweight convolutional neural network framework to classify the results into repaired and non-repaired states;

[0022] S22, labeling the instance segmentation samples of each tooth obtained in step S1, and dividing them into a restored state and a non-restored state;

[0023] S23, training a neural network model using the labeled instance segmentation samples of each tooth to obtain a rough tooth classification model;

[0024] S24, using the trained tooth rough classification model to classify the tooth instance segmentation samples into two categories: restoration state and non-restoration state.

[0025] Preferably, step S3 comprises the following steps:

[0026] S31, annotating the non-restoration tooth samples distinguished by the rough tooth classification model to obtain the enamel, dentin, pulp cavity and caries area of ​​each tooth sample;

[0027] S32, using the tooth sample annotated in step S31 to train a U-Net deep learning segmentation model to obtain a trained tooth tissue semantic segmentation model;

[0028] S33, use the trained tooth tissue semantic segmentation model to perform tissue segmentation on the tooth sample to obtain the enamel, dentin, pulp cavity and caries area of ​​the tooth sample.

[0029] Preferably, step S4 comprises the following steps:

[0030] S41, for each tooth instance segmentation sample of the panoramic film, the second-order moment is calculated according to the obtained tooth external contour, the main axis direction of the tooth is determined, and the coordinate system of each tooth is established with the center of gravity of the tooth contour as the origin and the main and secondary axes as the coordinate axes;

[0031] S42, taking the center of gravity of the dental pulp cavity area as the origin, taking the radius of the outer circle of the tooth contour as the radius, and generating a scanning line starting from the origin in the range of (0, π) with an angle interval of Δθ;

[0032] S43, traversing the tooth tissue region to which each point belongs on each scanning line, combining the results of the three regions of enamel, dentin, and pulp cavity obtained by the tooth tissue semantic segmentation model, and obtaining the inner and outer boundaries of dentin; wherein the inner and outer boundaries of dentin include the enamel-dentin boundary line and the boundary line of the dentin pulp cavity;

[0033] S44, based on the obtained inner and outer boundary lines of dentin, the unit circle rolling tracking method is used to extract the center line of 1 / 2 dentin;

[0034] S45, with the boundary curves of the outer contour of enamel, the boundary line between enamel and dentin, the center line of 1 / 2 dentin, and the outer contour line of the pulp cavity as edges, the first and last endpoints of the outer contour line of the pulp cavity are connected to form closed areas such as enamel, outer 1 / 2 dentin, and inner 1 / 2 dentin; the enamel area is defined as the shallow caries decision area, the outer 1 / 2 dentin is defined as the medium caries decision area, and the inner 1 / 2 dentin is defined as the deep caries decision area;

[0035] S46, spatially AND the caries area obtained by the tooth tissue semantic segmentation model with the deep caries decision area, medium caries decision area, and shallow caries decision area obtained in step S45 to determine whether they overlap with the caries area and determine the diagnosis result of the tooth caries grade.

[0036] Preferably, step S43 includes the following steps:

[0037] If the areas before and after the current traversal point on the scan line belong to different regions, the current traversal point is considered to be a boundary line.

[0038] Preferably, step S46 also includes the following steps:

[0039] If the sum of the operation is not 0, it is judged that there is overlap, and the corresponding level of dental caries grade diagnosis result is determined; if there is no overlap with the deep caries decision area, the medium caries decision area, and the shallow caries decision area, it is judged that the teeth are healthy.

[0040] The present invention also provides an automatic dental caries grading system based on panoramic films, comprising:

[0041] A tooth instance segmentation and cropping module is used to segment and crop the tooth instances in the panoramic image to obtain instance segmentation samples of a single tooth in the panoramic image;

[0042] The tooth rough classification module uses the trained tooth rough classification model to segment the obtained tooth instance samples and classify them into restoration state and non-restoration state;

[0043] The tooth tissue semantic segmentation module is used to segment the unrestored tooth samples using the trained tooth tissue semantic segmentation model to obtain several different regional tissues of the segmented teeth.

[0044] The caries grading module divides the tissue conditions of different areas of each tooth into four levels: healthy, shallow caries, moderate caries, and deep caries according to the caries grading model and the caries grading diagnostic standards.

[0045] Compared with the prior art, the present invention has the following advantages: (1) the present invention can realize the classification of dental health status and automatic grading of caries; (2) the present invention provides a visual diagnostic process for clinical dentists, which can assist dentists in clinical diagnosis and at the same time, help promote the establishment and management of residents' oral health records. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 A flow chart of the automatic caries grading method based on panoramic films of the present invention;

[0047] Figure 2 A schematic diagram of the panoramic data annotation result in the method of the present invention;

[0048] Figure 3 A schematic diagram of a Mask R-CNN network framework in the method of the present invention;

[0049] Figure 4 A schematic diagram of the result of the segmentation of a tooth instance of a panoramic film in the method of the present invention;

[0050] Figure 5 A schematic diagram of the cutting result of a single tooth in the method of the present invention;

[0051] Figure 6 A schematic diagram of a tooth sample in a restored state in the method of the present invention;

[0052] Figure 7 A schematic diagram of the tooth tissue region labeling result in the method of the present invention;

[0053] Figure 8 A schematic diagram of a framework of the U-Net semantic segmentation model in the method of the present invention;

[0054] Fig. 9 A schematic diagram of a coordinate system and representative scanning lines for cutting tooth samples in the method of the present invention;

[0055] Fig.10 A schematic diagram of points on the inner and outer boundary contour lines of dentin extracted by the method of the present invention;

[0056] Fig.11 A schematic diagram of the extraction result of the dentin 1 / 2 center line in the method of the present invention;

[0057] Fig.12 The figure is a process schematic diagram of the practical application of the automatic caries grading method based on panoramic films of the present invention. DETAILED DESCRIPTION

[0058] In order to more clearly illustrate the embodiments of the present invention, the specific implementation methods of the present invention will be described below with reference to the accompanying drawings. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those of ordinary skill in the art, other accompanying drawings and other implementation methods can be obtained based on these accompanying drawings without creative work.

[0059] Embodiment 1:

[0060] like Figure 1 As shown in the figure, the present invention uses deep learning technology to obtain instance segmentation of panoramic teeth and tissue segmentation results of each tooth. On this basis, the automatic classification of caries is achieved by using the overlap between carious areas and different areas of healthy tooth tissue. The specific process is as follows Figure 1 shown.

[0061] The method for automatic dental caries grading based on panoramic films comprises the following steps:

[0062] S1, segment and crop the tooth instances in the panoramic image to obtain instance segmentation samples of a single tooth in the panoramic image;

[0063] S2, using the trained tooth rough classification model, segmenting the tooth instance samples obtained in step S1 and classifying the restored state and the non-restored state;

[0064] S3, subjecting the unrestored tooth sample in step S2 to a trained tooth tissue semantic segmentation model to obtain a plurality of segmented tooth tissues in different regions;

[0065] S4, according to the tissue conditions of different areas of each tooth obtained in step S3, the dental caries grading model is used to classify the dental caries into four grades: healthy, shallow caries, moderate caries, and deep caries according to the dental caries grade diagnosis standard.

[0066] Step S1 specifically includes the following steps:

[0067] Oral panoramic radiographs were collected and all the data were cleaned to remove bad samples such as those that were too dark or too bright, had poor imaging effects, were seriously missing teeth, and could not be distinguished by the human eye. Finally, 1,000 samples were obtained. Two doctors with rich clinical experience used VGG Image Annotator to annotate the teeth and obtain the external contour and health status of each tooth. The health status was classified into five categories: healthy, deep caries, shallow caries, medium caries, and restored. The annotation results are shown in the figure below. Figure 2 shown.

[0068] Mask R-CNN is an effective deep learning framework for instance segmentation. It introduces FPN (Feature Pyramid Networks) into the backbone network of Faster R-CNN, improves ROIPooling to ROI Align, and adds an FCN branch to achieve mask segmentation. Figure 3 As shown. It mainly includes the following parts: The model training uses the SGD (Stochastic Gradient Descent) optimizer, the initial learning rate is set to 0.025, the momentum parameter is set to 0.9, the regularization parameter of the loss function is 0.0001, and the learning rate adjustment strategy uses linear hot start and equal spacing attenuation. The training process uses four graphics cards, and the number of training images for each card is 2, that is, the batch size is 8. A total of 30 rounds of training are performed to continuously fit the network parameters.

[0069] Figure 4 The results of instance segmentation of panoramas using the trained Mask R-CNN instance segmentation model are given.

[0070] Based on the tooth contours obtained from each example, the maximum rectangular bounding box of the tooth's outer contour is calculated. Figure 4In order to crop each tooth sample, the boundary point coordinates of the largest rectangular bounding box of the contour are set as P i (x i ,y i ), with 25 pixels as the number of external extended pixels, the specific boundary calculation method is as follows:

[0071] x start =min(x i )-25i=0,1,...,n

[0072] y start =min(y i )-25i=0,1,...,n

[0073] w=max(x i )-min(x i )+50i=0,1,...,n

[0074] h=max(y i )-min(y i )+50i=0,1,...,n

[0075] Where (x start ,y start ) are the coordinates of the starting point of the straight-edged rectangular box, and w and h are the width and height of the rectangular box.

[0076] According to the above formula, the starting point x of the cropping box is calculated start ,y start , and the corresponding width w and height h, the cropping results of each tooth sample are obtained by cropping on the original panoramic film. Typical results are as follows Figure 5 shown.

[0077] Step S2 specifically includes the following steps:

[0078] For the sample classification problem of two categories of teeth, restored and non-restored, a lightweight convolutional neural network framework is used because the restored teeth have obvious highlight areas and obvious features. This framework has the advantages of clear structure, few parameters and low training complexity. The designed lightweight convolutional neural network has a total of 15 layers, including 9 convolutional layers, 4 maximum pooling layers and 2 fully connected layers. The final output layer is the result of 2 classifications. BN layer and ReLU activation function are performed in each convolutional layer. During training, the Dropout layer is inserted into the fully connected layer with a probability of 0.5 to reduce overfitting. The minimum cross entropy loss function is selected as the optimization goal. Table 1 below gives the kernel size, number, stride, padding and output of each layer.

[0079] Table 2 Lightweight convolutional neural network framework

[0080]

[0081]

[0082] The cut tooth samples are categorized and labeled. The root canal treatment, crown restoration and other tooth samples are labeled as restoration state, and other types of teeth, including healthy and caries samples, are labeled as non-restoration state. Figure 6 shown.

[0083] Using the labeled data set, the lightweight convolutional neural network shown in Table 1 is trained to obtain a two-classification model that distinguishes between repaired and non-repaired states.

[0084] The trained model is used for the segmented tooth samples to predict the category of the teeth in order to distinguish between restored and non-restored tooth samples.

[0085] Step S3 specifically includes the following steps:

[0086] A clinical dentist annotates the tooth tissue area of ​​a cut single tooth sample by drawing a closed outline along the boundaries of the enamel area, dentin area, pulp cavity area, and caries area, and annotating the four areas: enamel area, dentin area, pulp cavity area, and caries area. Figure 7 shown.

[0087] Based on the labeled data, the U-Net semantic segmentation model is used to segment different tissue regions of a single tooth. The U-Net structure is as follows: Figure 8 As shown in the figure, the structure uses four downsamplings to continuously obtain high-level semantic features, and then performs four upsamplings to restore the features to the resolution of the original image. For the two parts of downsampling and upsampling, a direct connection structure is used to fuse the corresponding feature maps of the same scale, so that the feature maps restored by upsampling incorporate many low-order features and multi-scale features. Features of different scales in medical images are useful for segmentation. By fusing different features, the information restored by upsampling is made more refined.

[0088] During the training process, the stochastic gradient descent optimizer was used for training. The initial learning rate was set to 0.01, the momentum parameter was set to 0.9, the regularization parameter of the loss function was 0.0004, and the batch size was 4. Cosine annealing hot restart was used as the learning rate adjustment strategy. The number of iterations for the first restart was set to 750, and the number of iterations between two hot restarts in SGDR (stochastic gradient descent with hot restart) was set to 2.

[0089] Step S4 specifically includes the following steps:

[0090] For the tooth instance segmentation of the panoramic film and the cropped tooth samples, the second-order moment is calculated according to the obtained tooth external contour, the main axis direction of the tooth is determined, and the covariance matrix is ​​constructed as follows:

[0091]

[0092]

[0093]

[0094]

[0095]

[0096] Where I(x,y) represents the intensity of the grayscale image pixel, Denotes the coordinates of the geometric center of the contour. The contour orientation angle is obtained from the angle of the eigenvector associated with the largest eigenvalue and the direction of the closest axis.

[0097] The specific calculation formula is as follows:

[0098]

[0099] After the main axis direction θ is determined, the direction of the minor axis can be determined according to the characteristic that the minor axis is perpendicular to it. A coordinate system is established with the center of gravity of the tooth contour as the origin and the main and minor axes as the coordinate axes. With the center of gravity of the pulp cavity area as the origin and the radius of the outer circle of the tooth contour as the radius, a scan line starting from the origin is generated along the range (0, π) at intervals of 1 degree. The scan lines of two angles are as follows Fig. 9 As shown, Fig. 9 (a) is the coordinate system established on the cropped tooth sample and the scan line in the 100 degree direction. Fig. 9 (b) and (c) show the intersection of the scan line and the inner and outer boundaries of dentin, respectively.

[0100] Extract each pixel point on the scan line at different angles, and assign the pixel point to the tooth tissue area segmentation result. That is, when the search point detects a jump from the dentin label to the enamel label, it is regarded as an outer boundary point. Similarly, the jump between the pulp cavity and the dentin is regarded as an inner boundary point. Save the results of the inner and outer boundary points and display them in the figure. The results are as follows: Fig.10 shown.

[0101] Connect the points on the extracted inner and outer boundary contour lines of dentin to obtain the inner and outer boundary contour lines of dentin respectively. Use the unit circle rolling tracking method to extract the 1 / 2 center line of dentin. In the specific implementation process, the radius of the search unit circle is set to 10 pixels, and the termination condition of a single search is that the difference between the closest distances from the point on the unit circle to the two boundaries is less than 2 pixels. The direction of the first search is set to the positive direction of the tooth axis, and then each updated search direction is the direction from the center of the previous circle to the center of the current circle. After setting the initial parameters, iterate continuously until the termination point is within the unit circle search radius area and then stop the search. Fig.11 The results of the midline extraction of the 1 / 2 dentin area are shown.

[0102] The caries area obtained by the semantic segmentation model of tooth tissue is spatially ANDed with the deep caries decision area, medium caries decision area, and shallow caries decision area obtained in the previous step to determine whether they overlap with the caries area. If the sum of the AND operation is not 0, it is determined to be overlapped, and the corresponding level of diagnosis is determined. If there is no overlap with the three decision areas, it is determined to be healthy. In this way, the classification of different caries levels such as healthy teeth, shallow caries, medium caries, and deep caries is achieved.

[0103] In summary, the automatic caries grading strategy described in the present invention mainly includes: on the basis of using the deep learning segmentation model to obtain the panoramic tooth instance segmentation and the tissue area segmentation of a single tooth, a coordinate system is established according to the tooth morphology, the inner and outer boundary search lines of the dentin are generated, the inner and outer boundary discrete points of the dentin are searched, the 1 / 2 midline of the dentin area is obtained, the caries level decision area is generated, and the caries level decision is made. The intermediate process results of each stage are shown in the figure below. Fig.12 shown.

[0104] The present invention also provides an automatic dental caries grading system based on panoramic films, comprising:

[0105] A tooth instance segmentation and cropping module is used to segment and crop the tooth instances in the panoramic image to obtain instance segmentation samples of a single tooth in the panoramic image;

[0106] The tooth rough classification module uses the trained tooth rough classification model to segment the obtained tooth instance samples and classify them into restoration state and non-restoration state;

[0107] The tooth tissue semantic segmentation module is used to segment the unrestored tooth samples using the trained tooth tissue semantic segmentation model to obtain several different regional tissues of the segmented teeth.

[0108] The caries grading module divides the tissue conditions of different areas of each tooth into four levels: healthy, shallow caries, moderate caries, and deep caries according to the caries grading model and the caries grading diagnostic standards.

[0109] The above four modules correspond to the four steps of the method respectively.

[0110] The present invention proposes an automatic caries grading technology combined with deep learning for panoramic films widely used in community hospitals. The panoramic film is input, and each tooth can be segmented using the deep learning instance segmentation model, and then each tooth is segmented to determine the lesion area, enamel, dentin, pulp cavity and other different areas. Considering that the restored teeth have obvious brightness differences, obvious features and are easy to distinguish, a two-stage hierarchical classification network is designed for the classification of teeth in five different states: healthy, deep caries, medium caries, shallow caries, and restored. First, a CNN model is constructed to distinguish between restored and non-restored states. The restored teeth with obvious brightness features are extracted. The remaining tooth samples are segmented to determine different tissue areas such as enamel, dentin, pulp cavity, and caries. Then the boundary lines of each area are determined to provide regional decision support for the automatic grading of caries.

[0111] Based on tooth tissue segmentation, the present invention uses the ray scanning intersection method to extract the boundary lines of each tooth tissue area, and realizes automatic caries classification by judging the intersection of the carious area and the various tooth tissue areas. Different from the dark box characteristics of the deep learning classification model technical solution, it provides a visual diagnostic process for clinicians, which can assist dentists in clinical diagnosis. In particular, the system of the present invention fills the gap in the field of caries classification of panoramic images, and has guiding significance for subsequent caries classification related work. The system and method of the present invention will greatly reduce the workload of clinicians, help promote the establishment of residents' oral health records, promote oral health intelligence and big data management, and have great significance for the high-quality development of the oral health service industry.

[0112] The above description is only a detailed description of the preferred embodiments and principles of the present invention. For ordinary technicians in this field, according to the ideas provided by the present invention, there will be changes in the specific implementation methods, and these changes should also be regarded as the protection scope of the present invention.

Claims

1. An automatic dental caries grading method based on panoramic films, characterized in that: The steps include: S1, segment and crop the tooth instances in the panoramic image to obtain instance segmentation samples of a single tooth in the panoramic image; S2, using the trained tooth rough classification model, segmenting the tooth instance samples obtained in step S1 and classifying the restored state and the non-restored state; S3, subjecting the unrestored tooth sample in step S2 to a trained tooth tissue semantic segmentation model to obtain a plurality of segmented tooth tissues in different regions; S4, according to the tissue conditions of different areas of each tooth obtained in step S3, the dental caries grading model is used to classify the dental caries into four grades: healthy, shallow caries, moderate caries, and deep caries according to the dental caries grade diagnosis standard; Step S2 includes the following steps: S21, design a lightweight convolutional neural network framework to classify the results into repaired and non-repaired states; S22, labeling the instance segmentation samples of each tooth obtained in step S1, and dividing them into a restored state and a non-restored state; S23, training a neural network model using the labeled instance segmentation samples of each tooth to obtain a rough tooth classification model; S24, using the trained tooth rough classification model to classify the tooth instance segmentation samples into two categories: restoration state and non-restoration state.

2. The method for automatic dental caries grading based on panoramic films according to claim 1, characterized in that ,Step S1 includes the following steps: S11, manually annotating tooth instances in the collected panoramic data set; S12, uses the Mask R-CNN instance segmentation model to train on a manually annotated panoramic dataset; S13, using the trained Mask R-CNN instance segmentation model to perform tooth instance segmentation on the tooth instances in the panoramic image to obtain the external contour of each tooth; S14, calculating the maximum rectangular bounding box for the obtained tooth outer contour; S15, expanding the boundary of the largest rectangular bounding box outward along the major and minor axes, and cropping it to obtain an instance segmentation sample of a single tooth in the panoramic image.

3. The method for automatic dental caries grading based on panoramic films according to claim 1, characterized in that: Step S3 includes the following steps: S31, annotating the non-restoration tooth samples distinguished by the rough tooth classification model to obtain the enamel, dentin, pulp cavity and caries area of ​​each tooth sample; S32, using the tooth sample annotated in step S31 to train a U-Net deep learning segmentation model to obtain a trained tooth tissue semantic segmentation model; S33, use the trained tooth tissue semantic segmentation model to perform tissue segmentation on the tooth sample to obtain the enamel, dentin, pulp cavity and caries area of ​​the tooth sample.

4. The method for automatic dental caries grading based on panoramic films according to claim 1, characterized in that: Step S4 includes the following steps: S41, for each tooth instance segmentation sample of the panoramic film, the second-order moment is calculated according to the obtained tooth external contour, the main axis direction of the tooth is determined, and the coordinate system of each tooth is established with the center of gravity of the tooth contour as the origin and the main and secondary axes as the coordinate axes; S42, taking the center of gravity of the dental pulp cavity area as the origin, taking the radius of the outer circle of the tooth contour as the radius, and generating a scanning line starting from the origin in the range of (0, π) with an angle interval of Δθ; S43, traversing the tooth tissue region to which each point belongs on each scanning line, combining the results of the three regions of enamel, dentin, and pulp cavity obtained by the tooth tissue semantic segmentation model, and obtaining the inner and outer boundaries of dentin; wherein the inner and outer boundaries of dentin include the enamel-dentin boundary line and the boundary line of the dentin pulp cavity; S44, based on the obtained inner and outer boundary lines of dentin, the unit circle rolling tracking method is used to extract the center line of 1 / 2 dentin; S45, with the boundary curves of the outer contour of enamel, the boundary line between enamel and dentin, the center line of 1 / 2 dentin, and the outer contour line of the pulp cavity as edges, the first and last endpoints of the outer contour line of the pulp cavity are connected to form closed areas such as enamel, outer 1 / 2 dentin, and inner 1 / 2 dentin; the enamel area is defined as the shallow caries decision area, the outer 1 / 2 dentin is defined as the medium caries decision area, and the inner 1 / 2 dentin is defined as the deep caries decision area; S46, spatially AND the caries area obtained by the tooth tissue semantic segmentation model with the deep caries decision area, medium caries decision area, and shallow caries decision area obtained in step S45 to determine whether they overlap with the caries area and determine the diagnosis result of the tooth caries grade.

5. The method for automatic dental caries grading based on panoramic films according to claim 4, characterized in that: Step S43 includes the following steps: If the areas before and after the current traversal point on the scan line belong to different regions, the current traversal point is considered to be a boundary line.

6. The method for automatic dental caries grading based on panoramic films according to claim 4, characterized in that: Step S46 also includes the following steps: If the sum of the operation is not 0, it is judged that there is overlap, and the corresponding level of dental caries grade diagnosis result is determined; if there is no overlap with the deep caries decision area, the medium caries decision area, and the shallow caries decision area, it is judged that the teeth are healthy.

7. Automatic dental caries grading system based on panoramic films, characterized by: include: A tooth instance segmentation and cropping module is used to segment and crop the tooth instances in the panoramic image to obtain instance segmentation samples of a single tooth in the panoramic image; The tooth rough classification module uses the trained tooth rough classification model to segment the obtained tooth instance samples and classify them into restoration state and non-restoration state; The tooth tissue semantic segmentation module is used to segment the unrestored tooth samples using the trained tooth tissue semantic segmentation model to obtain several different regional tissues of the segmented teeth. The dental caries grading module is based on the dental caries grading model and dental caries grading criteria, which are divided into four grades: healthy, shallow caries, moderate caries, and deep caries. The tooth rough classification module is as follows: Design a lightweight convolutional neural network framework to classify the results into repaired and non-repaired states; Label the instance segmentation samples of each tooth obtained and divide them into restoration state and non-restoration state; The neural network model is trained using the labeled instance segmentation samples of each tooth to obtain a rough tooth classification model; The trained tooth rough classification model is used to classify the tooth instance segmentation samples into two categories: restoration state and non-restoration state.

Images