Deep learning-based dental trauma imaging diagnosis system and method

Abstract

The present invention provides a dental trauma imaging diagnosis system and method based on deep learning, belonging to the field of dental diagnosis and treatment. The method includes: performing data augmentation on a number of collected first CBCT image data to obtain training data with a higher degree of style diversification, and using the training data to train a segmentation model and a classification model based on Swin Transformer; using the trained segmentation model to perform dental image segmentation processing on the second CBCT image data of the target patient to obtain multiple single-tooth images, and extracting the single-tooth features and common features of each tooth from each single-tooth image; using the classification model to classify the single-tooth features to obtain a number of classification results, and deleting some of the classification results in each classification result based on the common features; generating corresponding treatment plans based on the remaining classification results after deletion. The present invention realizes the accurate diagnosis of dental trauma and can assist doctors in improving the diagnosis and treatment efficiency.

Description

Technical Field

[0001] The present invention relates to the field of dental diagnosis and treatment, and more particularly, to a dental trauma imaging diagnosis system and method based on deep learning. Background Art

[0002] Dental trauma is an acute injury that commonly occurs in adolescents. However, due to its complex classification and inconsistent treatment standards, it is prone to poor prognosis and may cause permanent physical and mental damage to children. Therefore, designing an intelligent diagnosis scheme for dental trauma based on a deep learning model will have broad application prospects.

[0003] However, the existing intelligent diagnosis schemes for dental trauma based on deep learning models still have the following several main problems: (1) Deep learning models rely heavily on large-scale and high-quality annotated datasets. However, due to privacy issues and annotation costs, standard datasets with large scales, diverse styles, and detailed annotations are generally insufficient, limiting the accuracy and robustness of deep learning models; (2) Existing dental image segmentation methods still have problems with unclear segmentation regions and are difficult to refine the blurred boundaries between adjacent teeth; (3) The accuracy of dental trauma classification still needs to be improved.

[0004] Therefore, designing an improved intelligent diagnosis scheme for dental trauma based on a deep learning model to solve the above technical problems is an urgent technical problem to be solved currently. Summary of the Invention

[0005] In view of the above technical problems, the present invention provides a dental trauma imaging diagnosis method, system, electronic device, computer storage medium, and computer program product based on deep learning.

[0006] The present invention discloses a dental trauma imaging diagnosis method based on deep learning, and the method includes the following steps:

[0007] Performing data augmentation on a number of collected first CBCT image data to obtain training data with a higher degree of style diversity, and using the training data to train a segmentation model and a classification model based on Swin Transformer;

[0008] Using the trained segmentation model to perform dental image segmentation processing on the second CBCT image data of a target patient to obtain multiple single-tooth images, and extracting the single-tooth features and common features of each tooth from each of the single-tooth images;

[0009] Using the classification model to classify the single-tooth features to obtain a number of classification results, and deleting some of the classification results in each of the classification results based on the common features;

[0010] Generate corresponding treatment plans based on each of the remaining classification results after deletion.

[0011] Further, perform data augmentation on a number of collected first CBCT image data to obtain training data with a higher degree of style diversity, including:

[0012] Perform Fourier transform processing on the original first CBCT image data to obtain frequency-domain image data;

[0013] Select noise data from other original first CBCT image data and convert it to the frequency domain to obtain frequency-domain noise data, and superimpose the frequency-domain noise data on the frequency-domain image data to obtain new frequency-domain image data;

[0014] Perform inverse Fourier transform on each new frequency-domain image data to obtain multiple third CBCT image data, and construct multiple training data based on each of the first CBCT image data, the third CBCT image data, and the corresponding class labels.

[0015] Further, superimpose the frequency-domain noise data on the frequency-domain image data to obtain new frequency-domain image data, including:

[0016] Determine the low-frequency part and high-frequency part of the frequency-domain image data, and for the low-frequency part and high-frequency part, weaken the frequency-domain noise data using a first addition ratio and a second addition ratio respectively; wherein, the first addition ratio is less than the second addition ratio;

[0017] Superimpose the weakened frequency-domain noise data on the frequency-domain image data to obtain new frequency-domain image data.

[0018] Further, before superimposing the frequency-domain noise data on the frequency-domain image data, the method further includes:

[0019] Determine the overall similarity between the first CBCT image data and other first CBCT image data, and match the reinforcement quantity according to the overall similarity; wherein, the reinforcement quantity is positively correlated with the overall similarity;

[0020] Generate multiple groups of addition ratios corresponding to the reinforcement quantity for the first CBCT image data, and each group of addition ratios includes a first addition ratio and a second addition ratio.

[0021] Further, use the classification model to classify the monomer features to obtain a number of classification results, and delete some of the classification results in each of the classification results based on the common features, including:

[0022] Classify the monomer features using a classification model to obtain several first classification results;

[0023] Extract the position and attitude information of the common features of each tooth from the second CBCT image data, evaluate the consistency of each position and attitude information. If the consistency evaluation result is higher than the preset determination condition, it is determined that the common feature is non-dental trauma;

[0024] Determine the first classification results belonging to non-dental trauma as the second classification results, and delete the second classification results in each of the first classification results.

[0025] The present invention also discloses a dental trauma image diagnosis system based on deep learning. The system includes a training module, a segmentation module, a classification module, and an output module;

[0026] The training module is used to perform data augmentation on a number of collected first CBCT image data to obtain training data with a higher degree of style diversification, and use the training data to train a segmentation model and a classification model based on Swin Transformer;

[0027] The segmentation module is used to perform tooth image segmentation processing on the second CBCT image data of the target patient using the trained segmentation model to obtain multiple monomer tooth images, and extract the monomer features and common features of each tooth from each of the monomer tooth images;

[0028] The classification module is used to classify the monomer features using a classification model to obtain several classification results, and delete some of the classification results in each of the classification results based on the common features;

[0029] The output module is used to generate corresponding treatment plans based on the remaining classification results after deletion.

[0030] Further, the training module is used for:

[0031] Perform Fourier transform processing on the original first CBCT image data to obtain frequency-domain image data;

[0032] Select noise data from other original first CBCT image data and convert it to the frequency domain to obtain frequency-domain noise data, and superimpose the frequency-domain noise data on the frequency-domain image data to obtain new frequency-domain image data;

[0033] Perform inverse Fourier transform on each new frequency-domain image data to obtain multiple third CBCT image data, and construct multiple training data based on each of the first CBCT image data, the third CBCT image data, and the corresponding class labels.

[0034] The present invention also discloses an electronic device, including one or more processors, a memory, and one or more computer programs. Among them, the one or more computer programs are stored in the memory, and the one or more computer programs include instructions that, when run by the processor, implement the deep learning-based dental trauma image diagnosis method described in any one of the above.

[0035] The present invention also discloses a computer storage medium, including computer instructions that can run on an electronic device. When the computer instructions are run, they implement the deep learning-based dental trauma image diagnosis method described in any one of the above.

[0036] The present invention also discloses a computer program product that can run on an electronic device to implement the deep learning-based dental trauma image diagnosis method described in any one of the above.

[0037] The beneficial effects of the present invention are at least as follows:

[0038] The solution of the present invention solves the problems of insufficient training data and low diversity of the segmentation model through data augmentation technology, enabling the trained segmentation model to adapt to actual situations such as blurred boundaries between adjacent teeth, with higher segmentation accuracy and robustness.

[0039] The solution of the present invention can also accurately classify the types of dental trauma based on the individual and common features of individual teeth, thereby improving the accuracy of generating subsequent treatment plans and assisting in improving the decision-making efficiency of clinicians in the diagnosis, treatment, and prognosis assessment of dental trauma. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as limiting the scope. For those of ordinary skill in the art, other related drawings can be obtained based on these drawings without creative efforts.

[0041] Figure 1 is a schematic diagram of the main process of a deep learning-based dental trauma image diagnosis method disclosed in an embodiment of the present invention;

[0042] Figure 2 is a schematic diagram of the process of data augmentation for CBCT image data disclosed in an embodiment of the present invention;

[0043] Figure 3 is a schematic diagram of the scenario of a deep learning-based dental trauma image diagnosis system disclosed in an embodiment of the present invention;

[0044] Figure 4 It is a schematic structural diagram of a dental trauma image diagnosis system based on deep learning disclosed in an embodiment of the present invention. Detailed implementation manners

[0045] It should be noted that, without conflict, the embodiments in the present application and the features in the embodiments may be combined with each other. The present invention will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0046] In order to enable those skilled in the art to better understand the solution of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0047] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and do not have to be used to describe a specific order or sequence. It should be understood that such data can be interchanged under appropriate circumstances so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "comprising" and "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device comprising a series of steps or units does not have to be limited to those steps or units clearly listed, but may include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0048] As Figure 1 shown, in response to the above technical problems, an embodiment of the present invention discloses a dental trauma image diagnosis method based on deep learning, and the method includes the following steps:

[0049] S10. Perform data augmentation on a plurality of collected first CBCT image data to obtain training data with a higher degree of style diversification, and use the training data to train a segmentation model and a classification model based on Swin Transformer.

[0050] The present invention uses a segmentation model based on Swin Transformer to segment the images of each tooth. Before that, it is necessary to first construct training data to fully train the segmentation model. As described in the background art, due to privacy issues and annotation costs, there is a general shortage of large-scale standard datasets with diverse styles and detailed annotations, which limits the accuracy and robustness of deep learning models. In response to this, the present invention uses dental image data augmentation technology to obtain sufficient and more diverse training data. Specifically as follows:

[0051] Collect a number of first CBCT image data. CBCT images, that is, cone beam CT images, can clearly show the three-dimensional structure of teeth and their surrounding tissues. For example, collect the CBCT images of 100 adolescent patients with dental trauma, and these images cover different types of dental trauma situations, such as tooth fracture, dislocation, tooth surface scratches, etc.

[0052] Then perform data augmentation. Data augmentation is to make the data more rich and diverse by performing some transformation operations on the original data. Conventional augmentation methods are, for example, rotating the collected CBCT images, rotating the images clockwise or counterclockwise by a certain angle, such as 15 degrees, 30 degrees, etc.; scaling operations can also be performed, magnifying or reducing the images by a certain proportion, such as reducing the image to 80% of the original or magnifying it to 120%; and brightness adjustment, brightening or dimming the images, etc. Through these operations, the original 100 image data can become 300 or more, which is beneficial to constructing a sufficient amount of training data with a higher degree of style diversification.

[0053] Finally, use the CBCT image data obtained after data augmentation to construct a sufficient amount of the first set of training data (the training data includes the CBCT image data and the marked boundary marks between teeth, and this boundary mark is used as a label), and use it to train the segmentation model based on Swin Transformer. Swin Transformer is a model that performs well in the field of image segmentation. Input various tooth image data after data augmentation into the segmentation model, and let it continuously learn how to distinguish and identify the shape and boundary of teeth, etc., so as to accurately segment tooth images in the future. Among them, the classification model also needs to be pre-trained, and the training data it uses is also constructed based on the CBCT image data obtained after data augmentation, which will be explained in detail later.

[0054] It should be noted that the label can be marked in the CBCT image data (or the segmented CBCT sub-image data) in the form of a pixel-level pseudo-mask.

[0055] S20. Use the trained segmentation model to perform dental image segmentation on the second CBCT image data of the target patient, obtain multiple single-tooth images, and extract the individual features and common features of each tooth from each of the single-tooth images.

[0056] Receive the second CBCT image data taken of the target patient, and use the segmentation model trained in the aforementioned step S20 to perform dental image segmentation on it, so that each tooth in the CBCT image of the entire oral cavity is separately segmented, obtaining multiple single-tooth images. For example, if there are 32 teeth (including wisdom teeth) in the oral cavity, 32 separate tooth images will be obtained after segmentation, and each image only contains one tooth.

[0057] Extract the individual features and common features of each tooth from each single-tooth image. The individual features are all the extractable features of each tooth. For example, a certain front tooth may have an obvious fracture mark. The common features refer to some general features that all (or most) teeth have, such as structural features like enamel and dentin, and surface features of teeth. These features are extracted from these single-tooth images by using methods such as convolutional networks for subsequent analysis.

[0058] S30. Use the classification model to classify the individual features, obtain several classification results, and delete some of the classification results in each of the classification results based on the common features.

[0059] The present invention also pre-constructs and trains a classification model (preferably based on a neural network algorithm). Using the classification model can achieve the classification of individual features, that is, determine the possible trauma types of each tooth. For example, according to the individual features such as the shape and size of the fracture mark of a front tooth, it is judged whether this front tooth belongs to different dental trauma classifications such as crown fracture, root fracture, or combined crown and root fracture, and conventional surface scratches. The recognition features corresponding to each dental trauma classification are preset, and the classification results obtained by the classification model may include multiple ones. These classification results are all classification results above the confidence threshold. That is, the classification model may not be able to uniquely determine the trauma type to which a certain tooth belongs, but outputs all the classification results above the confidence threshold.

[0060] Since there may be incorrect situations such as misidentification in the above multiple classification results. For example, the classification result of a tooth shows that the enamel part presents an abnormal shape, which is very different from the common features of the enamel of other normal teeth and injured teeth. If it is determined that this classification result is incorrect through judgment based on the common features, it will be deleted from all the classification results to ensure that the remaining classification results are more reliable.

[0061] It should be noted that the classification model also needs to be pre-trained, and the second set of training data used for training is also constructed based on the CBCT image data obtained in step S10. The difference is that the second set of training data contains monomer images of multiple teeth and their labels, and the labels are types of dental trauma.

[0062] S40. Based on each of the remaining classification results after deletion, generate corresponding treatment plans.

[0063] Based on each of the remaining classification results after deletion, automatically generate corresponding treatment plans. For example, if the classification result is that a target patient has one tooth with a crown fracture and another tooth with a dislocation. Then for the tooth with a crown fracture, if the fracture is not serious, filling and restoration treatment can be carried out; for the dislocated tooth, reduction and fixation treatment can be carried out, etc.

[0064] This process can also be implemented through a treatment plan automatic generation model. The treatment plan automatic generation model can learn the standard diagnosis and treatment methods for different types of dental trauma, and combine the results of evidence-based medicine to conduct prognostic evaluation, and finally realize the intelligent generation of treatment plans. The treatment plan automatic generation model can be constructed using conventional classification algorithms (such as SVM, random forest, etc.), or can be constructed based on deep learning algorithms. The present invention does not make specific limitations.

[0065] The solution of the present invention solves the problems of insufficient training data and low diversification degree of the segmentation model through data augmentation technology, so that the trained segmentation model can adapt to actual situations such as blurred boundaries between adjacent teeth, and has higher segmentation accuracy and robustness. At the same time, the solution of the present invention can also accurately classify the types of dental trauma based on the monomeric features and common features of monomer teeth, thereby improving the accuracy of generating subsequent treatment plans, and can assist in improving the decision-making efficiency of clinicians in the diagnosis, treatment, and prognostic evaluation of dental trauma.

[0066] As Figure 3 shown, after segmenting and classifying the dental image data, output the corresponding dental trauma classification results, treatment plans, etc. to the human-computer interaction platform for doctors to refer to. The output results can also include the positional relationship between the traumatized tooth and the alveolar socket, or directly mark the traumatized tooth and its classification results on the CBCT image data. No specific limitations are made in this regard.

[0067] Optionally, in order to alleviate problems such as insufficient dental image data volume and single style, the present invention adopts a data augmentation technology based on Fourier transform, adding noise from other data samples to the training samples to generate new data with different styles, which helps the model learn high-level semantic information from diverse data distributions and improves the model's versatility and clinical applicability. Refer to Figure 2 , specifically as follows:

[0068] Data augmentation is performed on a number of collected first CBCT image data to obtain training data with a higher degree of style diversity, including:

[0069] S11. Perform Fourier transform processing on the original first CBCT image data to obtain frequency-domain image data.

[0070] First, perform Fourier transform on the original first CBCT image data (such as images of teeth and surrounding tissues), which is to convert the image from the representation form of spatial pixels to the representation form of frequency components. Through Fourier transform, the image can be decomposed into a combination of sine and cosine waves of different frequencies, and each frequency component carries information about different aspects of the image. The low-frequency part reflects the general outline of the image, and the high-frequency part reflects the detailed features of the image.

[0071] CBCT images are essentially two-dimensional images. When performing Fourier transform on them, two-dimensional discrete Fourier transform (2D-DFT) is usually used. Let the CBCT image data be , where ; ; and are the number of rows and columns of the CBCT image data respectively. The formula for its two-dimensional discrete Fourier transform is:

[0072] ;

[0073] In the formula, is the frequency-domain image data after transformation, , , is the imaginary unit .

[0074] In actual calculation, starting from the top-left pixel of the image, for each pixel point , multiply its pixel value by the complex exponential term , and then accumulate over all pixel points of the entire image. As varies, transformation results at different frequencies will be obtained, thus constructing the complete frequency-domain image data. For example, when , represents the DC component of the image , that is, the sum of all pixel values. And as and increase in absolute value, the corresponding frequency component is higher, reflecting the detailed information in the CBCT image.

[0075] S12. Select noise data from other original first CBCT image data, convert it to the frequency domain to obtain frequency-domain noise data, and superimpose the frequency-domain noise data on the frequency-domain image data to obtain new frequency-domain image data.

[0076] Select noise data from other original first CBCT image data. These noise data refer to image segments that are different from the current first CBCT image data but contain similar elements, and these image segments can come from multiple other first CBCT image data. The selected noise data is also converted to the frequency domain in the aforementioned manner to obtain frequency-domain noise data.

[0077] Then, superimpose the frequency-domain noise data on the previously obtained frequency-domain image data. By superimposing noise, new features and variations can be introduced in the frequency domain, making the finally generated images more diverse in style. Different noise data will bring different changes to the frequency-domain image, thus generating new frequency-domain image data with multiple styles. For example, if the frequency-domain noise data has more high-frequency components, then the high-frequency part of the new frequency-domain image data after superimposition will be enhanced, making the finally generated third CBCT image data show more changes in details, such as more complex textures on the tooth surface.

[0078] S13. Perform inverse Fourier transform on each new frequency-domain image data to obtain multiple third CBCT image data, and construct multiple training data based on each of the first CBCT image data, the third CBCT image data, and the corresponding class labels.

[0079] Perform inverse Fourier transform on the superimposed new frequency-domain image data to convert the new frequency-domain data back to the spatial domain and obtain third CBCT image data that can be used for training. The inverse Fourier transform is the inverse process of the Fourier transform, which recombines the signal in the frequency domain into the pixel form of the image. The formula for the inverse transform is as follows:

[0080] ;

[0081] When calculating the inverse transform, the same operation is performed on each point of the frequency-domain image by multiplying the frequency-domain value by the complex exponential term , then accumulating all the , and finally dividing by to obtain the pixel value of the original image at . Through this process, the frequency information contained in the frequency domain is recombined into an image in the spatial domain, restoring a CBCT image with practical significance.

[0082] Based on the original first CBCT image data, the generated third CBCT image data and their corresponding category labels (such as the first category label marking the tooth boundary, or the second category label of normal teeth, tooth trauma type, etc.), multiple training data are constructed. These training data contain the original data and the new data after data enhancement, which enriches the diversity of the training set and helps to improve the training effect and generalization ability of the model. It should be noted that the generated training data is divided into two groups, one for the segmentation model and the other for the classification model, which will not be repeated here.

[0083] Among them, after the inverse Fourier transform, the generated new data sample, i.e., the third CBCT image data, needs to be subjected to boundary clipping and filtering processing to remove edge outliers and high-frequency noise that may be generated by the noise addition and transformation process, so as to improve the quality of the new data sample.

[0084] Optionally, superimposing the frequency domain noise data with the frequency domain image data to obtain new frequency domain image data includes:

[0085] Determine a low-frequency part and a high-frequency part of the frequency-domain image data, and for the low-frequency part and the high-frequency part, respectively use a first adding ratio and a second adding ratio to weaken the frequency-domain noise data; wherein the first adding ratio is smaller than the second adding ratio;

[0086] The weakened frequency domain noise data is superimposed on the frequency domain image data to obtain new frequency domain image data.

[0087] In this embodiment, the frequency domain image data contains information of different frequencies, wherein the low frequency part mainly reflects the overall features of the teeth in the image, such as the general outline and shape, such as the overall shape of the teeth, the general structure of the alveolar bone, etc.; the high frequency part focuses on the details of the teeth in the image, such as the texture of the tooth surface, tiny damage marks, etc. The low frequency and high frequency parts can be distinguished according to the frequency range or through a specific filtering algorithm. For example, a low-pass filter can be used to extract the low frequency part in the frequency domain image data, and a high-pass filter can be used to obtain the high frequency part.

[0088] Since the low-frequency part has a great influence on the overall structure and main features of the dental CBCT image, if too much low-frequency noise is added, the basic shape and contour of the image may be greatly changed, affecting the authenticity and availability of the data. For example, if too much noise is added to the low-frequency part, the overall shape of the tooth may appear distorted, which is not conducive to subsequent diagnosis and other operations based on the image. Therefore, the present invention adopts a smaller first addition ratio so that the impact of the added low-frequency noise on the main structure of the image is controlled within a smaller range, while increasing the diversity of the data and retaining the key features of the original image to the greatest extent.

[0089] Meanwhile, considering that the high-frequency part mainly reflects the details of the image. Appropriately increasing the amount of added noise in the high-frequency part can introduce more detail variations and enhance the diversity of the data. For example, details such as tiny cracks and wear marks on the tooth surface are reflected in the high-frequency part. Appropriately increasing the high-frequency noise can simulate more different detail situations, enabling the segmentation model to learn richer features. However, this ratio cannot be too large, otherwise it may introduce excessive and messy high-frequency noise, masking the true detail information.

[0090] The frequency-domain noise data weakened by different ratios are respectively superimposed on the low-frequency and high-frequency parts of the frequency-domain image data. For the low-frequency part, the noise weakened according to the first addition ratio is added to the low-frequency image data, and for the high-frequency part, the noise weakened according to the second addition ratio is added to the high-frequency image data, finally obtaining the new frequency-domain image data. The new frequency-domain image data obtained after such superimposition not only maintains similarity with the original image in terms of overall structure but also increases diversity in the detail part, providing rich frequency-domain information for subsequent generation of CBCT images with diverse styles through inverse Fourier transform, and helping to improve the training effect and generalization ability of the segmentation model.

[0091] It should be noted that the marked positions of the tooth boundary labels are actually unchanged before and after the superimposition, so the newly generated third CBCT image data also contains the first type of labels corresponding to the first CBCT image data.

[0092] Optionally, before superimposing the frequency-domain noise data on the frequency-domain image data, the method further includes:

[0093] Determine the overall similarity of the first CBCT image data with other first CBCT image data, and obtain the enhancement quantity according to the overall similarity; wherein, the enhancement quantity is positively correlated with the overall similarity;

[0094] Generate multiple sets of addition ratios corresponding to the enhancement quantity for the first CBCT image data, and each set of addition ratios includes the first addition ratio and the second addition ratio.

[0095] In this embodiment, the degree of style diversification of the first CBCT image data can be improved through data augmentation processing, thereby improving the degree of style diversification of the training data. However, there are also differences in the salience of different first CBCT image data. For example, some first CBCT image data are significantly different from other first CBCT image data, while some other first CBCT image data are relatively similar to some other first CBCT image data. If too much data augmentation processing is performed on the latter, the third CBCT image data obtained through the augmentation processing is actually very similar to the original first CBCT image data, which will instead reduce the overall degree of style diversification of the first CBCT image data collected in this batch.

[0096] To address the above technical problem, the present invention first calculates the similarity between each first CBCT image data and the remaining other first CBCT image data, and then aggregates all the similarities (such as calculating the average value, the mode value, or the median value) to obtain the overall similarity. The higher the overall similarity, the greater the style gap of this first CBCT image data compared to other first CBCT image data. Conversely, it indicates that the style of this first CBCT image data is closer to that of other first CBCT image data.

[0097] Further, a reinforcement quantity is obtained based on the overall similarity obtained above. This matching process is based on a preset correlation relationship, that is, a positive correlation relationship between the reinforcement quantity and the overall similarity is established in advance, and then the overall similarity is matched and calculated with this positive correlation relationship to obtain the corresponding reinforcement quantity.

[0098] In this way, a larger reinforcement quantity is set for the first CBCT image data with an independent style, that is, more groups of addition ratios are generated for it. In other words, based on this first CBCT image data, more third CBCT image data are generated subsequently. Each group of addition ratios includes different first addition ratios and second addition ratios. However, both the first addition ratio and the second addition ratio should be within the preset reasonable range, which will not be elaborated here.

[0099] In this embodiment of the present invention, the first CBCT image data with a significant style and the first CBCT image data with an insignificant style are distinguished, and based on this, different numbers of third CBCT image data are generated. In this way, the first CBCT image data with a significant style can be mainly used to generate the third CBCT image data. Since the first CBCT image data has a higher degree of style salience, it has more room for augmentation processing, and the style gap between the obtained third CBCT image data will be relatively larger, which will not lead to a reduction in the overall degree of style diversification of the first CBCT image data collected in this batch.

[0100] Optionally, classifying the monomeric features using the classification model to obtain several classification results, and deleting some of the classification results in each of the classification results based on the common features, including:

[0101] Classifying the monomeric features using the classification model to obtain several first classification results;

[0102] Extracting the position and pose information of the common features of each tooth based on the second CBCT image data, performing a consistency evaluation on each of the position and pose information, and if the consistency evaluation result is higher than a preset determination condition, determining that the common feature is non-dental trauma;

[0103] Determining the first classification results belonging to non-dental trauma as second classification results, and deleting the second classification results in each of the first classification results.

[0104] In this embodiment, the classification model analyzes and judges the monomeric features extracted from the monomeric tooth images. The monomeric features of each tooth contain unique information of the tooth, such as cracks on the tooth surface, the shape and position of defects, etc. The classification model classifies these monomeric features into different categories according to the pre-learned knowledge and patterns, so as to obtain a series of first classification results. These results may include different types of dental trauma, such as crown fracture, root fracture, etc., or may include non-dental trauma situations, that is, mis-identified non-dental trauma classifications.

[0105] The common features refer to some general features that all teeth possess, such as the arrangement, relative position and angle of teeth in the oral cavity, etc. In this embodiment, the common features refer to surface traces with a specific shape and distribution law that exist on each tooth (or most teeth). These surface traces may be caused by trauma, that is, under the action of trauma, similar scars appear on the surfaces of multiple teeth, such as long strip scratches formed by sharp metal on multiple teeth. The above situation belongs to the dental trauma situation.

[0106] However, these surface marks may also be caused by non-dental trauma situations. For example, teenagers may wear braces for a long time. The brackets and arch wires of fixed braces are connected to the teeth, and the enamel at the adhesion site of the brackets may show demineralization due to continuous external force and changes in the oral microenvironment, manifested as local color changes and the formation of chalky patches. During the wearing and removal of removable braces, they will frequently rub against the teeth, possibly causing fine scratches on the tooth surface. In addition, wearing braces will also change the cleaning difficulty in the oral cavity, making it easier for food residues and bacteria to accumulate in the area where the braces contact the teeth, accelerating the demineralization of the tooth surface and forming obvious marks over time. Such marks will be clearly presented in the CBCT images, causing the classification model to misclassify them as the corresponding dental trauma types, such as surface enamel damage. Obviously, these surface marks belong to non-dental trauma situations, and the corresponding dental trauma classification results are incorrect.

[0107] In order to identify whether there are situations such as surface marks caused by wearing braces in the above classification results, the present invention projects these common features onto the second CBCT image data, and then the position and posture of these common features can be determined. The position is the distribution area of these surface marks on each tooth, and the posture is the orientation of these surface marks on each tooth. If the above common features (i.e., the position and posture of the surface marks) of all teeth have a high degree of consistency, it indicates that the distribution of these surface marks on the patient's teeth is highly consistent. At this time, it is determined that they are surface marks caused by wearing braces, that is, it is determined that they do not belong to the dental trauma situation, and the corresponding classification result is incorrect and should be deleted. On the contrary, if the consistency of the above common features of all teeth, that is, the position and posture of the surface marks, is low, it indicates that these surface marks do not have consistency and it is impossible for them to be surface marks caused by wearing braces, and it is determined that they belong to the dental trauma situation, and the corresponding classification result is retained.

[0108] With such a setting, the present invention realizes a more accurate classification of dental trauma situations, and a low proportion of incorrect classification results can significantly reduce the ineffective workload of doctors, which is more practically significant in the case of classifying dental trauma for a large number of patients.

[0109] It can be understood that the consistency evaluation result can be a consistency evaluation value. The higher the consistency evaluation value, the higher the consistency of these common features, and vice versa. Correspondingly, the preset determination condition is, for example, that the consistency evaluation value is higher than the evaluation threshold.

[0110] An embodiment of the present invention also discloses a dental trauma image diagnosis system based on deep learning, as Figure 4 shown. The system includes a training module, a segmentation module, a classification module, and an output module;

[0111] The training module is used to perform data augmentation on a number of collected first CBCT image data to obtain training data with a higher degree of style diversity, and use the training data to train a segmentation model and a classification model based on Swin Transformer;

[0112] The segmentation module is used to perform dental image segmentation on the second CBCT image data of the target patient using the trained segmentation model to obtain multiple single-tooth images, and extract the single-tooth features and common features of each tooth from each of the single-tooth images;

[0113] The classification module is used to classify the single-tooth features using the classification model to obtain a number of classification results, and delete some of the classification results in each of the classification results based on the common features;

[0114] The output module is used to generate corresponding treatment plans based on the remaining classification results after deletion.

[0115] Optionally, the training module is used to:

[0116] Perform Fourier transform on the original first CBCT image data to obtain frequency-domain image data;

[0117] Select noise data from other original first CBCT image data and convert it to the frequency domain to obtain frequency-domain noise data, and superimpose the frequency-domain noise data on the frequency-domain image data to obtain new frequency-domain image data;

[0118] Perform inverse Fourier transform on each new frequency-domain image data to obtain multiple third CBCT image data, and construct multiple training data based on each of the first CBCT image data, the third CBCT image data, and the corresponding class labels.

[0119] Optionally, the training module is used to:

[0120] Determine the low-frequency part and high-frequency part of the frequency-domain image data, and for the low-frequency part and high-frequency part, weaken the frequency-domain noise data using a first addition ratio and a second addition ratio respectively; wherein, the first addition ratio is less than the second addition ratio;

[0121] Superimpose the weakened frequency-domain noise data on the frequency-domain image data to obtain new frequency-domain image data.

[0122] Optionally, the training module is further used to: before superimposing the frequency-domain noise data on the frequency-domain image data:

[0123] Determine the overall similarity between the first CBCT image data and the other first CBCT image data, and obtain the enhancement quantity according to the overall similarity; wherein, the enhancement quantity is positively correlated with the overall similarity;

[0124] Generate multiple groups of addition ratios corresponding to the enhancement quantity for the first CBCT image data, and each group of addition ratios includes a first addition ratio and a second addition ratio.

[0125] Optionally, the classification module is used for:

[0126] Use a classification model to classify the monomer features and obtain several first classification results;

[0127] Extract the position and pose information of the common features of each tooth based on the second CBCT image data, perform a consistency evaluation on each position and pose information, and if the consistency evaluation result is higher than a preset determination condition, determine that the common feature is non-dental trauma;

[0128] Determine the first classification results belonging to non-dental trauma as the second classification results, and delete the second classification results in each of the first classification results.

[0129] An embodiment of the present invention also discloses an electronic device, including one or more processors, a memory, and one or more computer programs, wherein one or more computer programs are stored in the memory, and one or more computer programs include instructions, and when the instructions are run by the processor, the method described in any one of the preceding items is implemented.

[0130] An embodiment of the present invention also discloses a computer storage medium, including computer instructions that can be run on an electronic device, and when the computer instructions are run, the method described in any one of the preceding items is implemented.

[0131] An embodiment of the present invention also discloses a computer program product, which can be run on an electronic device to implement the method described in any one of the preceding items.

[0132] In each embodiment of the present invention, each functional unit can be integrated in a processing unit, or each unit can exist physically alone, or two or more units can be integrated in one unit. The above integrated unit can be implemented in the form of hardware or in the form of a software functional unit.

[0133] When the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as USB flash drives, read-only memories (ROMs), random access memories (RAMs), mobile hard disks, magnetic disks, or optical discs.

[0134] The above are only the preferred embodiments of the present invention. It should be noted that for those of ordinary skill in the art, without departing from the principle of the present invention, several improvements and refinements can still be made, and these improvements and refinements should also be regarded as the protection scope of the present invention.

Claims

1. A method for diagnosing dental trauma images based on deep learning, characterized in that: The method includes the following steps: Perform data augmentation on a number of collected first CBCT image data to obtain training data with a higher degree of style diversity, and use the training data to train a segmentation model and a classification model based on Swin Transformer; Use the trained segmentation model to perform tooth image segmentation on the second CBCT image data of the target patient to obtain multiple single-tooth images, and extract the single-tooth features and common features of each tooth from each of the single-tooth images; The common features refer to some general features that all or most teeth have; Use the classification model to classify the single-tooth features to obtain a number of classification results, and delete some of the classification results in each of the classification results based on the common features; Generate corresponding treatment plans based on the remaining classification results after deletion; Performing data augmentation on a number of collected first CBCT image data to obtain training data with a higher degree of style diversity, including: Perform Fourier transform processing on the original first CBCT image data to obtain frequency-domain image data; Select noise data from other original first CBCT image data and convert it to the frequency domain to obtain frequency-domain noise data, and superimpose the frequency-domain noise data on the frequency-domain image data to obtain new frequency-domain image data; Perform inverse Fourier transform on each new frequency-domain image data to obtain multiple third CBCT image data, and construct multiple training data based on each of the first CBCT image data, the third CBCT image data, and the corresponding class labels; Superimposing the frequency-domain noise data on the frequency-domain image data to obtain new frequency-domain image data, including: Determine the low-frequency part and the high-frequency part of the frequency-domain image data, and for the low-frequency part and the high-frequency part, weaken the frequency-domain noise data using a first addition ratio and a second addition ratio respectively; wherein, the first addition ratio is less than the second addition ratio; Superimpose the weakened frequency-domain noise data on the frequency-domain image data to obtain new frequency-domain image data; Before superimposing the frequency-domain noise data on the frequency-domain image data, the method further includes: Determine the overall similarity between the first CBCT image data and other first CBCT image data, and match the reinforcement quantity according to the overall similarity; wherein, the reinforcement quantity is negatively correlated with the overall similarity; Generate multiple groups of addition ratios corresponding to the reinforcement quantity for the first CBCT image data, and each group of the addition ratios includes a first addition ratio and a second addition ratio.

2. The method for diagnosing dental trauma images based on deep learning according to claim 1, wherein: Using the classification model to classify the single-tooth features to obtain a number of classification results, and deleting some of the classification results in each of the classification results based on the common features, including: Use the classification model to classify the single-tooth features to obtain a number of first classification results; Extract the position and pose information of the common features of each tooth based on the second CBCT image data, perform a consistency evaluation on each piece of position and pose information, and if the consistency evaluation result is higher than the preset determination condition, determine that the common feature is non-dental trauma; Determine the first classification results belonging to non-dental trauma as the second classification results, and delete the second classification results in each of the first classification results.

3. A dental trauma imaging diagnosis system based on deep learning, characterized in that: The system includes a training module, a segmentation module, a classification module, and an output module; The training module is used to perform data augmentation on a number of collected first CBCT image data to obtain training data with a higher degree of style diversification, and use the training data to train a segmentation model and a classification model based on Swin Transformer; The segmentation module is used to perform tooth image segmentation processing on the second CBCT image data of the target patient using the trained segmentation model to obtain multiple single-tooth images, and extract the single features and common features of each tooth from each of the single-tooth images; The classification module is used to classify the single features using the classification model to obtain a number of classification results, and delete some of the classification results in each of the classification results based on the common features; The output module is used to generate corresponding treatment plans based on the remaining classification results after deletion; The training module is used for: Perform Fourier transform processing on the original first CBCT image data to obtain frequency-domain image data; Select noise data from other original first CBCT image data and convert it to the frequency domain to obtain frequency-domain noise data, and superimpose the frequency-domain noise data on the frequency-domain image data to obtain new frequency-domain image data; Perform inverse Fourier transform on each new frequency-domain image data to obtain multiple third CBCT image data, and construct multiple pieces of training data based on each of the first CBCT image data, the third CBCT image data, and the corresponding class labels; It is also used for: determining the low-frequency part and high-frequency part of the frequency-domain image data, and respectively weakening the frequency-domain noise data using a first addition ratio and a second addition ratio for the low-frequency part and the high-frequency part; wherein, the first addition ratio is less than the second addition ratio; Superimpose the weakened frequency-domain noise data on the frequency-domain image data to obtain new frequency-domain image data; It is also used for: before superimposing the frequency-domain noise data on the frequency-domain image data: Determine the overall similarity between the first CBCT image data and other first CBCT image data, and match the reinforcement quantity according to the overall similarity; wherein, the reinforcement quantity is negatively correlated with the overall similarity; Generate multiple groups of addition ratios corresponding to the reinforcement quantity for the first CBCT image data, and each group of addition ratios includes a first addition ratio and a second addition ratio.

4. An electronic device, characterized in that: Comprising one or more processors, a memory, and one or more computer programs, wherein the one or more computer programs are stored in the memory, and the one or more computer programs include instructions which, when run by the processor, implement the deep learning-based dental trauma image diagnosis method according to claim 1 or 2.

5. A computer storage medium, characterized in that: Comprising computer instructions executable on an electronic device, which, when run, implement the deep learning-based dental trauma image diagnosis method according to claim 1 or 2.

6. A computer program product, characterized in that: The computer program product is executable on an electronic device to implement the deep learning-based dental trauma image diagnosis method according to claim 1 or 2.

Images