Tooth Image Data Enhancement Method, Device, Terminal Device and Storage Medium

Abstract

The present invention belongs to the technical field of image enhancement, and provides a method, a device, a terminal device and a storage medium for enhancing dental image data. The method includes: receiving a plurality of dental image data with marking information, and dividing each dental image data into a first category and a second category according to the marking information; for each dental image data belonging to the second category, extracting typical abnormal features from the dental image data as enhancement conditions; respectively generating a plurality of first dental image enhancement data and second dental image enhancement data based on the dental image data of the first category and the second category; combining each first dental image enhancement data and second dental image enhancement data with each dental image data to obtain a dental image enhancement data set. The present invention uses a generative adversarial network to perform adversarial iteration on two types of data respectively to generate enhancement data, greatly enriching the diversity of the data.

Description

Technical Field

[0001] The present invention relates to the technical field of image enhancement, and more specifically, to a method, device, terminal device and storage medium for enhancing dental image data. Background Art

[0002] In the field of modern dentistry, deep learning-based dental image segmentation technology plays a crucial role in accurate diagnosis and treatment plan formulation. The performance of deep learning models highly depends on large-scale, high-quality and detailed-annotated datasets to learn the complex features and patterns of dental images. However, in practical applications, the dental segmentation task faces severe challenges. On the one hand, due to strict requirements for patient privacy protection, there are many restrictions on obtaining a large number of real dental image data; on the other hand, accurate annotation of dental images requires professional dental medical knowledge and a large amount of time and effort, which leads to extremely high annotation costs.

[0003] All of the above factors make the current dental segmentation task seriously lack large-scale and diverse standard datasets, greatly limiting the accuracy and robustness of dental segmentation models. The models cannot fully learn all the features of dental images under different individuals and different shooting conditions, and are prone to inaccurate segmentation when facing new and complex dental images.

[0004] To alleviate the above problems, data augmentation strategies are widely used. Among many data augmentation strategies, adversarial data augmentation strategies stand out. Through technologies such as generative adversarial networks, they can effectively enrich the dataset, enable the model to learn more diverse features, and thus improve the generalization of the model. When using adversarial data augmentation strategies, existing methods often regularize the augmentation process based on prior knowledge to achieve a certain degree of control in order to avoid generating meaningless augmented images. However, the regularization based on prior knowledge requires professional domain knowledge for setting and is difficult to adapt to complex and changing dental image data.

[0005] Therefore, how to better utilize adversarial data augmentation strategies to generate effective, reliable and diverse dental image augmented images has become a key problem to be solved urgently. Summary of the Invention

[0006] In response to this, the present invention provides a method, device, terminal device, storage medium and computer program product for enhancing dental image data to solve the above technical problems.

[0007] The present invention discloses a method for enhancing dental image data, the method comprising the following steps:

[0008] Receive a number of dental image data with marked information, and divide each of the dental image data into a first classification with significant disease characteristics and a second classification with insignificant disease characteristics according to the marked information;

[0009] For each of the dental image data belonging to the second classification, extract typical abnormal features from the dental image data, and use the typical abnormal features as enhancement conditions;

[0010] Randomly select a batch of data from the first classification as the first real samples, the first generator generates a batch of first false samples corresponding to the first real samples according to random noise, and the first discriminator and the first generator perform adversarial iteration based on the first real samples and the first false samples to generate a number of first dental image enhancement data;

[0011] Randomly select a batch of data from the second classification as the second real samples, the second generator generates a batch of second false samples corresponding to the second real samples according to random noise, and the second discriminator and the second generator perform adversarial iteration based on the second real samples, the second false samples and the enhancement conditions to generate a number of second dental image enhancement data;

[0012] Combine each of the first dental image enhancement data, the second dental image enhancement data with each of the dental image data to obtain a dental image enhancement data set.

[0013] In some embodiments, the dividing each of the dental image data into a first classification with significant disease characteristics and a second classification with insignificant disease characteristics according to the marked information includes:

[0014] Use a classifier to classify the types of dental lesions or injuries and the severity of lesions or injuries included in the marked information to obtain the significant level of disease characteristics of the corresponding dental image data;

[0015] Determine the dental image data with a significant level of disease characteristics higher than the level threshold as the first classification with significant disease characteristics, and determine the dental image data with a significant level of disease characteristics not higher than the level threshold as the second classification with insignificant disease characteristics.

[0016] In some embodiments, the extracting typical abnormal features from each of the dental image data belonging to the second classification and using the typical abnormal features as enhancement conditions includes:

[0017] Adopt a multi-scale feature extraction algorithm to extract features from each of the dental image data belonging to the second classification, and the extracted features include fine textures, local features, and overall structural features;

[0018] Calculate the feature distribution differences of the fine texture, the local features, and the overall structural features in the corresponding tooth image data and several normal tooth images, and use the features with feature distribution differences exceeding a preset significance threshold as the typical abnormal features;

[0019] Use the quantized and encoded typical abnormal features as enhancement conditions.

[0020] In some embodiments, calculating the feature distribution differences of the fine texture, the local features, and the overall structural features in the corresponding tooth image data and several normal tooth images includes:

[0021] Statistically obtain the sum of the quantities of the fine texture and the local features extracted, that is, the first quantity, match the second quantity based on the first quantity, and randomly select the second quantity of normal tooth images;

[0022] Calculate the feature distribution differences of the fine texture, the local features, and the overall structural features in the corresponding tooth image data and each normal tooth image respectively, and calculate the average value of all feature distribution differences.

[0023] In some embodiments, the second discriminator and the second generator perform adversarial iteration based on the second real samples, the second fake samples, and the enhancement conditions to generate several second tooth image enhancement data, including:

[0024] Integrate the enhancement conditions into the input layer or the intermediate layer of the second generator. The second generator performs nonlinear transformation and feature combination on the input random noise and the enhancement conditions through its internal multi-layer neural network structure to initially generate a batch of second fake samples;

[0025] The second discriminator matches and discriminates the typical abnormal features in the second fake samples and the corresponding second real samples to determine whether they meet the enhancement conditions, and scores based on the judgment results; the second generator adjusts its network parameters based on the scores until the adjusted second fake samples that meet the enhancement conditions are generated;

[0026] The second discriminator performs an overall authenticity discrimination on the second fake samples, and feeds back the discrimination results to the second generator to further adjust its network parameters;

[0027] Iterate the above process until the adversarial iteration termination condition is met. At this time, a batch of second fake samples generated by the second generator are several second tooth image enhancement data.

[0028] The present invention also discloses a tooth image data enhancement device, which includes a receiving module, an enhancement condition generation module, and an image enhancement module;

[0029] The receiving module receives a number of dental image data with marking information, and divides each of the dental image data into a first classification with significant disease characteristics and a second classification with insignificant disease characteristics according to the marking information;

[0030] The enhancement condition generation module extracts typical abnormal features from each of the dental image data belonging to the second classification, and uses the typical abnormal features as enhancement conditions;

[0031] The image enhancement module randomly selects a batch of data from the first classification as the first real samples, the first generator generates a batch of first false samples corresponding to the first real samples according to random noise, and the first discriminator and the first generator perform adversarial iterations based on the first real samples and the first false samples to generate a number of first dental image enhancement data;

[0032] Randomly select a batch of data from the second classification as the second real samples, the second generator generates a batch of second false samples corresponding to the second real samples according to random noise, and the second discriminator and the second generator perform adversarial iterations based on the second real samples, the second false samples and the enhancement conditions to generate a number of second dental image enhancement data;

[0033] Combine each of the first dental image enhancement data, the second dental image enhancement data with each of the dental image data to obtain a dental image enhancement data set.

[0034] In some embodiments, the receiving module is specifically configured to:

[0035] Use a classifier to classify the types of lesions or injuries and the severity of lesions or injuries of the teeth included in the marking information to obtain the significant level of the disease characteristics of the corresponding dental image data;

[0036] Determine the dental image data with the significant level of disease characteristics higher than the level threshold as the first classification with significant disease characteristics, and determine the dental image data with the significant level of disease characteristics not higher than the level threshold as the second classification with insignificant disease characteristics.

[0037] The present invention also discloses a terminal device, including: at least one processor, a memory, and a computer program stored in the memory and executable on the at least one processor, and the processor executes the computer program to implement the method as described in any one of the above.

[0038] The present invention also discloses a storage medium, the storage medium stores a computer program, and the computer program is executed by a processor to implement the method as described in any one of the above.

[0039] The present invention also discloses a computer program product, which, when run on a terminal device, enables the implementation of the method described in any of the above.

[0040] The beneficial effects of the present invention are as follows:

[0041] By classifying dental image data according to the significance degree of disease characteristics, extracting typical abnormal features from the second classified data with insignificant disease characteristics and using them as enhancement conditions, the present invention can effectively mine hidden disease information. By using a generative adversarial network, adversarial iteration is performed on the two types of data respectively to generate enhanced data, greatly enriching the diversity of the data. The finally formed dental image enhancement dataset provides more comprehensive and targeted data support for the dental segmentation model, significantly improving the performance and generalization ability of the model, and helping to more accurately identify dental lesions in actual clinical applications and assist dentists in formulating precise diagnosis and treatment plans. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following will briefly introduce the drawings required for the embodiments. It should be understood that the following drawings only show some embodiments of the present invention, and therefore should not be regarded as limiting the scope. For those of ordinary skill in the art, other related drawings can be obtained based on these drawings without creative efforts.

[0043] Figure 1 is a schematic flowchart of a method for enhancing dental image data disclosed in an embodiment of the present invention;

[0044] Figure 2 is a schematic structural diagram of a generative adversarial network disclosed in an embodiment of the present invention;

[0045] Figure 3 is a schematic structural diagram of a device for enhancing dental image data disclosed in an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0046] The following specific embodiments illustrate the implementation manners of the present application. Those skilled in the art can easily understand other advantages and effects of the present application from the content disclosed in this specification. Obviously, the described embodiments are part of the embodiments of the present application, rather than all of them. All other embodiments obtained by those of ordinary skill in the art without creative efforts based on the embodiments in the present application belong to the scope of protection of the present application.

[0047] In addition, the technical features involved in different implementation manners of the present application described below can be combined with each other as long as they do not conflict with each other.

[0048] Such asFigure 1 As shown, an embodiment of the present invention discloses a method for enhancing dental image data. The method includes the following steps:

[0049] S10. Receive a number of dental image data with marking information, and divide each of the dental image data into a first classification with significant disease characteristics and a second classification with insignificant disease characteristics according to the marking information.

[0050] The solution of the present invention is applied to terminals such as devices, systems, or servers dedicated to enhancing dental image data. These terminals receive a batch of dental image data samples, and then expand this batch of samples through data enhancement to obtain more and diverse samples.

[0051] In this step, the above-mentioned terminal receives a batch of dental image data uploaded by the user and stores it in the sample library. These data all carry corresponding marking information. The marking information can be manually marked by a professional dentist during the diagnosis process, or automatically generated by the image acquisition device, and includes markings related to tooth type, tooth lesion or injury type, lesion or injury severity, etc. For example, the type of tooth (such as incisors, molars, etc.), whether there are lesions such as dental caries, periodontitis, etc.

[0052] Based on the above-mentioned marking information, taking whether the disease characteristics are significant as the classification criterion, all dental image data are clearly classified into two categories. The first classification is data with significant disease characteristics. Such dental images clearly show obvious lesions, such as large-area dental caries and alveolar bone resorption caused by severe periodontitis, presenting intuitive and prominent abnormal features in the images, which have a key and direct guiding role in disease diagnosis. The second classification is data with insignificant disease characteristics. The disease manifestations in the images are relatively hidden. For example, very early dental caries only show extremely subtle density changes, or mild periodontal tissue inflammation, and it is difficult to quickly and accurately identify the difference from the normal state from the image features.

[0053] S20. For each of the dental image data belonging to the second classification, extract typical abnormal features from the dental image data, and use the typical abnormal features as enhancement conditions.

[0054] In this step, since the disease characteristics of the second-classification data are not significant and these typical abnormal characteristics are often hidden and subtle, the present invention is configured to extract typical abnormal characteristics from the dental image data of the second classification. For example, through a grayscale analysis algorithm, it is possible to identify the subtle grayscale difference between the extremely early dental caries site and the normal tooth tissue, such as the grayscale value of a certain local area being slightly lower than the normal range and presenting a specific shape and distribution pattern. Or, by means of an edge detection algorithm, it is found that there are minor irregular changes in the periodontal ligament boundary when there is mild periodontitis, such as the boundary line being slightly tortuous or locally blurred, while the normal periodontal ligament boundary is clear and smooth.

[0055] The extracted typical abnormal characteristics are used as the constraint conditions for subsequent data augmentation. When augmenting the second-classification data, the process of the second generator generating new data will closely revolve around these typical abnormal characteristics. When the generator generates new dental image data, based on the attribute information such as the position, shape, and grayscale of these characteristics in the image, the input random noise is subjected to corresponding transformation and adjustment to ensure that the generated data contains both the typical abnormal characteristics similar to the original data and can, to a certain extent, simulate and expand these characteristics diversely.

[0056] It should be noted that if enhancement conditions are set for the first-classification data with significant disease characteristics, the generated enhanced data may deviate from the true pattern of significant disease characteristics to a certain extent, which may cause the model to overfit to these enhanced "atypical" data during the training process, increasing the risk of overfitting. Therefore, the present invention only sets enhancement conditions for the second-classification data, that is, the method of conditional generation is implemented in the subsequent adversarial data augmentation strategy.

[0057] S30, randomly select a batch of data from the first classification as the first real samples, the first generator generates a batch of first false samples corresponding to the first real samples according to random noise, and the first discriminator and the first generator perform adversarial iteration based on the first real samples and the first false samples to generate a number of first dental image enhanced data.

[0058] In this step, the present invention uses an adversarial data augmentation strategy to augment the initial samples, that is, dental image data. The structure of the generative adversarial network adopted is as Figure 2 shown.

[0059] First, randomly select a batch of data from the first-classification data as the first real samples, and these samples truly reflect the dental image conditions with significant disease characteristics.

[0060] The first generator, based on the principle of generative adversarial networks, takes random noise as input and generates a batch of first fake samples that are consistent with the first real samples in terms of dimension and format. These fake samples are the result of the generator simulating real samples, but they are not real dental image data.

[0061] The task of the first discriminator is to accurately distinguish whether the input data is a first real sample or a first fake sample. The first discriminator and the first generator are trained through continuous adversarial iterations. In each iteration, the first discriminator judges the input first real samples and first fake samples, and adjusts its own parameters according to the judgment results to improve the discrimination ability. At the same time, the first generator adjusts its parameters according to the feedback of the first discriminator to generate more realistic fake samples, making it difficult for the first discriminator to distinguish. After multiple adversarial iterations, the first generator generates a batch of first dental image enhancement data that is similar to the first classification data but has diversity.

[0062] Repeating the above process, a number of first dental image enhancement data can be obtained.

[0063] S40, randomly select a batch of data from the second classification as the second real samples. The second generator generates a batch of second fake samples corresponding to the second real samples according to the random noise. The second discriminator and the second generator perform adversarial iterations based on the second real samples, the second fake samples, and the enhancement conditions, and generate a number of second dental image enhancement data.

[0064] In this step, a batch of data is randomly selected from the second classification data as the second real samples, and these samples have the typical abnormal features extracted before. The second generator also takes random noise as input and generates a batch of second fake samples corresponding to the second real samples.

[0065] The second discriminator and the second generator perform adversarial iterations. The difference from the first classification is that this adversarial iteration needs to consider the previously extracted typical abnormal features as the enhancement conditions. When the second discriminator distinguishes the second real samples and the second fake samples, it needs to judge both the authenticity of the data and whether the data meets the enhancement conditions (that is, whether it has the corresponding typical abnormal features). When the second generator generates the second fake samples, it adjusts the generated samples according to the enhancement conditions, so that the generated samples are both authentic and meet the requirements of specific typical abnormal features. Through this adversarial iteration, a batch of second dental image enhancement data that conforms to the characteristics of the second classification data is generated.

[0066] Repeating the above process, a number of second dental image enhancement data can be obtained.

[0067] S50, combine each of the first dental image enhancement data, the second dental image enhancement data, and each of the dental image data to obtain a dental image enhancement dataset.

[0068] In this step, the first dental image enhancement data generated in step S30 and the second dental image enhancement data generated in step S40 are combined with all the original dental image data initially received (i.e., the dental image data of the first classification and the second classification). After combination, a dental image enhancement data set containing the original data and the enhancement data is obtained.

[0069] This dental image enhancement data set greatly enriches the data diversity and can be used to train a dental segmentation model, significantly improving the performance and generalization ability of the model.

[0070] In the present invention, by classifying dental image data according to the significance degree of disease characteristics, extracting typical abnormal characteristics of the second classification data with insignificant disease characteristics as enhancement conditions, the hidden disease information can be effectively mined. Using a generative adversarial network, adversarial iteration is performed on the two types of data respectively to generate enhancement data, greatly enriching the data diversity. The finally formed dental image enhancement data set provides more comprehensive and targeted data support for the dental segmentation model, significantly improving the performance and generalization ability of the model, and helping to more accurately identify dental lesions in actual clinical applications and assist dentists in formulating precise diagnosis and treatment plans.

[0071] In some embodiments, the dividing of each of the dental image data into a first classification with significant disease characteristics and a second classification with insignificant disease characteristics according to the marking information includes:

[0072] Using a classifier to classify the lesion or injury type of the teeth and the severity of the lesion or injury included in the marking information, so as to obtain the significant level of the disease characteristics of the corresponding dental image data;

[0073] The dental image data with the significant level of the disease characteristics higher than the level threshold is determined as the first classification with significant disease characteristics, and the dental image data with the significant level of the disease characteristics not higher than the level threshold is determined as the second classification with insignificant disease characteristics.

[0074] In the embodiments of the present invention, a classifier is introduced, and its function is to comprehensively analyze and classify the lesion or injury type of the teeth (such as lesion types such as dental caries, periodontitis, pulpitis, or injury types such as tooth fracture, wear, etc.) and the severity of the lesion or injury (such as the area size of dental caries, the degree of alveolar bone resorption caused by periodontitis, the degree of tooth wear, etc.) in the marking information.

[0075] The classifier can be a model constructed based on machine learning algorithms, such as decision trees, support vector machines, or more complex deep learning models like convolutional neural networks. After being trained with a large amount of accurately labeled dental image data, the classifier has the ability to judge the significance level of dental disease characteristics based on the input labeled information.

[0076] After processing the type and severity of lesions or injuries, the classifier will output a corresponding significance level of disease characteristics, which is a quantitative assessment of the disease condition in the dental image data. For example, the level can be represented by numerical values 1 - 5, and the larger the value, the more significant the disease characteristics. If a tooth has extensive dental caries accompanied by severe pulp inflammation, the classifier will give a higher level, such as 4 or 5; while for a tooth with only mild gingival inflammation, the classifier will give a lower level, such as 1 or 2.

[0077] Compare the significance level of disease characteristics obtained from the classifier with a preset level threshold (for example, 3). If the significance level of disease characteristics of a certain dental image data is higher than this level threshold, then the image data of this tooth is determined to be the first classification with significant disease characteristics. Conversely, if the significance level of disease characteristics is not higher than the level threshold (i.e., less than or equal to the level threshold), the dental image data is determined to be the second classification with insignificant disease characteristics. In this way, through a clear quantitative standard and comparison process, the accurate classification of dental image data based on the significance level of disease characteristics is achieved.

[0078] In some embodiments, for each of the dental image data belonging to the second classification, extracting typical abnormal features from the dental image data and using the typical abnormal features as enhancement conditions includes:

[0079] Using a multi-scale feature extraction algorithm to extract features from each of the dental image data belonging to the second classification, and the extracted features include fine textures, local features, and overall structure features;

[0080] Calculating the feature distribution differences of the fine textures, the local features, and the overall structure features in the corresponding dental image data and several normal dental images, and taking the features with feature distribution differences exceeding a preset significance threshold as the typical abnormal features;

[0081] Using the quantified and encoded typical abnormal features as enhancement conditions.

[0082] In the embodiments of the present invention, in order to capture feature information of different sizes and levels from dental image data, the present invention adopts a multi-scale feature extraction algorithm to process the dental image data of the second classification. The multi-scale feature extraction algorithm uses convolution kernels or filters of different scales to extract features from the image in multiple scale spaces, obtaining fine textures, local features, and overall structural features.

[0083] Among them, fine textures and local features are crucial for detecting early and minor pathological abnormalities. For example, early dental caries may only show subtle texture changes or gray-scale differences in local areas in the image, and these features can be accurately captured by small-scale convolution kernels (where local features refer to other features in the local area besides fine textures, such as the morphology and contour of cusp tips, pits, and fissures, and the local morphology changes during tooth wear). The overall structural features, on the other hand, help to grasp abnormalities in the overall morphology of teeth. For example, for minor tooth torsion, displacement, etc., the change information of the overall structural features can be better obtained through large-scale convolution kernels.

[0084] Calculate the distribution differences between the fine textures, local features, and overall structural features extracted from the dental image data of the second classification and the corresponding features in a number of normal dental images. That is, a reference data set is composed of a number of normal dental images, and by comparing the differences in statistical distributions between the features of the second classification images and the normal image features, it is determined which features may belong to typical abnormal features. When the feature distribution difference between the dental image data of the second classification and the normal dental image data exceeds a preset significance threshold, the feature is determined to be a typical abnormal feature.

[0085] Convert the selected typical abnormal features into specific numerical forms for quantification, and then further organize these numerical values into digital feature vectors suitable for computer processing to complete the encoding. The obtained encoded vectors contain various attribute information of the features in the image, such as position, size, gray-scale value, shape, etc. For example, for a typical abnormal feature of local gingival thickening caused by suspected early periodontitis, its digital feature vector may include the coordinate position of the thickened area in the image, the area size, the change amount of the gray-scale value compared with the normal gingival area, and the shape features of the area are characterized by some shape description parameters (such as circularity, perimeter-to-area ratio, etc.).

[0086] The typical abnormal features after quantization and encoding will be used as enhancement conditions. When the second generator generates the second fake sample, these typical abnormal features will be incorporated into the generation process as constraint conditions. The second generator will generate a fake sample that is both similar to the original second real sample and can highlight these typical abnormal features by adjusting the input random noise or internal network parameters according to the constraint conditions (which can be multiple). In this way, the generated second dental image enhancement data not only enriches the data diversity but also can specifically strengthen these abnormal conditions with insignificant disease features, providing more valuable training data for the dental segmentation model.

[0087] In some embodiments, calculating the feature distribution differences of the fine texture, the local features, and the overall structural features in the corresponding dental image data and a plurality of normal dental images includes:

[0088] Counting the sum of the quantities of the fine texture and the local features obtained by extraction (i.e., the first quantity), matching the second quantity based on the first quantity, and randomly selecting the second quantity of normal dental images;

[0089] Calculating the feature distribution differences of the fine texture, the local features, and the overall structural features in the corresponding dental image data and each normal dental image respectively, and calculating the average value of all the feature distribution differences.

[0090] In the embodiments of the present invention, the sum of the quantities of the fine texture and the local features extracted from the second classified dental image data is counted and denoted as the first quantity. The fine texture and the local features contain a large amount of detailed information in the dental image and are crucial for discovering early and subtle pathological abnormalities. The first quantity characterizes the richness of the detailed features in the dental image data.

[0091] When the detailed features contained in the dental image data are more abundant, more normal dental image data are randomly selected from the normal dental image database. In this way, the average value of the calculated more feature distribution differences is closer to the real situation, and the subsequent recognition result of the typical abnormal features is more accurate. On the contrary, fewer normal dental image data are randomly selected from the normal dental image database. This can not only ensure that there are enough normal images for comparison but also avoid excessive computational complexity caused by too many comparison samples. Among them, a control data table of the first quantity and the second quantity is established in advance, and the appropriate second quantity can be obtained by querying this table, which will not be elaborated here.

[0092] For the subtle textures, local features, and overall structural features extracted from the second-classified tooth image data, calculate the distribution differences between them and the corresponding features in the randomly selected normal tooth images. The calculation of feature distribution differences can adopt a variety of statistical methods, such as calculating the difference between two feature sets in statistics such as mean, variance, and histogram. Taking the grayscale value feature as an example, if the grayscale value mean of a local area in the second-classified tooth image is μ1 and the variance is σ1, and the grayscale value mean of the corresponding area in the normal tooth image is μ2 and the variance is σ2, then the distribution difference of the grayscale value features of the two areas can be measured by calculating indicators such as |μ1-μ2| and |σ1-σ2|.

[0093] After calculating the distribution differences of all features (fine texture, local features, and overall structural features) from each normal tooth image, all these calculated feature distribution difference values are summarized and their average values are calculated. This average value can comprehensively reflect the degree of difference between the current second-classified tooth image data and the normal tooth image data in the overall feature distribution. By using the average value as a measurement standard, the impact of individual abnormal comparison results on the final judgment can be reduced, making the obtained feature distribution difference results more stable and reliable, thereby providing a more valuable reference data basis for the subsequent determination of typical abnormal features.

[0094] In some embodiments, the second discriminator and the second generator perform adversarial iteration based on the second real sample, the second false sample and the enhancement condition to generate a plurality of second dental image enhancement data, including:

[0095] The enhancement condition is integrated into the input layer or the middle layer of the second generator, and the second generator performs nonlinear transformation and feature combination on the input random noise and the enhancement condition through its internal multi-layer neural network structure to preliminarily generate a batch of second false samples;

[0096] The second discriminator matches and discriminates the typical abnormal features in the second false sample and the corresponding second real sample to determine whether the enhancement condition is met, and scores the second sample based on the determination result; the second generator adjusts its network parameters based on the score until the adjusted second false sample that meets the enhancement condition is generated;

[0097] The second discriminator performs overall authenticity discrimination on the second false sample, and feeds the discrimination result back to the second generator so that it further adjusts its network parameters;

[0098] The above process is iterated until the adversarial iteration termination condition is met. At this time, a batch of second false samples generated by the second generator are a number of second tooth image enhancement data.

[0099] In the embodiments of the present invention, the typically abnormal features extracted and quantized and encoded from the second-class dental image data are used as enhancement conditions and incorporated into the input layer or intermediate layer of the second generator. In this way, the generator can generate data based on these typically abnormal features at the initial stage of data generation, ensuring that the generated data has the typically abnormal features related to the original second-class data.

[0100] The second generator receives random noise as input and performs non-linear transformation and feature combination on the random noise and the enhancement conditions through its internal multi-layer neural network structure. The random noise provides a basis for the diversity of the generated data, while the enhancement conditions guide the generation direction. In this process, each layer of the neural network operates on the input through activation functions and other means, fusing information from different sources to initially generate a batch of second false samples.

[0101] The second discriminator matches and discriminates the typically abnormal features in the initially generated second false samples with those in the corresponding second true samples. The discrimination process mainly relies on the quantization information about the typically abnormal features in the enhancement conditions, such as attributes like the position, size, gray value, shape, etc. of the features. The second discriminator checks whether the features in the second false samples match these quantization criteria, determines whether they meet the enhancement conditions, and scores based on the judgment results. If the typically abnormal features in the second false samples highly match the enhancement conditions, the score will be higher; conversely, if the features deviate greatly, the score will be lower.

[0102] The second generator receives the score given by the second discriminator and adjusts its network parameters according to the score. A lower score means that the generated samples do not meet the enhancement conditions. The generator needs to adjust parameters such as the weights and biases of each layer in the neural network through algorithms like backpropagation to optimize the generation process of the typically abnormal features, so that the subsequent generated second false samples can more accurately meet the enhancement conditions. This process will be repeated continuously until the generator can generate adjusted second false samples that meet the enhancement conditions.

[0103] After the second false samples meet the enhancement conditions, the second discriminator performs an overall authenticity discrimination on them. At this time, the discriminator no longer only focuses on the typically abnormal features, but comprehensively judges from multiple aspects such as the overall visual effect, texture, and structure of the samples whether the samples are from the real second-class dental image data or the second false samples generated by the second generator. The second discriminator judges the authenticity of the second false samples by learning the overall feature distribution of the real samples.

[0104] The second discriminator feeds back the overall authenticity discrimination result to the second generator. If the discrimination result shows that the second fake sample is easily recognized as fake, the second generator will further adjust its network parameters and optimize the generation process again to make the generated sample more realistic as a whole and difficult to be distinguished by the discriminator. This adjustment process is also achieved by adjusting the neural network parameters, aiming to make the samples generated by the generator not only meet the requirements in terms of typical abnormal features but also be indistinguishable from real samples in terms of overall appearance.

[0105] The above processes of enhanced condition-based matching discrimination, generator adjustment, overall authenticity discrimination, and generator readjustment will be continuously iterated. In each iteration, the second discriminator and the second generator improve their performance in confrontation with each other. The second discriminator continuously enhances its ability to distinguish real samples from fake samples, while the second generator continuously improves the quality of the generated samples to make them closer to real samples and highlight typical abnormal features. When the preset confrontation iteration termination condition is met, the iteration process ends. The termination condition can be reaching a certain number of iterations, or the accuracy of the second discriminator stabilizing within a preset threshold range, etc. At this time, a batch of second fake samples generated by the second generator are several second dental image enhancement data.

[0106] These enhancement data not only contain typical abnormal features similar to the original second classification data but also have high authenticity as a whole, which can effectively expand the second classification dental image dataset and provide richer and more valuable data for the training of the dental segmentation model.

[0107] As Figure 3 shown, an embodiment of the present invention also discloses a dental image data enhancement device, which includes a receiving module, an enhancement condition generation module, and an image enhancement module;

[0108] The receiving module receives a number of dental image data with marking information, and divides each of the dental image data into a first classification with significant disease characteristics and a second classification with insignificant disease characteristics according to the marking information;

[0109] The enhancement condition generation module extracts typical abnormal features from each of the dental image data belonging to the second classification and uses the typical abnormal features as enhancement conditions;

[0110] The image enhancement module randomly selects a batch of data from the first classification as the first real samples, the first generator generates a batch of first fake samples corresponding to the first real samples according to random noise, and the first discriminator and the first generator perform confrontation iteration based on the first real samples and the first fake samples to generate a number of first dental image enhancement data;

[0111] Randomly select a batch of data from the second classification as the second real samples. The second generator generates a batch of second fake samples corresponding to the second real samples according to random noise. The second discriminator and the second generator perform adversarial iteration based on the second real samples, the second fake samples, and the enhancement conditions to generate a number of second dental image enhancement data;

[0112] Combine each of the first dental image enhancement data and the second dental image enhancement data with each of the dental image data to obtain a dental image enhancement data set.

[0113] An embodiment of the present invention also discloses a terminal device, including: at least one processor, a memory, and a computer program stored in the memory and executable on the at least one processor, where the processor executes the computer program to implement the method as described in the foregoing embodiments.

[0114] An embodiment of the present invention also discloses a storage medium storing a computer program, where the computer program is executed by a processor to implement the method as described in the foregoing embodiments.

[0115] An embodiment of the present invention also discloses a computer program product, which, when run on a terminal device, enables the implementation of the method as described in any one of the above.

[0116] The present invention is described with reference to the flowcharts and / or block diagrams of methods, apparatuses, and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowcharts and / or block diagrams, and the combination of processes and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to the processors of general-purpose computers, special-purpose computers, embedded processors, or other programmable data processing devices to generate a machine, such that the instructions executed by the processors of the computer or other programmable data processing devices generate a device for implementing the functions specified in Figure 1 one process or multiple processes and / or blocks Figure 1 one block or multiple blocks.

[0117] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, such that the instructions stored in the computer-readable memory generate a manufactured article including an instruction device that implements the functions specified in Figure 1 one process or multiple processes and / or blocks Figure 1 one block or multiple blocks.

[0118] The above is only a preferred embodiment of the present invention and is not intended to limit the protection scope of the present invention.

Claims

1. A method for enhancing dental image data, characterized in that: The method includes the following steps: Receiving a plurality of dental image data with marking information, and dividing each of the dental image data into a first classification with significant disease characteristics and a second classification with insignificant disease characteristics according to the marking information; For each of the dental image data belonging to the second classification, extracting typical abnormal features from the dental image data, and using the typical abnormal features as enhancement conditions; Randomly selecting a batch of data from the first classification as the first real samples, the first generator generating a batch of first fake samples corresponding to the first real samples according to random noise, and the first discriminator and the first generator performing adversarial iteration based on the first real samples and the first fake samples to generate a plurality of first dental image enhancement data; Randomly selecting a batch of data from the second classification as the second real samples, the second generator generating a batch of second fake samples corresponding to the second real samples according to random noise, and the second discriminator and the second generator performing adversarial iteration based on the second real samples, the second fake samples and the enhancement conditions to generate a plurality of second dental image enhancement data; Combining each of the first dental image enhancement data, the second dental image enhancement data with each of the dental image data to obtain a dental image enhancement data set.

2. The dental image data enhancement method according to claim 1, characterized in that: Dividing each of the dental image data into a first classification with significant disease characteristics and a second classification with insignificant disease characteristics according to the marking information, including: Using a classifier to classify the type of lesion or injury and the severity of the lesion or injury of the teeth included in the marking information to obtain the significant level of the disease characteristics of the corresponding dental image data; Determining the dental image data with the significant level of the disease characteristics higher than the level threshold as the first classification with significant disease characteristics, and determining the dental image data with the significant level of the disease characteristics not higher than the level threshold as the second classification with insignificant disease characteristics.

3. The dental image data enhancement method according to claim 2, wherein: For each of the dental image data belonging to the second classification, extracting typical abnormal features from the dental image data, and using the typical abnormal features as enhancement conditions, including: Adopting a multi-scale feature extraction algorithm to extract features from each of the dental image data belonging to the second classification, and the extracted features include fine textures, local features, and overall structure features; Calculating the feature distribution differences of the fine textures, the local features, and the overall structure features in the corresponding dental image data and a plurality of normal dental images, and using the features with the feature distribution differences exceeding the preset significance threshold as the typical abnormal features; Using the quantified and encoded typical abnormal features as enhancement conditions.

4. A method for enhancing dental image data according to claim 3, characterized in that: Calculating the feature distribution differences of the fine textures, the local features, and the overall structure features in the corresponding dental image data and a plurality of normal dental images, including: Counting the sum of the quantities of the extracted fine textures and the local features, that is, the first quantity, matching the second quantity based on the first quantity, and randomly selecting the second quantity of normal dental images; Calculate the feature distribution differences of the fine texture, the local features, and the overall structural features in the corresponding tooth image data and each normal tooth image respectively, and calculate the average value of all the feature distribution differences.

5. A method for enhancing dental image data according to claim 1, characterized in that: The second generator generates a batch of second fake samples corresponding to the second real samples according to random noise. The second discriminator and the second generator perform adversarial iterations based on the second real samples, the second fake samples, and the enhancement conditions to generate a number of second tooth image enhancement data, including: Integrate the enhancement conditions into the input layer or the intermediate layer of the second generator. The second generator performs non-linear transformation and feature combination on the input random noise and the enhancement conditions through its internal multi-layer neural network structure to initially generate a batch of second fake samples; The second discriminator matches and discriminates the typical abnormal features in the second fake samples and the corresponding second real samples to determine whether they meet the enhancement conditions, and scores based on the judgment results; the second generator adjusts its network parameters based on the scores until the adjusted second fake samples that meet the enhancement conditions are generated; The second discriminator performs overall authenticity discrimination on the adjusted second fake samples, and feeds back the discrimination results to the second generator to further adjust its network parameters; Iterate the above process until the adversarial iteration termination condition is met. At this time, a batch of second fake samples generated by the second generator are the number of second tooth image enhancement data.

6. A dental image data enhancement device, characterized in that: The device includes a receiving module, an enhancement condition generation module, and an image enhancement module; The receiving module receives a number of tooth image data with marking information, and divides each tooth image data into a first classification with significant disease characteristics and a second classification with insignificant disease characteristics according to the marking information; The enhancement condition generation module extracts the typical abnormal features from each tooth image data belonging to the second classification, and uses the typical abnormal features as the enhancement conditions; The image enhancement module randomly selects a batch of data from the first classification as the first real samples. The first generator generates a batch of first fake samples corresponding to the first real samples according to random noise. The first discriminator and the first generator perform adversarial iterations based on the first real samples and the first fake samples to generate a number of first tooth image enhancement data; Randomly select a batch of data from the second classification as the second real samples. The second generator generates a batch of second fake samples corresponding to the second real samples according to random noise. The second discriminator and the second generator perform adversarial iterations based on the second real samples, the second fake samples, and the enhancement conditions to generate a number of second tooth image enhancement data; Combine each of the first tooth image enhancement data, the second tooth image enhancement data, and each of the tooth image data to obtain a tooth image enhancement data set.

7. The dental image data enhancement device according to claim 6, wherein: The receiving module is specifically used for: Use a classifier to classify the type of tooth lesion or injury and the severity of the lesion or injury included in the marking information to obtain the significant level of the disease characteristics of the corresponding tooth image data; The dental image data with the disease feature significance level higher than the level threshold is determined as the first classification with significant disease features, and the dental image data with the disease feature significance level not higher than the level threshold is determined as the second classification with insignificant disease features.

8. A terminal device, comprising: At least one processor, a memory, and a computer program stored in the memory and executable on the at least one processor, wherein: the processor executes the computer program to implement the method according to any one of claims 1-5.

9. A storage medium storing a computer program, characterized in that: The computer program is executed by a processor to implement the method according to any one of claims 1-5.

10. A computer program product, when the computer program product is run by a terminal device, is characterized in that: So as to implement the method according to any one of claims 1-5.

Images