Resnet-transformer-based automatic detection system and method for malformation central cusp

Abstract

The application discloses an automatic detection system and method for malformation central cusp based on ResNet-Transformer, relates to the technical field of tooth morphological dysplasia detection, and is suitable for early screening of tooth morphological dysplasia. The system comprises an image acquisition module, a preprocessing unit, a feature extraction module based on ResNet-18 and a classification module based on Transformer. Through preprocessing, feature extraction and classification on the target region in the panoramic film of a patient, the system can accurately identify the malformation central cusp. The feature extraction module extracts the deep features in the image by using ResNet-18, the classification module classifies the features based on the self-attention mechanism of Transformer, and the visualization of model decision is realized by combining Grad-CAM, so that the accuracy and consistency of diagnosis are improved. The system can effectively reduce the missed detection rate of artificial film reading and the subjectivity of diagnosis, is especially suitable for the high-load clinical environment of the department of pediatric dentistry, and has a wide application prospect.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of tooth morphological dysplasia detection, in particular to an automatic detection system and method for dens evaginatus based on ResNet-Transformer. BACKGROUND

[0002] Dens evaginatus is a kind of tooth morphological dysplasia, mainly manifested as a conical dental cusp protruding from the buccal triangular ridge of the central fossa or near the central fossa of the premolar. The prevalence rate of Asian people is 0.5% to 4.3%. Dens evaginatus is often asymptomatic, so it is easy to be ignored. If not diagnosed and treated in time, dens evaginatus may cause pulp exposure due to wear or breakage, leading to pulp infection and apical inflammation, and further affecting the oral health of patients. In addition, since most of the broken teeth with dens evaginatus are young permanent teeth, they often affect the normal development of the roots, causing short roots and thin root canal walls, leading to root fracture and even the inability to retain the affected teeth. Although there are treatment methods such as root apex induction in the current clinical practice, there are certain limitations, and the affected teeth cannot develop normally. Therefore, early detection and early treatment are crucial for preventing dens evaginatus breakage and protecting healthy pulp. If the existence of dens evaginatus can be found during the tooth germ period, timely intervention can be made after the affected tooth erupts, and the serious consequences caused by dens evaginatus breakage can be avoided to the greatest extent.

[0003] At present, the examination of tooth germs by doctors focuses more on whether the number of tooth germs is abnormal, and the screening of tooth germ morphology development is often neglected, resulting in missed detection of dens evaginatus in tooth germs. In China, due to the shortage of children's oral medical resources and the heavy pressure on doctors, this phenomenon of missed detection is more prominent. For many Chinese children, due to the limited number of oral examinations, regular check-ups are not possible, and in this limited examination, the missed detection of dens evaginatus in tooth germs cannot provide early warning to doctors and parents, which may lead to the failure to take timely measures when the permanent tooth erupts, resulting in the breakage of dens evaginatus and the occurrence of pulp infection, and causing serious consequences.

[0004] The imaging diagnosis of dens evaginatus requires doctors to have rich clinical experience, and only after careful analysis and identification of X-ray films can correct diagnosis be made. However, image quality, excessive number of images causing fatigue during reading, and personal subjectivity can all affect the judgment of doctors on the images of dens evaginatus, and there may be large differences in the diagnosis results between different doctors, making it difficult to ensure the consistency of diagnosis. Especially in the case of weak morphological changes or unclear features, doctors may make different judgments, increasing the uncertainty of the diagnosis results. In addition, there are large differences in medical level in different regions, and the medical level in remote areas is backward. It is of great significance to bridge the gap and enable children in remote areas to enjoy high-quality medical services.

[0005] With the rapid development of artificial intelligence technology, especially the successful application of deep learning in image processing, medical image analysis gradually introduces deep learning models to improve the automation and accuracy of diagnosis. Deep learning models, especially convolutional neural networks (CNN), have shown good results in radiological imaging, pathological detection and other fields. However, existing single convolutional neural networks have certain limitations in processing complex image structures, and it is difficult to effectively capture long-distance dependencies in images. Therefore, models relying solely on CNNs may have insufficient recognition accuracy when dealing with morphological abnormalities such as central cusp.

[0006] In addition, another problem faced by the introduction of artificial intelligence technology is the "black box" characteristic. Most deep learning models cannot explain their decision-making process, and doctors cannot intuitively understand the basis for the model's judgment, making it difficult for the model to gain the full trust of doctors in clinical applications. In recent years, the emergence of some visualization techniques such as Grad-CAM (Gradient-weighted Class Activation Mapping) has provided an important perspective for explaining the model's decision-making process

[0007] To solve the above problems, the present application proposes a ResNet-Transformer-based automatic detection system and method for central cusp abnormalities. SUMMARY

[0008] The present application aims to address the shortcomings of the prior art by providing a ResNet-Transformer-based automatic detection system and method for central cusp abnormalities to solve the problems raised in the background art.

[0009] To achieve the above-mentioned purpose, the present application provides the following technical solution: a ResNet-Transformer-based automatic detection system for central cusp abnormalities, characterized by comprising the following modules:

[0010] Image acquisition module: used to acquire panoramic X-ray images of the patient's oral cavity;

[0011] Preprocessing unit: used to adjust the size and normalize the acquired images;

[0012] Feature extraction module: based on ResNet-18 architecture, deep features are extracted from images through multiple convolutional layers;

[0013] Classification module: based on the Transformer architecture, the features are input into the multi-head self-attention mechanism for processing to determine whether there is a central cusp abnormality.

[0014] As a preferred technical scheme of the present application, the image resolution acquired by the image acquisition module is 2904*1536 pixels, and the image is taken by an oral panoramic X-ray machine with a tube voltage of 80 kV and a tube current of 10 mA after the patient performs head position standard placement.

[0015] As a preferred technical scheme of the present application, the preprocessing unit adjusts the size of the image, adjusts each region of interest (ROI) to 130*110 pixels, and performs normalization processing to map the pixel value to the range of 0 to 1.

[0016] As a preferred technical scheme of the present application, the feature extraction module adopts a ResNet-18 architecture, including multiple convolutional layers, for mapping input data to a latent space to generate a feature map with a size of 4*5 and a channel number of 512.

[0017] As a preferred technical scheme of the present application, the classification module includes the following algorithm steps: for the feature map F from ResNet-18 with a size of C*H*W (where C=512, H=4, W=5), each channel is regarded as a "word" in a sequence, and the spatial dimension is flattened into a one-dimensional vector, and the specific steps are as follows:

[0018] F c =flatten(F[c,:,:])

[0019] where c∈{1,2,…,512}, the two-dimensional array F[c,:,:] is reshaped into a 20-dimensional vector.

[0020] As a preferred technical scheme of the present application, the matrix size of the flattened feature input sequence T=[F1,F2,…,F 512 ] is 512*20, each row corresponds to a token, and each token is composed of a 20-dimensional vector obtained by flattening the spatial dimension of the corresponding channel, and the Transformer module processes the sequence through a multi-head self-attention mechanism.

[0021] As a preferred technical scheme of the present application, the classification module finally outputs a classification result of a target class, and the classification layer includes a fully connected layer with a Softmax activation function for judging the existence of a central cusp.

[0022] The present application also proposes a ResNet-Transformer-based automatic detection method for central cusp, which includes the following steps:

[0023] Obtain the oral panoramic X-ray image of the patient and perform preprocessing;

[0024] The preprocessed image is input into a feature extraction module to generate a feature map through a ResNet-18 architecture;

[0025] The extracted features are classified using a Transformer architecture, and a judgment result of whether a malformation central cusp exists is output.

[0026] As a preferred technical solution of the application, in the feature extraction module, the size of the feature map generated by the ResNet-18 model is 4x5, the number of channels is 512, and after flattening and conversion, an input sequence T of the Transformer is formed:

[0027] T = [F1, F2, …, F 512 ]

[0028] Wherein, F i represents a 20-dimensional vector after flattening.

[0029] As a preferred technical solution of the application, in the detection method, a training configuration with a batch size of 32, a learning rate of 0.001, and a cross-entropy loss function is used, and through model training on panoramic X-ray images of 1410 patients, high sensitivity and specificity detection of malformation central cusp is realized.

[0030] Compared with the prior art, the application has the following advantages:

[0031] The application develops an automatic detection system based on ResNet-Transformer. The system combines the advantages of convolutional neural network (CNN) and Transformer model, not only can extract the spatial features of the image, but also can effectively capture the long-distance dependence relationship in the image through the self-attention mechanism of the Transformer, thereby improving the accuracy and specificity of the detection. The application uses ResNet-18 architecture for feature extraction, and then classifies the extracted features through the Transformer module, effectively making up for the shortcomings of single CNN model in detecting long-distance dependence relationship. In addition, the Grad-CAM technology is used to generate a visual heat map, so that doctors can understand the key areas that the model focuses on, enhance the understanding of the model decision, and thus improve the credibility of the model in clinical application. BRIEF DESCRIPTION OF DRAWINGS

[0032] Figure 1 The system block diagram of the automatic detection system and method of malformation central cusp based on ResNet-Transformer proposed by the application;

[0033] Figure 2 The method flowchart of the automatic detection system and method of malformation central cusp based on ResNet-Transformer proposed by the application. DETAILED DESCRIPTION

[0034] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by a person of ordinary skill in the art without creative work fall within the protection scope of the present application.

[0035] The specific embodiments of the present application are described in detail below with reference to the drawings in conjunction with the embodiments of the present application. Figure 1 、 Figure 2 The specific embodiments of the present application are described in detail below with reference to the drawings in conjunction with the embodiments of the present application.

[0036] Embodiment 1: Standard image preprocessing and detection

[0037] Step 1: Image acquisition and preprocessing

[0038] The panoramic X-ray image of the patient's oral cavity is acquired using a Veraviewepocs panoramic X-ray machine, with a resolution of 2904x1536 pixels, a tube voltage of 80 kilovolts, and a current of 10 milliamperes.

[0039] The preprocessing of each panoramic X-ray image includes:

[0040] Image cropping, extracting the premolar region as the region of interest (ROI).

[0041] Adjust the ROI to 130x110 pixels in size and perform normalization to map the pixel values to the range of 0 to 1.

[0042] Step 2: Deep feature extraction

[0043] The preprocessed ROI image is input into the ResNet-18 feature extraction module. The multi-layer convolutional layers in the ResNet-18 model convert the input image into a deep feature map, with the output feature map size being 4x5 and the channel number being 512.

[0044] Step 3: Feature flattening and classification

[0045] The feature map generated by ResNet-18 is flattened to obtain a 20-dimensional vector representation:

[0046] F c = flatten(F[ c ,:,:])

[0047] where c∈{1,2,…,512}. This generates 512 flattened vectors T = [F1, F2, …, F 512 ], forming an input matrix with a size of 512x20.

[0048] The matrix is input into the Transformer classification module, the correlation between channels is captured through multi-head self-attention mechanism, and finally the judgment result of whether there is a malformation central cusp is output through a fully connected layer.

[0049] Effect: This embodiment realizes accurate detection of malformation central cusp, and the system sensitivity reaches 0.8600, the specificity reaches 0.9200, and the F1 score is 0.8866.

[0050] Embodiment 2: Early detection of malformation central cusp for screening of abnormal tooth morphology development in children

[0051] Background: This embodiment is specially designed for early tooth morphology development screening of children aged 3-12 years old. In the tooth germ development stage, the image discrimination of malformation central cusp is a great challenge, and a timely automatic detection system can help doctors to detect malformation central cusp early, so as to take timely preventive intervention and avoid its breakage.

[0052] Step 1: X-ray image acquisition and processing in tooth germ stage

[0053] Use standard panoramic radiographic image acquisition process to obtain panoramic radiograph of children's oral cavity, and focus on the tooth germ of premolar.

[0054] Crop and adjust the image, extract the target ROI region, and adjust its size to 130x110 pixels and normalize the pixel value.

[0055] Step 2: Deep learning assisted detection

[0056] Use the pre-trained ResNet-18 and Transformer hybrid model to extract features and classify the ROI region in the tooth germ stage. The feature map generated by ResNet-18 is processed by the Transformer classification module to generate classification judgment for each ROI region.

[0057] Effect: In this process, through deep learning assistance, early warning detection of malformation central cusp of unerupted teeth can be realized, which alerts parents and doctors to take preventive measures in time after the affected tooth erupts, avoiding the risk of pulp infection and root development stagnation caused by malformation central cusp breakage. At the same time, with the help of the model, doctors reduce the missed detection rate of malformation central cusp, and improve the consistency and accuracy of diagnosis.

[0058] Embodiment 3: Model decision visualization combined with Grad-CAM

[0059] Background: Due to the "black box" nature of deep learning models, Grad-CAM (Gradient-weighted Class Activation Mapping) tool is introduced in this embodiment to visualize the decision-making process of the model. In this way, it can help doctors understand the specific areas that the model focuses on when detecting malposed central cusps.

[0060] Step 1: Grad-CAM heat map generation

[0061] For X-ray images that have detected malposed central cusps, Grad-CAM generates a heat map. This heat map highlights the areas that the model considers most important for classification decision, and presents them to the doctor in an image overlay manner.

[0062] Step 2: Clinical decision assistance

[0063] When doctors view images that the system has determined to be malposed central cusps, they can refer to the Grad-CAM heat map to confirm whether the specific cusp areas that the system focuses on are consistent with actual judgment, thereby enhancing trust and diagnostic consistency.

[0064] Effect: This embodiment plays an important role in improving diagnostic accuracy and transparency. The application of Grad-CAM enables doctors to intuitively understand the basis of model decision-making, thereby reducing the possibility of misjudgment.

[0065] Embodiment 4: Automatic detection and automatic screening workflow

[0066] Background: In some resource-limited clinical environments, doctors have a heavy burden. This embodiment demonstrates an automatic processing mode that improves work efficiency through automatic screening and detection.

[0067] Step 1: Automatic image input

[0068] Input the panoramic film image data of the patient into the system, automatically locate the premolar germ region, and perform size adjustment and normalization through the preprocessing unit.

[0069] Step 2: Automatic detection and screening

[0070] All preprocessed ROI regions enter the ResNet-Transformer model at the same time, and the system outputs the judgment result of whether there is a malposed central cusp in each panoramic film image. The detection result is displayed to the doctor through the interface, and the doctor only needs to review the abnormal images marked by the system.

[0071] Effect: In this mode, the system realizes rapid screening and improves clinical work efficiency. Batch detection not only improves workflow efficiency, but also ensures high accuracy, especially suitable for high-load dental clinics.

[0072] Embodiment 5: Model verification of multi-center dataset

[0073] Background: To verify the generalization and robustness of the model, this example tests panoramic films from different sources, including multiple medical center datasets, ensuring that the model performs well in diverse data.

[0074] Step 1: Data collection and preprocessing

[0075] Panoramic films collected from different medical centers, including images from different devices and imaging parameters, are uniformly converted to a standard format, with ROI size and pixel normalization adjusted to ensure input consistency.

[0076] Step 2: Model detection and verification

[0077] Batch input images from multi-center datasets into the ResNet-Transformer model to verify the performance indicators, including sensitivity, specificity, positive predictive value, negative predictive value, F1 score, and AUC.

[0078] Effect: This example shows the generalization and robustness of the model, with excellent detection results in different data sources. Through multi-center testing, the system is suitable for different medical institutions and has strong practicality.

[0079] The content not described in detail in this specification belongs to the existing technology known to those skilled in the art, although the present invention has been described in detail with reference to the foregoing examples, those skilled in the art can still modify the technical solutions recorded in the foregoing examples, or make equivalent replacement for part of the technical features, any modification, equivalent replacement, improvement, etc. within the spirit and principles of the present invention shall be included within the protection scope of the present invention.

Claims

1. An automatic detection system for malformation central cusp based on ResNet-Transformer, characterized in that, The system comprises the following modules: An image acquisition module for acquiring a panoramic image of a patient's oral cavity; A preprocessing unit for resizing and normalizing the acquired image; A feature extraction module based on the ResNet-18 architecture, which extracts feature vectors from the image through multiple convolutional layers; The feature extraction module adopts the ResNet-18 architecture, which includes multiple convolutional layers for mapping input data to a latent space and generating a feature map with a size of 4x5 and a channel number of 512; The classification module: based on the Transformer architecture, input the features into the multi-head self-attention mechanism for processing to determine whether there is a malformation central cusp; the classification module includes the following algorithm steps: inputting the feature map from the ResNet-18 into the multi-head self-attention mechanism , the size of which is (Wherein , , ), each channel is regarded as a "word" in a sequence, and the spatial dimension is flattened into a one-dimensional vector, and the specific steps are as follows: ； wherein The two-dimensional array is reshaped into a 20-dimensional vector; Flattened sequence of feature inputs The matrix size is Each row corresponds to a token, each token consisting of a 20-dimensional vector resulting from flattening the spatial dimensions of the corresponding channel. The Transformer module processes the sequence through a multi-head self-attention mechanism, and the classification module finally outputs a classification result of a target class, with the classification layer including a fully connected layer with a Softmax activation function for determining the presence or absence of a central sharp deformity.

2. The ResNet-Transformer based automatic detection system of malformation central cusp according to claim 1, wherein, The image acquisition module acquires an image with a resolution of 2904x1536 pixels, which is captured by an oral panoramic X-ray machine with a tube voltage of 80 kV and a tube current of 10 mA after the patient's head position is properly placed.

3. The ResNet-Transformer based automatic detection system of malformation central cusp according to claim 1, wherein, The preprocessing unit resizes each region of interest (ROI) to 130x110 pixels and normalizes the image to map pixel values to the range of 0 to 1.

4. The ResNet-Transformer-based automatic detection method of the central sharp deformity according to claim 1, characterized in that, In the feature extraction module, the feature map generated by the ResNet-18 model has a size of 4x5 and a channel number of 512, and after flattening and conversion, an input sequence of the Transformer is formed : ； wherein, represents the flattened 20-dimensional vector.

5. An automatic detection method for central sharp deformity based on the system of any one of claims 1-4, comprising the following steps: Acquire a panoramic X-ray image of a patient's oral cavity and preprocess it; Input the preprocessed image into the feature extraction module to generate a feature map through the ResNet-18 architecture; Classify the extracted features using the Transformer architecture and output the judgment result of whether there is a central sharp deformity.

6. The ResNet-Transformer-based automatic detection method of the central sharp deformity according to claim 5, characterized in that, In the detection method, the training configuration with a batch size of 32, a learning rate of 0.001, and a cross-entropy loss function is used to train the model on 1410 panoramic X-ray images of patients, achieving high sensitivity and specificity in detecting central sharp deformities.

Images