ResNet-Transformer-based automatic detection system and method for deformed central tip

Abstract

The invention discloses an automatic abnormal central tip detection system and method based on ResNet-Transformer, relates to the technical field of detection of abnormal tooth morphological development, and is suitable for early screening of abnormal tooth morphological development. The system comprises an image acquisition module, a preprocessing unit, a feature extraction module based on ResNet-18 and a classification module based on Transform. Through preprocessing, feature extraction and classification of the target area in the patient panorama, the system can accurately identify the malformed central tip. The feature extraction module is used for extracting depth features in an image by using ResNet-18, the classification module is used for classifying the features based on a self-attention mechanism of Transform, and visualization of model decision is realized in combination with Grad-CAM, so that the accuracy and consistency of diagnosis are improved. The system can effectively reduce the omission ratio and diagnosis subjectivity of manual film reading, is especially suitable for a high-load clinical environment of the department of stomatology of children, and has a wide application prospect.

Description

Technical Field

[0001] The present invention relates to the technical field of tooth morphological developmental abnormality detection, and specifically to a system and method for automatically detecting a deformed central cusp based on ResNet-Transformer. Background Art

[0002] Dens Evaginatus is a type of tooth morphological developmental abnormality, which is mainly manifested as a conical cusp protruding from the central fossa of the premolar or the triangular ridge of the buccal cusp near the central fossa. The prevalence rate in Asians is 0.5% to 4.3%. Dens Evaginatus often has no obvious symptoms, so it is easy to be ignored. If not diagnosed and treated in time, the deformed central cusp may cause pulp exposure due to wear or fracture, leading to pulp infection and apical inflammation, which will have an adverse effect on the patient's oral health. In addition, since most teeth with deformed central cusp fractures are young permanent teeth, it often affects the normal development of the tooth root, resulting in short roots and thin root canal walls, leading to root fractures and even the inability to retain the affected tooth. Although there are treatment methods such as apex induction in clinical practice, they all have certain limitations and cannot make the affected teeth develop like normal teeth. Therefore, early detection and early treatment are crucial to preventing the fracture of the deformed central cusp and protecting healthy dental pulp. If the presence of the deformed central cusp can be found in the tooth germ stage, timely intervention can be made after the affected tooth erupts, thus avoiding the serious consequences caused by the fracture of the deformed central cusp to the greatest extent.

[0003] At present, doctors focus more on whether the number of tooth germs is abnormal when examining tooth germs, and often ignore the screening of tooth germ morphological development, which leads to missed detection of deformed central cusps in tooth germs. In China, due to the shortage of children's oral medical resources and the high pressure on doctors to see patients, this phenomenon of missed detection is more prominent. For many Chinese children, the number of oral examinations is very limited and regular examinations are not possible. In these limited examinations, missed detection of deformed central cusps in tooth germs will not be able to warn doctors and parents, which may result in failure to promptly handle the eruption of permanent teeth, leading to fracture of deformed central cusps and pulp infection, with serious consequences.

[0004] The imaging identification of the deformed central cusp requires doctors to have rich clinical experience, and a correct diagnosis can only be made after careful analysis and identification of the X-ray. However, image quality, reading fatigue caused by too many images, and personal subjectivity can affect the doctor's judgment of the deformed central cusp image. The diagnostic results between different doctors may vary greatly, and diagnostic consistency is difficult to ensure. Especially when the morphological changes are subtle or the features are not obvious, doctors may make different judgments, which increases the uncertainty of the diagnostic results. In addition, the medical level varies greatly in different regions, and the medical level in remote areas is backward. How to level the baseline so that children in remote areas can enjoy high-quality medical services is of great significance.

[0005] With the rapid development of artificial intelligence technology, especially the successful application of deep learning in image processing, deep learning models are gradually introduced into medical image analysis to improve the automation and accuracy of diagnosis. Deep learning models, especially convolutional neural networks (CNNs), have shown good results in the fields of radiological imaging and pathological detection. However, the existing single convolutional neural network has certain limitations in processing complex image structures and it is difficult to effectively capture long-distance dependencies in the image. Therefore, models that rely only on CNN may have insufficient recognition accuracy when dealing with morphological abnormality detection tasks such as deformed central cusps.

[0006] In addition, another problem faced by the introduction of artificial intelligence technology is the "black box" feature. Most deep learning models cannot explain their decision-making process, and doctors cannot intuitively understand the basis for the model's judgment, which makes it difficult for the model to gain full trust from doctors in clinical applications. In recent years, the emergence of some visualization technologies such as Grad-CAM (Gradient Weighted Class Activation Mapping) has provided an important perspective for explaining the model's decision-making process.

[0007] To solve the above problems, the present invention proposes a system and method for automatic detection of deformed central cusp based on ResNet-Transformer to solve the above problems. Summary of the invention

[0008] The purpose of the present invention is to address the deficiencies of the prior art and provide a system and method for automatic detection of deformed central cusp based on ResNet-Transformer to solve the problems raised in the background technology.

[0009] To achieve the above object, the present invention provides the following technical solution: a deformed central cusp automatic detection system based on ResNet-Transformer, characterized in that it includes the following modules:

[0010] Image acquisition module: used to acquire the patient's oral panoramic X-ray image;

[0011] Preprocessing unit: used to resize and normalize the collected images;

[0012] Feature extraction module: Based on the ResNet-18 architecture, it extracts deep features from images through multiple convolutional layers;

[0013] Classification module: Based on the Transformer architecture, the features are input into the multi-head self-attention mechanism for processing to determine whether there is a deformed central cusp.

[0014] As a preferred technical solution of the present invention, the image acquisition module acquires an image with a resolution of 2904×1536 pixels. After the patient's head is placed in a standard position, the image is taken by an oral panoramic X-ray machine with a tube voltage of 80 kV and a tube current of 10 mA.

[0015] As a preferred technical solution of the present invention, the preprocessing unit resizes the image, adjusts each region of interest (ROI) to a size of 130×110 pixels, and performs normalization processing to map pixel values ​​to a range of 0 to 1.

[0016] As a preferred technical solution of the present invention, the feature extraction module adopts the ResNet-18 architecture, including multiple layers of convolutional layers, which are used to map the input data to the latent space and generate a feature map with a size of 4×5 and a channel number of 512.

[0017] As a preferred technical solution of the present invention, the classification module includes the following algorithm steps: for the feature map F from ResNet-18, whose size is C×H×W (where C=512, H=4, W=5), each channel is regarded as a "word" in the sequence, and its spatial dimension is flattened into a one-dimensional vector. The specific steps are:

[0018] F c = flatten(F[c,:,:])

[0019] Where c∈{1,2,…,512}, reshape the two-dimensional array F[c,:,:] into a 20-dimensional vector.

[0020] As a preferred technical solution of the present invention, the flattened feature input sequence T=[F 1 ,F 2 ,…,F 512 ] has a matrix size of 512×20, each row corresponds to a token, and each token is composed of a 20-dimensional vector obtained by flattening the spatial dimension of the corresponding channel. The Transformer module processes the sequence through a multi-head self-attention mechanism.

[0021] As a preferred technical solution of the present invention, the classification module finally outputs a classification result of a target category, and its classification layer includes a fully connected layer with a Softmax activation function, which is used to judge whether the deformed central cusp exists or not.

[0022] The present invention also proposes a method for automatically detecting a deformed central cusp based on ResNet-Transformer, comprising the following steps:

[0023] Obtain and pre-process the patient's oral panoramic X-ray images;

[0024] The preprocessed image is input into the feature extraction module to generate feature maps through the ResNet-18 architecture;

[0025] The Transformer architecture is used to classify the extracted features and output the judgment result of whether there is a deformed central cusp.

[0026] As a preferred technical solution of the present invention, in the feature extraction module, the size of the feature map generated by the ResNet-18 model is 4×5, the number of channels is 512, and after flattening and conversion, the input sequence T of the Transformer is formed:

[0027] T=[F 1 ,F 2 ,…,F 512 ]

[0028] Among them, F i Represents a flattened 20-dimensional vector.

[0029] As a preferred technical solution of the present invention, in the detection method, a training configuration with a batch size of 32, a learning rate of 0.001, and a loss function of cross entropy loss is adopted. By training the model on panoramic X-ray images of 1,410 patients, high sensitivity and specificity detection of the deformed central cusp is achieved.

[0030] Compared with the prior art, the present invention has the following beneficial effects:

[0031] The present invention has developed an automatic detection system based on ResNet-Transformer. The system combines the advantages of convolutional neural networks (CNN) and Transformer models, and can not only extract the spatial features of images, but also effectively capture long-distance dependencies in images through the self-attention mechanism of Transformer, thereby improving the accuracy and specificity of detection. The present invention uses the ResNet-18 architecture for feature extraction, and then classifies the extracted features through the Transformer module, which effectively makes up for the shortcomings of a single CNN model in detecting long-distance dependencies. In addition, by generating a visual heat map through Grad-CAM technology, doctors can understand the key areas of concern of the model and enhance their understanding of model decisions, thereby improving its credibility in clinical applications. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 This is a system block diagram of the ResNet-Transformer-based deformed central cusp automatic detection system and method proposed in the present invention;

[0033] Figure 2This is a flow chart of the method for the automatic detection system and method of deformed central cusp based on ResNet-Transformer proposed in the present invention. DETAILED DESCRIPTION

[0034] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0035] The following is combined with Figure 1 , Figure 2 With reference to the accompanying drawings and a number of embodiments, specific embodiments of the present invention are described in detail.

[0036] Example 1: Standard image preprocessing and detection

[0037] Step 1: Image acquisition and preprocessing

[0038] A Veraviewepocs panoramic X-ray machine was used to acquire oral panoramic X-ray images of the patients, with a resolution of 2904 × 1536 pixels, a tube voltage of 80 kV, and a current of 10 mA.

[0039] Pre-process each panoramic X-ray image, including:

[0040] The image was cropped and the premolar area was extracted as the region of interest (ROI).

[0041] The ROI was resized to 130 × 110 pixels and normalized to map pixel values ​​to the range of 0 to 1.

[0042] Step 2: Deep feature extraction

[0043] The preprocessed ROI image is input into the ResNet-18 feature extraction module. The multi-layer convolutional layers in the ResNet-18 model convert the input image into a deep feature map, and the output feature map size is 4×5 and the number of channels is 512.

[0044] Step 3: Feature Flattening and Classification

[0045] The feature map generated by ResNet-18 is flattened to obtain a 20-dimensional vector representation:

[0046] F c = flatten(F[ c ,:,:])

[0047] where c∈{1,2,…,512}. This generates 512 flattened vectors T=[F 1 ,F 2 ,…,F 512 ], forming an input matrix of size 512x20.

[0048] The matrix is ​​input into the Transformer classification module, and the correlation between channels is captured through the multi-head self-attention mechanism. Finally, the judgment result of whether there is a deformed central cusp is output through the fully connected layer.

[0049] Effect: This embodiment achieves accurate detection of the deformed central cusp, with a system sensitivity of 0.8600, a specificity of 0.9200, and an F1 score of 0.8866.

[0050] Example 2: Early detection of deformed central cusp for screening of abnormal tooth morphology in children

[0051] Background: This embodiment is specifically for early tooth morphology development screening for children aged 3 to 12 years. During the tooth germ development stage, the image identification of the deformed central cusp is quite challenging. A timely automatic detection system can help doctors detect the deformed central cusp early, thereby performing timely preventive intervention to avoid its fracture.

[0052] Step 1: X-ray image acquisition and processing at the tooth germ stage

[0053] Panoramic radiographs of the child's oral cavity were obtained using a standard panoramic radiographic acquisition process, focusing on the tooth germs of the premolars.

[0054] The image was cropped and adjusted, the target ROI area was extracted, and its size was adjusted to 130 × 110 pixels, and the pixel values ​​were normalized.

[0055] Step 2: Deep learning-assisted detection

[0056] The pre-trained ResNet-18 and Transformer hybrid model was used to extract features and classify the ROI region at the tooth germ stage. The feature map generated by ResNet-18 was processed by the Transformer classification module to generate classification judgments for each ROI region.

[0057] Effect: In this process, deep learning can be used to detect the deformed central cusp of unerupted teeth, and parents and doctors can be warned to take preventive measures in time after the eruption of the affected teeth to avoid risks such as pulp infection and root development stagnation caused by the fracture of the deformed central cusp. At the same time, with the help of this model, doctors have reduced the missed detection rate of the deformed central cusp and improved the consistency and accuracy of diagnosis.

[0058] Example 3: Model decision visualization combined with Grad-CAM

[0059] Background: Due to the "black box" characteristics of deep learning models, the Grad-CAM (gradient weighted class activation mapping) tool is introduced in this embodiment to visualize the decision-making process of the model. In this way, it can help doctors understand the specific areas that the model focuses on when detecting the central apex of the deformity.

[0060] Step 1: Grad-CAM heatmap generation

[0061] For X-ray images with detected deformed central cusps, Grad-CAM generates a heat map that highlights the areas that the model considers most important for classification decisions and presents them to doctors as an image overlay.

[0062] Step 2: Clinical decision support

[0063] When doctors review images that the system has determined to be malformed central cusps, they can refer to the Grad-CAM heat map to confirm whether the specific cusp area that the system focuses on is consistent with the actual judgment, thereby enhancing trust and diagnostic consistency.

[0064] Effect: This embodiment plays an important role in improving diagnostic accuracy and transparency. The application of Grad-CAM enables doctors to intuitively understand the basis of model decisions, thereby reducing the possibility of misjudgment.

[0065] Example 4: Automatic detection and automatic screening workflow

[0066] Background: In some clinical settings with limited resources, doctors are under heavy burden. This embodiment demonstrates the automatic processing mode to improve work efficiency through automatic screening and detection.

[0067] Step 1: Automatic Image Input

[0068] The patient's panoramic radiographic image data is input into the system, the premolar tooth germ area is automatically located, and the data is resized and normalized through the preprocessing unit.

[0069] Step 2: Automated detection and screening

[0070] All preprocessed ROI areas enter the ResNet-Transformer model at the same time, and the system outputs the judgment results of whether all premolars in each panoramic image have deformed central cusps. The test results are displayed to the doctor through the interface, and the doctor only needs to review the abnormal images marked by the system.

[0071] Effect: In this mode, the system achieves rapid screening and improves clinical work efficiency. Batch testing not only improves workflow efficiency but also ensures high accuracy, which is particularly suitable for high-load dental clinics.

[0072] Example 5: Model validation on a multicenter dataset

[0073] Background: To verify the generalization and robustness of the model, this embodiment tests panoramic images from different sources, including datasets from multiple medical centers, to ensure that the model has good performance in diverse data.

[0074] Step 1: Data collection and preprocessing

[0075] Panoramic images collected from different medical centers, including images with different equipment and imaging parameters, were uniformly converted into a standard format, and the ROI size and pixel normalization were adjusted to ensure the consistency of the input.

[0076] Step 2: Model testing and verification

[0077] The images in the multicenter dataset were batch-inputted into the ResNet-Transformer model to verify the performance indicators of the model, including sensitivity, specificity, positive predictive value, negative predictive value, F1 score, and AUC.

[0078] Effect: This example shows the generalization and robustness of the model, and the detection effect is excellent in different data sources. Through multi-center testing, this system is suitable for different medical institutions and has strong practicality.

[0079] The contents not described in detail in this specification belong to the prior art known to professional and technical personnel in the field. Although the present invention has been described in detail with reference to the aforementioned embodiments, it is still possible for those skilled in the art to modify the technical solutions described in the aforementioned embodiments, or to make equivalent substitutions for some of the technical features therein. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. The ResNet-Transformer-based automatic detection system for deformed central cusps is characterized by: Includes the following modules: Image acquisition module: used to acquire the patient's oral panoramic film images; Preprocessing unit: used to resize and normalize the collected images; Feature extraction module: Based on the ResNet-18 architecture, feature vectors are extracted from images through multiple convolutional layers; Classification module: Based on the Transformer architecture, the features are input into the multi-head self-attention mechanism for processing to determine whether there is a deformed central cusp.

2. The ResNet-Transformer-based automatic detection system for deformed central cusp according to claim 1, characterized in that: The image acquisition module acquires an image with a resolution of 2904×1536 pixels. The image is taken by an oral panoramic X-ray machine with a tube voltage of 80 kV and a tube current of 10 mA after the patient's head is placed in a standard position.

3. The ResNet-Transformer-based automatic detection system for deformed central cusp according to claim 1, characterized in that: The preprocessing unit resizes the image, adjusts the size of each region of interest (ROI) to 130×110 pixels, and performs normalization processing to map the pixel values ​​to a range of 0 to 1.

4. The ResNet-Transformer-based automatic detection system for deformed central cusp according to claim 1, characterized in that: The feature extraction module adopts the ResNet-18 architecture, including multiple convolutional layers, which is used to map the input data to the latent space and generate a feature map with a size of 4×5 and 512 channels.

5. The ResNet-Transformer-based automatic detection system for deformed central cusp according to claim 1, characterized in that: The classification module includes the following algorithmic steps: for the feature map F from ResNet-18, whose size is C×H×W (where C=512, H=4, W=5), each channel is regarded as a "word" in the sequence, and its spatial dimension is flattened into a one-dimensional vector. The specific steps are: F c =flatten(F[c,:,:]) Where c∈{1,2,…,512}, reshape the two-dimensional array F[c,:,:] into a 20-dimensional vector.

6. The ResNet-Transformer-based automatic detection system for deformed central cusp according to claim 5, characterized in that: The flattened feature input sequence T = [F1, F2, ..., F 512 ] has a matrix size of 512×20, each row corresponds to a token, and each token is composed of a 20-dimensional vector obtained by flattening the spatial dimension of the corresponding channel. The Transformer module processes the sequence through a multi-head self-attention mechanism.

7. The ResNet-Transformer-based automatic detection system for deformed central cusp according to claim 6, characterized in that: The classification module finally outputs a classification result of a target category, and its classification layer includes a fully connected layer with a Softmax activation function, which is used to judge whether the deformed central cusp exists or not.

8. The ResNet-Transformer-based automatic detection method for the deformed central cusp includes the following steps: Obtain and pre-process the patient's oral panoramic X-ray images; The preprocessed image is input into the feature extraction module to generate feature maps through the ResNet-18 architecture; The Transformer architecture is used to classify the extracted features and output the judgment result of whether there is a deformed central cusp.

9. The ResNet-Transformer-based automatic detection method for deformed central cusp according to claim 8, characterized in that: In the feature extraction module, the feature map generated by the ResNet-18 model has a size of 4×5 and a channel number of 512. After flattening and conversion, it forms the input sequence T of the Transformer: T=[F1,F2,…,F 512 ] Among them, F i Represents a flattened 20-dimensional vector.

10. The ResNet-Transformer-based automatic detection method for deformed central cusp according to claim 8, characterized in that: In the detection method, a training configuration with a batch size of 32, a learning rate of 0.001, and a loss function of cross entropy loss was adopted. The model was trained on panoramic X-ray images of 1410 patients to achieve high sensitivity and specificity detection of the deformed central cusp.

Images