Automatic small-scale tooth disease detection method and device based on Cascade R-CNN

Abstract

The invention discloses a small-scale tooth disease automatic detection method and device based on a Cascade R-CNN, and the method comprises the steps: enabling Attack to combine deformation convolution with a global attention mechanism of deformation attention, and enabling a model to extract small target foreground features more accurately under the complex background of a panoramic X-ray film; and the FAM is provided to dynamically align the upper and lower features to improve the generalization ability of the model to different small-scale diseases. The Point-to-Patch Module adopts key point detection and cascade dynamic patches to dynamically adjust the focus boundary box, so that enough high-quality suggestions are provided for small-scale tooth diseases, and the performance of the detector is improved.

Description

Technical Field

[0001] The present invention relates to the field of dental detection technology, and in particular to a method and device for automatic detection of small-scale dental diseases based on Cascade R-CNN. Background Art

[0002] Dental disease is one of the most common oral diseases in humans, affecting the health of many people. Dental panoramic X-rays, also known as panoramic radiographs, are important diagnostic tools widely used in dental practice. With the rapid development of deep learning, automated detection based on deep learning can automatically detect dental diseases in panoramic X-rays and improve the accuracy and efficiency of doctors' diagnosis.

[0003] With the rapid development of deep learning, the automated detection method of dental diseases based on convolutional neural networks has made many breakthrough advances. However, due to the complexity of the data distribution of panoramic oral X-rays, the large scale differences between different lesions and the existence of a large number of small-scale diseases, the existing technology is not satisfactory for the detection of small-scale dental diseases. Summary of the invention

[0004] In order to solve the above technical problems, the present invention provides a small-scale dental disease automatic detection method and device based on Cascade R-CNN, which can adapt to the complex background and data distribution characteristics of oral panoramic X-rays and achieve more accurate small-scale dental disease detection.

[0005] In order to achieve the above object, the technical solution provided by the present invention is:

[0006] A small-scale automated detection method for dental diseases based on Cascade R-CNN, comprising the following steps:

[0007] Step 1: Collect clinical oral panoramic X-rays, annotate the collected oral panoramic X-rays, create an oral disease detection data set, and randomly divide the training set and test set according to a certain ratio;

[0008] Step 2: Build an end-to-end Cascade R-CNN improved model. The first-stage network is the ResNet50+Attneck feature extraction network, the second-stage network is the FPN+FAM feature pyramid structure, the third-stage network is the RPN and RoIAlign network, and the fourth-stage network is the improved cascade detector structure.

[0009] Step 3: Use the divided oral disease detection dataset to train the improved Cascade R-CNN model;

[0010] Step 4: Input the oral panoramic X-ray to be detected into the trained improved Cascade R-CNN best model for detection.

[0011] In one embodiment of the present invention, step 1 specifically includes: collecting a large number of clinical oral panoramic X-ray images, and saving all images in JPG format to ensure the consistency of the data format and facilitate subsequent processing. Next, invite professional dentists with rich clinical experience to manually annotate these X-rays, and annotate four common dental diseases such as high-density shadows, low-density shadows, root tip shadows and impacted teeth; the annotation process will strictly follow the COCO (Common Objects in Context) target detection format, including accurate bounding box annotation of the area of ​​interest in the image, and attach corresponding category labels, and the annotated English labels are 'high density shadow', 'lowdensity shadow', 'root tip shadow', 'obstructed teeth'; after the annotation is completed, all the annotated image data are divided into a training set and a test set in a ratio of 8:2; the training set is used for model training to ensure that the model can learn from a large number of samples and optimize its detection performance; the test set is used to evaluate the generalization ability and actual application effect of the model to ensure that the model performs well on unseen data.

[0012] In one embodiment of the present invention, the step 2 specifically includes:

[0013] Step 2.1: Select the ResNet50 network to extract dental disease features, and use Attneck to improve its bottleneck block based on ResNet50. Attneck uses Deformable Convolution to extract local foreground features and combines Transformer Attention with global context information around the target to enhance the existing ResNet50, which can help the model better obtain small-scale disease features;

[0014] Step 2.2: Use the FPN feature pyramid structure, but FPN recursively uses downsampling to perform feature map fusion, which will lead to feature misalignment. Therefore, add the FAM module on the basis of FPN to align the upper and lower feature maps, improve the generalization ability of different small-scale dental diseases, and improve the robustness of the model.

[0015] Step 2.3: An improved cascade detector is used to predict nine key points of the disease bounding box using a fully convolutional network, and cascade detection is performed through a dynamic patch strategy to achieve accurate positioning of small-scale dental diseases.

[0016] In one embodiment of the present invention, step 3 specifically includes: sending the divided oral disease detection data set to the improved Cascade R-CNN network to start training, during the training process, setting the size of all image data to 1333×800, and performing data enhancement processing on the training data by horizontal flipping. The IOU values ​​of the improved cascade detector are set to 0.5, 0.6, and 0.7 respectively.

[0017] In one embodiment of the present invention, step 4 specifically includes: inputting the oral panoramic X-ray to be detected into the improved Cascade R-CNN optimal model trained in step 3, inputting the detection result image, the disease detection result image includes a rectangular box marking the disease location, and displaying the disease name information corresponding to the rectangular box.

[0018] The second aspect of the present invention relates to an automatic detection device for small-scale dental diseases based on Cascade R-CNN, comprising a memory and one or more processors, wherein the memory stores executable code, and when the one or more processors execute the executable code, they are used to implement an automatic detection method for small-scale dental diseases based on Cascade R-CNN of the present invention.

[0019] A third aspect of the present invention relates to a computer program product, comprising a computer program, which, when executed by a processor, implements an automatic detection method for small-scale dental diseases based on Cascade R-CNN of the present invention.

[0020] Attneck combines deformable convolution with the global attention mechanism of deformable attention, enabling the model to more accurately extract small target foreground features in the complex background of panoramic X-rays, and proposes FAM to dynamically align upper and lower features to improve the model's generalization ability for different small-scale diseases. Point-to-Patch Module uses key point detection and cascaded dynamic patches to dynamically adjust the lesion bounding box to ensure that there are enough high-quality suggestions for small-scale dental diseases, thereby improving detector performance.

[0021] The present invention improves the ResNet50 network, which can more effectively extract small-scale dental disease features, adds a FAM module to align the FPN feature pyramid network, which is more conducive to obtaining more accurate position feature information, and finally improves the detection capability of small-scale dental diseases through the improved cascade detector iteration, thereby improving the detection accuracy of the model.

[0022] The advantages of the present invention are: effectively extracting small-scale tooth disease characteristics, improving the detection capability of small-scale tooth diseases, and improving the detection accuracy of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 It is a schematic diagram of the overall process of the present invention.

[0024] Figure 2(a) and Figure 2(b) are core architecture diagrams of Attneck provided by the present invention, wherein Figure 2(a) is the Attneck of the present invention, and Figure 2(b) is the original Bottleneck structure of the ResNet network.

[0025] Figure 3 This is the core architecture diagram of the FAM module provided by the present invention. DETAILED DESCRIPTION

[0026] The technical solution of the present invention is further described below in conjunction with the accompanying drawings.

[0027] Example 1

[0028] The present invention provides a small-scale automated detection method for dental diseases based on Cascade R-CNN. Figure 1 As shown, the following steps are included:

[0029] Step 1: Collect a large number of clinical oral panoramic X-ray images, and save all images in JPG format to ensure the consistency of data format and facilitate subsequent processing. Next, invite professional dentists with rich clinical experience to manually annotate these X-rays, mainly annotating four common dental diseases such as high-density shadows, low-density shadows, root tip shadows and impacted teeth. The annotation process will strictly follow the format of COCO (Common Objects in Context) target detection, including accurate bounding box annotation of the area of ​​interest in the image, and attach corresponding category labels. The annotated English labels are 'high density shadow', 'low density shadow', 'root tip shadow', 'obstructed teeth'. After the annotation is completed, all the annotated image data are divided into training set and test set in a ratio of 8:2. The training set is used for model training to ensure that the model can learn from a large number of samples and optimize its detection performance; the test set is used to evaluate the generalization ability and actual application effect of the model to ensure that the model performs well on unseen data.

[0030] Step 2: Build an end-to-end Cascade R-CNN improved model. The first-stage network is the ResNet50+Attneck feature extraction network, the second-stage network is the FPN+FAM feature pyramid structure, the third-stage network is the RPN and RoIAlign network, and the fourth-stage network is the improved cascade detector structure. Specifically, the following steps are included:

[0031] The feature extraction module Contextual Attention Alignment Network consists of ResNet50 network + Attneck and FPN feature pyramid + FAM to extract small-scale dental disease features. Dental diseases in oral panoramic X-rays are characterized by complex data distribution, similar lesion features to the background, and small scale. The standard ResNet50 network is difficult to capture the key features of the disease. Replace the bottleneck block of ResNet50 with Attneck, as shown in Figure 2. Attneck combines the global attention mechanism of Deformable Convolution and transformer attention, enabling the model to more accurately extract small target foreground features in complex backgrounds such as panoramic X-rays. Figure 1 As shown in the figure, the input data is a three-channel RGB image of 1333×800. The data first passes through a 7×7 convolution and a convolution layer with a stride of 2 for downsampling, and the length and width are reduced to 666×400, while the number of channels is increased to 64, followed by maximum pooling. Further, the data passes through 3 bottlenecks, the length and width are reduced to 333×200, and the number of channels is increased to 256. Further, the data passes through 4 Attnecks, the length and width are reduced to 166×100, and the number of channels is increased to 512. Further, the data passes through 6 Attnecks, the length and width are reduced to 83×50, and the number of channels is increased to 1024. Further, the data passes through 3 Attnecks, the length and width are reduced to 41×25, and the number of channels is increased to 2048. Finally, feature maps C2-C5 of different scales are obtained.

[0032] The FPN+FAM feature pyramid structure includes the FPN feature pyramid module and the FAM feature alignment module. In FPN, the upsampled low-resolution feature map and the high-resolution feature map are directly fused at the element level, which will result in foreseeable spatial misalignment, which will be converted into misclassification in prediction. Therefore, feature alignment must be performed before feature fusion. The structure of FAM is as follows: Figure 3As shown in the figure, specifically, a 1×1 convolution is used to adjust the number of channels of the feature maps Ci obtained at different levels to a uniform 256, defined as Pi. After Pi is upsampled by a bilinear interpolation layer to obtain Ui, it is connected to Pi-1 and a 1×1 convolution is performed to obtain the offset Δi. In this work, spatial information is represented by a two-dimensional feature map, where each offset can be considered as the displacement distance between each point in Pi-1 and the corresponding point in Pi-1 in two-dimensional space. When deformable convolution is performed on Ui, the offset Δi between Ui and Pi-1 is used as a reference. Deformable convolution can adjust the position of its convolution kernel according to the spatial distance between Pi and Pi-1 to align Pi. The aligned output is defined as Ni, which enhances the model's ability to extract rich contextual information from multi-layer features and improves the model's generalization ability for diseases of different scales.

[0033]

[0034] The RPN network is responsible for efficiently generating candidate target regions (i.e., bounding boxes) for subsequent target classification and regression. The output of the RPN includes two branches: the target classification branch and the bounding box regression branch. The target classification branch is used to predict whether the anchor box contains the target (foreground) or the background. A binary softmax layer is used to output the category probability of each anchor box. The bounding box regression branch is used to adjust the position and size of the anchor box. Through a regression network, the coordinate offset of each anchor box relative to the true target is output. Taking the feature map Ni obtained above as input, the RPN generates multiple anchor boxes in a sliding window manner on the input feature map. The size and aspect ratio of the anchor box are pre-set, and multiple scales and ratios are usually used to accommodate targets of different sizes and shapes. At each sliding window position, the RPN classifies and regresses each anchor box to ensure that the target object can be effectively detected. The generated anchor boxes will be processed by non-maximum suppression (NMS) to remove redundant overlapping boxes, and finally obtain high-quality candidate regions. These candidate regions will be passed to the subsequent target classification and regression network to further refine the detection results. Finally, RoIAlign is used to extract the positive sample features corresponding to the anchor box for subsequent further detection.

[0035] The improved cascade detector structure is as follows Figure 1 As shown in the Point-to-Patch Module. For each positive sample feature, we first use a 3×3 convolution layer with a stride of 2 to generate a feature with a resolution of 7×7. Then, we use seven 3×3 dilated convolutions to expand the receptive field. The resulting features are divided into 9 groups, each corresponding to a key point region.

[0036] v j =M PG (x i),j=1,…,9 (2)

[0037] Due to the spatial sensitivity of the fully convolutional network, each keypoint contains certain spatial information. Therefore, adjacent points can correct each other's positions. For each keypoint feature, select the node with a distance of 2 (the distance between two adjacent points is 1). Then apply a 5x5 depth convolution to minimize the difference between the keypoint and the node, and fuse the convolution node with the keypoint to obtain a new keypoint region. Finally, use two deconvolution layers and sigmoid to generate a 28x28 probability heat map.

[0038]

[0039] h j =sigmoid(deconv(deconv(c j ))) (4)

[0040] The h with the highest confidence is selected from the probability heat map of each positive sample feature as the key point. The key point is mapped back to the proposal area by multiplying the scale factor from the proposal to the heat map and adding the upper left corner of the proposal as an offset. However, considering that the original proposal may not completely contain small-scale diseases, it is mapped back to an area twice the size of the original proposal, and then the nine coordinate points of the final detection box are obtained.

[0041]

[0042] Because the bounding box of the target disease is four points, we need to map 9 key points on the original image to determine the boundary of the target box. Define the bounding box B = (x min ,y min ,x max ,y max ), representing the coordinate points of the upper left corner and lower right corner of the bounding box respectively. Specifically, we use the weighted average of the three leftmost points E1 and the three rightmost points E2 to predict x min and x max Similarly, we use the weighted average of the top three points E3 and the bottom three points E4 to predict y min and max. Unlike Cascade R-CNN, we do not simply pass all detected bounding boxes to the next stage detector for training. We believe that for small target diseases, the detected bounding boxes often cannot accurately surround them and are full of confusing background information. Therefore, we design a region-based dynamic patch strategy to ensure that diseases of different sizes have accurate potential anchor boxes. We set a threshold of 20 based on the average target size in the dental disease dataset, and it can be adjusted according to different datasets. For bounding boxes with an area greater than 20, we pass them directly to the next stage. If the area is less than 20, we divide the bounding box into four equal parts and pass them to the next stage. The present invention uses a three-stage detector to achieve classification and regression of dental diseases. The loss weights of the three stages are {1, 0.75, 0.5}, and the IoU thresholds are {0.5, 0.6, 0.7}, respectively.

[0043] Step 3: The divided oral disease detection dataset is fed into the improved Cascade R-CNN network to start training. During the training process, the input image is scaled or normalized, and data enhancement is performed by rotation, scaling, flipping, etc. The model is trained for 100 epochs.

[0044] Step 4: Input the oral panoramic X-ray to be tested into the improved Cascade R-CNN optimal model trained in step 3, and input the detection result image. The disease detection result image contains a rectangular box marking the disease location and displays the disease name information corresponding to the rectangular box.

[0045] This paper mainly aims at the problem of automatic detection of small-scale dental diseases, and proposes an automatic detection method for small-scale dental diseases based on Cascade R-CNN. This model uses deformable convolution to extract local foreground features while combining Transformer Attention with global contextual information around the target to enhance the existing ResNet50. In order to solve the problem of feature difference caused by recursive downsampling of FPN, we proposed FAM to dynamically align upper and lower features. In addition, inspired by the cascade structure and key point detection, we designed the Point-to-patch Module. This module uses key point detection to replace traditional regression, and dynamically adjusts the bounding box through dynamic patch strategy and cascade structure to improve the detection accuracy of small-scale dental diseases.

[0046] Example 2

[0047] The present embodiment relates to an automatic detection device for small-scale dental diseases based on Cascade R-CNN, comprising a memory and one or more processors, wherein the memory stores executable code, and when the one or more processors execute the executable code, they are used to implement an automatic detection method for small-scale dental diseases based on Cascade R-CNN of Example 1.

[0048] Example 3

[0049] This embodiment relates to a computer program product, including a computer program, which, when executed by a processor, implements an automatic detection method for small-scale dental diseases based on Cascade R-CNN of embodiment 1.

[0050] The present invention has been described through the above examples, but it is obvious that the examples here are only for the purpose of explanation, and the present invention is not limited to the scope of the examples. Researchers in the technical field of the present invention can make non-substantial changes in form and content according to the above steps without departing from the scope of the substantial protection of the present invention. Therefore, the present invention is not limited to the above specific forms and details.

Claims

1. A small-scale automated detection method for dental diseases based on Cascade R-CNN, comprising the following steps: Step 1: Collect clinical oral panoramic X-rays, annotate the collected oral panoramic X-rays, create an oral disease detection dataset, and randomly divide the dataset into training and test sets; Step 2: Build an end-to-end Cascade R-CNN improved model. The first-stage network is the ResNet50+Attneck feature extraction network, the second-stage network is the FPN+FAM feature pyramid structure, the third-stage network is the RPN and RoIAlign network, and the fourth-stage network is the improved cascade detector structure. Step 3: Use the divided oral disease detection dataset to train the improved Cascade R-CNN model; Step 4: Input the oral panoramic X-ray to be detected into the trained improved Cascade R-CNN best model for detection.

2. According to the method for automatic detection of small-scale dental diseases based on Cascade R-CNN in claim 1, the step 1 specifically comprises: Collect a large number of clinical oral panoramic X-ray images, and save all images in JPG format to ensure the consistency of data format and facilitate subsequent processing; Next, we invite professional dentists with rich clinical experience to manually annotate these X-rays and annotate four common dental diseases, including high-density shadows, low-density shadows, root tip shadows, and impacted teeth. The annotation process will strictly follow the COCO (Common Objects in Context) target detection format, including precise bounding box annotation of the area of ​​interest in the image, and attach corresponding category labels. The annotated English labels are 'high density shadow', 'low density shadow', 'root tip shadow', 'obstructed teeth'. After labeling, all labeled image data are divided into training set and test set in a ratio of 8:2; the training set is used to train the model to ensure that the model can learn from a large number of samples and optimize its detection performance; the test set is used to evaluate the generalization ability and actual application effect of the model to ensure that the model performs well on unseen data.

3. According to the method for automatic detection of small-scale dental diseases based on Cascade R-CNN in claim 1, step 2 specifically comprises: Step 2.1: Select the ResNet50 network to extract dental disease features, and use Attneck to improve its bottleneck block based on ResNet50; Attneck uses Deformable Convolution to extract local foreground features and combines Transformer Attention with global context information around the target to enhance the existing ResNet50, which can help the model better obtain the features of small-scale diseases; Step 2.2: Using the FPN feature pyramid structure, add a FAM module on the basis of FPN to align the upper and lower feature maps; Step 2.3: An improved cascade detector is used to predict nine key points of the disease bounding box using a fully convolutional network, and cascade detection is performed through a dynamic patch strategy to achieve accurate positioning of small-scale dental diseases.

4. According to the method for automatic detection of small-scale dental diseases based on Cascade R-CNN in claim 1, step 3 specifically comprises: The divided oral disease detection dataset is sent to the improved Cascade R-CNN network to start training. During the training process, the size of all image data is set to 1333×800, and the training data is enhanced by horizontal flipping; The IOU values ​​of the improved cascade detector are set to 0.5, 0.6, and 0.7 respectively.

5. According to the method for automatic detection of small-scale dental diseases based on Cascade R-CNN in claim 1, step 4 specifically comprises: The oral panoramic X-ray to be tested is input into the improved Cascade R-CNN optimal model trained in step 3, and the detection result image is input. The disease detection result image includes a rectangular box marking the disease location and displays the disease name information corresponding to the rectangular box.

6. A small-scale automatic detection device for dental diseases based on Cascade R-CNN, characterized in that: It comprises a memory and one or more processors, wherein the memory stores executable code, and when the one or more processors execute the executable code, it is used to implement an automatic detection method for small-scale dental diseases based on Cascade R-CNN as described in any one of claims 1 to 5.

7. A computer program product, characterized in that It includes a computer program, which, when executed by a processor, implements an automatic detection method for small-scale dental diseases based on Cascade R-CNN as described in any one of claims 1 to 5.

Images