A dental caries detection method based on channel attention and multi-feature fusion

Abstract

The application discloses a caries detection method based on channel attention and multi-feature fusion, belongs to the technical field of image recognition processing, selects a YOLOv7 algorithm as a basic algorithm model, and improves the YOLOv7 algorithm for a caries detection problem, adds a module combined with an attention mechanism and a lightweight neural network in a backbone network, effectively improves caries feature extraction capability, and reduces model parameter quantity. In the feature map output part, self-adaptive feature fusion is introduced, so that the model adaptively learns the spatial weight of each scale feature map when fusing, fully utilizes feature information of different scales, and thus improves the globality and accuracy of caries detection. The improved recognition algorithm, the application adopts the above method, better adjusts the overlap degree between candidate frames, effectively improves the problems of caries missed detection and mis-detection. The improved YOLOv7 algorithm realizes rapid and accurate detection of caries in an oral cavity photo, saves medical resources, and is convenient for wide use.

Description

Technical Field

[0001] This invention relates to the technical field of image recognition processing, and in particular to a method for detecting dental caries based on channel attention and multi-feature fusion. Background Technology

[0002] Dental caries is a localized disease of the hard tissues of the teeth caused by microorganisms in dental plaque, which seriously affects an individual's health and quality of life. Therefore, utilizing electronic information technology for the detection and prevention of dental caries has become a popular research topic. As an important component of artificial intelligence, deep learning technology has achieved excellent results in the field of assisted medical diagnosis. Compared to traditional methods, deep learning learns features from more data, enabling it to more accurately identify lesions or abnormalities. The convolutional neural networks in deep learning models can effectively and automatically extract features from images without manual extraction. Moreover, through multi-layered network structures, they can extract more abstract and complex features layer by layer, thereby achieving more accurate target detection.

[0003] Limitations of Existing Technologies: Most current research on dental caries detection using deep learning techniques is conducted on dental X-rays. These X-rays require specialized equipment and professionally trained medical personnel. In areas with limited medical resources, the need for such training and the expense of the equipment further complicate the acquisition of dental X-rays. Research on dental caries detection using mobile phone-captured images is relatively limited, with most relying on existing deep learning methods for caries identification. However, the oral environment is complex. In some oral images, the target areas for caries are small and numerous, with dispersed locations, and teeth may be misaligned or overlapping. Furthermore, caries detection is easily affected by factors such as dental stains and pulpitis. Existing deep learning models lack specificity and cannot fully capture the subtle features of caries in the oral cavity, resulting in insufficient detection accuracy. Additionally, existing deep learning models are typically complex, making deployment on mobile devices difficult and consuming significant computational resources. Summary of the Invention

[0004] The purpose of this invention is to provide a caries detection method based on channel attention and multi-feature fusion, which can accurately detect caries in oral photographs, reduce the number of model parameters, and facilitate subsequent deployment on mobile devices.

[0005] To achieve the above objectives, this invention provides a dental caries detection method based on channel attention and multi-feature fusion, comprising the following steps:

[0006] S1: Collect images of cavities in the human oral cavity, and combine them with professional annotation boxes for the cavities in the images. Then, use data augmentation techniques to enhance the annotated images of cavities to obtain a cavity dataset.

[0007] S2: Improve the YOLOv7 network by replacing the original ELAN module of the YOLOv7 backbone network with the ECA-MobileOne lightweight network module, and add the adaptive feature fusion ASFF module to the feature map output part to obtain the recognition network model. The recognition network model is trained using the dental caries dataset.

[0008] S3: The recognition process involves feeding an oral photograph containing cavities into a trained recognition network model. Through forward propagation, the model locates and identifies the position and shape of the cavities and outputs a feature map with corresponding target candidate boxes.

[0009] S4: The generated target candidate boxes are processed using the soft-NMS algorithm. By adjusting the overlap between the candidate boxes, the final output target boxes are obtained.

[0010] Preferably, in step S2, the improved process is as follows:

[0011] The YOLOv7 network consists of three parts: the input layer, the backbone network, and the head. The improvement process includes adding the ECA-MobileOne lightweight network module to the backbone network and adding the adaptive feature fusion (ASFF) module to the feature map output part of the head.

[0012] The ECA-MobileOne lightweight network module consists of multiple MobileOne Blocks, each of which includes depthwise convolution and pointwise convolution.

[0013] The deep convolutional structure consists of three branches: 1×1 convolution, overparameterized 3×3 convolution, and skip connections with batch normalization layers. The 1×1 and 3×3 convolutions are both deep convolutions. The point convolution consists of two branches: overparameterized 1×1 convolution and skip connections with normalization layers. The number of output channels is adjusted and fused with the feature map output by the deep convolution. An ECA attention mechanism is added after the ReLU activation function to form the ECA-MobileOne network module.

[0014] Preferably, in step S3, the identification process is as follows:

[0015] For the ECA attention mechanism module structure diagram, the input feature map has a size of W×H and the number of channels is C. After global average pooling compression, the feature values ​​of each channel are averaged to obtain a tensor of size 1×1×C. A one-dimensional convolutional kernel of size K is used to capture the information interaction between channels. Finally, the output feature map is multiplied element-wise with the input feature map to obtain a weighted feature map. The size K of the one-dimensional convolutional kernel is proportional to the number of channels C, as shown in the following formula:

[0016]

[0017] In the above formula, the mapping parameter γ is 2 and b is 1;

[0018] For the Adaptive Feature Fusion (ASFF) module, the three output feature maps are F1, F2, and F3, which have different resolutions and number of channels, representing 1 / 8, 1 / 16, and 1 / 32 of the original image, respectively.

[0019] To generate the ASFF-1 feature map, both F2 and F3 undergo convolution operations to adjust the number of channels to match F1. Then, interpolation is performed to adjust the size. Finally, each scale of the feature map is multiplied by a learned weight, and then a weighted sum is calculated to generate the fused feature map ASFF-1. The calculation formula is as follows:

[0020]

[0021] In the above formula, F ij 2→1 and F ij 3→1 This represents the feature vector at position (i,j) corresponding to layer F1 after adjusting the number of channels and size of the feature maps F2 and F3. This indicates that the spatial fusion weights of the three different feature layers learned adaptively by the network are used in the first layer. Similarly, the ASFF-2 and ASFF-3 feature maps are generated through the above steps. The output ASFF-1, ASFF-2, and ASFF-3 feature maps all contain target candidate boxes.

[0022] Preferably, in step S4, the process of processing the generated target candidate boxes using the soft-NMS algorithm is as follows: Gaussian weighting is used to solve the continuity problem. The calculation method is as follows:

[0023]

[0024]

[0025] In the above formula, s i b represents the score for each detection box. iLet M represent each detection box, and M represent the detection box with the highest score.

[0026] Therefore, the present invention employs the above-mentioned caries detection method based on channel attention and multi-feature fusion, which has the following advantages:

[0027] (1) In this invention, the ECA-MobileOne network module is introduced into the backbone network of the YOLOv7 algorithm to replace the original high-efficiency layer aggregation network (ELAN) module, thereby reducing the number of model parameters and improving the effective extraction of caries features.

[0028] (2) In this invention, adaptive feature fusion (ASFF) is used in the head feature map output layer to fully acquire key features at different scales in the oral cavity image, thereby improving the globality and accuracy of detection.

[0029] (3) In this invention, the soft-NMS algorithm is used to replace the original non-maximum suppression algorithm (NMS). By adjusting the overlap between candidate boxes, the final output target box can be controlled more precisely, which can more effectively improve the detection effect in cases of tooth displacement or overlap.

[0030] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0031] Figure 1 This is a flowchart illustrating the overall workflow of a dental caries detection method based on channel attention and multi-feature fusion according to the present invention.

[0032] Figure 2 This is a diagram of the improved YOLOv7 network structure in a dental caries detection method based on channel attention and multi-feature fusion according to the present invention.

[0033] Figure 3 This is a structural diagram of the ECA attention mechanism in a dental caries detection method based on channel attention and multi-feature fusion according to the present invention;

[0034] Figure 4 This is a diagram of the ECA-MobileOne network structure in a dental caries detection method based on channel attention and multi-feature fusion according to the present invention.

[0035] Figure 5 This is a diagram of the adaptive feature fusion structure in a dental caries detection method based on channel attention and multi-feature fusion according to the present invention. Detailed Implementation

[0036] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Specific model specifications need to be selected and determined according to the actual specifications of the device, etc. The specific selection calculation method adopts existing technology in the art, and therefore will not be described in detail.

[0037] Example

[0038] like Figure 1 As shown, this invention provides a dental caries detection method based on channel attention and multi-feature fusion, comprising the following steps:

[0039] S1: Collect images of cavities in the human oral cavity, and combine them with professional identification and annotation boxes for the cavities in the images. Then, use data augmentation techniques to enhance the annotated images of cavities, including rotation, flipping, brightness adjustment, and noise addition, to finally obtain the cavities dataset.

[0040] S2: Improve the YOLOv7 network by replacing the original ELAN module of the YOLOv7 backbone network with the ECA-MobileOne lightweight network module, and add an adaptive feature fusion (ASFF) module to the feature map output to obtain the recognition network model, such as... Figure 2 The diagram shown is of the improved recognition network structure. The improvement process is as follows:

[0041] The YOLOv7 network consists of three parts: the input layer, the backbone network, and the head. The improvement process includes adding the ECA-MobileOne lightweight network module to the backbone network and adding the adaptive feature fusion (ASFF) module to the feature map output part of the head.

[0042] like Figure 4 The ECA-MobileOne lightweight network module consists of multiple MobileOne Blocks, each of which includes depthwise convolution and pointwise convolution.

[0043] The deep convolutional structure comprises three branches: a 1×1 convolution, an overparameterized 3×3 convolution, and a skip connection with a batch normalization (BN) layer. Both the 1×1 and 3×3 convolutions are deep convolutions. The point convolution consists of two branches: an overparameterized 1×1 convolution and a skip connection with a BN layer. Its main function is to allow for flexible adjustment of the number of output channels and to fuse them with the feature maps output by the deep convolution. Adding the ECA attention mechanism to the ReLU activation function forms the ECA-MobileOne network module. During model training, ECA-MobileOne consists of... Figure 4 The left side shows a complete structure composed of stacked blocks of depthwise convolutions and pointwise convolutions, which extracts more features related to tooth decay. After model training, the multi-branch structure is transformed into a single-branch structure through reparameterization, simplifying the model, reducing the number of parameters, and improving inference speed. The recognition network model is then trained using a tooth decay dataset.

[0044] S3: The recognition process involves feeding an oral photograph containing cavities as input data into a trained recognition network model. Through forward propagation, the model locates and identifies the position and shape of the cavities, and outputs a feature map containing corresponding candidate bounding boxes. The recognition process is as follows:

[0045] like Figure 3 For the ECA attention mechanism module structure diagram, the input feature map has a size of W×H and the number of channels is C. After global average pooling compression, the feature values ​​of each channel are averaged to obtain a tensor of size 1×1×C. A one-dimensional convolutional kernel of size K is used to capture the information interaction between channels, thereby capturing the correlation between global channels. Finally, the output feature map is multiplied element-wise with the input feature map to obtain a weighted feature map. By applying attention weights to the feature map of each channel, important features are highlighted and invalid features are suppressed. The size of the one-dimensional convolutional kernel K is proportional to the number of channels C: the formula is as follows:

[0046]

[0047] In the above formula, the mapping parameter γ is 2 and b is 1;

[0048] like Figure 5 For the Adaptive Feature Fusion (ASFF) module, similar to other object detection models, YOLOv7 also outputs multi-layer features through methods such as direct concatenation and direct addition. However, since high-level features contain low-level features, these methods cannot fully utilize data at different scales. Furthermore, oral cavity images are complex and prone to background noise issues with feature redundancy. Therefore, an Adaptive Feature Fusion (ASFF) method is used in the head output section to effectively fuse multi-scale features of the image through adaptively learning spatial weights.

[0049] When the three output feature maps are F1, F2, and F3, these three feature maps have different resolutions and number of channels, which are 1 / 8, 1 / 16, and 1 / 32 of the original image, respectively.

[0050] To generate the ASFF-1 feature map, both F2 and F3 undergo convolution operations to adjust the number of channels to match F1. Then, interpolation is performed to adjust the size. Finally, each scale of the feature map is multiplied by a learned weight, and then a weighted sum is calculated to generate the fused feature map ASFF-1. The calculation formula is as follows:

[0051]

[0052] In the above formula, F ij 2→1 and F ij 3→1 This represents the feature vector at position (i,j) corresponding to layer F1 after adjusting the number of channels and size of the feature maps F2 and F3. This indicates that the spatial fusion weights of the three different feature layers learned adaptively by the network are used in the first layer. Similarly, the ASFF-2 and ASFF-3 feature maps are generated through the above steps. The output ASFF-1, ASFF-2, and ASFF-3 feature maps all contain target candidate boxes.

[0053] S4: The generated target candidate boxes are processed using the soft-NMS algorithm. By adjusting the overlap between the candidate boxes, the final output target boxes are obtained. The process of processing the generated target candidate boxes using the soft-NMS algorithm is as follows: Gaussian weighting is used to solve the continuity problem. The calculation method is as follows:

[0054]

[0055]

[0056] In the above formula, s i b represents the score for each detection box. i Let M represent each detection box, and M represent the detection box with the highest score.

[0057] Therefore, this invention employs a caries detection method based on channel attention and multi-feature fusion. The YOLOv7 algorithm is selected as the base algorithm model, and improvements are made specifically for caries detection. An attention mechanism combined with a lightweight neural network module is added to the backbone network, effectively enhancing caries feature extraction capabilities and reducing the number of model parameters. Adaptive feature fusion is introduced in the feature map output, allowing the model to adaptively learn the spatial weights of feature maps at different scales during fusion, fully utilizing feature information at different scales, thereby improving the globality and accuracy of caries detection. The improved recognition algorithm can better adjust the overlap between candidate boxes, effectively improving the problems of missed and false detections of caries. The improved YOLOv7 algorithm achieves fast and accurate detection of caries in oral photographs, saving medical resources and facilitating widespread use.

[0058] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the technical solutions of the present invention, and these modifications or equivalent substitutions cannot cause the modified technical solutions to deviate from the spirit and scope of the technical solutions of the present invention.

Claims

1. A caries detection method based on channel attention and multi-feature fusion, characterized in that: Includes the following steps: S1: Collect images of cavities in the human oral cavity, and combine them with professional identification boxes for the cavities in the images. Then, use data augmentation techniques to enhance the images of the cavities to obtain a cavity dataset. S2: Improve the YOLOv7 network by replacing the original ELAN module of the YOLOv7 backbone network with the ECA-MobileOne lightweight network module, and add the adaptive feature fusion ASFF module to the feature map output part to obtain the recognition network model. The recognition network model is trained using the dental caries dataset. S3: The recognition process involves feeding an oral photograph containing cavities into a trained recognition network model. Through forward propagation, the model locates and identifies the position and shape of the cavities and outputs a feature map with corresponding target candidate boxes. S4: The generated target candidate boxes are processed using the soft-NMS algorithm. By adjusting the overlap between the candidate boxes, the final output target boxes are obtained.

2. The dental caries detection method based on channel attention and multi-feature fusion according to claim 1, characterized in that: In step S2, the improved process is as follows: The YOLOv7 network consists of three parts: the input layer, the backbone network, and the head. The improvement process includes adding the ECA-MobileOne lightweight network module to the backbone network and adding the adaptive feature fusion (ASFF) module to the feature map output part of the head. The ECA-MobileOne lightweight network module consists of multiple MobileOneBlocks, each of which includes depthwise convolution and pointwise convolution. The deep convolutional structure consists of three branches: 1×1 convolution, overparameterized 3×3 convolution, and skip connections with batch normalization layers. The 1×1 and 3×3 convolutions are both deep convolutions. The point convolution consists of two branches: overparameterized 1×1 convolution and skip connections with normalization layers. The number of output channels is adjusted and fused with the feature map output by the deep convolution. An ECA attention mechanism is added after the ReLU activation function to form the ECA-MobileOne network module.

3. The dental caries detection method based on channel attention and multi-feature fusion according to claim 2, characterized in that: In step S3, the identification process is as follows: For the ECA attention mechanism module structure diagram, the input feature map has a size of W×H and the number of channels is C. After global average pooling compression, the feature values ​​of each channel are averaged to obtain a tensor of size 1×1×C. A one-dimensional convolutional kernel of size K is used to capture the information interaction between channels. Finally, the output feature map is multiplied element-wise with the input feature map to obtain a weighted feature map. The size K of the one-dimensional convolutional kernel is proportional to the number of channels C, as shown in the following formula: In the above formula, the mapping parameter γ is 2 and b is 1; For the adaptive feature fusion (ASFF) module, the output feature maps are F1, F2, and F3, which have different resolutions and number of channels. They are generated by taking up 1 / 8, 1 / 16, and 1 / 32 of the original image, respectively. F2 and F3 are first subjected to convolutional operations to adjust the number of channels to match F1, and then interpolation operations are performed to adjust their size. Finally, each feature map is multiplied by a learned weight, and then a weighted sum is performed to generate the fused feature map ASFF-1. The calculation formula is as follows: In the above formula, and This represents the feature vector at position (i,j) corresponding to layer F1 after adjusting the number of channels and size of the feature maps F2 and F3. This indicates that the spatial fusion weights of the three different feature layers learned adaptively by the network are used in the first layer. Similarly, the ASFF-2 and ASFF-3 feature maps are generated through the above steps. The output ASFF-1, ASFF-2, and ASFF-3 feature maps all contain target candidate boxes.

4. The caries detection method based on channel attention and multi-feature fusion according to claim 3, characterized in that: In step S4, the process of processing the generated target candidate boxes using the soft-NMS algorithm is as follows: Gaussian weighting is used to solve the continuity problem, and the calculation method is as follows: In the above formula, s i b represents the score for each detection box. i Let M represent each detection box, and M represent the detection box with the highest score.

Images