Tooth image detection device and method based on YOLOv5 model

Abstract

The invention discloses a tooth image detection device and method based on a YOLOv5 model, applied to the field of medical images, and the method comprises the steps: obtaining a pre-trained tooth image detection model; according to the tooth image detection model, an attention mechanism is introduced on the basis that a YOLOv5 convolutional neural network is used as a framework, a HardSwsh activation function is used as an activation function, and an EIOU loss function is used as a loss function in a model training process; inputting a tooth image to be detected into the trained tooth image detection model, and detecting whether a focus area exists or not; if yes, displaying the lesion area in a rectangular frame labeling mode, and displaying lesion information; the lesion information at least comprises lesion type, number, size and position information. According to the method, the tooth image detection speed and precision of the YOLOv5 convolutional neural network are improved, and the detection effect is obviously improved.

Description

Technical Field

[0001] The present invention relates to the field of medical images, and in particular to a tooth image detection device and method based on a YOLOv5 model. Background Art

[0002] Nowadays, medical technology has achieved unprecedented development, and medical equipment has become increasingly intelligent. However, there are still many challenges in dental examinations. Traditional dental CT (Computed Tomography) image detection methods require a lot of manpower and material resources, which may easily lead to misdiagnosis or missed diagnosis, causing great difficulties for doctors and patients. With the rapid development of computer vision, deep learning has begun to be widely used in the medical field, but due to the particularity and complexity of dental images, the current deep learning-based detection methods still have problems with poor accuracy and low efficiency. Summary of the invention

[0003] In view of this, the purpose of the present invention is to provide a tooth image detection device and method based on the YOLOv5 model, which solves the problems of inaccurate tooth image detection and low detection efficiency in the prior art.

[0004] In order to solve the above technical problems, the present invention provides a tooth image detection device based on the YOLOv5 model, comprising:

[0005] A model acquisition module, used to acquire a pre-trained tooth image detection model; the tooth image detection model introduces an attention mechanism based on the YOLOv5 convolutional neural network architecture, and uses the HardSwish activation function as the activation function and the EIOU loss function as the loss function in the model training process;

[0006] A detection module, used for inputting the tooth image to be detected into the trained tooth image detection model to detect whether there is a lesion area;

[0007] The detection result display module is used to display the lesion area by marking it with a rectangular frame if it exists, and display the lesion information; the lesion information at least includes the type, number, size and location information of the lesion.

[0008] Optionally, the formula of the HardSwish activation function in the model acquisition module is:

[0009] ;

[0010] Among them, in the HardSwish activation function of the output layer, x represents the weight coefficients of different lesions predicted by the tooth image detection model.

[0011] Optionally, the formula of the EIOU loss function in the model acquisition module is:

[0012] ;

[0013] Among them, A is the anchor box, B is the target box, b is the center of the anchor box, and C is the minimum union area of ​​the target box and the anchor box. is the center of the target box, is the height of the target frame, is the width of the target box, h is the height of the anchor box, and w is the width of the anchor box. is the height of the minimum union area, is the width of the minimum union region, is the Euclidean distance, is the IOU loss value at the AB box, is the EIOU loss value.

[0014] Optionally, the model acquisition module includes:

[0015] An attention mechanism unit is used to use the YOLOv5 convolutional neural network as the basic architecture of the tooth image detection model and introduce an attention mechanism; wherein the attention mechanism is any one of the SEnet attention mechanism, the CBAM attention mechanism and the ECAnet attention mechanism.

[0016] Optionally, the model acquisition module includes:

[0017] A training data set acquisition unit is used to acquire a training data set, wherein the training data set includes various tooth images and corresponding label information; the label information is a label identification of the lesion area in the tooth image; the label identification includes an inflammation label, a wisdom tooth label and a tooth loss label;

[0018] A storage unit, used to store the image name, lesion condition and lesion location of each tooth image in the training data set;

[0019] The training unit is used to train the tooth image detection model using the training data set. When the training end condition is met, the training ends and the trained tooth image detection model is obtained.

[0020] Optionally, it also includes:

[0021] A data enhancement module, used to perform data enhancement processing on the tooth images in the training data set and generate corresponding label information to obtain an expanded training data set; the data enhancement processing at least includes image flipping, adding noise and masking processing;

[0022] Accordingly, the training unit includes:

[0023] The training subunit is used to train the tooth image detection model using the expanded training data set.

[0024] Optionally, it also includes:

[0025] A preprocessing module performs a preprocessing operation on the tooth image to be detected to obtain the processed tooth image; the preprocessing operation at least includes size adjustment and flip correction processing;

[0026] Correspondingly, the detection module includes:

[0027] The detection unit is used to input the processed tooth image into the trained tooth image detection model.

[0028] The present invention also provides a tooth image detection method based on the YOLOv5 model, which is applied to a processor and includes:

[0029] Obtain a pre-trained tooth image detection model; the tooth image detection model introduces an attention mechanism based on the YOLOv5 convolutional neural network architecture, and uses the HardSwish activation function as the activation function and the EIOU loss function as the loss function in the model training process;

[0030] Inputting the tooth image to be detected into the trained tooth image detection model to detect whether there is a lesion area;

[0031] If it exists, the lesion area is displayed by marking with a rectangular frame, and the lesion information is displayed; the lesion information at least includes the type, number, size and location information of the lesion.

[0032] The present invention also provides a tooth image detection device based on the YOLOv5 model, comprising:

[0033] Memory for storing computer programs;

[0034] A processor is used to implement the above-mentioned tooth image detection method based on the YOLOv5 model when executing the computer program.

[0035] The present invention also provides a computer-readable storage medium, in which computer-executable instructions are stored. When the computer-executable instructions are loaded and executed by a processor, the tooth image detection method based on the YOLOv5 model as described above is implemented.

[0036] It can be seen that the present invention uses a model acquisition module to obtain a pre-trained tooth image detection model; the tooth image detection model introduces an attention mechanism based on the YOLOv5 convolutional neural network architecture, and uses the HardSwish activation function as the activation function, and the EIOU loss function as the loss function in the model training process; the detection module is used to input the tooth image to be detected into the trained tooth image detection model to detect whether there is a lesion area; the detection result display module is used to display the lesion area by marking it with a rectangular frame if it exists, and display the lesion information; the lesion information at least includes the type, quantity, size and location information of the lesion. The present invention uses the YOLOv5 convolutional neural network to detect dental images, which can effectively help medical professionals to make auxiliary diagnoses, reduce misdiagnosis and missed diagnosis problems, reduce patient waiting time, and improve detection efficiency. In order to further improve the speed and accuracy of the YOLOv5 convolutional neural network in detecting dental images, an attention mechanism is added on the basis of YOLOv5, so that the model pays more attention to the area of ​​interest, thereby improving the detection accuracy. At the same time, the activation function is also modified to speed up the detection speed, which significantly improves the detection effect. In addition, during the model training process, the EIOU loss function can better ensure the accuracy of the model training.

[0037] In addition, the present invention also provides a tooth image detection method, device and computer-readable storage medium based on the YOLOv5 model, which also have the above-mentioned beneficial effects. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0039] Figure 1 A flowchart of a tooth image detection method based on the YOLOv5 model provided in an embodiment of the present invention;

[0040] Figure 2 A schematic diagram of the structure of a Focus network provided by an embodiment of the present invention;

[0041] Figure 3 A schematic diagram of the structure of an SPP network provided by an embodiment of the present invention;

[0042] Figure 4 A schematic diagram of the structure of a Neck network provided in an embodiment of the present invention;

[0043] Figure 5A schematic diagram of a SEnet attention mechanism provided by an embodiment of the present invention;

[0044] Figure 6 and Figure 7 A schematic diagram of a CBAM attention mechanism provided by an embodiment of the present invention;

[0045] Figure 8 A schematic diagram of an ECAnet attention mechanism provided by an embodiment of the present invention;

[0046] Fig. 9 A schematic diagram of a HardSwish function provided by an embodiment of the present invention;

[0047] Fig.10 A schematic diagram of an EIOU loss function provided by an embodiment of the present invention;

[0048] Fig.11 An example diagram of a tooth image detection model structure based on the YOLOv5 model provided in an embodiment of the present invention;

[0049] Figure 12 to Figure 13 A schematic diagram of detection results of a tooth image detection method based on the YOLOv5 model provided in an embodiment of the present invention;

[0050] Figures 14 to 17 A schematic diagram of the results of a tooth image processing method provided by an embodiment of the present invention;

[0051] Fig.18 A flowchart of a tooth image detection method provided by an embodiment of the present invention;

[0052] Fig.19 A schematic diagram of the structure of a tooth image detection device based on the YOLOv5 model provided in an embodiment of the present invention;

[0053] Fig. 20 A schematic diagram of the structure of a tooth image detection device based on the YOLOv5 model provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0054] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0055] Compared with the traditional manual segmentation method, the automatic image detection method based on deep learning has higher accuracy and efficiency, which can greatly reduce the working time of doctors and improve the quality of medical services. However, due to the particularity and complexity of dental images, there are still some problems and challenges in the dental image detection method based on deep learning. For example, there are still certain limitations in the analysis and processing of local details. Therefore, how to further improve the accuracy and efficiency of dental image detection is the current problem. The present invention provides a dental image detection method based on the YOLOv5 model. For details, please refer to Figure 1 , Figure 1 A flowchart of a tooth image detection method based on the YOLOv5 model provided in an embodiment of the present invention. The method may include:

[0056] S101: Obtain a pre-trained tooth image detection model; the tooth image detection model introduces an attention mechanism based on the YOLOv5 convolutional neural network architecture, and uses the HardSwish activation function as the activation function and the EIOU loss function as the loss function in the model training process.

[0057] It should be noted that this embodiment uses the YOLOv5 (You Only Look Once v5, the fifth version of the target detection algorithm) convolutional neural network as the basic architecture. The YOLOv5 convolutional neural network consists of four parts: input layer (Input), backbone network (Backbone), neck (Neck) network and output layer (Output). The input layer passes the image as input to the network, but before entering the network, some preprocessing operations are usually performed, including image scaling, size padding, normalization, etc., to ensure that the input images have the same format and size, such as 416×416 or 640×640.

[0058] The execution subject of this embodiment is a processor. The input layer of the YOLOv5 convolutional neural network in this embodiment mainly consists of three parts: Mosaic data enhancement, adaptive anchor box calculation, and adaptive image scaling. Among them, (1) Mosaic data enhancement is a data enhancement operation that achieves data enhancement by splicing the images in the image data set in a random scaling, random cropping, and random arrangement manner. Mosaic data enhancement can effectively solve the problem of uneven distribution of small targets in the data set, increase the number of small targets in the data set, and make the model more fully trained for small targets. (2) Adaptive anchor box calculation: Unlike the previous YOLO series algorithms, YOLOv5 has the ability to adapt anchor box parameters, which can be automatically updated during the training process and calculate the best anchor box value to adapt to different training situations. Each object is assigned to a different grid according to the center point, and then the IoU (Intersection over Union) of each anchor box is looked at, and the one with the highest IoU is selected and used for prediction. (3) Adaptive image scaling: An important factor in the speed improvement of the YOLOv5 convolutional neural network is the improvement of adaptive image scaling at the input layer. In the target detection task, the network input is mostly a fixed value, which makes different images scaled to the same size. Therefore, some redundant information will be added to the input image, affecting the training speed. The YOLOv5 convolutional neural network uses the minimum black border strategy according to the aspect ratio of the original image to reduce the amount of calculation, thereby greatly improving the model operation speed.

[0059] The backbone network of the YOLOv5 convolutional neural network in this embodiment is a residual neural network based on CSPDarknet53 (a deep learning model that combines DarkNet53 (an efficient convolutional neural network model) and CSP (Cross Stage Partial, cross-stage partial) structure), and is stacked through a series of convolutional layers, pooling layers and activation functions. The convolutional layers in the backbone network have different convolution kernel sizes and step sizes, which can effectively capture multi-scale feature information. This method enables the YOLOv5 convolutional neural network to effectively handle targets of different sizes and have a wider receptive field. Through the downsampling operation of the pooling layer, the size of the feature map is further reduced, and the computational efficiency is improved. The activation function introduces nonlinear properties, which helps to better capture the nonlinear relationship in the input data, prevent network collapse, and enhance the network's expressive power. In addition, the backbone network also includes some special modules, such as the Focus network structure, the SPP (Spatial PyramidPooling, spatial pyramid pooling) module, and the C3 (Cross Stage Network, cross-stage network) module. Among them, (1) Focus network structure: It is used to reduce the amount of calculation and extract global features, aiming to reduce the calculation cost by changing the input and output channels. Figure 2 As shown, the specific function can be regarded as slicing in the picture. First, the input image is sampled at intervals, that is, a value is obtained for every other pixel in each image, and the four independent feature layers obtained at this time are stacked to complete the four-fold channel expansion. Then, through the convolution operation, a double-downsampled feature map with no feature information loss can be obtained. This method increases the dimension of the channel, improves the model training speed, and has a larger receptive field, which helps to extract more global features. (2) The SPP module is a derivative of the spatial pyramid. By borrowing the idea of ​​the spatial pyramid, the maximum pooling operation is used to fuse the receptive fields of different feature areas. The purpose is to convert feature maps of any size into feature vectors of fixed dimensionality. The SPP module is as follows Figure 3As shown in the figure. First, the number of input channels is halved through a standard deviation convolution module, followed by three maximum pooling operations. The results of the three pooling operations are concatenated with the original input data without pooling operations, and finally the data with twice the number of channels is synthesized. The sizes of the three convolution kernels of the maximum pooling are 5, 9, and 13 respectively. Convolution kernels of different sizes can capture different feature information of the input data. Since spatial pyramid pooling can divide the input data into sub-regions of different scales and perform pooling operations on each sub-region, it can capture richer feature information at different scales. The advantage of spatial pyramid pooling is that it can process input feature maps of any scale, while ensuring the size of the receptive field, extracting richer contextual information and enhancing the network's expression ability. (3) C3 module: It is the main module for feature learning in the residual neural network, which can ensure that the model learns more features. The front-end feature map of this module is mainly divided into two parts. First, 1x1 convolution is used to reduce the dimension of the channel to reduce the number of parameters and computational loss. Subsequently, 3x3 convolution is used to extract features. By stacking multiple 3x3 convolutions, the receptive field of view can be increased and richer feature information can be extracted. Finally, the input feature map after 1x1 convolution is residually connected with the feature map after 3x3 convolution, and the original features are added to the learned features to achieve information transmission and fusion. The C3 module is a cross-scale connection, which strengthens the gradient information of back propagation, enhances the learning ability of the network, and better solves the problems of overfitting and gradient disappearance caused by the deep number of neural network layers. The C3 module can increase the expressive power and receptive field size of features while maintaining the network calculation and parameter quantity. By introducing the residual network, the C3 module can better transmit and fuse feature information, enable the backbone network to extract richer features, and provide more accurate feature representation for subsequent target detection tasks.

[0060] In this embodiment, the neck network (also called the feature fusion layer) of the YOLOv5 convolutional neural network is located between the backbone network and the detection head, and plays a role in feature fusion and scale adaptation. It combines the path aggregation network PAN (Path Aggregation Network) with the FPN (Feature Pyramid Network). The neck network structure is as follows: Figure 4As shown. Among them, FPN mainly constructs a feature pyramid and fuses high-level feature information by upsampling. PAN transmits features from bottom to top on the basis of PFN, and fuses the bottom-level features with the high-level features, so that the rich information of the bottom layer can also be used by the top-level feature map, avoiding the loss of bottom-level features. This structure combines the two to integrate parameters of different detection layers and complement each other, which can effectively improve the detection effect of the target. The neck network of the YOLOv5 convolutional neural network in this embodiment is CSP-FPN (Cross Stage Partial Feature Pyramid Network, cross-stage partial feature pyramid network), which combines the ideas of CSP module and FPN. CSP-FPN realizes the cross-stage partial connection and information transmission of features through the CSP module, and then fuses features through FPN to obtain a multi-scale feature pyramid. CSP-FPN can effectively fuse feature maps of different resolutions and provide multi-scale feature representations to meet the detection needs of targets of different scales. Through the design of feature fusion and pyramid structure, the Neck network can provide rich semantic information and contextual information, which helps to improve the accuracy and robustness of the target detection algorithm.

[0061] The output layer of the YOLOv5 convolutional neural network in this embodiment is the final stage of the entire target detection task. It consists of a convolutional layer, an activation function, a prediction layer, a loss function, and NMS (Non-Maximum Suppression). At this stage, YOLOv5 converts the feature map into a target detection result and performs post-processing to obtain the final detection output. Among them, (1) the convolutional layer plays a role in further processing the feature map and extracting high-level feature representations in the output layer. These convolutional layers usually use a smaller convolution kernel size to capture the details and local features of the target. By stacking multiple convolutional layers, the neural network can obtain a deeper and more complex feature representation, thereby improving the accuracy of target detection. (2) The activation function is applied to the output of the convolutional layer due to its nonlinear nature. This nonlinear activation function can well enhance the network's expression ability and improve the modeling ability of the target. Through element-by-element nonlinear operations, the linear activation function can handle negative values ​​and nonlinear relationships in the feature map, thereby better adapting to the feature distribution of the target. (3) The prediction layer is a key component of the output layer and is used to generate target detection results. The prediction layer usually consists of a set of convolutional layers and fully connected layers. These layers are used to predict information such as the category, position, and confidence of the target. For each target, the prediction layer generates a vector containing the prediction results such as the position of the prediction box, the category label, and the confidence. (4) When calculating the position error between the target box and the prediction box, YOLOv5 uses the CIOU_Loss loss function. Compared with the traditional IoU loss function, CIOU_Loss can more accurately evaluate the position error between the target box and the prediction box, avoiding the problem of slow convergence caused by the non-overlapping bounding boxes. This loss function can provide better training results and more accurate target detection results. (5) In order to select the confidence of the predicted category and eliminate overlapping bounding boxes, YOLOv5 uses NMS non-maximum suppression. The NMS algorithm evaluates the degree of overlap between the predicted box and other boxes by calculating the IoU between them. If the IoU value exceeds the given threshold, it is considered that the two boxes predict the same object and are considered as part of the final prediction result. If the IoU value is less than the threshold, the predicted box does not belong to the object identified by the target detection task. Through the application of NMS, YOLOv5 can select the predicted box with higher confidence in the output result and eliminate the overlapping bounding boxes. This can greatly reduce repeated detection and redundant output, thereby improving the accuracy and efficiency of target detection. Therefore, the output layer converts the feature map into the final target detection result, and accurately predicts and adjusts the position, category and confidence of the predicted box to achieve efficient and accurate target detection.

[0062] Furthermore, considering that a large amount of information will appear when using the YOLOv5 convolutional neural network to process images, but not all information is helpful for analysis and processing, the YOLOv5 convolutional neural network does not need to pay attention to all the information, but pays attention to the areas that should be paid attention to, and can adaptively pay attention to objects. The attention mechanism network is a way of adaptive objects, which can be divided into spatial attention mechanism, channel attention mechanism, and a combination of the two. In this way, by adding the attention mechanism, the model can pay more attention to the area of ​​interest. Among them, the attention mechanism is any one of the SEnet attention mechanism, the CBAM attention mechanism, and the ECAnet attention mechanism.

[0063] In the SEnet attention mechanism, the network focuses on the weight value of each channel in the input layer. First, the input feature layer is averaged and pooled, followed by two full connections. The first full connection has fewer neurons, and the second full connection has the same number of neurons as the input layer. Then, through the Sigmoid activation function, the output value is fixed to a probability between 0 and 1, and then the weight probability value of each channel is obtained. Finally, this weight value is multiplied by the original input feature layer. The network feature diagram is as follows: Figure 5 shown.

[0064] CBAM takes into account the influence of spatial weights and combines the spatial attention mechanism with the channel attention mechanism, such as Figure 6 and Figure 7 shown. Figure 6 It is a channel attention mechanism. The input single-layer feature values ​​are subjected to global maximum pooling and global average pooling, and then the results of the two are added together through the Sigmoid function to obtain a weight in the interval of 0-1, and finally the obtained weight is multiplied by the corresponding original input layer. Figure 7 It is a spatial attention mechanism. The maximum and average values ​​of the channels of each feature point in the input feature layer are taken, and then the results are stacked. The number of channels is adjusted by a convolution with a channel number of 1, and then the weights in the interval of 0-1 are obtained through the Sigmoid activation function. Finally, the obtained weights are multiplied by the original input layer.

[0065] ECAnet is also a channel attention mechanism. It removes the fully connected layer based on SEnet and directly performs 1D convolution on the feature layer after global pooling. Figure 8 shown.

[0066] Furthermore, the activation function in the tooth image detection model is the HardSwish activation function. That is to say, in this embodiment, the activation function in the output layer of the YOLOv5 convolutional neural network is modified to the HardSwish activation function. The HardSwish activation function is an activation function used in neural networks, which has low computational complexity and high computational accuracy, and speeds up the detection speed. Fig. 9 As shown, the formula is:

[0067] ;

[0068] Among them, in the HardSwish activation function of the output layer, x represents the weight coefficients of different lesions predicted by the tooth image detection model.

[0069] Compared with other common activation functions, the HardSwish activation function only requires some simple comparison, addition and multiplication operations, and is also nonlinear, differentiable and low saturation. Specifically, compared with the ReLU activation function, the HardSwish function is also nonlinear, which can better adapt to complex data distribution and effectively suppress dimensionality collapse; the HardSwish function is continuously differentiable, which makes it possible to calculate the gradient well during the back propagation process; compared with functions such as Sigmoid and Tanh, the HardSwish activation function has relatively low saturation, which can well avoid the problem of gradient disappearance. Therefore, compared with other activation functions, it has obvious advantages.

[0070] Furthermore, considering that the loss function is a very important part of the network structure, it can be used to measure the gap between the predicted value and the true value. The minimum loss function can be optimized by adjusting the parameters, so that the predicted value is closer to the true value. Therefore, this embodiment modifies the original CIOU_Loss loss function in the output layer of the YOLOv5 convolutional neural network to the EIOU loss function. The EIOU loss function is a target loss function that is further optimized based on loss functions such as IoU and CIOU. It mainly separates the influencing factors of the aspect ratio between the target box and the anchor box to calculate the overlap loss and width and height loss more accurately. Specifically, the EIOU loss function includes the following three parts:

[0071] (1) Overlap loss: used to measure the degree of overlap between the target box and the anchor box, that is, the ratio of their intersection area to their union area.

[0072] (2) Center distance loss: used to measure the distance between the center point of the target box and the anchor box. The distance is normalized and used to calculate the overall position loss.

[0073] (3) Width-height loss: used to calculate the width difference between the target box and the anchor box. It is calculated by taking the square sum of the difference between the width and height of the target box and the anchor box, so that the model can predict the size of the target box more accurately.

[0074] like Fig.10 As shown, the calculation formula of the EIOU loss function is:

[0075] ;

[0076] Among them, A is the anchor box, B is the target box, b is the center of the anchor box, and C is the minimum union area of ​​the target box and the anchor box. is the center of the target box, is the height of the target frame, is the width of the target box, h is the height of the anchor box, and w is the width of the anchor box. is the height of the minimum union area, is the width of the minimum union region, is the Euclidean distance, is the IOU loss value at the AB box, is the EIOU loss value.

[0077] Based on the above description of the YOLOv5 convolutional neural network structure, the activation function and loss function in the YOLOv5 convolutional neural network have been modified, and the attention mechanism has been added. Therefore, the structure diagram of the optimized YOLOv5 convolutional neural network (i.e., the tooth image detection model) can be referred to Fig.11 This embodiment describes the optimized YOLOv5 convolutional neural network structure in combination with the image processing process, which may specifically include:

[0078] (1) Input layer: Dental images (such as dental CT images or dental X-rays) are first input into the optimized YOLOv5 convolutional neural network. Dental images may be preprocessed, such as resizing or normalizing, to meet the input requirements of the network.

[0079] (2) Backbone: Responsible for extracting important features from the input image. The backbone network usually consists of multiple convolutional layers (Conv2D), combined with batch normalization and activation functions (BN_SiLU), and C3STR (Cross-column Convolutional Neural Networks) modules. These components help the model focus on key features in the image, such as edges, shapes, and textures, which are crucial for detecting teeth and related lesions.

[0080] (3) Neck network: After the backbone network extracts image features, the “neck” part further refines these features to improve detection performance. It includes upsampling and downsampling operations to manage feature maps at different scales. Through the C2F (cross-stage partial network) module and spatial pyramid pooling (SPPF), the model can combine features of different resolutions to improve its ability to detect dental problems at different scales.

[0081] (4) Prediction Head: In this stage, the processed feature map is passed through additional convolutional layers to generate bounding boxes, segmentation masks, and category labels for the detection objects (such as teeth, caries, and dental diseases). Multiple convolutional layers (Conv2D) are used to output the final detection results.

[0082] This embodiment integrates an attention mechanism (such as the CBAM module) in the YOLOv5 implementation to help the model focus on key areas in the image and prioritize relevant spatial and channel information. In addition, the Swin Transformer (an image processing technology based on the Transformer architecture) architecture is used to extract features from different scales, and continuous window segmentation and shifted windowing techniques are used to help capture more effective local and global contextual information, further improving the generalization ability and performance of the model. At the same time, an EIOU loss function is designed specifically for dental images, which can optimize prediction accuracy when multiple lesion labels are annotated. The input image undergoes multi-layer convolution processing, extracts and refines features at each stage, and makes predictions, using attention mechanisms and customized loss functions to address the specific challenges of dental image recognition.

[0083] S102: Input the tooth image to be detected into the trained tooth image detection model to detect whether there is a lesion area.

[0084] In this embodiment, the trained tooth image detection model is used to detect the tooth image to be detected, and the detection result can be obtained. It should be noted that the tooth image in this embodiment refers to a tooth CT image or a tooth X-ray.

[0085] Furthermore, in order to improve the detection accuracy, before inputting the tooth image to be detected into the trained tooth image detection model, the following steps may be further included:

[0086] Performing a preprocessing operation on the tooth image to be detected to obtain a processed tooth image; the preprocessing operation at least includes size adjustment and flip correction processing;

[0087] Accordingly, the tooth image to be detected is input into the trained tooth image detection model, including:

[0088] The processed tooth image is input into the trained tooth image detection model.

[0089] It should be noted that, before the formal detection, the tooth image to be detected can also be preprocessed, such as resizing and flipping correction, to ensure that the size and format of the image to be detected input into the model meet the model requirements.

[0090] S103: If there is, the lesion area is displayed by marking with a rectangular frame, and the lesion information is displayed; the lesion information at least includes the type, number, size and location information of the lesion.

[0091] When the trained tooth image detection model detects that there is a lesion area in the tooth image to be detected, it is displayed in the form of a rectangular frame annotation, such as Fig.12 and Fig.13 Furthermore, an automatic tooth image detection platform can be built, which can input one or more tooth images and output the lesion analysis of the tooth image, including the type, number, size and location of the lesion inflammation, which can provide medical reference for doctors.

[0092] The tooth image detection method based on the YOLOv5 model provided in the embodiment of the present invention is applied, by obtaining a pre-trained tooth image detection model; the tooth image detection model introduces an attention mechanism based on the YOLOv5 convolutional neural network architecture, and uses the HardSwish activation function as the activation function, and uses the EIOU loss function as the loss function in the model training process; the tooth image to be detected is input into the trained tooth image detection model to detect whether there is a lesion area; if so, the lesion area is displayed by marking with a rectangular frame, and the lesion information is displayed; the lesion information at least includes the type, quantity, size and location information of the lesion. The present invention uses the YOLOv5 convolutional neural network to detect dental images, which can effectively help medical professionals to make auxiliary diagnoses, reduce misdiagnosis and missed diagnosis problems, reduce patient waiting time, and improve detection efficiency. In order to further improve the speed and accuracy of the YOLOv5 convolutional neural network in detecting dental images, an attention mechanism is added on the basis of YOLOv5, so that the model pays more attention to the area of ​​interest, thereby improving the detection accuracy. At the same time, the activation function is also modified to speed up the detection speed, which significantly improves the detection effect. In addition, during the model training process, the EIOU loss function can better ensure the accuracy of the model training.

[0093] Based on the technology of the above embodiment, this embodiment further illustrates and optimizes the technical solution. Specifically, in this embodiment, the training process of the tooth image detection model may include the following steps:

[0094] Obtain a training data set, which includes various tooth images and corresponding label information; the label information is a label identification of the lesion area in the tooth image; the label identification includes an inflammation label, a wisdom tooth label, and a tooth loss label;

[0095] The image name, lesion condition and lesion location of each tooth image in the training data set are stored;

[0096] The tooth image detection model is trained using the training data set. When the training end condition is reached, the training ends and a trained tooth image detection model is obtained.

[0097] It should be noted that in this embodiment, all the dental images used for training need to be labeled. This embodiment may adopt a manual labeling method; or it may adopt an automatic labeling method. For the reliability of the training data set, this embodiment adopts manual labeling, and uses an online data set labeling platform to select the lesion area of ​​the dental image, set labels, and generate a corresponding xml (a markup language used to mark electronic files to make them structured) file. Among them, the lesion type corresponds to the label one by one, including inflammation, wisdom teeth and tooth loss. This embodiment does not limit the training end conditions. For example, the number of iterations can be set to a preset number; or the training error satisfies the preset error.

[0098] Based on the technology of the above embodiment, this embodiment further illustrates and optimizes the technical solution. Specifically, in this embodiment, in order to improve the accuracy of model training, after obtaining the training data set, the following steps may be further included:

[0099] Performing data enhancement processing on the tooth images in the training data set and generating corresponding label information to obtain an expanded training data set; the data enhancement processing at least includes image flipping, adding noise and masking processing;

[0100] Accordingly, training the tooth image detection model using the training data set may include the following steps:

[0101] The tooth image detection model is trained using the expanded training dataset.

[0102] It should be noted that since model training requires a large amount of sample data, some samples in the training data set can be randomly selected for data enhancement to expand the training sample set. The specific steps are as follows, which can be referred to Figures 14 to 17 , Fig.14 is the original tooth image; Fig.15 This is the tooth image after mask processing; Fig.16 This is the tooth image after blurring; Fig.17 This is the tooth image after flipping the image:

[0103] (1) Image flipping: The tooth image is flipped with the center as the origin;

[0104] (2) Blurring: Add Gaussian noise to the tooth image with a mean of zero and a variance of 0.01;

[0105] (3) Mask processing: The original tooth image is multiplied by the template image to extract the desired target area.

[0106] When using the expanded training data set for model training, the entire data set can be randomly selected, with 80% of the data selected as the training set, 10% as the test set, and 10% as the validation set. The above information is also stored in folders, each of which contains the image file name and is arranged in the order of the file name. Model training can be started by reading the text information in the folder, i.e. the label file.

[0107] In order to make the present invention easier to understand, please refer to Fig.18 , Fig.18 An example flow chart of a tooth image detection method provided by an embodiment of the present invention may specifically include:

[0108] The dental image is acquired and preprocessed, such as resizing and flip correction. The improved YOLOv5 neural network model (i.e., the trained dental image detection model) is used for image detection. The detection results are then output. The detection results can be fed back to the doctor as a reference for further judgment and treatment of the patient's condition.

[0109] In addition, the present invention also conducts experiments on various models based on the above-mentioned training data set. Specifically, the size of the input image during training is set to 640×640, the initial learning rate is 0.001, the optimization algorithm uses Adam (Adaptive Moment Estimation, a first-order optimization algorithm that can replace the traditional stochastic gradient descent process, iteratively updating the neural network weights based on training data), the IoU threshold is 0.5, the epoch (refers to the process in which the entire training data set passes through the neural network once and returns) training iteration round is 300 rounds, and the batchsize (the number of samples used for a gradient descent update) of the input sample for each iteration is 4. Among them, mAP (mean Average Precision) is selected as the evaluation index of the target detection algorithm. The following table is a performance comparison table of various algorithms. It can be seen from the experimental results that compared with the traditional target detection algorithm, the mAP value of the improved YOLOv5 target detection algorithm proposed in the present invention reaches 81.9%, which has a higher detection accuracy.

[0110] Table 1 Algorithm performance comparison

[0111]

[0112] The following is an introduction to a tooth image detection device based on the YOLOv5 model provided in an embodiment of the present invention. The tooth image detection device based on the YOLOv5 model described below and the tooth image detection method based on the YOLOv5 model described above can refer to each other.

[0113] Please refer to Fig.19 , Fig.19 A schematic diagram of the structure of a tooth image detection device based on a YOLOv5 model provided in an embodiment of the present invention may include:

[0114] The model acquisition module 100 is used to acquire a pre-trained tooth image detection model; the tooth image detection model introduces an attention mechanism based on the YOLOv5 convolutional neural network architecture, and uses the HardSwish activation function as the activation function and the EIOU loss function as the loss function in the model training process;

[0115] The detection module 200 is used to input the tooth image to be detected into the trained tooth image detection model to detect whether there is a lesion area;

[0116] The detection result display module 300 is used to display the lesion area by marking it with a rectangular frame if it exists, and to display the lesion information; the lesion information at least includes the type, number, size and location information of the lesion.

[0117] Based on the above embodiment, the formula of the HardSwish activation function in the model acquisition module 100 is:

[0118] ;

[0119] Among them, in the HardSwish activation function of the output layer, x represents the weight coefficients of different lesions predicted by the tooth image detection model.

[0120] Based on the above embodiment, the formula of the EIOU loss function in the model acquisition module 100 is:

[0121] ;

[0122] Among them, A is the anchor box, B is the target box, b is the center of the anchor box, and C is the minimum union area of ​​the target box and the anchor box. is the center of the target box, is the height of the target frame, is the width of the target box, h is the height of the anchor box, and w is the width of the anchor box. is the height of the minimum union area, is the width of the minimum union region, is the Euclidean distance, is the IOU loss value at the AB box, is the EIOU loss value.

[0123] Based on the above embodiment, the model acquisition module 100 may include:

[0124] An attention mechanism unit is used to use the YOLOv5 convolutional neural network as the basic architecture of the tooth image detection model and introduce an attention mechanism; wherein the attention mechanism is any one of the SEnet attention mechanism, the CBAM attention mechanism and the ECAnet attention mechanism.

[0125] Based on the above embodiment, the model acquisition module 100 may include:

[0126] A training data set acquisition unit is used to acquire a training data set, wherein the training data set includes various tooth images and corresponding label information; the label information is a label identification of the lesion area in the tooth image; the label identification includes an inflammation label, a wisdom tooth label and a tooth loss label;

[0127] A storage unit, used to store the image name, lesion condition and lesion location of each tooth image in the training data set;

[0128] The training unit is used to train the tooth image detection model using the training data set. When the training end condition is met, the training ends and the trained tooth image detection model is obtained.

[0129] Based on the above embodiment, the tooth image detection device based on the YOLOv5 model may further include:

[0130] A data enhancement module, used to perform data enhancement processing on the tooth images in the training data set and generate corresponding label information to obtain an expanded training data set; the data enhancement processing at least includes image flipping, adding noise and masking processing;

[0131] Accordingly, the training unit may specifically include:

[0132] The training subunit is used to train the tooth image detection model using the expanded training data set.

[0133] Based on the above embodiment, the tooth image detection device based on the YOLOv5 model may further include:

[0134] A preprocessing module, used for performing a preprocessing operation on the tooth image to be detected to obtain the processed tooth image; the preprocessing operation at least includes size adjustment and flip correction processing;

[0135] Accordingly, the detection module 200 may include:

[0136] The detection unit is used to input the processed tooth image into the trained tooth image detection model.

[0137] It should be noted that the order of the modules and units in the above-mentioned tooth image detection device based on the YOLOv5 model can be changed without affecting the logic.

[0138] The tooth image detection device based on the YOLOv5 model provided in the embodiment of the present invention is applied, and the model acquisition module 100 is used to obtain a pre-trained tooth image detection model; the tooth image detection model introduces an attention mechanism based on the YOLOv5 convolutional neural network architecture, and uses the HardSwish activation function as the activation function, and uses the EIOU loss function as the loss function in the model training process; the detection module 200 is used to input the tooth image to be detected into the trained tooth image detection model to detect whether there is a lesion area; the detection result display module 300 is used to display the lesion area by marking it with a rectangular frame if it exists, and display the lesion information; the lesion information at least includes the type, quantity, size and location information of the lesion. The present invention uses the YOLOv5 convolutional neural network to detect dental images, which can effectively help medical professionals to make auxiliary diagnoses, reduce misdiagnosis and missed diagnosis problems, reduce patient waiting time, and improve detection efficiency. In order to further improve the speed and accuracy of the YOLOv5 convolutional neural network in detecting dental images, an attention mechanism is added on the basis of YOLOv5, so that the model pays more attention to the area of ​​interest, thereby improving the detection accuracy. At the same time, the activation function is also modified to speed up the detection speed, which significantly improves the detection effect. In addition, during the model training process, the EIOU loss function can better ensure the accuracy of the model training.

[0139] The following is an introduction to a tooth image detection device based on the YOLOv5 model provided in an embodiment of the present invention. The tooth image detection device based on the YOLOv5 model described below and the tooth image detection method based on the YOLOv5 model described above can refer to each other.

[0140] Please refer to Fig. 20 , Fig. 20 A structural schematic diagram of a tooth image detection device based on a YOLOv5 model provided in an embodiment of the present invention may include:

[0141] A memory 10, used for storing computer programs;

[0142] The processor 20 is used to execute a computer program to implement the above-mentioned tooth image detection method based on the YOLOv5 model.

[0143] The memory 10 , the processor 20 , and the communication interface 31 all communicate with each other via the communication bus 32 .

[0144] In the embodiment of the present invention, the memory 10 is used to store one or more programs, and the program may include program code, and the program code includes computer operation instructions. In the embodiment of the present invention, the memory 10 may store programs for implementing the following functions:

[0145] Obtain a pre-trained tooth image detection model; the tooth image detection model introduces an attention mechanism based on the YOLOv5 convolutional neural network architecture, and uses the HardSwish activation function as the activation function and the EIOU loss function as the loss function in the model training process;

[0146] Input the tooth image to be detected into the trained tooth image detection model to detect whether there is a lesion area;

[0147] If it exists, the lesion area is displayed by marking with a rectangular frame, and the lesion information is displayed; the lesion information at least includes the type, number, size and location information of the lesion.

[0148] In a possible implementation, the memory 10 may include a program storage area and a data storage area, wherein the program storage area may store an operating system and an application required for at least one function, etc.; the data storage area may store data created during use.

[0149] In addition, the memory 10 may include a read-only memory and a random access memory, and provide instructions and data to the processor. A portion of the memory may also include an NVRAM. The memory stores an operating system and operating instructions, executable modules or data structures, or a subset thereof, or an extended set thereof, wherein the operating instructions may include various operating instructions for implementing various operations. The operating system may include various system programs for implementing various basic tasks and processing hardware-based tasks.

[0150] The processor 20 may be a central processing unit (CPU), an application specific integrated circuit, a digital signal processor, a field programmable gate array or other programmable logic device, a microprocessor or any conventional processor, etc. The processor 20 may call a program stored in the memory 10 .

[0151] The communication interface 31 may be an interface of a communication module, and is used to connect to other devices or systems.

[0152] Of course, it should be noted that Fig. 20 The structure shown does not constitute a limitation on the tooth image detection device based on the YOLOv5 model in the embodiment of the present invention. In actual applications, the tooth image detection device based on the YOLOv5 model may include Fig. 20 More or fewer components than shown, or combinations of certain components.

[0153] The following is an introduction to the readable storage medium provided in an embodiment of the present invention. The readable storage medium described below and the tooth image detection method based on the YOLOv5 model described above can be referenced to each other.

[0154] The present invention also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the above-mentioned tooth image detection method based on the YOLOv5 model are implemented.

[0155] The computer-readable storage medium may include: a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and other media that can store program codes.

[0156] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0157] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0158] Finally, it should be noted that, in this article, relationships such as first and second, etc. are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprises" or any other variations are intended to cover non-exclusive inclusion, so that a process, method, article or device that includes a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.

[0159] The above is a detailed introduction to a tooth image detection method, device, equipment and computer-readable storage medium based on the YOLOv5 model provided by the present invention. Specific examples are used in this article to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea; at the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.

Claims

1. A tooth image detection device based on the YOLOv5 model, characterized in that: include: A model acquisition module, used to acquire a pre-trained tooth image detection model; the tooth image detection model introduces an attention mechanism based on the YOLOv5 convolutional neural network architecture, and uses the HardSwish activation function as the activation function and the EIOU loss function as the loss function in the model training process; A detection module, used for inputting the tooth image to be detected into the trained tooth image detection model to detect whether there is a lesion area; The detection result display module is used to display the lesion area by marking it with a rectangular frame if it exists, and display the lesion information; the lesion information at least includes the type, number, size and location information of the lesion.

2. The tooth image detection device based on the YOLOv5 model according to claim 1, characterized in that: The formula of the HardSwish activation function in the model acquisition module is: ； Among them, in the HardSwish activation function of the output layer, x represents the weight coefficients of different lesions predicted by the tooth image detection model.

3. The tooth image detection device based on the YOLOv5 model according to claim 1, characterized in that: The formula of the EIOU loss function in the model acquisition module is: ； Among them, A is the anchor box, B is the target box, b is the center of the anchor box, and C is the minimum union area of ​​the target box and the anchor box. is the center of the target box, is the height of the target frame, is the width of the target box, h is the height of the anchor box, and w is the width of the anchor box. is the height of the minimum union area, is the width of the minimum union region, is the Euclidean distance, is the IOU loss value at the AB box, is the EIOU loss value.

4. The tooth image detection device based on the YOLOv5 model according to claim 1, characterized in that: Model acquisition module, including: An attention mechanism unit is used to use the YOLOv5 convolutional neural network as the basic architecture of the tooth image detection model and introduce an attention mechanism; wherein the attention mechanism is any one of the SEnet attention mechanism, the CBAM attention mechanism and the ECAnet attention mechanism.

5. The tooth image detection device based on the YOLOv5 model according to claim 1, characterized in that: Model acquisition module, including: A training data set acquisition unit is used to acquire a training data set, wherein the training data set includes various tooth images and corresponding label information; the label information is a label identification of the lesion area in the tooth image; the label identification includes an inflammation label, a wisdom tooth label and a tooth loss label; A storage unit, used to store the image name, lesion condition and lesion location of each tooth image in the training data set; The training unit is used to train the tooth image detection model using the training data set. When the training end condition is met, the training ends and the trained tooth image detection model is obtained.

6. The tooth image detection device based on the YOLOv5 model according to claim 5, characterized in that: Also includes: A data enhancement module, used to perform data enhancement processing on the tooth images in the training data set and generate corresponding label information to obtain an expanded training data set; The data enhancement processing at least includes image flipping, adding noise and masking processing; Accordingly, the training unit includes: The training subunit is used to train the tooth image detection model using the expanded training data set.

7. The tooth image detection device based on the YOLOv5 model according to claim 1, characterized in that: Also includes: A preprocessing module performs a preprocessing operation on the tooth image to be detected to obtain the processed tooth image; The preprocessing operation includes at least resizing and flip correction processing; Correspondingly, the detection module includes: The detection unit is used to input the processed tooth image into the trained tooth image detection model.

8. A tooth image detection method based on the YOLOv5 model, characterized in that: Applicable to processors, including: Obtain a pre-trained tooth image detection model; the tooth image detection model introduces an attention mechanism based on the YOLOv5 convolutional neural network architecture, and uses the HardSwish activation function as the activation function and the EIOU loss function as the loss function in the model training process; Inputting the tooth image to be detected into the trained tooth image detection model to detect whether there is a lesion area; If it exists, the lesion area is displayed by marking with a rectangular frame, and the lesion information is displayed; the lesion information at least includes the type, number, size and location information of the lesion.

9. A tooth image detection device based on the YOLOv5 model, characterized in that: include: Memory for storing computer programs; A processor, configured to implement the tooth image detection method based on the YOLOv5 model as claimed in claim 8 when executing the computer program.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, and when the computer-executable instructions are loaded and executed by the processor, the tooth image detection method based on the YOLOv5 model as described in claim 8 is implemented.

Images