A sputum sample cell detection method based on YOLOV8

Abstract

This invention provides a method for cell detection in sputum specimens based on YOLOv8, comprising the following steps: Step 1, acquiring a dataset of Gram-stained slide images and preprocessing it into a training set, a validation set, and a test set; Step 2, improving YOLOv8 by combining the convolutional attention mechanism module CBAM and CIoU GrayMSE Loss to obtain an improved network model; Step 3, inputting the training set into the network model for iterative training to obtain the optimal target detection model; Step 4, inputting the test set into the optimal network model for prediction, and obtaining the target detection result through the tested target detection network model; This invention can solve the problems of false detection and false negative detection of cells in sputum slide sample microscopic images under complex backgrounds in the prior art.

Description

Technical Field

[0001] The present invention relates to the field of image processing and computer vision technology, and in particular to a sputum sample cell detection method based on YOLOV8. Background Art

[0002] Compared with traditional visual algorithms, image semantic segmentation technology based on deep convolutional neural networks is widely used in cell detection. The application prospects of deep learning in the medical field are very broad. It can help doctors improve the accuracy and efficiency of diagnosis. With the continuous development of technology, the application of deep learning in the medical field will continue to be improved and perfected. Under the current technical background, the sputum slide samples currently used have complex backgrounds and diverse cell morphologies after imaging. Due to the large amount of interference, there are some false detections and missed detections in the cell detection method for sputum slide imaging images, which leads to reduced detection accuracy of white blood cells and epithelial cells.

[0003] Therefore, how to provide a sputum sample cell detection method for YOLOV8 is a problem that technical personnel in this field urgently need to solve. Summary of the invention

[0004] The purpose of the present invention is to provide a sputum specimen cell detection method based on YOLOV8, which, in actual use, can solve the problems of false detection and missed detection of cells in microscopic images of sputum slide samples under complex backgrounds in the prior art.

[0005] In order to solve the above technical problems, the technical solution adopted by the present invention is:

[0006] A sputum sample cell detection method based on YOLOV8, comprising the following steps:

[0007] Step 1: Obtain a Gram-stained slide image dataset and preprocess it into a training set, a validation set, and a test set;

[0008] Step 2: Improve YOLOV8 by combining the convolutional attention mechanism module CBAM and CIoU GrayMSE Loss to obtain an improved network model;

[0009] Step 3: Input the training set into the network model for iterative training to obtain the optimal target detection network model;

[0010] Step 4: input the test set into the optimal network model for prediction, and obtain the target detection result through the tested target detection network model.

[0011] Furthermore, in step 1, the training set is used to train the network model so that the network model can learn the features and patterns in the image through the training set data for prediction; the validation set is used to adjust the hyperparameters of the network model and select the best network model; and the test set is used to finally evaluate the performance of the network model.

[0012] Furthermore, in step 1, the ratio of the training set, the validation set and the test set is 6:2:2.

[0013] Furthermore, in step 2, the CBAM includes a channel attention module and a spatial attention module; the channel attention module and the spatial attention module are used to be embedded in different layers of CNN respectively to enhance feature representation.

[0014] Furthermore, the channel attention module is used to aggregate the spatial information of the input feature map F through average pooling and maximum pooling operations and then pass it through a shared network. Finally, the obtained eigenvalues ​​are added and the activation function is used to obtain the weight coefficient of the channel domain; the channel domain weight coefficient is multiplied by the input feature map to obtain the channel attention module feature map Fˊ.

[0015] Furthermore, the spatial attention module is used to obtain two channel feature matrices by average pooling and maximum pooling of the feature map Fˊ of the input channel attention module, superimpose them and then obtain the spatial domain weight coefficient through convolution and activation function; the spatial domain weight coefficient is multiplied by the channel attention module feature map Fˊ to obtain the CBAM attention mechanism module feature map Fˊˊ.

[0016] Furthermore, the CIoU GrayMSE Loss is used as a bounding box regression loss function to improve the overall performance of the detector.

[0017] Compared with the prior art, the present invention has the following beneficial effects:

[0018] Based on the existing YOLOV8, the present invention adds an attention mechanism for the features of white blood cells and epithelial cells on Gram-stained sputum slide sample images, and uses the characteristics of cells containing nuclei to improve the loss function and introduce the features of nuclei. After the training set is input into the network model, the optimal network model is obtained through multiple iterations of training, thereby improving the accuracy of white blood cell and epithelial cell detection in Gram-stained sputum slide sample microscopic images. The problem of false detection and missed detection of cells in sputum slide sample microscopic images under complex backgrounds is solved. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0020] Figure 1 This is a flow chart of the sputum sample cell detection method based on YOLOV8 in the present invention.

[0021] Figure 2 This is a flow chart comparing the improved network model in the present invention with the original YOLOV8 model.

[0022] Figure 3 It is a schematic diagram of the structure of the CBAM module in the present invention.

[0023] Figure 4 This is a workflow diagram of the channel attention module in the present invention.

[0024] Figure 5 This is a workflow diagram of the spatial attention module in the present invention.

[0025] Figure 6 This is a diagram showing the cell recognition effect of white blood cells in the present invention.

[0026] Figure 7 This is a diagram showing the cell recognition effect of leukocytes and epithelial cells in the present invention. DETAILED DESCRIPTION

[0027] In the following, only some exemplary embodiments are briefly described. As those skilled in the art will appreciate, the described embodiments may be modified in various ways without departing from the spirit or scope of the embodiments of the present invention. Therefore, the drawings and descriptions are considered to be exemplary and non-restrictive in nature.

[0028] In the description of the embodiments of the present invention, it should be understood that the terms "length", "vertical", "horizontal", "top", "bottom", etc. indicating the orientation or positional relationship are based on the orientation or positional relationship shown in the accompanying drawings, and are only for the convenience of describing the embodiments of the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the embodiments of the present invention.

[0029] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of the embodiments of the present invention, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

[0030] In the embodiments of the present invention, unless otherwise clearly specified and limited, the terms "installed", "connected", "connected", "fixed" and the like should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection, an electrical connection, or a communication; it can be a direct connection, or an indirect connection through an intermediate medium, and it can be the internal connection of two elements or the interaction relationship between two elements. For ordinary technicians in this field, the specific meanings of the above terms in the embodiments of the present invention can be understood according to specific circumstances.

[0031] In the embodiments of the present invention, unless otherwise clearly specified and limited, a first feature being "above" or "below" a second feature may include the first and second features being in direct contact, or may include the first and second features being in contact not directly but through another feature between them. Moreover, a first feature being "above", "above" and "above" a second feature includes the first feature being directly above and obliquely above the second feature, or simply indicates that the first feature is higher in level than the second feature. A first feature being "below", "below" and "below" a second feature includes the first feature being directly below and obliquely below the second feature, or simply indicates that the first feature is lower in level than the second feature.

[0032] The disclosure below provides many different embodiments or examples to implement different structures of the embodiments of the present invention. In order to simplify the disclosure of the embodiments of the present invention, the parts and settings of specific examples are described below. Of course, they are only examples, and the purpose is not to limit the embodiments of the present invention. In addition, the embodiments of the present invention can repeat reference numbers and / or reference letters in different examples, and this repetition is for the purpose of simplification and clarity, which itself does not indicate the relationship between the various embodiments and / or settings discussed.

[0033] The embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0034] Embodiment 1

[0035] See also Figure 1-Figure 7 ,like Figure 1 As shown, this embodiment discloses a sputum sample cell detection method based on YOLOV8, comprising the following steps:

[0036] Step 1: Obtain a Gram-stained slide image dataset and preprocess it into a training set, a validation set, and a test set;

[0037] Step 2: Improve YOLOV8 by combining the convolutional attention mechanism module CBAM and CIoU GrayMSE Loss to obtain an improved network model;

[0038] Step 3: Input the training set into the network model for iterative training to obtain the optimal target detection network model;

[0039] Step 4: input the test set into the optimal network model for prediction, and obtain the target detection result through the tested target detection network model.

[0040] In step 1, first obtain a dataset of microscopic images of Gram-stained slides and divide it into a training set, a validation set, and a test set in a ratio of 6:2:2.

[0041] like Figure 2 As shown, in step 2, the existing network model is improved;

[0042] The workflow of the network model of the prior art before improvement is:

[0043] The image enters ConV convolution layer → ConV convolution layer → C2f layer → ConV convolution layer → C2f layer → ConV convolution layer → C2f layer → ConV convolution layer → C2f layer → ConV convolution layer → C2f layer → SPPF;

[0044] The workflow of the improved network model is:

[0045] The image enters ConV convolution layer → ConV convolution layer → C2f layer → CBAM layer → ConV convolution layer → C2f layer → CBAM convolution layer → ConV convolution layer → C2f layer → ConV convolution layer → C2f convolution layer → SPPF;

[0046] After passing through two layers of Conv convolution modules, it is connected to the C2f module and then enters the CBAM module to strengthen channel and spatial attention. Then it enters a Conv convolution module and a C2f module and then enters the CBAM module again to strengthen channel and spatial attention. Then it enters a Conv convolution module, a C2f module, a Conv convolution module, a C2f module, and finally enters the SPPF module to fuse the features in feature channels of different scales.

[0047] The target feature extraction network backbone in the network model adds a CBAM attention mechanism module to extract features from the image. After adding the convolutional attention mechanism module CBAM, the impact of redundant information introduced by various complex backgrounds on the detection accuracy can be reduced, so that attention can be focused on the area of ​​interest during subsequent detection.

[0048] Among them, the Conv module is a convolutional layer, which usually includes the number of convolution kernels, the size of the convolution kernels and the stride of the convolution operation.

[0049] C2f module: It is a complex module composed of multiple convolutional layers and activation functions, usually including two main convolution operations, whose outputs are merged. C2f is used in YOLOV8 to increase the expressive power of the network model and capture richer features while reducing the amount of computation.

[0050] SPPF module: It adopts the idea of ​​spatial pyramid pooling and integrates the information of feature maps through multi-scale pooling operations. SPPF means using 1 or more channels and a pooling layer to enhance the detection ability of objects of different scales.

[0051] like Figure 3 As shown in the figure, the CBAM consists of two key parts: the channel attention module (C-channel) and the spatial attention module (S-channel). The channel attention module (C-channel) and the spatial attention module (S-channel) can be embedded in different layers of CNN respectively to enhance feature representation. The weight of each channel is adjusted by calculating the channel importance to emphasize the feature response to a specific channel. The spatial distribution of the feature map is adjusted by calculating the weight of each spatial position to focus on important areas in the image.

[0052] Among them, Figure 4 As shown, the channel attention module (S-channel) is used to aggregate the spatial information of the input feature map F through average pooling and maximum pooling operations and then pass it through a shared network. Finally, the obtained feature values ​​are added and the activation function is used to obtain the channel domain weight coefficient; the channel domain weight coefficient is multiplied by the input feature map to obtain the channel attention module (S-channel) feature map Fˊ.

[0053] Among them, Figure 5As shown in the figure, the feature map Fˊ of the channel attention module (S-channel) is used as the input of the spatial attention module (S-channel). Two channel feature matrices are obtained through average pooling and maximum pooling, which are superimposed and then subjected to convolution and activation functions to obtain the spatial domain weight coefficient. The spatial domain weight coefficient is multiplied by the feature map Fˊ of the channel attention module (S-channel) to obtain the feature map Fˊˊ of the CBAM attention mechanism module.

[0054] Wherein, in step 1, the ratio of the training set, validation set and test set is 6:2:2. The 60% of the training set enables the network model to obtain enough samples for learning, so as to capture the patterns in the data. The 20% of the validation set is used to adjust the network model hyperparameters and select the best network model. This part of the data does not participate in the weight update during the training process, which helps monitor the performance of the network model and prevent overfitting. The 20% of the test set is used to finally evaluate the performance of the network model. This part of the data can provide an objective performance indicator for the network model. This allocation method can maintain a certain balance between training, validation and testing, and ensure that the amount of data at each stage is sufficient for effective evaluation and tuning.

[0055] Embodiment 2

[0056] See also Figure 1-Figure 7 In order to improve the detection accuracy of white blood cells and epithelial cells in Gram-stained sputum slide sample microscopic images, the present invention newly adds a CIoU GrayMSE Loss loss function calculation method; in step 2, after the target detection layer in the network model predicts the target area with the features extracted by the network model, the improved loss function module CIoUGrayMSE Loss is introduced as the border regression loss function, which can improve the overall performance of the detector.

[0057] The original YOLOV8 loss function evaluation calculation methods include IoU, DIoU, GIoU, and CIoU.

[0058] IoU: A measure of the degree of overlap between two bounding boxes. It is calculated by dividing the area of ​​the overlapping region by the area of ​​the union of the two boxes.

[0059] DIoU: Based on IoU, the distance between the center points of the bounding boxes is added. The distance between boxes is taken into account, so that the box with a longer distance is penalized more, which improves the convergence speed and accuracy.

[0060] GIoU: An extension of IoU, improved by considering the minimum bounding box of two boxes. It adds additional area measurements on top of IoU to handle situations where IoU is close to 0.

[0061] CIoU: further improves DIoU by introducing the aspect ratio consistency of the bounding box. The stability and accuracy of the regression are improved by considering the center distance, aspect ratio and IoU of the box (iou is the iou value calculated by one of the above methods).

[0062] Loss=((1.0-iou)*weight).sum() / target_scores_sum;

[0063] Weight: Multiply the IoU loss by the weight, which is usually used to weight the importance or priority of each sample.

[0064] Sum: sum of weighted IoU losses of all samples;

[0065] target_scores_sum: Normalizes by the sum of target scores to ensure that the loss is averaged over the number of targets.

[0066] Considering that both white blood cells and epithelium contain nuclei in Gram-stained sputum slide microscopy images, by separating the image RGB, on the green channel (G), the nuclei are darker in color and can be distinguished under complex background interference. Therefore, when identifying features, it is necessary to consider whether the identified cell BBOX contains a darker nucleus, and whether the nucleus has a large gray value difference with the cytoplasm and background. This is to prevent the area with dye from being identified as a cell (without a nucleus). Therefore, the iou value is obtained after calculating CIoU. It is also necessary to calculate the gray value difference between the predicted box and the real box.

[0067] The specific steps of calculating the CIoU GrayMSE Loss function are as follows

[0068] Step 1: Gray value difference calculation

[0069] First, a mean filter with a 3x3 convolution kernel is applied to the image in the frame to remove noise interference.

[0070] filtered_image = F.avg_pool2d(image, kernel_size=kernel_size, stride=1,padding=kernel_size / / 2);

[0071] Then, calculate the maximum and minimum grayscale values ​​respectively.

[0072] max_val, _ = filtered_image.max(dim=(1, 2));

[0073] min_val, _ = filtered_image.min(dim=(1, 2));

[0074] Finally, we get the difference:

[0075] diff = max_val - min_val;

[0076] Step 2: Calculate the mean square MSE of the gray value differences

[0077] gray_diff_loss = F.mse_loss(torch.stack(pred_gray_diffs), torch.stack(true_gray_diffs));

[0078] pred_gray_diffs: the gray value difference of the predicted box;

[0079] true_gray_diffs: the gray value difference of the true frame;

[0080] Step 3: Integrate the difference loss into the original loss to form a new CIoU GrayMSE Loss.

[0081] Before improvement: Loss = ((1.0-iou) * weight).sum() / target_scores_sum;

[0082] After improvement: CIoU GrayMSE Loss= ((1.0 - iou) * weight).sum() / target_scores_sum+ weight_gray* gray_diff_loss;

[0083] weight_gray: is the influence weight coefficient of the grayscale difference.

[0084] In step 3, the network model is trained on the PC side. After 300 epochs of iterative training, the network model file with the lowest loss is saved as the optimal network model for this training. The specific method flow of iterative training is as follows:

[0085] Initialization: Select the initial network model structure and hyperparameters.

[0086] Training: Use the training set data to train the network model and update the network model parameters.

[0087] Validation: Use the validation set to evaluate the performance of the network model and calculate indicators such as loss value or accuracy.

[0088] Adjustment: Adjust the network model hyperparameters or structure based on the validation results.

[0089] Repeat: Repeat training, validation, and adjustment until the network model performance meets the requirements or reaches the stopping criteria.

[0090] The performance of the network model on the validation set and the test set can be used to determine whether the optimal network model is obtained. Select the network model configuration that performs best on the validation set. Indicators such as accuracy, loss value, and F1 score can help determine the best network model.

[0091] In this embodiment, the network model can learn the features and patterns in the image through the training set data; adjusting the network model hyperparameters according to the verification results specifically means that during the training process, the verification set is used to verify the network model, and adaptive adjustments (learning rate, optimizer, regularization parameters, etc.) are made based on the verification results.

[0092] In step 4, the test set is input into the optimal network model for prediction, and the target detection result is obtained through the tested target detection network model.

[0093] Although the preferred embodiments of the present invention have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.

[0094] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. It should be pointed out that any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A sputum sample cell detection method based on YOLOV8, characterized in that: The following steps are involved: Step 1: Obtain a Gram-stained slide image dataset and preprocess it into a training set, a validation set, and a test set; Step 2: Improve YOLOV8 by combining the convolutional attention mechanism module CBAM and CIoU GrayMSE Loss to obtain an improved network model; After passing through two layers of Conv convolution modules, it is connected to the C2f module, and then enters the CBAM module to strengthen the attention of channels and space. Then it enters a Conv convolution module, a C2f module, and then enters the CBAM module again to strengthen the attention of channels and space. Then it enters a Conv convolution module, a C2f module, a Conv convolution module, a C2f module, and finally enters the SPPF module to fuse the features in feature channels of different scales. In step 2, after the target detection layer in the network model predicts the target area with the features extracted by the network model, the improved loss function module CIoU GrayMSE Loss is introduced as the border regression loss function; The specific steps of calculating the CIoU GrayMSE Loss function are as follows Step 1: Gray value difference calculation First, a mean filter with a 3x3 convolution kernel is applied to the image in the frame to remove noise interference; filtered_image = F.avg_pool2d(image, kernel_size=kernel_size, stride=1,padding=kernel_size / / 2); Then, calculate the maximum and minimum grayscale values ​​respectively. max_val, _ = filtered_image.max(dim=(1, 2)); min_val, _ = filtered_image.min(dim=(1, 2)); Finally, we get the difference: diff = max_val - min_val; Step 2: Calculate the mean square MSE of the gray value differences gray_diff_loss=F.mse_loss(torch.stack(pred_gray_diffs),torch.stack(true_gray_diffs)); pred_gray_diffs: the gray value difference of the predicted box; true_gray_diffs: the gray value difference of the true frame; Step 3: Integrate the difference loss into the original loss to form a new CIoU GrayMSE Loss; Before improvement: Loss = ((1.0-iou) * weight).sum() / target_scores_sum; After improvement: CIoU GrayMSE Loss= ((1.0-iou) * weight).sum() / target_scores_sum+weight_gray* gray_diff_loss; weight_gray: is the influence weight coefficient of the grayscale difference; Step 3: Input the training set into the network model for iterative training to obtain the optimal target detection network model; Step 4: input the test set into the optimal network model for prediction, and obtain the target detection result through the tested target detection network model.

2. A sputum sample cell detection method based on YOLOV8 according to claim 1, characterized in that: In step 1, the training set is used to train the network model, so that the network model can learn the features and patterns in the image through the training set data for prediction; The validation set is used to adjust the hyperparameters of the network model and select the best network model; the test set is used to finally evaluate the performance of the network model.

3. A sputum sample cell detection method based on YOLOV8 according to claim 2, characterized in that: In step 1, the ratio of the training set, the validation set and the test set is 6:2:

2.

4. A sputum sample cell detection method based on YOLOV8 according to claim 1, characterized in that: In step 2, the CBAM includes a channel attention module and a spatial attention module; the channel attention module and the spatial attention module are used to be embedded in different layers of CNN respectively to enhance feature representation.

5. A sputum sample cell detection method based on YOLOV8 according to claim 4, characterized in that: The channel attention module is used to aggregate the spatial information of the input feature map F through average pooling and maximum pooling operations and then pass it through a shared network. Finally, the obtained feature values ​​are added and the channel domain weight coefficient is obtained by using an activation function; the channel domain weight coefficient is multiplied by the input feature map to obtain the channel attention module feature map Fˊ.

6. A sputum sample cell detection method based on YOLOV8 according to claim 5, characterized in that: The spatial attention module is used to obtain two channel feature matrices by average pooling and maximum pooling of the feature map Fˊ of the input channel attention module, superimpose them and then obtain the spatial domain weight coefficient through convolution and activation function; the spatial domain weight coefficient is multiplied by the channel attention module feature map Fˊ to obtain the CBAM attention mechanism module feature map Fˊˊ.

7. A sputum sample cell detection method based on YOLOV8 according to claim 1, characterized in that: The CIoU GrayMSE Loss is used as a bounding box regression loss function to improve the overall performance of the detector.

Images