Cell detection performance improving method based on YOLOv8

Abstract

Collecting image data of the medical image cells, and marking the screened image data; preprocessing image data, dividing a data set, modifying a backbone network and a feature fusion network, and sending a training set to a network model for training; the verification set verifies a training result and updates a weight file of the network model to obtain an optimal weight file of the cell image detection model; and testing the network model, and evaluating the performance. The method has the beneficial effects that the cell target detection performance and the generalization ability and accuracy of the model are improved, and meanwhile, the perception ability and expression ability of the model are enhanced.

Description

Technical Field

[0001] The present invention relates to the technical fields of target detection, image processing, and multi-scale fusion, and in particular to a cell detection method of YOLOv8 based on an attention mechanism and multi-scale feature fusion. Background Art

[0002] YOLO is an object detection algorithm that uses a single neural network model to simultaneously locate and classify objects. Compared to traditional object detection methods, YOLO offers superior real-time performance. YOLOv8 is the latest version of the YOLO family, improving upon YOLOv5. YOLOv8 utilizes a deeper neural network architecture and introduces additional technical details and design improvements to enhance object detection accuracy and performance.

[0003] Cell visual inspection utilizes computer vision and image processing technologies to automatically analyze and inspect biological cells. It has significant applications in medical diagnosis, drug development, and biological research. Traditionally, cell visual inspection uses a manual microscope to observe and record cell images. These images are then analyzed and measured manually. With the rapid development of computer technology, cell visual inspection technology based on computer vision and image processing has gradually emerged. This new technology combines image acquisition, image processing, and pattern recognition to achieve automated cell analysis and inspection. It can help doctors quickly and accurately detect and diagnose diseases in medical diagnosis. Furthermore, it can be used in drug development to assess the effects and efficacy of new drugs on cells. Furthermore, biological research can use cell visual inspection technology to study cellular physiological activities and pathological changes. However, cell visual inspection technology also faces challenges in terms of accuracy. Cell images can be affected by noise, blur, and other quality issues. Extracting meaningful features from cell images is crucial for classification and recognition. Therefore, improving recognition accuracy and overall performance has become a pressing technical challenge. Summary of the Invention

[0004] This invention aims to overcome the shortcomings of the existing technology by providing a cell detection method based on the YOLOv8 attention mechanism and multi-scale feature fusion, improving the following aspects: target detection accuracy, target localization precision, target recognition capability, and model generalization capability. This invention is implemented using the following technical solution, which includes the following steps: image acquisition, image preprocessing, cell segmentation, feature extraction, and classification and recognition.

[0005] Step S1, obtaining cell images through a high-resolution microscope or other equipment, cleaning the image data and screening invalid image data.

[0006] In step S2, the image is preprocessed, including denoising, enhancement, and image enhancement. By increasing the diversity of training data, the generalization ability of the model can be improved. Techniques such as flipping are used to increase the number and diversity of training samples to improve image quality.

[0007] In step S3, the cells are separated from the background and labeled, and the labeled image dataset is divided into training set, validation set, and test set.

[0008] In step S4, the attention mechanism is added to the YOLOv8 model network to perform multi-scale feature fusion, residual connections are introduced, and the improved loss function is WIoU.

[0009] In step S5, the training parameters of the improved YOLOv8 network model are set and the divided training set is fed into the improved YOLOv8 network model for training. The model then classifies and identifies cells, identifying specific cell types. The validation set verifies the results of the improved YOLOv8 network model training and continuously updates the training parameters to obtain the optimal weight file. The final training results are then verified on the test set.

[0010] Furthermore, the specific process of step S1 is as follows:

[0011] In step S11, a high-resolution microscope and an appropriate slide are prepared. The cell sample to be observed is placed on the slide. The slide is placed on the microscope stage and the microscope focus is adjusted to ensure a clear image. A camera or image capture device is used to capture the image. Ensure that appropriate exposure time, focus, and image parameter settings are used to obtain a high-quality image.

[0012] In step S12, the images are pre-processed, such as by adjusting brightness, contrast, or color balance, to ensure consistent image quality. The images are inspected for noise or artifacts, and image processing algorithms or filters are used to reduce these artifacts. Low-quality images, such as blurred, overexposed, or underexposed images, are deleted to prevent them from interfering with subsequent analysis.

[0013] Step S13 defines criteria for filtering invalid images, such as cell integrity and background interference. Using image processing algorithms or machine learning methods, images are compared against these criteria and invalid images are identified. Invalid images are removed from the dataset, retaining only valid images for subsequent analysis.

[0014] Furthermore, the specific process of step S2 is as follows:

[0015] Step S21, denoising: using a denoising algorithm such as median filtering to reduce noise in the image, thereby improving the clarity and details of the image.

[0016] Step S22, enhancement: increasing the contrast, brightness and color saturation of the image through adaptive histogram equalization to improve the visibility of the image.

[0017] Step S23, flip: flip the image horizontally or vertically to increase the amount of training data and help the model learn features of different angles and directions.

[0018] Furthermore, the specific process of step S3 is as follows:

[0019] In step S31, the red blood cell image data is annotated using a labeling tool (Labelimg) to generate a VOC format annotation file. The VOC format file is converted into a YOLO format suitable for the YOLOv8 network model using code. The YOLO format file includes the category name of the target to which the image data belongs, the center coordinates of the target's true frame, and the width and height.

[0020] Step S32 , the .xml tag files and .jpg image files in the labeled red blood cell dataset are divided into a training set, a validation set, and a test set in a ratio of 8:1:1.

[0021] Furthermore, the improvement of the backbone network in step S4 is as follows:

[0022] Step S41: First, in the YOLOv8 network, the backbone network Darknet53 is replaced with EfficientNet. The backbone network has more layers and parameters and can provide better feature representation capabilities.

[0023] In step S42, the basic convolution block of the backbone network and the CBAM (Channel Attention and Spatial Attention) attention mechanism are combined to form a new attention convolution module (CACBS, CBAM Attention Convolution Block).

[0024] Step S43: Increase the scale of the output feature map. YOLOv8 only has three output feature maps (P3, P4, and P5). Add more output feature maps, P6 and P7. This provides more feature maps of different scales, which helps detect objects of different sizes.

[0025] In step S44, all downsampling modules of the backbone network are replaced with maximum pooling modules (DP) with a kernel size of 2x2. Data enhancement and regularization techniques such as Dropout are used to increase data diversity and the generalization ability of the model, as shown in the following formula.

[0026] ; ; ; ; The Bernoulli function is used to randomly generate a vector of 0 and 1 with a probability p (usually set to 0.5 or 0.3). It is to make a neuron inactive with a probability of p (activation value is 0).

[0027] Step S45: Introduce a residual connection between the Convolutional layer and the Route layer. Both layers can be dimensionally matched, making residual connections easier to implement. In the backbone network, a residual connection is introduced between the Convolutional layer of each module and the Route layer of the next module. Residual Connection: ; in Variables representing inputs; Layer Normalization: ; ; ; in is the vector representation of the neurons in this layer, is the number of hidden neurons in the layer, To add a small value to the variance to prevent division by zero, is a trainable scale parameter, is a trainable bias parameter.

[0028] In step S46, the improved loss function WIoU takes into account the differences in relative size and position between objects and introduces a self-weight adjustment mechanism to adaptively adjust the weight coefficients and improve the flexibility and robustness of object detection. The WIoU is calculated as follows: ; ; ; in , Indicates the coordinates of the center point of the actual frame, , It represents the height and width of the minimum enclosing box formed by the predicted box and the actual box, and IoU represents the intersection over union ratio. is the bounding box regression loss.

[0029] Furthermore, the specific process of model training in step S5 is as follows:

[0030] In step S51, the training set is fed into the improved YOLOv8 network model for training, wherein the category variables are stored in three categories: WBC, RBC, and Platelets. That is, WBC represents white blood cells, RBC represents red blood cells, and Platelets represents platelets, so as to classify and identify cells.

[0031] In step S52, the trained improved YOLOv8 network model is tested on the processed test set to calculate the probability of predicted positive samples to all correct predictions, i.e., precision, the ratio of predicted positive samples to correctly predicted positive samples and correctly predicted negative samples, i.e., recall, the harmonic mean F1 of precision and recall, and the average precision (mAP) of the category to evaluate the performance of the model.

[0032] Beneficial effects of the present invention: The benefit of introducing the CBAM attention mechanism in YOLOv8 is that it improves the model's ability to focus on objects. The CBAM attention mechanism automatically learns which features in the feature map are more important for object detection and adjusts weights to enhance these important features. Combining the CBAM attention mechanism with the basic convolutional blocks of the backbone network to form a new attention convolution module allows for more targeted feature weighting during the feature extraction phase. This improves the model's perception and representation capabilities, thereby enhancing object detection accuracy. Replacing the backbone network with EfficientNet offers the benefits of improved model lightweighting and computational efficiency. EfficientNet's network design, based on network scaling factors, reduces parameters and computational overhead while maintaining model performance. This makes YOLOv8 more feasible and scalable for object detection tasks. Increasing the scale of the output feature map improves the model's detection capabilities for objects of varying scales. By fusing multi-scale feature maps, the model can better capture objects of varying sizes and more accurately localize and classify them. Regularization techniques can help prevent overfitting and improve model generalization. Regularization techniques such as dropout and L1 / L2 regularization can reduce model complexity, prevent overfitting of the training data, and enable the model to learn features with greater generalizability. Introducing residual connections can help avoid the vanishing gradient problem, making it easier to optimize the model and maintain effective gradient flow during training. Residual connections also facilitate information transfer, helping the model learn deeper feature representations and improving object detection performance. BRIEF DESCRIPTION OF THE DRAWINGS Figure 1 Flow chart of the method of the present invention.

[0033] References: Srivastava, Nitish, Hinton, Geoffrey, Krizhevsky, Alex, Sutskever, Ilya, Salakhutdinov, Ruslan. Dropout: A Simple Way to Prevent Neural Networks from Overtting Ioffe S, Szegedy C. Batch normalization: Accelerating deep network training by reducing internal covariate shift[J]. arXiv preprint arXiv:1502.03167, 2015. Gang Wang, Yanfei Chen, Pei An, Hanyu Hong, Jinghu Hu and Tiange Huang. UAV-YOLOv8: A Small-Object-Detection Model Based on Improved YOLOv8 for UAV Aerial Photography Scenarios[J]. Sensors 2023, 23, 7190.

Claims

1. The present invention aims to overcome the shortcomings of the prior art by providing a cell detection method using YOLOv8 based on an attention mechanism and multi-scale feature fusion to improve the following aspects: target detection accuracy, target localization precision, target recognition capability, and model generalization capability. The present invention is implemented using the following technical solution, including the following steps: Including image acquisition, image preprocessing, cell segmentation, feature extraction and classification recognition; Step S1, acquiring cell images through a high-resolution microscope or other equipment, cleaning the image data and screening invalid image data; Step S2: Preprocess the image, including denoising, enhancement, and image enhancement. By increasing the diversity of training data, the generalization ability of the model can be improved. Techniques such as flipping are used to increase the number and diversity of training samples to improve image quality. Step S3: Separate the cells from the background and label them, and divide the labeled image dataset into training set, validation set, and test set; Step S4: Add an attention mechanism to the YOLOv8 model network and perform multi-scale feature fusion, introduce residual connections, and improve the loss function to WIoU; In step S5, the training parameters of the improved YOLOv8 network model are set and the divided training set is fed into the improved YOLOv8 network model for training. The model then classifies and identifies cells, identifying specific cell types. The validation set verifies the results of the improved YOLOv8 network model training and continuously updates the training parameters to obtain the optimal weight file. The final training results are then verified on the test set.

2. Further, the specific process of step S1 is as follows: Step S1.1: Prepare a high-resolution microscope and appropriate slides, and place the cell sample to be observed on the slide. Place the slide on the microscope stage and adjust the microscope focus to ensure a clear image. Use a camera or image capture device to capture the image. Ensure that appropriate exposure time, focus, and imaging parameters are used to obtain high-quality images. Step S1.2: Preprocess the images, such as adjusting brightness, contrast, or color balance, to ensure consistent image quality. Check for noise or artifacts in the images and use image processing algorithms or filters to reduce these artifacts. Remove low-quality images, such as blurred, overexposed, or underexposed images, to prevent them from interfering with subsequent analysis. In step S1.3, define criteria for filtering invalid images, such as cell integrity and background interference. Using image processing algorithms or machine learning methods, compare images against these criteria and identify invalid images. Invalid images are removed from the dataset, retaining only valid images for subsequent analysis.

3. Further, the specific process of step S2 is as follows: Step S2.1, Denoising: Use a denoising algorithm such as median filtering to reduce noise in the image to improve image clarity and details; Step S2.2, enhancement: improving the image visibility by increasing the contrast, brightness and color saturation of the image through adaptive histogram equalization; Step S2.3, Flip: Flip the image horizontally or vertically to increase the amount of training data and help the model learn features from different angles and directions.

4. Further, the specific process of step S3 is as follows: Step S3.1: Use the labeling tool (Labelimg) to label the red blood cell image data and generate a VOC format annotation file. This VOC format file is converted into the YOLO format suitable for the YOLOv8 network model using code. The YOLO format file includes the category name of the target in the image data, the center coordinates of the target's ground-truth box, and the width and height. Step S3.2: Divide the .xml tag files and .jpg image files in the labeled red blood cell dataset into a training set, a validation set, and a test set in a ratio of 8:1:

1.

5. Further, the improvement of the backbone network in step S4 is as follows: Step S4.1: First, in the YOLOv8 network, replace the backbone network Darknet53 with EfficientNet. This backbone network has more layers and parameters and can provide better feature representation capabilities. Step S4.2: Combine the basic convolutional block of the backbone network with the CBAM (Channel Attention and Spatial Attention) attention mechanism to form a new attention convolution block (CACBS, CBAM Attention Convolution Block); Step S4.3: Increase the scale of the output feature map. YOLOv8 only has three output feature maps (P3, P4, and P5). Add more output feature maps, P6 and P7. This provides more feature maps of different scales, which helps detect objects of different sizes. In step S4.4, all downsampling modules in the backbone network are replaced with only a 2x2 max pooling module (DP). Data augmentation and regularization techniques, such as Dropout, are used to increase data diversity and the generalization ability of the model, as shown below. ； ； ； ； The Bernoulli function is used to randomly generate a vector of 0 and 1 with a probability p (usually set to 0.5 or 0.3). It is to make a neuron inactive with a probability of p (activation value is 0) In step S4.5, a residual connection is introduced between the Convolutional layer and the Route layer. Both layers are dimensionally compatible, making residual connections easier to implement. In the backbone network, a residual connection is introduced between the Convolutional layer of each module and the Route layer of the next module. Residual Connection: ,in Variables representing inputs; Layer Normalization: ； ； ,in is the vector representation of the neurons in this layer, is the number of hidden neurons in the layer, To add a small value to the variance to prevent division by zero, is a trainable scale parameter, is a trainable bias parameter; In step S4.6, the improved loss function WIoU takes into account the differences in relative size and position between objects and introduces a self-weight adjustment mechanism to adaptively adjust the weight coefficients and improve the flexibility and robustness of object detection. WIoU is calculated as follows: ； ； ,in , Indicates the coordinates of the center point of the actual frame, , It represents the height and width of the minimum enclosing box formed by the predicted box and the actual box, and IoU represents the intersection over union ratio. is the bounding box regression loss.

6. Further, the model training in step S5 is as follows: In step S5.1, the training set is fed into the improved YOLOv8 network model for training. The categorical variables are stored in three categories: WBC, RBC, and Platelets. WBC represents white blood cells, RBC represents red blood cells, and Platelets represents platelets. This allows for cell classification and identification. In step S5.2, the trained improved YOLOv8 network model is tested on the processed test set. The accuracy (the probability of predicted positive samples to all correct predictions) and the recall (the ratio of predicted positive samples to correctly predicted positive samples and correctly predicted negative samples) are calculated, as well as the harmonic mean F1 of the precision and recall and the average precision (mAP) of the categories to evaluate the performance of the model.

Images