A method for bacterial detection in sputum specimens based on YOLOV8

Abstract

The present invention provides a sputum sample bacteria detection method based on YOLOV8, which belongs to the field of image processing and computer vision technology; specifically comprises the following steps: step 1: obtaining a Gram-stained slide image data set, and preprocessing the obtained Gram-stained slide image data set; step 2, data labeling: performing rotating frame data labeling on the preprocessed data in step 1, and dividing the data into a training set, a validation set, and a test set; step 3, improving the network model: modifying the loss function, improving YOLOV8 in the feature extraction and feature fusion stages, and obtaining an improved model; step 4, training the model: inputting the training set into the network model for iterative training, and obtaining an optimal target detection model; step 5, evaluating the model. The present invention realizes accurate and rapid bacterial instance segmentation and recognition, and can significantly improve the detection accuracy and efficiency of bacteria, and improve accuracy and credibility.

Description

Technical Field

[0001] The present invention relates to the field of image processing and computer vision technology, and in particular to a sputum sample bacteria detection method based on YOLOV8. Background Art

[0002] YOLOv8 is a SOTA model that builds on the success of the previous YOLO series models and introduces new features and improvements to further improve performance and flexibility. Specific innovations include: a new backbone network, a new Ancher-Free detection head, and a new loss function that can run on a variety of hardware platforms from CPU to GPU. Note that ultralytics did not directly name the open source library YOLOv8, but directly used the word Ultralytics. The reason is that the Ultralytics library is positioned as an algorithm framework rather than a specific algorithm. This library can not only be used for the YOLO series models, but also supports other visual tasks such as image classification, instance segmentation, etc.

[0003] In the microbial world, bacteria are about 0.5-5 microns in size, and the wavelength of the light source used for microscopic imaging is generally 0.4-0.8 microns. After microscopic imaging, the minimum pixel area occupied by bacteria is 7x7 pixels. COCO defines objects with a pixel area less than 32*32 pixels as small targets.

[0004] In comparison, bacteria are actually ultra-small targets in the field of computer vision. Ultra-small target individuals contain very few discriminative features due to too little imaging information. At the same time, bacteria are aggregated during the reproduction process. Overall, bacteria appear as dense tiny targets after microscopic imaging, which poses a huge challenge to object data labeling and target recognition, resulting in low detection accuracy and efficiency of bacteria in bacterial instance segmentation and recognition. Summary of the invention

[0005] The present invention discloses a sputum sample bacteria detection method based on YOLOV8, which realizes accurate and rapid bacteria instance segmentation and recognition, and can significantly improve the detection accuracy and efficiency of bacteria, and enhance the accuracy and credibility.

[0006] In order to solve the above technical problems, the technical solution adopted by the present invention is:

[0007] A sputum sample bacteria detection method based on YOLOV8 specifically includes the following steps:

[0008] Step 1: Obtain a Gram-stained slide image dataset, and preprocess the obtained Gram-stained slide image dataset;

[0009] Step 2, data labeling: label the data preprocessed in step 1 with a rotating box data, and divide the data into a training set, a validation set, and a test set;

[0010] Step 3, improve the network model: modify the loss function, improve YOLOV8 in the feature extraction and feature fusion stages, and obtain an improved model;

[0011] Step 4, training model: inputting the training set into the network model for iterative training to obtain the optimal target detection model;

[0012] Step 5, evaluating the model: inputting the test set into the optimal network model for prediction, and obtaining the target detection result through the tested target detection network model.

[0013] Among them, step 3 specifically includes the following steps, and the specific improvements based on the original YoLoV8-P2 model are as follows:

[0014] S101, improved feature extraction network Backbone;

[0015] S102, improve feature fusion and detection network Head;

[0016] S103, improvement of loss function;

[0017] In S102,

[0018] In the original network structure, the output features of the 9th layer are upsampled (nn.Upsample), the upsampling results are concatenated (Concat) with the output features of the 6th layer, and then the features are merged (C2f) through convolution as the input of the next building block. The network structure is changed to deconvolve the output features of the 9th layer, convolve the output features of the 4th layer, and perform scale sequence feature fusion (SSFF) on the output features of the above two layers and the output features of the 6th layer. Finally, the fused features are decomposed into multiple branches through a branched convolution structure to extract features and integrate information.

[0019] Furthermore, in S101, in the feature extraction network Backbone, the C2f_ConvNeXtv2 module is used to replace the C2F module of the original network.

[0020] Furthermore, in S103, the bounding box is first converted to a Gaussian distribution, and the center points of the two Gaussian distributions are moved to be close, and then the distribution function of the overlapping area is obtained by Gaussian product, and finally the obtained distribution function is converted into a rotated bounding box to calculate the overlapping area, SkewIoU and loss.

[0021] Further, the detailed steps of step 103 are as follows:

[0022] 1) Calculate Gaussian distribution:

[0023] ,

[0024] Covariance matrix Through the rotation matrix The covariance matrix describes the statistical characteristics of the shape and orientation of the bounding box after rotation.

[0025] ,

[0026] is the mean vector, indicating that the center point coordinates (x, y) of the bounding box are used as the mean vector;

[0027] ,

[0028] Rotation Matrix Used to transform the bounding box from the standard position (unrotated) to the actual rotated position, using cosine and sine functions to achieve the angle Rotation of

[0029] ;

[0030] Scaling Matrix describes the contribution of the bounding box width and height to the covariance. Here and One quarter of the width and one quarter of the height, respectively, are used to calculate the diffusion or expansion of the bounding box in the horizontal and vertical directions;

[0031] Bounding Box , where x, y are the coordinates of the center point, w, h are the width and height, For the angle, calculate the Gaussian distribution ; is the covariance matrix, describing the shape and orientation of the bounding box.

[0032] 2) Calculate KFIoU by Gaussian product;

[0033] Gaussian distribution calculated for two bounding boxes , calculate the Gaussian product to get the distribution function of the intersection area:

[0034] ;

[0035] 3) Calculate KFIoU:

[0036] ;

[0037] Among them, B1, B2, and B3 are three bounding boxes.

[0038] ;

[0039] represents the area covariance matrix of bounding box No. 1, represents the area covariance matrix of bounding box No. 2, represents the area covariance matrix of bounding box No. 3;

[0040] 4) Loss function:

[0041] .

[0042] in, : Intermediate variable value, no physical meaning;

[0043] : is Gaussian distribution;

[0044] : is the mean vector of bounding box No. 1, indicating that the coordinates (x, y) of the center point of bounding box No. 1 are used as the mean vector;

[0045] : is the mean vector of bounding box No. 2, indicating that the coordinates (x, y) of the center point of bounding box No. 2 are used as the mean vector;

[0046] : is the covariance matrix of bounding box No. 1;

[0047] : is the covariance matrix of bounding box No. 2;

[0048] Furthermore, in step 1, the acquired Gram-stained slide image dataset is segmented into images of size 320x320.

[0049] Furthermore, the ratio of training set, validation set and test set is 6:2:2.

[0050] Compared with the prior art, the present invention has the following beneficial effects:

[0051] The present invention proposes a method for detecting bacteria in sputum specimens based on YOLOV8. By analyzing the bacteria on the Gram-stained sputum slide sample image, the overall performance is dense and tiny targets. The traditional horizontal object detection cannot meet the requirements, and the rotation object detection needs to be used. Since bacteria show obvious arrangement characteristics during the reproduction process, such as diplococci, streptococci, and staphylococci, these feature prompts can improve the ability to detect tiny objects through the contextual attention mechanism. Therefore, a rotation box is introduced for annotation in the data annotation stage, and the corresponding detection head of YOLOV8 is changed to rotation object detection, and the loss function is modified for rotation object detection. The feature extraction adds a global attention mechanism, and the feature fusion stage adopts an enhanced bidirectional pyramid feature fusion algorithm, combined with spatial and scale features, to achieve accurate and fast bacterial instance segmentation and recognition. In summary, the above innovations can significantly improve the detection accuracy and efficiency of bacteria, and improve accuracy and credibility. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments. It should be understood that the following drawings only show certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.

[0053] Figure 1 It is the overall principle diagram of the present invention.

[0054] Figure 2 Improved network schematic for the original YoLoV8-P2 model.

[0055] Figure 3 This is the improved network schematic diagram of the YoLoV8-P2 model modified by the present invention.

[0056] Figure 4 This is the principle diagram before and after the improvement of the feature fusion and detection network Head of the present invention.

[0057] Figure 5 This is a schematic diagram of the present invention after BiFusion replaces SSFF on the 20th layer.

[0058] Figure 6 This is one of the detection images used in the present invention to mark small targets.

[0059] Figure 7 This is the second detection image of the present invention for marking small targets.

[0060] Figure 8 This is the bacterial map detected by the original network structure of YoloV8-P2.

[0061] Fig. 9 Bacteria graph detected by improving the network structure for YoloV8-P2. DETAILED DESCRIPTION

[0062] In the following, only certain exemplary embodiments are briefly described. As will be appreciated by those skilled in the art, the described embodiments may be modified in various ways without departing from the spirit or scope of the embodiments of the present invention. Therefore, the drawings and description are to be regarded as illustrative and non-restrictive in nature. They do not in themselves indicate the relationship between the various embodiments and / or arrangements discussed.

[0063] The embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0064] Embodiment 1

[0065] This embodiment discloses a sputum sample bacteria detection method based on YOLOV8, which specifically includes the following steps:

[0066] Step 1: Obtain a Gram-stained slide image dataset and preprocess the acquired Gram-stained slide image dataset; in actual use, in order to balance detection accuracy and computational efficiency, the acquired Gram-stained slide image dataset is segmented into images of 320x320 size.

[0067] Step 2, data labeling: label the data preprocessed in step 1 with a rotating box data, and divide the data into a training set, a validation set, and a test set;

[0068] Step 3, improve the network model: modify the loss function, improve YOLOV8 in the feature extraction and feature fusion stages, and obtain an improved model;

[0069] Step 4, training model: inputting the training set into the network model for iterative training to obtain the optimal target detection model;

[0070] Step 5, evaluating the model: inputting the test set into the optimal network model for prediction, and obtaining the target detection result through the tested target detection network model.

[0071] Among them, step 3 specifically includes the following steps, and the specific improvements based on the original YoLoV8-P2 model are as follows:

[0072] S101, improved feature extraction network Backbone;

[0073] S102, improve feature fusion and detection network Head;

[0074] S103, improvement of loss function;

[0075] In S102,

[0076] In the original network structure, the output features of the 9th layer are upsampled (nn.Upsample), the upsampling results are concatenated (Concat) with the output features of the 6th layer, and then the features are merged (C2f) through convolution as the input of the next building block. The network structure is changed to deconvolve the output features of the 9th layer, convolve the output features of the 4th layer, and perform scale sequence feature fusion (SSFF) on the output features of the above two layers and the output features of the 6th layer. Finally, the fused features are decomposed into multiple branches through a branched convolution structure to extract features and integrate information.

[0077] Furthermore, in S101, in the feature extraction network Backbone, the C2f_ConvNeXtv2 module is used to replace the C2F module of the original network.

[0078] Improvements in the feature extraction and feature fusion stages are mainly aimed at improving the accuracy, speed and robustness of the model. These improvements help improve the performance of object detection tasks.

[0079] Furthermore, in S103, the bounding box is first converted to a Gaussian distribution, and the center points of the two Gaussian distributions are moved to be close, and then the distribution function of the overlapping area is obtained by Gaussian product, and finally the obtained distribution function is converted into a rotated bounding box to calculate the overlapping area, SkewIoU and loss.

[0080] The detailed steps of step 103 are as follows:

[0081] 1) Calculate Gaussian distribution:

[0082] ,

[0083] Covariance matrix Through the rotation matrix The covariance matrix describes the statistical characteristics of the shape and orientation of the bounding box after rotation.

[0084] ,

[0085] is the mean vector, indicating that the center point coordinates (x, y) of the bounding box are used as the mean vector;

[0086] ,

[0087] Rotation Matrix Used to transform the bounding box from the standard position (unrotated) to the actual rotated position, using cosine and sine functions to achieve the angle Rotation of

[0088] ;

[0089] Scaling Matrix describes the contribution of the width and height of the bounding box to the covariance. Here and One quarter of the width and one quarter of the height, respectively, are used to calculate the diffusion or expansion of the bounding box in the horizontal and vertical directions;

[0090] Bounding Box , where x, y are the coordinates of the center point, w, h are the width and height, For the angle, calculate the Gaussian distribution ; is the covariance matrix, describing the shape and orientation of the bounding box.

[0091] 2) Calculate KFIoU by Gaussian product;

[0092] Gaussian distribution calculated for two bounding boxes , , calculate the Gaussian product to get the distribution function of the intersection area:

[0093] ;

[0094] 3) Calculate KFIoU:

[0095] ;

[0096] Among them, B1, B2, and B3 are three bounding boxes.

[0097] ;

[0098] represents the area covariance matrix of bounding box No. 1, represents the area covariance matrix of bounding box No. 2, represents the area covariance matrix of bounding box No. 3;

[0099] 4) Loss function:

[0100] .

[0101] in, : Intermediate variable value, no physical meaning;

[0102] : is Gaussian distribution;

[0103] : is the mean vector of bounding box No. 1, indicating that the coordinates (x, y) of the center point of bounding box No. 1 are used as the mean vector;

[0104] : is the mean vector of bounding box No. 2, indicating that the coordinates (x, y) of the center point of bounding box No. 2 are used as the mean vector;

[0105] : is the covariance matrix of bounding box No. 1;

[0106] : is the covariance matrix of bounding box No. 2.

[0107] Furthermore, the ratio of training set, validation set and test set is 6:2:2.

[0108] In order to facilitate those skilled in the art to further understand the present invention, the present invention is further described below in conjunction with specific implementation cases:

[0109] See also Figure 1 , a sputum sample bacteria detection method based on YOLOV8, the detailed scheme is as follows:

[0110] s1, prepare Gram-stained slide microscopic image dataset: obtain the Gram-stained slide image dataset and segment it into images of size 320x320.

[0111] s2, data labeling: label the rotating box data and divide the data into training set, validation set and test set;

[0112] The purpose of splitting the dataset into training, validation, and test sets is to ensure that the model can generalize effectively to unseen data and is properly tuned and evaluated.

[0113] The specific functions of the three parts are as follows:

[0114] The training set is used to train the model. The model learns features and patterns from this part of the data in order to make predictions and help the model find patterns in the data. At this stage, the model will adjust its internal parameters to minimize prediction errors.

[0115] The validation set is used to adjust the model's hyperparameters and select the best model. The validation set is a "neutral" data set used for model selection and parameter adjustment. It is not used to train the model, but is used to evaluate the model's performance on this data. To help prevent overfitting, the model may perform well on the training set but perform poorly in real scenarios. By evaluating the performance of the model on the validation set, the structure or parameters of the model can be adjusted to improve generalization capabilities.

[0116] The test set is used to finally evaluate the performance of the model. The test set is a data set that is completely independent of the training process. The model should not see the test data at this stage; in order to provide a final evaluation of the generalization ability of the model, the evaluation results of the test set can reflect the performance of the model in actual applications, rather than the performance in the training or verification stage.

[0117] s3, Improved network model:

[0118] Modify the loss function and improve YOLOV8 in the feature extraction and feature fusion stages to obtain an improved model;

[0119] S4, training model:

[0120] The training set is input into the network model for iterative training to obtain the optimal target detection model.

[0121] The specific iterative training steps are as follows:

[0122] 1. Initialization: Select the initial model structure and hyperparameters.

[0123] 2. Training: Use the training set data to train the model and update the model parameters.

[0124] 3. Validation: Use the validation set to evaluate the model performance and calculate metrics such as loss or accuracy.

[0125] 4. Adjustment: Adjust model hyperparameters or structure based on the validation results.

[0126] 5. Repeat: Train, validate, and adjust repeatedly until the model performance meets the requirements or reaches the stopping criteria.

[0127] In actual use, the optimal target mainly refers to the performance of the validation set and the test set.

[0128] Validation set performance: Select the model configuration that performs best on the validation set. Metrics such as accuracy, loss, and F1 score can help determine the best model.

[0129] Test set performance: Ensuring that the final model performs well on the test set is a key step in evaluating the model’s generalization ability.

[0130] S4: Evaluate the model:

[0131] The test set is input into the optimal network model for prediction, and the target detection result is obtained through the tested target detection network model.

[0132] The original YoLoV8-P2 model is improved by the network reference Figure 2 ;

[0133] Modified YoLoV8-P2 model improved network reference Figure 3 ,exist Figure 2 and Figure 3 In the figure, each rectangular box represents an algorithm module, and the arrows indicate the input and output directions of the data.

[0134] 1. Improve the feature extraction network Backbone: Due to the focal length, bacteria show large, medium and small features on the spatial scale in microscopic imaging. Therefore, in the feature extraction network Backbone, the C2f_ConvNeXtv2 module is used to replace the C2F module of the original network.

[0135] The differences between the two modules are as follows:

[0136] The C2f module is a complex module consisting of multiple convolutional layers and activation functions, usually including two main convolution operations whose outputs are merged. C2f is used in YOLOV8 to increase the expressive power of the network and capture richer features while reducing the amount of computation.

[0137] C2f_ConvNeXtv2 replaces Conv with ConvNeXt V2 in the convolutional layer. ConvNeXt V2 is a new convolutional neural network architecture that combines self-supervised learning technology and architectural improvements, especially the addition of a fully convolutional masked autoencoder framework and a global response normalization (GRN) layer, which significantly improves the performance of large, medium and small object detection.

[0138] 2. Improve feature fusion and detection network head: During the training process, the image will be subjected to feature fusion and detection network head;

[0139] See also Figure 4 ,In this stage, improvements are made by building blocks for feature fusion.

[0140] The original P2 layer results are as follows Figure 4 As shown on the left, after the image passes through the c2f module, the features are concatenated with the image upsampled by SPPF and then output by C2F.

[0141] After modification:

[0142] C2f_ConvNeXtv2 is used to replace Conv with ConvNeXt V2 in the convolutional layer, and then it passes through SimConV to reach the splicing layer. Then SPPF is changed to pass through ConV to reach the splicing layer and finally pass through RepBlock for output.

[0143] In the original network structure, the output features of the 9th layer are upsampled (nn.Upsample), the upsampling results are concatenated (Concat) with the output features of the 6th layer, and then the features are merged through convolution (C2f) as the input of the next building block.

[0144] The improved network structure deconvolves the output features of the 9th layer, convolves the output features of the 4th layer, and performs scale sequence feature fusion (SSFF) on the output features of the above two layers with the output features of the 6th layer. Finally, the fused features are decomposed into multiple branches through the branched convolution structure to decompose the feature extraction and information integration process, so that the model can better adapt to features of different scales and directions (RepBlock).

[0145] See also Figure 5 ,In the 20th layer, SSFF is replaced by BiFusion, and the residual pyramid is used to further improve the detection ability of small objects by bidirectionally fusing deep and shallow features.

[0146] The functions of the main modules involved in the improvement are briefly described as follows:

[0147] SPPF module: Adopts the idea of ​​spatial pyramid pooling to integrate the information of feature maps through multi-scale pooling operations. SPPF means using several channels and a pooling layer to enhance the detection ability of objects of different scales.

[0148] RepBlock module: The multi-branch convolutional structure decomposes the process of feature extraction and information integration into multiple branches, so that the model can better adapt to features of different scales and directions.

[0149] BiFusion module: Uses residual pyramid to improve the detection of small and large objects by bidirectionally fusing deep and shallow features.

[0150] SSFF module: The scale sequence feature fusion (SSFF) module is used to process multi-scale information. Its main function is to enhance the ability of neural networks in extracting features of different scales.

[0151] By aggregating features from different layers of the network, the SSFF module is able to provide richer and more detailed feature representations, which helps improve the model's performance when dealing with objects of different sizes. This fusion strategy is particularly suitable for tasks that require accurate positioning and identification of multi-scale objects in images. With SSFF, the model can more effectively identify and understand the details in the image, thereby improving the overall detection and recognition performance.

[0152] 3. Improvement of loss function:

[0153] Horizontal object detection can be well suited for detection metrics by computing IoU-based losses, however rotated object detection often implies a more complex SkewIoU-based loss calculation, which is not conducive to gradient-based training. SkewIoU between large aspect ratio objects is very sensitive to deviations in object position, which will lead to inconsistencies between metrics (based on SkewIoU) and regression losses (such as ln-norms).

[0154] Due to the inconsistency between SkewIoU and Smooth L1 Loss, specifically, when the deviation angle remains unchanged, SkewIoU decreases sharply as the aspect ratio increases, while Smooth L1 loss remains unchanged. Similarly, when SkewIoU remains unchanged, Smooth L1 loss increases as the angle difference increases. For the above reasons, for rotation detection, instead of the SkewIoU loss, which is cumbersome to calculate accurately and is not conducive to differentiable learning, the easy-to-implement relative loss KFIoU is used. Because it is fully differentiable and can handle disjoint scenes, the results are very good.

[0155] (1) Calculate Gaussian distribution:

[0156] Bounding Box Represents a rotated rectangular box, where x, y are the center coordinates, w, h are the width and height, is the angle.

[0157] Calculate Gaussian distribution Probability distribution used to describe the rotation bounding box:

[0158] , ;

[0159] is the mean vector, representing the coordinates of the center point of the bounding box.

[0160] is the covariance matrix, describing the shape and orientation of the bounding box.

[0161] (2) Calculate KFIoU by Gaussian product;

[0162] Gaussian distribution calculated for two bounding boxes , calculate the Gaussian product to get the distribution function of the intersection area:

[0163]

[0164] ;

[0165] Calculate KFIoU:

[0166] , where B1, B2, and B3 are three bounding boxes.

[0167] .

[0168] Loss Function

[0169] .

[0170] In actual use, first obtain the microscopic image dataset of Gram-stained slides, split it into 320x320 images, annotate the dataset with rotation boxes, and divide it into training set, validation set, and test set in a ratio of 6:2:2. Edit the yolov8-p2-RepBiSF.yaml configuration file according to the improved network structure, and specify the use of the rotation detection loss function KFLoss:

[0171] nc: 1 # number of classes

[0172] scales: # model compound scaling constants, ie 'model=yolov8n.yaml' will call yolov8.yaml with scale 'n'

[0173] # [depth, width, max_channels]

[0174] n: [0.33, 0.25, 1024] # YOLOv8n summary: 225 layers, 3157200parameters, 3157184 gradients, 8.9 GFLOPs

[0175] loss: KFloss

[0176] # YOLOv8.0n backbone

[0177] backbone:

[0178] # [from, repeats, module, args]

[0179] - [-1, 1, Conv, [64, 3, 2]] # 0-P1 / 2

[0180] - [-1, 1, Conv, [128, 3, 2]] # 1-P2 / 4

[0181] - [-1, 3, C2f_ConvNeXtv2,

[128] ] # 2

[0182] - [-1, 1, Conv, [256, 3, 2]] # 3-P3 / 8

[0183] - [-1, 6, C2f_ConvNeXtv2,

[256] ] # 4

[0184] - [-1, 1, Conv, [512, 3, 2]] # 5-P4 / 16

[0185] - [-1, 6, C2f_ConvNeXtv2,

[512] ] # 6

[0186] - [-1, 1, Conv, [1024, 3, 2]] # 7-P5 / 32

[0187] - [-1, 3, C2f_ConvNeXtv2,

[1024] ] # 8

[0188] - [-1, 1, SPPF, [1024, 5]] # 9

[0189] # YOLOv8.0n head

[0190] head:

[0191] - [-1, 1, Conv, [512, 1, 1]] # 10

[0192] - [-7, 1, SimConv, [512, 1, 1]] # 11 SC-P3

[0193] - [[-1, 6, 10], 1, SSFF,

[512] ] # 12 SSFF-P4<--- SC-P3,P4,P5

[0194] - [-1, 3, RepBlock,

[512] ] # 13-->RB-P4

[0195] - [-1, 1, Conv, [256, 1, 1]] # 14

[0196] - [2, 1, SimConv, [256, 1, 1]] # 15 SC-P2

[0197] - [[-1, 4, -2], 1, SSFF,

[256] ] # 16 SSFF-P3<--- SC-P2,P3,RB-P4

[0198] - [-1, 3, RepBlock,

[256] ] # 17-->RB-p3

[0199] - [-1, 1, Conv, [128, 1, 1]] # 18

[0200] - [0, 1, SimConv, [128, 1, 1]] # 19 SC-P1

[0201] - [[-1, 2, -2], 1, BiFusion,

[128] ]# 20 BF-P2<---- SC-P1,P2,RB-p3

[0202] - [-1, 3, RepBlock,

[128] ] # 21-->RB-P2 -xs small)

[0203] - [-1, 1, SimConv, [128, 3, 2]] # 22

[0204] - [[-1,17], 1, Concat, [1]] # 23 cat RB-P2 + RB-P3

[0205] - [-1, 3, RepBlock,

[256] ] # 24 -->RB-P3

[0206] - [-1, 1, SimConv, [256, 3, 2]] # 25

[0207] - [[-1, 13], 1, Concat, [1]] # 26 cat RB-P3 + RB-P4

[0208] - [-1, 3, RepBlock,

[512] ] # 27-->RB-P4

[0209] - [-1, 1, SimConv, [512, 3, 2]] # 28

[0210] - [[-1, 9], 1, Concat, [1]] # 29 cat RB-P4,P5

[0211] - [-1, 3, RepBlock,

[1024] ] # 30 -->RB-P5 (P5 / 32-large)

[0212] - [[21,24,27,30], 1,OBB, [nc,1]] # obb(P2,P3,P4,P5)

[0213] Prepare the data configuration file data.yaml:

[0214] # dataset path

[0215] train: D: / ultralytics-main / ultralytics-main / datasets / images / train

[0216] val: D: / ultralytics-main / ultralytics-main / datasets / images / val

[0217] test: D: / ultralytics-main / ultralytics-main / datasets / images / test

[0218] # number of classes

[0219] nc: 1

[0220] # class names

[0221] names:

[0222] 0: coccus

[0223] Edit the p2-RepBiSF.yaml training parameter file:

[0224] # Ultralytics YOLO, AGPL-3.0 license

[0225] # Default training settings and hyperparameters for medium-augmentation COCO training

[0226] task: detect # (str) YOLO task, i.e. detect, segment, classify, pose

[0227] mode: train # (str) YOLO mode, i.e. train, val, predict, export,track, benchmark

[0228] # Train settings-------------------------------------------------------------------------------------------------------

[0229] model: D: / ultralytics-main / ultralytics-main / ultralytics / cfg / models / v8 / yolov8-p2-RepBiSF.yaml#(str, optional) path to model file, i.e.yolov8n.pt, yolov8n.yaml,yolov8-p2.yaml

[0230] data: D: / ultralytics-main / ultralytics-main / data.yaml # (str,optional)path to data file, i.e. coco128.yaml

[0231] epochs: 6000 # (int) number of epochs to train for

[0232] patience: 0 # (int) epochs to wait for no observable improvement forearly stopping of training

[0233] batch: 8 # (int) number of images per batch (-1 for AutoBatch)

[0234] imgsz: 320 # (int | list) input images size as int for train and valmodes, or list[w,h]for predict and export modes

[0235] save: True # (bool) save train checkpoints and predict results

[0236] save_period: 2000 # (int) Save checkpoint every x epochs (disabled if<1) ......

[0237] Enter the command in the command line window to start training:

[0238] model = YOLO(r'D:\ultralytics-main\ultralytics-main\ultralytics\cfg\models\v8\yolov8-p2-RepBiSF.yaml')

[0239] model.train(cfg='D: / ultralytics-main / ultralytics-main / ultralytics / cfg / p2-RepBiSF.yaml',data = 'D: / ultralytics-main / ultralytics-main / data.yaml',resume=True)

[0240] After 2000 epochs of iterative training, the model file with the lowest loss is saved as the optimal model for this training. Finally, the optimal model is used for prediction and inference to identify bacteria in Gram-stained sputum slide microscopy images.

[0241] In order to further reflect the present invention, the rotation object annotation and detection effect are adopted. The specific results are shown in the following table:

[0242]

[0243] The object information marked by rotation detection on dense small objects is more accurate, the model training converges quickly, the number of training times is greatly reduced, and the accuracy is improved.

[0244] The detection results of the original YoloV8-P2 network structure and the improved YoloV8-P2 network structure are as follows: Figure 8 and Fig. 9, the original network structure of YoloV8-P2 detected a total of 7138 bacteria, and the improved network structure of YoloV8-P2 detected a total of 10109 bacteria; after adopting the improved YOLOV8 network, the detection of tiny targets is more accurate.

[0245] See also Figure 6 , for the detection of tiny targets, in the left image (using horizontal box), the horizontal box marks the bacillus as a square-like shape, while in the right image (using rotating box), it is more accurate. Bacteria, especially bacilli, have directionality under the microscope. Using rotating box annotation and training can significantly improve the recognition ability of the model. More importantly, accurate rotating boxes are more accurate for the recognition of coccobacilli based on the aspect ratio of the bounding box, which is of significant significance.

[0246] See also Figure 7 , by using the KFLoss of the present invention, rotation detection is more accurate.

[0247] The present invention proposes a method for detecting bacteria in sputum specimens based on YOLOV8. Through analysis, the bacteria on the Gram-stained sputum slide sample image generally appear as dense tiny targets. The traditional horizontal object detection cannot meet the requirements, and the rotation object detection needs to be used. Since bacteria show obvious arrangement characteristics during the reproduction process, such as diplococci, streptococci, and staphylococci, these feature prompts can improve the ability to detect tiny objects through the contextual attention mechanism. Therefore, a rotation box is introduced for annotation in the data annotation stage, and the corresponding detection head of YOLOV8 is changed to rotation object detection, and the loss function is modified for rotation object detection. The feature extraction adds a global attention mechanism, and the feature fusion stage adopts an enhanced bidirectional pyramid feature fusion algorithm, which combines spatial and scale features to achieve accurate and fast bacterial instance segmentation and recognition. In summary, the above innovations can significantly improve the detection accuracy and efficiency of bacteria, and improve accuracy and credibility.

[0248] Although the preferred embodiments of the present invention have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.

[0249] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. It should be pointed out that any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for detecting bacteria in sputum specimens based on YOLOV8, characterized in that: The specific steps include: Step 1: Obtain a Gram-stained slide image dataset, and preprocess the obtained Gram-stained slide image dataset; Step 2, data labeling: label the data preprocessed in step 1 with a rotating box data, and divide the data into a training set, a validation set, and a test set; Step 3, improve the network model: modify the loss function, improve YOLOV8 in the feature extraction and feature fusion stages, and obtain an improved model; Step 4, training model: inputting the training set into the network model for iterative training to obtain the optimal target detection model; Step 5, evaluating the model: inputting the test set into the optimal network model for prediction, and obtaining the target detection result through the tested target detection network model; Among them, step 3 specifically includes the following steps, and the specific improvements based on the original YoLoV8-P2 model are as follows: S101, improved feature extraction network Backbone; S102, improve feature fusion and detection network Head; S103, improvement of loss function; In S102, the output features of the 9th layer in the original network structure are upsampled, the upsampled results are concatenated with the output features of the 6th layer, and then the convolution merge feature C2f is used as the input of the next building block. The network structure is changed to deconvolve the output features of the 9th layer, convolve the output features of the 4th layer, and fuse the scale sequence features of the output features of the above two layers with the output features of the 6th layer. Finally, the fused features are decomposed into multiple branches through the branch convolution structure to extract features and integrate information; In S101, in the feature extraction network Backbone, the C2f_ConvNeXtv2 module is used to replace the C2F module of the original network; In S103, the bounding box is first converted to a Gaussian distribution, and the center points of the two Gaussian distributions are moved to make them close. Next, the distribution function of the overlapping area is obtained by Gaussian product. Finally, the obtained distribution function is converted into a rotated bounding box to calculate the overlapping area, SkewIoU and loss; The detailed steps of step 103 are as follows: 1) Calculate Gaussian distribution: ， is the covariance matrix, describing the shape and orientation of the bounding box. Through the rotation matrix The covariance matrix describes the statistical characteristics of the shape and orientation of the bounding box after rotation. ， is the mean vector, representing the coordinates of the center point of the bounding box as the mean vector; ， Rotation Matrix Used to convert the bounding box from the standard position to the actual rotation position, using cosine and sine functions to achieve the angle Rotation of ； Scaling Matrix Describes the contribution of the bounding box width and height to the covariance; and One quarter of the width and one quarter of the height, respectively, are used to calculate the diffusion or expansion of the bounding box in the horizontal and vertical directions; Bounding Box , where x, y are the coordinates of the center point, w, h are the width and height, For the angle, calculate the Gaussian distribution ; 2) Calculate KFIoU by Gaussian product; Gaussian distribution calculated for two bounding boxes , calculate the Gaussian product to get the distribution function of the intersection area: ， : is Gaussian distribution, : Intermediate variable value, no physical meaning; ； ， : is the mean vector of bounding box No. 1, indicating the coordinates of the center point of bounding box No. 1 as the mean vector; : is the mean vector of bounding box No. 2, indicating the coordinates of the center point of bounding box No. 2 as the mean vector; : is the covariance matrix of bounding box No. 1; : is the covariance matrix of bounding box No. 2; 3) Calculate KFIoU: ； Among them, B1, B2, and B3 are three bounding boxes. represents the area covariance matrix of bounding box No. 1, represents the area covariance matrix of bounding box No. 2, represents the area covariance matrix of bounding box No. 3; ； 4) Loss function: 。 2. A sputum sample bacteria detection method based on YOLOV8 according to claim 1, characterized in that: In step 1, the acquired Gram-stained slide image dataset is segmented into images of size 320x320.

3. A sputum sample bacteria detection method based on YOLOV8 according to claim 1, characterized in that: The ratio of training set, validation set and test set is 6:2:2.

Images