Blood cell automatic detection method based on grading strategy

Abstract

The invention relates to a medical image processing and recognition technology, in particular to a blood cell automatic detection method based on a grading strategy, which comprises the following steps: acquiring a blood cell image set with a label, performing data enhancement on the data set to obtain a first training data set, and selecting a minority class of samples in the first training data set as a second training data set; training a secondary classification network by using the second training data set; constructing an improved YOLOv8 target detection algorithm model, and if the category output by a detection head in the YOLOv8 target detection algorithm model is not a minority category, directly outputting a detection result; and if the category of the single cell image output by the detection head is a minority category, inputting the single cell image into a pre-trained secondary classification network, and obtaining a final category probability by combining the confidence of the secondary classification network with an improved YOLOv8 target detection algorithm model. The method can improve the detection accuracy and the detection rate, and is suitable for the scene of peripheral blood cell detection.

Description

Technical Field

[0001] The invention relates to medical image processing and recognition technology, and in particular to an automatic blood cell detection method based on a classification strategy. Background Art

[0002] In the medical field, blood cell testing is the core technology for the identification of blood components and is crucial for the diagnosis and treatment of various diseases. There are three main types of cells in the blood: white blood cells (WBC), red blood cells (RBC), and platelets. Abnormal blood cell counts may indicate a variety of health problems. For example, abnormalities in white blood cells may indicate infection, inflammation, or immune system problems; changes in red blood cells are often associated with anemia due to iron deficiency or chronic diseases; abnormalities in platelets may indicate coagulation disorders or bleeding tendencies, such as thrombocytopenia. Therefore, rapid and accurate testing of various blood cells is crucial for doctors to assess patients' health and develop treatment plans, thereby significantly improving treatment outcomes and patients' quality of life.

[0003] Traditional blood cell detection methods mainly rely on manual microscopic observation, which is time-consuming, laborious, and prone to subjective errors. The advancement of digital image processing technology has promoted the application of computer vision technology in blood cell detection, but it still requires manual supervision, such as automatic cell recognition technology based on Hough transform, watershed method based on distance transform, and threshold-based methods, whose efficiency and accuracy are restricted. The emergence of machine learning has brought new opportunities for blood cell detection, which can learn from data and automatically adapt, thereby improving the degree of automation of detection. In recent years, the rapid development of deep learning technology, especially the rise of the YOLO (You Only Look Once) algorithm, has made real-time and efficient blood cell detection possible. With its end-to-end detection capabilities, YOLO has achieved a good balance between speed and accuracy and has been widely used in blood cell detection.

[0004] At present, blood cell detection technology still has many shortcomings in meeting clinical needs, which are mainly reflected in two aspects. First, the quality and complexity of peripheral blood microscopic images are an important challenge. Due to the dense distribution of cells, a wide variety of types and different morphologies, there are many occlusions and overlaps in the image, which makes it difficult for the target detection algorithm to distinguish cell types, even for YOLO. In addition, the complex background and the uneven distribution of cell numbers make it difficult for traditional image processing technology to extract effective features, which in turn affects the accuracy of detection. Secondly, current research usually limits blood cell classification to three categories (red blood cells, white blood cells and platelets) or only focuses on subtypes of white blood cells. This classification method fails to meet the meticulous clinical requirements for blood testing to some extent. In contrast, subdividing cell categories helps to achieve more accurate diagnosis and treatment. However, another problem brought about by subdividing categories is the imbalance between categories, especially when there are fewer samples of certain types of cells, which may lead to insufficient recognition of these categories by the detection model during the learning process, thereby affecting the overall detection effect. Therefore, to address these issues, more flexible and precise detection technologies need to be developed to achieve efficient identification and accurate detection of subtyped cells, thereby improving the effectiveness and clinical application value of blood cell testing. Summary of the invention

[0005] In order to improve the detection efficiency, detection accuracy and detection stability of cell detection, the present invention proposes an automatic blood cell detection method based on a classification strategy, which specifically includes the following steps:

[0006] Obtain a labeled blood cell image set, perform data enhancement on the data set to obtain a first training data set, and select minority class samples from the first training data set as a second training data set;

[0007] training a secondary classification network using the second training data set;

[0008] Build an improved YOLOv8 target detection algorithm model. If the category output by the detection head in the YOLOv8 target detection algorithm model is not a minority class, the detection result is directly output;

[0009] If the category of a single cell image output by the detection head is a minority class, the single cell image is input into a pre-trained secondary classification network, and the final category probability is obtained by combining the confidence of the secondary classification network with the improved YOLOv8 target detection algorithm model.

[0010] Furthermore, in the improved YOLOv8 target detection algorithm model, the FasterNet module is used to replace the bottleneck in the C2f module of the YOLOv8 target detection algorithm model.

[0011] Furthermore, the FasterNet module includes a cascaded split function layer, three FasterNet units and a splicing module. The splicing module splices the outputs of the split function layer and the three FasterNet units together as the output of the FasterNet module; each FasterNet unit includes a cascaded 3×3 local convolution unit, a 1×1 convolution module and a 1×1 convolution unit, and splices the feature map of the input FasterNet unit and the feature map processed by the 1×1 convolution unit together as the output of the FasterNet unit; each convolution module is composed of a cascaded convolution layer, a normalization layer and a SiLU activation function layer.

[0012] Furthermore, in the improved YOLOv8 target detection algorithm model, the SimSPPF module is used to replace the SPPF module of the YOLOv8 target detection algorithm model.

[0013] Furthermore, the SimSPPF module consists of a cascaded convolution module, a 5×5 maximum pooling layer, a 9×9 maximum pooling layer, a 13×13 maximum pooling layer, a splicing unit, and a convolution module. The splicing module splices the output of each maximum pooling layer together as the input of the last level convolution module.

[0014] Furthermore, in the improved YOLOv8 target detection algorithm model, the neck network processes the three feature maps of different sizes output by the backbone network, and outputs the three feature maps of different sizes to the detection head for target detection. The neck network is composed of a cascaded first upsampling layer, a first splicing layer, a first C2f module, a second upsampling layer, a second splicing layer, a second C2f module, a first convolution module, a third splicing layer, a third C2f module, a second convolution module, a fourth splicing layer and a fourth C2f module, wherein:

[0015] The small-size feature map output by the neck network is used as the input of the first upsampling layer. The feature map output by the first upsampling layer and the medium-size feature map output by the neck network are concatenated in the first concatenation layer.

[0016] The feature map output by the second upsampling layer and the large-size feature map output by the neck network are spliced ​​in the second splicing layer;

[0017] The output of the first C2f module is used as the large-size feature map output by the neck network;

[0018] The feature map output by the first convolutional module and the medium-sized feature map output by the neck network are concatenated in the third concatenation layer;

[0019] The output of the third C2f module is used as the medium-sized feature map output by the neck network;

[0020] The feature map output by the second convolution module and the small-size feature map output by the neck network are concatenated in the fourth concatenation layer;

[0021] The output of the fourth C2f module is used as the small-size feature map output by the neck network.

[0022] Furthermore, in the improved YOLOv8 target detection algorithm model, the detection head is composed of three cascaded Dyheadblocks, and each Dyhead block is composed of a cascaded scale-aware attention module, a space-aware attention module, and a task-aware attention module.

[0023] Furthermore, the loss function of the improved YOLOv8 target detection algorithm model during training includes classification loss, positioning loss and confidence loss. The classification loss calculates whether the anchor box and the corresponding calibration classification are correct, the positioning loss calculates the error between the predicted box and the calibration box, and the confidence loss calculates the confidence of the network. The target confidence loss and classification loss use a binary cross entropy function with sigmoid.

[0024] Furthermore, the two-level classification network consists of a cascaded 3×3 convolutional layer, a 2×2 maximum pooling layer, two residual blocks, a global average pooling layer, a fully connected layer and a Softmax layer. Each residual module is cascaded by two 3×3 convolutional layers and the input and output of the residual module are added through skip connections.

[0025] Furthermore, the final category probability acquisition process of the minority class is expressed as:

[0026]

[0027] C final,i =C yolo,i ×C cls,i

[0028] Among them, P final (i) represents the final probability that a single cell image belongs to category i; C final represents the final confidence that a single cell image belongs to category i, C final,k represents the final confidence that a single cell image belongs to category k; C yolo,i represents the confidence of the improved YOLOv8 target detection algorithm model predicting that a single cell image belongs to category k; C cls,i Represents the confidence of the secondary classification network predicting that a single cell image belongs to category k.

[0029] On the one hand, the present invention significantly improves the efficiency of blood cell detection and the processing ability of multi-scale features by improving the YOLOv8 model structure and introducing optimization modules such as FasterNet, SimSPPF and BiFPN; on the other hand, the Dect_Dyhead dynamic detection head and the secondary classification network are used to achieve the refined classification of a few types of blood cells, significantly improving the detection accuracy; at the same time, the improved model structure and training method enhance the detection stability, effectively solve the problem caused by the imbalance of sample quantity, and improve the generalization ability of the model in different scenarios. The present invention also performs data enhancement on a few types of blood cell images and optimizes the segmentation strategy of the data set to ensure that good detection results can still be achieved when data is scarce. In addition, the use of efficient optimizers and loss functions makes model training more efficient, converges quickly and obtains the detection model weight with the highest mAP value, which improves the detection accuracy and efficiency as a whole. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 It is the technical roadmap of the present invention;

[0031] Figure 2 This is a network model diagram of the present invention;

[0032] Figure 3 It is a C2f_f structural model diagram in the yolov8 backbone network of the present invention;

[0033] Figure 4 It is the structural model diagram of SimSPPF in the yolov8 backbone network of the present invention;

[0034] Figure 5 This is the structural model diagram of the Detect_Dyhead part of the yolov8 detection head of the present invention;

[0035] Figure 6 It is a two-level classification network structure model diagram of the present invention;

[0036] Figure 7 This is a diagram showing the effect of blood cell detection according to the present invention. DETAILED DESCRIPTION

[0037] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0038] The present invention proposes a blood cell automatic detection method based on a classification strategy, which specifically includes the following steps:

[0039] Obtain a labeled blood cell image set, perform data enhancement on the data set to obtain a first training data set, and select minority class samples from the first training data set as a second training data set;

[0040] training a secondary classification network using the second training data set;

[0041] Build an improved YOLOv8 target detection algorithm model. If the category output by the detection head in the YOLOv8 target detection algorithm model is not a minority class, the detection result is directly output;

[0042] If the category of a single cell image output by the detection head is a minority class, the single cell image is input into a pre-trained secondary classification network, and the final category probability is obtained by combining the confidence of the secondary classification network with the improved YOLOv8 target detection algorithm model.

[0043] like Figure 1 In this embodiment, a blood cell automatic detection method based on a classification strategy specifically includes the following steps:

[0044] S1: collect peripheral blood smear microscopic images, preprocess and label the collected blood cell images, and generate a data set, which contains blood cell samples of different categories;

[0045] S2: Perform data enhancement on the blood cell image to obtain data set 1, and divide the processed data set into a training set, a test set, and a validation set according to a preset ratio;

[0046] S3: extracting cell images of categories with a small number of samples as data set 2 of the classification network, inputting the cell images of the data set 2 into the secondary classification network, and completing network training;

[0047] S4: Improve the YOLOv8 target detection algorithm model, and improve the modules and network structure as follows:

[0048] S41: Introduce the FasterNet architecture to replace the bottleneck in the C2f module to form the C2f_f module to improve the efficiency of feature extraction;

[0049] S42: The SimSPPF module is used to replace the original SPPF to improve the speed of the model;

[0050] S43: Use BiFPN to replace PAFPN in the neck part to optimize the feature fusion process and achieve more efficient feature transfer;

[0051] S44: Replace the detection head of the Head part with Dect_Dyhead to improve the accuracy and stability of target detection;

[0052] S45: Introduce the secondary classification network of trained cells in the Head part to perform refined classification on the detected minority cells.

[0053] S5: Input the training set into the network model in S4 for iterative training to obtain a target network model, update the weights, and obtain the optimal detection model weight with the highest mAP value;

[0054] S6: Test the trained model on the test set to verify the accuracy, efficiency, and confidence of cell detection and complete the detection of peripheral blood cells.

[0055] In step S1, by collecting the peripheral blood smear microscopic image, the collected blood cell image is preprocessed by the following method:

[0056] S11: Use a microscope to observe and take high-definition photos of peripheral blood smears to ensure the clarity and details of the images for subsequent analysis; screen the obtained blood cell images, remove low-quality images, perform denoising, and organize them into a peripheral blood cell dataset.

[0057] Specifically, the process of screening the obtained blood cell images includes:

[0058] Perform Laplace transform on the image and calculate its variance value. Generally, the variance value of a clear image is higher, while the variance value of a blurred image is lower.

[0059] A predetermined variance threshold is set, and images with variance values ​​below the threshold are considered to be of low quality and are eliminated, thus completing the screening.

[0060] Specifically, non-local mean filtering (NLM) is applied to the screened high-quality images for denoising. NLM filtering can effectively remove random noise in the image, especially noise around cells or in the background, by analyzing the similarity of local areas of the image, while maintaining the details of the blood cell edges and structures to avoid image blur. After processing, a peripheral blood cell dataset image with clear structure and no noise is generated.

[0061] S12: Use the same target detection labeling method as the YOLO format dataset, and use the dataset labeling software Labelimg to label the peripheral blood cell dataset obtained in step S11. By default, a rectangular box is used to label each cell, and different categories of cells are numbered, such as "102". Finally, a YOLO format dataset is obtained in which each jpg image corresponds to a labeled txt file, and a smaller number of categories are divided into one category for detection.

[0062] Specifically, the process of annotating an image includes:

[0063] In order to better reflect the functional differences of different cell types, the dataset was first divided into five categories, namely, erythrocyte system, granulocyte system, lymphocyte system, monocyte system, and megakaryocyte system;

[0064] In order to judge various pathological conditions, different cell systems are reclassified according to cell morphology (shape, distribution, developmental period), and different types of cells are numbered, such as "102", "103", etc. Finally, the peripheral blood cell system is divided into 46 types of cells, and the categories with smaller numbers (less than a certain value, such as 1000) are divided into one category for detection. The category name is unified as "undefined", and its label category information is modified.

[0065] As an optional implementation, with the assistance of experts, the same target detection labeling method as the YOLO format dataset is used to label the peripheral blood cell dataset obtained in step S11 using the dataset labeling software Labelimg, and a rectangular box is used to label each cell by default; finally, a YOLO format dataset is obtained in which each jpg image corresponds to a labeled txt file, and the bounding box is represented by a quaternion array, and the expression of the bounding box is:

[0066] (x,y,w,h)

[0067] Among them, x is the horizontal coordinate of the center point of the bounding box, y is the vertical coordinate of the center point of the bounding box, w is the width of the bounding box, and h is the height of the bounding box.

[0068] Step S2 performs data enhancement on the blood cell image to obtain a data set 1, and divides the processed data set into a training set, a test set, and a validation set according to a preset ratio, specifically including the following steps:

[0069] S21: Count the various types of blood cells in the data set and determine the number of samples in each category. Considering the imbalance in the number of various types of cells in the data set, the cell images containing the "undefined" category are selected according to the label information. Data enhancement operations such as rotation, translation, scaling, random cropping, flipping, and adding noise are used to expand the number of samples of minority cells. The enhanced images are added to the original data set to obtain data set 1.

[0070] As an optional implementation, the image may be enhanced by adding Gaussian noise. The enhancement process may be expressed as:

[0071] I′(x,y)=I(x,y)+N(0,σ 2 )

[0072] Among them, N(0,σ 2 ) means the mean is 0 and the variance is σ 2Gaussian noise, I(x,y) represents the pixel value of the image at (x,y), and I′(x,y) is the pixel value after adding Gaussian noise at position I(x,y).

[0073] S22: Divide dataset 1 into training set, test set and validation set in a ratio of 8:1:1, place the corresponding image files in the train, val and test folders under the images directory, and place the label files in the train, val and test folders under the labels directory. Finally, save the file path of the image in a txt file, and let the improved YOLOv8 blood cell detection model obtain the image path by reading the txt file for training.

[0074] Step S3 extracts cell images of categories with a small number of samples as data set 2 of the secondary classification network, inputs the cell images of the data set 2 into the secondary classification network, and completes the training of the secondary classification network, which specifically includes the following steps:

[0075] S31: According to the category and coordinate information of the label file, the cells marked as minority classes are cropped from the original image, a dataset folder is created, and the cropped single cell images are saved to subfolders in the dataset with different cell class names as file names to generate dataset 2; in this process, the cropping operation is only used to generate new minority class cell images, but the original images remain unchanged and will not be modified or deleted.

[0076] S32: Figure 6 As shown in the figure, the structure of the secondary classification network is as follows: the ResNet lightweight network is introduced as the backbone network, and the weights pre-trained on the ImageNet large dataset are used for initialization, which can effectively improve the feature extraction ability of the classification network and reduce the training time, especially when the number of samples is small; custom classification layers are added to the last few layers of the network to adapt to the blood cell classification task, specifically including a global average pooling layer (Global AveragePooling Layer) and a Softmax layer, which are used to output the multi-category classification results and confidence of minority cells.

[0077] like Figure 2 As shown in the figure, the YOLOv8 model is improved to better complete the blood cell detection task. The improved modules and network structure are as follows:

[0078] S41: Introduce the FasterNet architecture to replace the bottleneck in the C2f module to form the C2f_f module to improve the efficiency of feature extraction. Specifically:

[0079] FasterNet is introduced. This structure reduces redundant calculations and memory accesses by reducing FLOPs (floating-point operations). The relationship between latency and FLOPS (floating-point operations per second) is:

[0080]

[0081] Considering the case where the input and output feature maps have the same number of channels, the FLOPs of a PConv are only:

[0082]

[0083] Among them, h×w represents the spatial size (height and width) of the output feature map, k 2 is the size of the convolution kernel, is the number of partial channels involved in the calculation.

[0084] When using the commonly used value When the FLOPs of a PConv is one of that of a normal Conv Where r represents the partial ratio in the partial convolution (PConv), c is the total number of channels.

[0085] like Figure 3 As shown in Figure 2, FasterNet mainly consists of a 3×3 local convolution (Pconv) to extract local spatial features of the input feature map; two 1×1 convolutions are used for channel fusion and feature conversion. At the same time, there is a residual connection between the input and output to improve the efficiency of feature extraction and the stability of the network. Figure 3 c is the C2f_f module, which includes two front and rear CBS, a Split module and three FasterNet modules, which effectively reduces the time required for feature extraction.

[0086] like Figure 2 In this embodiment, the neck network includes a cascaded convolution module (CBS), a convolution module, a first C2f_f module, a convolution module, a second C2f_f module, a convolution module, a third C2f_f module, a convolution module, a fourth C2f_f module, and a SimSPPF module (or SPPF module). The second C2f_f module outputs a large-size feature map (P3_in), and the third C2f_f module outputs a medium-size feature map (P4_in_1 and P4_in_2, these two maps Figure 1The same as above, only the input stitching module is different), the SimSPPF module (or SPPF module) outputs a small-size feature map (P5_in).

[0087] S42: The SimSPPF module is used to replace the original SPPF module to improve the speed of the model.

[0088] The SimSPPF module can improve the speed and efficiency of blood cell testing without sacrificing accuracy. Figure 4 As shown in the figure, SimSPPF replaces the SiLU activation function in SPPF with ReLU. The speed of a single ConvBNReLU is 18% faster than ConvBNSiLU. The formula of ReLU is:

[0089] f(x)=max(0,x)

[0090] Among them, f(x) represents the output of the ReLU activation function; x represents the input value; max(0,x) means taking the larger value between the input value and zero.

[0091] The data processing flow of the SimSPPF module of the present invention includes: the input data passes through the first 1×1 CBR module, enters a 5×5 maximum pooling layer, then passes through a 9×9 maximum pooling layer, and then passes through a 13×13 maximum pooling layer, and then the above modules are residually connected, and finally input into the second 1×1 CBR module and input, effectively extracting multi-scale features, optimizing computing performance, and making the detection process more efficient.

[0092] S43: Use BiFPN to replace PAFPN in the neck part to optimize the feature fusion process and achieve more efficient feature transfer.

[0093] In the improved YOLOv8 target detection algorithm model, the neck network processes the three feature maps of different sizes output by the backbone network, and outputs three feature maps of different sizes to the detection head for target detection. The neck network consists of a cascaded first upsampling layer, a first splicing layer, a first C2f module, a second upsampling layer, a second splicing layer, a second C2f module, a first convolution module, a third splicing layer, a third C2f module, a second convolution module, a fourth splicing layer, and a fourth C2f module, where:

[0094] The small-size feature map output by the neck network is used as the input of the first upsampling layer. The feature map output by the first upsampling layer and the medium-size feature map output by the neck network are concatenated in the first concatenation layer.

[0095] The feature map output by the second upsampling layer and the large-size feature map output by the neck network are spliced ​​in the second splicing layer;

[0096] The output of the first C2f module is used as the large-size feature map output by the neck network;

[0097] The feature map output by the first convolutional module and the medium-sized feature map output by the neck network are concatenated in the third concatenation layer;

[0098] The output of the third C2f module is used as the medium-sized feature map output by the neck network;

[0099] The feature map output by the second convolution module and the small-size feature map output by the neck network are concatenated in the fourth concatenation layer;

[0100] The output of the fourth C2f module is used as the small-size feature map output by the neck network.

[0101] BiFPN introduces a weighted strategy, that is, features flow bidirectionally between different levels, and the features of each path are weighted, thereby enhancing the model's ability to process multi-scale features.

[0102] The calculation formula of the output layer feature of FPN is:

[0103]

[0104] in, is the output of the second C2f module; is the output of the second concatenation module; Conv(·) in this embodiment represents the operation of the C2f module; represents the output of the third splicing module; represents the output of the third C2f module; is the output of the fourth splicing module; is the output of the fourth C2f module; Resize(·) means changing the size of the feature map to keep it consistent with the size of the added feature map, usually the operation is upsampling (Upsampling) or downsampling (Downsampling).

[0105] Weighted Bidirectional Feature Pyramid Network: weighted plus a learnable weight; bidirectional is the final feature map combined with bidirectional scale links and fast normalization fusion.

[0106] BiFPN output layer feature weighted calculation formula:

[0107]

[0108] Among them, in each weight (including w 1 、w 2 、w 1 ′、w 2 ′、w 3Adding the ReLU activation function after ′ ensures that the weight value is greater than or equal to zero, and ε = 0.0001 is used to avoid numerical instability.

[0109] from Figure 2 As can be seen from the figure, the structure of BiFPN is as follows: the input three feature maps of different sizes p3, p4, and p5 are obtained, and then after 1×1 convolution adjustment channels, p3_in, p4_in_1, p4_in_2, and p5_in are obtained, and then p5_in is upsampled, and stacked with P4_in_1 after upsampling to obtain P4_td; then P4_td is upsampled, and stacked with P3_in_1 after upsampling to obtain P3_out; after obtaining P3_out, P4_td, and P5_in, P3_out needs to be downsampled, and after downsampling, it is stacked with P4_td and P4_in_2 to obtain P4_out; then P4_out is downsampled, and after downsampling, it is stacked with P5_in to obtain P5_out.

[0110] S44: Replace the detection head of the Head part with Dect_Dyhead to improve the accuracy and stability of target detection.

[0111] Replace the detection head of the Head part with Dect_Dyhead. The structure of the detection head is as follows: Figure 5 As shown, the output vector (L, S, C) is obtained from BiFPN and sent to three perceptually enhanced attention modules, where:

[0112] Scale-aware attention module (π L ): Dynamically fuse the features of different scales according to the semantic importance of different scales on the input L dimension, expressed as:

[0113]

[0114] Among them, F is the input feature, f is a linear function of a 1×1 convolution layer, and the activation function is a Hard_Sigmoid with the formula:

[0115]

[0116] The scale information of the features is extracted through the average pooling operation, and the 1×1 convolution is used for the channel dimension convolution processing. The nonlinear expression ability is increased through the ReLU activation function. Finally, the scale attention weight is generated through the Hard_Sigmoid function to adjust the scale of the input features.

[0117] Spatial perception attention module (π S): Use deformable convolution to make attention learning sparse, and then aggregate different levels of features at the same spatial position. The module operates in the S dimension of the input, expressed as:

[0118]

[0119] Where K is the number of sparse sampling locations, p k +Δp k is the position moved by the self-learned spatial offset, Δp k Learned by deformable convolution, focusing on ambiguous areas; F(l; p k +Δp k ; c) represents the feature map in L dimension, S dimension, C dimension, lth, pth k +Δp k There are c elements, w l,k represents the aggregation weight; Δm k Represents the position Δp k The importance of self-learning.

[0120] The spatial information of the features is extracted through offset learning, and the spatial correlation is further enhanced using a 3×3 convolutional layer, and the expressiveness of the spatial features is improved through ReLU activation.

[0121] Task-aware attention module (π C ): The switch controls whether the hyperparameter learns the activation threshold to automatically choose to open or close the channel to adapt to different tasks, expressed as:

[0122] π C (F)·F=max(α 1 (F)·F C +β 1 (F),α 2 (F)·F C +β 2 (F));

[0123] max() is a hyperparameter function that learns the threshold for activating the channel. C is the feature split at the cth channel, [α 1 ,α 2 ,β 1 ,β 2 ] is a hyperparameter used to control the activation threshold.

[0124] This module learns the task-related channel dimension attention through the fully connected layer, processes the features using the ReLU activation function and normalization operation, and finally outputs the features adjusted by the task attention.

[0125] The enhanced output is obtained by applying three attentions at different positions:

[0126] π(F)=π C (π S (π L (F)·F)·F)·F;

[0127] Among them, π(F) represents the final result obtained by continuously stacking and combining the three attention modules, and F is the feature map of the input attention module.

[0128] S45: Introduce the secondary classification network of trained cells in the Head part to perform refined classification on the detected minority cells.

[0129] In the Head part, the secondary classification network of cells is introduced. The network structure is as follows Figure 6 As shown in the figure, the network backbone adopts ResNet and uses the weights pre-trained in Imagenet. The processing process of the secondary classification network on the image includes:

[0130] The single cell image was input into the secondary classification network, and features were extracted through a 3×3 convolutional layer (Conv Layer 1), and the ReLU activation function was applied to introduce nonlinear operations;

[0131] The features pass through a 2×2 max pooling layer to reduce the size of the feature map and retain the main features;

[0132] Then, the image data enters a deep network consisting of two residual blocks. Each residual block consists of two 3×3 convolutional layers, and each layer has a ReLU activation function. Through skip connection, each residual block directly adds the input to the output of the convolution operation, thus forming a residual learning mechanism to help alleviate the gradient vanishing problem in deep networks.

[0133] After these convolutional and residual blocks, the network applies a global average pooling layer to compress the dimensions of the feature map into a single vector to preserve global information. The compression process is expressed as:

[0134]

[0135] Among them, H and W are the height and width of the feature map;

[0136] Next, the vector is fed into a fully connected layer and generates an output through a Softmax layer, which provides a probability distribution for each category, expressed as:

[0137]

[0138] Among them, P(y i ) represents the probability that the input image belongs to the i-th category; Z i , Z j They represent the features of the i-th and j-th categories output by the fully connected layer respectively; K is the number of categories;

[0139] Output the multi-category classification results and confidence of the cell, return the secondary classification network results, integrate them with the results of YOLO and output them.

[0140] In this embodiment, the training set is input into the improved YOLOv8 for training. The improved YOLOv8 processes the image including:

[0141] The image is passed through Backbone (feature extraction network) to extract the features of peripheral blood cells. Backbone consists of a series of convolutional layers and an improved C2f_f architecture, which can effectively extract cell features of different scales and complexities, ensuring that the cell morphology, contours and internal details are fully captured;

[0142] The features are fused through the Neck part. The improved Neck part adopts BiFPN (weighted bidirectional feature pyramid network), which can realize efficient information transmission and fusion between different levels. BiFPN enhances the processing ability of multi-scale features through weight learning, ensuring the detection accuracy of cells of different sizes. Especially in the detection of peripheral blood cells, the sizes of different cells vary significantly. BiFPN can effectively improve the network's detection ability of small cells and large cells.

[0143] In the Head part, the improved detection head generates the prediction box, category confidence and location information of each cell through the Dect_Dyhead structure. Dect_Dyhead is combined with a dynamic convolution head to improve the precise positioning of cell boundaries and shapes and enhance the stability of the prediction box.

[0144] In order to handle complex cell detection tasks, the Head part also introduces a trained secondary classification network. When some cells are difficult to distinguish due to their small number, the secondary classification network can perform fine-grained classification of minority cells to further improve the classification accuracy.

[0145] The improved yolov8 model in the present invention has three loss functions: classification loss, positioning loss and confidence loss. The classification loss calculates whether the anchor box and the corresponding calibration classification are correct, the positioning loss calculates the error between the prediction box and the calibration box, and the confidence loss calculates the confidence of the network. The target confidence loss and category loss use the binary cross entropy function BCEWithLogitsLoss with sigmoid to calculate the confidence of the network.

[0146] This embodiment uses the bounding box regression loss (CIoU loss) to optimize the difference between the predicted box and the true label box position. The calculation formula of CIoU is expressed as:

[0147]

[0148] Among them, IoU is the intersection-over-union ratio, which represents the ratio of the intersection area of ​​the predicted box and the true box to the union area; ρ 2 (b,b gt ) is the center point of the predicted box and the real box b and the real box b gt The Euclidean distance between the center points of ; c is the diagonal length of a bounding box; v is the aspect ratio penalty term, which is used to describe the difference in aspect ratio between the predicted box and the true box.

[0149] Therefore, the CIoU loss function ensures that the predicted box surrounds the target cell as accurately as possible, which can be expressed as:

[0150] CIoU_loss=1-CIoU

[0151] The classification loss is used to optimize the category prediction of each cell so that the model can accurately classify different types of blood cells. This embodiment uses binary cross entropy to calculate the confidence loss, which is expressed as:

[0152]

[0153] Where N is the number of bounding boxes, t j is the label of the target box (1 means it exists, 0 means it does not exist), is the confidence in the model prediction.

[0154] For the "undefined" category detected by the model, the initial confidence C of the target is first obtained through the YOLO model yolo Then, these “undefined” categories are classified more finely through the secondary classification network to obtain the category confidence C cls , the final confidence C final It is the product of the two, which is used to improve the accuracy of classification of minority categories. The final confidence is expressed as:

[0155] C final=C yolo ×C cls

[0156] The final class probability is expressed as:

[0157]

[0158] Among them, P final (i) is the final probability for category i, C final,k is the final confidence of each category, and K is the total number of categories.

[0159] The entire training process uses the adam optimizer for back propagation. The adam optimizer combines momentum and adaptive learning rate adjustment to make the model converge faster during training and effectively avoid problems such as gradient disappearance or explosion. Through multiple iterations of training, the model gradually updates the weights and finally obtains the detection model weight with the highest mAP value. The mAP value (Mean Average Precision) is an important indicator of the detection model performance. It comprehensively considers the detection accuracy and recall rate. During the training process, the hyperparameters and network structure are continuously adjusted to ensure that the mAP value reaches the optimal value, thereby completing the model training.

[0160] In step S6, the test set is evaluated to verify the accuracy and efficiency of the trained model in cell detection. Specifically, the test image is input into the network, the prediction box, category and confidence are generated, and the prediction accuracy and mAP value are calculated. The performance of the model in detecting peripheral blood cells is evaluated to ensure that it has good adaptability and accuracy in application scenarios of different cell types, and the detection of peripheral blood cells is completed. The test results are as follows: Figure 7 shown.

[0161] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A blood cell automatic detection method based on a classification strategy, characterized in that: The specific steps include: Obtain a labeled blood cell image set, perform data enhancement on the data set to obtain a first training data set, and select minority class samples from the first training data set as a second training data set; training a secondary classification network using the second training data set; Build an improved YOLOv8 target detection algorithm model. If the category output by the detection head in the YOLOv8 target detection algorithm model is not a minority class, the detection result is directly output; If the category of a single cell image output by the detection head is a minority class, the single cell image is input into a pre-trained secondary classification network, and the final category probability is obtained by combining the confidence of the secondary classification network with the improved YOLOv8 target detection algorithm model.

2. The method for automatic blood cell detection based on a classification strategy according to claim 1, characterized in that: In the improved YOLOv8 target detection algorithm model, the FasterNet module is used to replace the bottleneck in the C2f module of the YOLOv8 target detection algorithm model.

3. The automatic blood cell detection method based on a classification strategy according to claim 2 is characterized in that: The FasterNet module includes a cascaded split function layer, three FasterNet units and a splicing module. The splicing module splices the outputs of the split function layer and the three FasterNet units together as the output of the FasterNet module; each FasterNet unit includes a cascaded 3×3 local convolution unit, a 1×1 convolution module and a 1×1 convolution unit, and splices the feature map of the input FasterNet unit with the feature map processed by the 1×1 convolution unit as the output of the FasterNet unit; each convolution module consists of a cascaded convolution layer, a normalization layer and a SiLU activation function layer.

4. The automatic blood cell detection method based on a classification strategy according to claim 1, characterized in that: In the improved YOLOv8 target detection algorithm model, the SimSPPF module is used to replace the SPPF module of the YOLOv8 target detection algorithm model.

5. The method for automatic blood cell detection based on a classification strategy according to claim 4, characterized in that: The SimSPPF module consists of a cascaded convolution module, a 5×5 maximum pooling layer, a 9×9 maximum pooling layer, a 13×13 maximum pooling layer, a splicing unit, and a convolution module. The splicing module splices the output of each maximum pooling layer together as the input of the last level convolution module.

6. The method for automatic blood cell detection based on a classification strategy according to claim 1, characterized in that: In the improved YOLOv8 target detection algorithm model, the neck network processes the three feature maps of different sizes output by the backbone network, and outputs three feature maps of different sizes to the detection head for target detection. The neck network consists of a cascaded first upsampling layer, a first splicing layer, a first C2f module, a second upsampling layer, a second splicing layer, a second C2f module, a first convolution module, a third splicing layer, a third C2f module, a second convolution module, a fourth splicing layer, and a fourth C2f module, where: The small-size feature map output by the neck network is used as the input of the first upsampling layer. The feature map output by the first upsampling layer and the medium-size feature map output by the neck network are concatenated in the first concatenation layer. The feature map output by the second upsampling layer and the large-size feature map output by the neck network are spliced ​​in the second splicing layer; The output of the first C2f module is used as the large-size feature map output by the neck network; The feature map output by the first convolutional module and the medium-sized feature map output by the neck network are concatenated in the third concatenation layer; The output of the third C2f module is used as the medium-sized feature map output by the neck network; The feature map output by the second convolution module and the small-size feature map output by the neck network are concatenated in the fourth concatenation layer; The output of the fourth C2f module is used as the small-size feature map output by the neck network.

7. The automatic blood cell detection method based on a classification strategy according to claim 1, characterized in that: In the improved YOLOv8 target detection algorithm model, the detection head is a cascade of three Dyhead blocks, each of which is composed of a cascaded scale-aware attention module, a spatial-aware attention module, and a task-aware attention module.

8. The method for automatic blood cell detection based on a classification strategy according to claim 1, characterized in that: The loss functions of the improved YOLOv8 target detection algorithm model during training include classification loss, positioning loss and confidence loss. The classification loss calculates whether the anchor box and the corresponding calibration classification are correct, the positioning loss calculates the error between the predicted box and the calibration box, and the confidence loss calculates the confidence of the network. The target confidence loss and classification loss use the binary cross entropy function with sigmoid.

9. The method for automatic blood cell detection based on a classification strategy according to claim 1, characterized in that: The two-level classification network consists of a cascaded 3×3 convolutional layer, a 2×2 maximum pooling layer, two residual blocks, a global average pooling layer, a fully connected layer, and a Softmax layer. Each residual module is cascaded with two 3×3 convolutional layers and the input and output of the residual module are added through a skip connection.

10. The blood cell automatic detection method based on the classification strategy according to claim 1, characterized in that: The final category probability acquisition process of the minority class is expressed as: C final,i =C yolo,i ×C cls,i Among them, P final (i) represents the final probability that a single cell image belongs to category i; C final represents the final confidence that a single cell image belongs to category i, C final,k represents the final confidence that a single cell image belongs to category k; C yolo,i represents the confidence of the improved YOLOv8 target detection algorithm model predicting that a single cell image belongs to category k; C cls,i Represents the confidence of the secondary classification network predicting that a single cell image belongs to category k.

Images