A cell detection method and electronic device based on YOLOv8

Abstract

The present invention discloses a cell detection method and electronic device based on YOLOv8, which belongs to the application field of computer neural network technology. It includes: setting a new loss function S-MPDIoU loss function to replace the original loss function, and integrating the improved MDA attention mechanism module into the backbone of the yolov8 network, constructing a target detection model based on the improved yolov8, and obtaining an improved cell detection network model; obtaining the cell detection result of the image to be tested through the tested target cell detection network model. The present invention uses a novel attention module MDA and integrates the module into the backbone of the network to improve the ability of the network to extract fusion features, and replaces the original loss function with a novel S-MPDIoU loss function to improve the cell detection performance.

Description

Technical Field

[0001] The present invention relates to the application field of computer neural network technology, and more specifically to a cell detection method and electronic equipment based on YOLOv8. Background Art

[0002] Compared with traditional visual algorithms, image semantic segmentation technology based on deep convolutional neural networks is widely used in cell detection. The application prospects of deep learning in the medical field are very broad. It can help doctors improve the accuracy and efficiency of diagnosis. With the continuous development of technology, the application of deep learning in the medical field will continue to be improved and perfected. At present, the research on cell detection methods based on deep learning is mainly divided into two directions, based on classification algorithms and based on end-to-end algorithms. Zhao et al proposed a method based on convolutional neural networks to automatically detect and identify different types of white blood cells. This method first detects all white blood cells and then distinguishes the types of white blood cells through classification algorithms. Garcia et al used a deep convolutional network to detect lymphocytes. First, a lymphocyte dataset was created and a deep convolutional neural network was trained using this dataset. After the network training was completed, the original image was tested in the form of a sliding window, and the detected positive cells were selected. Finally, the non-maximum suppression method was used to delete redundant frames to obtain the final detection results. Linderet.al designed a two-stage classification strategy. First, it roughly judged whether an area was a lymphatic infiltration area. Then, according to the judgment result, the lymphatic infiltration area was reclassified to obtain the final refined lymphocyte detection result. The detection method adopted by the above researchers can efficiently process small-size images, but its detection effect is not good for large-size WSI. Object detection is an important task in the field of computer vision. Its purpose is to accurately find and identify the location and category of the target in an image or video. The current representative algorithms include Faster R-CNN, YOLO and SSD. These algorithms automatically learn and extract image features by training deep neural networks, have higher accuracy and better generalization ability, and can realize end-to-end cell detection of pathological images. Inspired by yolov3, Swiderska-Chadaj et.al proposed a method for lymphocyte detection based on the yolo architecture. Wijesinghe et.al proposed a yolo-based method to realize automatic identification and statistics of blood cells. However, there are problems such as insufficient detection accuracy and efficiency.

[0003] YOLOv8 includes a new backbone network, a new Ancher-Free detection head and a new loss function, and can run on a variety of hardware platforms from CPU to GPU. In addition, YOLOv8 is also scalable. Ultralytics did not directly name the open source library YOLOv8, but directly used "ultralytics" to position it as an algorithm framework rather than a specific algorithm. This also makes the YOLOv8 open source library not only applicable to the YOLO series models, but also supports non-YOLO models and various tasks such as classification, segmentation, and posture estimation.

[0004] Therefore, how to provide a cell detection method and electronic device based on YOLOv8 is a problem that technical personnel in this field need to solve urgently. Summary of the invention

[0005] In view of this, the present invention provides a cell detection method and electronic device based on YOLOv8, which are used to solve the above-mentioned prior art.

[0006] In order to achieve the above object, the present invention provides the following technical solutions:

[0007] A cell detection method based on YOLOv8, comprising the following steps:

[0008] Obtaining a test image data set, and preprocessing the data set, dividing the preprocessed image data into a training set and a test set;

[0009] Set a new loss function S-MPDIoU loss function to replace the original loss function, and integrate the improved MDA attention mechanism module into the backbone of the yolov8 network to build a target detection model based on the improved yolov8, and obtain an improved cell detection network model;

[0010] Inputting the training set into the cell detection network model for iterative training to obtain a target cell detection network model;

[0011] The test set is input into the target network cell detection network model for testing, and the cell detection result of the image to be tested is obtained through the tested target cell detection network model.

[0012] Preferably, the new loss function S-MPDIoU calculation steps are as follows:

[0013] Calculate the width of the predicted box and the distance between the coordinate centers of the real box, and the height of the predicted box and the distance between the coordinate centers of the real box;

[0014] Calculate the height and width, and calculate the angle between the predicted frame and the real frame coordinate center;

[0015] Calculate the distance between the upper left corner vertex of the predicted box and the upper left corner vertex of the real box, and the distance between the lower right corner vertex of the predicted box and the lower right corner vertex of the real box;

[0016] Calculate the baseline length based on the angle between the predicted box and the real box coordinate center;

[0017] Introduce a scaling factor scale into the loss function;

[0018] The new loss function S-MPDIoU is obtained according to the distance between the upper left corner vertex of the predicted box and the upper left corner vertex of the real box, the distance between the lower right corner vertex of the predicted box and the lower right corner vertex of the real box, the diagonal length of the feature map, and the scaling factor scale.

[0019] Preferably, the expression for calculating the width between the centers of the predicted frame and the real frame coordinates and the height between the centers of the predicted frame and the real frame coordinates is:

[0020]

[0021]

[0022] In the formula, C h Indicates the width between the predicted box and the real box coordinate center, C w It is expressed as the height between the center of the predicted box and the center of the real box coordinates. are the center coordinates of the predicted box and the true box respectively.

[0023] Preferably, the expression for calculating the height and width and the angle between the predicted frame and the real frame coordinate center is:

[0024]

[0025] Where α is the angle between the predicted box and the center of the real box coordinates.

[0026] Preferably, the expression for calculating the distance between the upper left corner vertex of the prediction box and the upper left corner vertex of the real box, and the distance between the lower right corner vertex of the prediction box and the lower right corner vertex of the real box is:

[0027]

[0028]

[0029] Where d1 is the distance between the upper left corner vertex of the predicted box and the upper left corner vertex of the real box, and d2 is the distance between the lower right corner vertex of the predicted box and the lower right corner vertex of the real box. They are represented as the coordinates of the upper left and lower right corners of the predicted box and the true box, respectively.

[0030] Preferably, the expression for calculating the length of the baseline based on the angle between the predicted frame and the real frame coordinate center is:

[0031] If the angle α between the predicted box and the true box coordinate center is less than π / 4:

[0032] w1=W,h1=W×tana;

[0033] If the angle α between the predicted box and the true box coordinate center is greater than π / 4:

[0034]

[0035] Combining w1 and h1 calculated above, the baseline length C is finally calculated as follows:

[0036]

[0037] In the formula, H, W represent the height and width of the predicted feature map respectively; w1, h1 represent the horizontal width and vertical height of the baseline respectively.

[0038] Preferably, the calculation formula of the scaling factor scale is:

[0039] scale = log2(downsample);

[0040] In the formula, downsample represents the downsampling ratio of the current detection feature map.

[0041] Preferably, the expression of the new loss function S-MPDIoU is:

[0042]

[0043] Where IoU represents the loss function.

[0044] Preferably, the structure of the improved MDA attention mechanism module includes: a one-dimensional convolution layer, an average pooling layer, and a Sigmoid activation function. The input first obtains channel-related feature statistics by performing a pooling operation in the spatial dimension, then obtains the correlation information between the statistics through a one-dimensional convolution, and then performs nonlinear processing through the Sigmoid activation function to complete the extraction of channel attention features. After that, the average pooling operation is performed on the width and height dimensions respectively to extract features, and then the Sigmoid activation function is used for nonlinear processing to obtain the feature statistics of these two dimensions, and finally the feature information is fed back to the original input to achieve feature fusion and enhancement.

[0045] An electronic device comprises: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements a cell detection method based on YOLOv8 when executing the computer program.

[0046] Through the above technical solution, the present invention has the following beneficial effects compared with the prior art:

[0047] 1. A novel attention module MDA is provided and integrated into the backbone of the network to enhance the network's ability to extract fusion features.

[0048] 2. A novel S-MPDIoU loss function is provided to replace the original loss function, which speeds up network training and improves detection effect.

[0049] Other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or understood by practicing the present invention. The purpose and other advantages of the present invention can be realized and obtained by the structures particularly pointed out in the written description, claims, and drawings.

[0050] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 It is a schematic diagram of the method flow of the present invention;

[0052] Figure 2 This is a schematic diagram of the improved yolov8 network framework of the present invention;

[0053] Figure 3 This is a calculation principle diagram of the new loss function S-MPDIoU of the present invention;

[0054] Figure 4 This is a diagram of the improved MDA attention mechanism model architecture of the present invention;

[0055] Figure 5(a) is a graph showing the convergence effects of various IoU loss functions in the horizontal case in IoU simulation experiment 1;

[0056] Figure 5(b) is a graph showing the convergence effects of various IoU loss functions in the diagonal case in IoU simulation experiment 1;

[0057] Figure 5(c) is a graph showing the convergence effects of various IoU loss functions in the vertical case in IoU simulation experiment 1;

[0058] Figure 6(a) is a simulation effect diagram of the feature map to be detected in IoU simulation experiment 2 with a size of 20*20;

[0059] Figure 6(b) is a simulation effect diagram of the feature map to be detected in IoU simulation experiment 2 with a size of 40*40;

[0060] Figure 7 Schematic diagram of manually labeled lymphocytes;

[0061] Figure 8 To improve the YOLOv8n detection effect map;

[0062] Fig. 9 A flowchart for auxiliary diagnosis;

[0063] Figure 10(a) shows the lesion marked by the doctor;

[0064] FIG10( b ) is a diagram showing the detection effect of the auxiliary diagnosis system. DETAILED DESCRIPTION

[0065] The following is a further description of a cell detection method and electronic device based on YOLOv8 of the present invention in conjunction with the accompanying drawings and specific embodiments.

[0066] Example 1

[0067] See also Figure 1 The embodiment of the present invention discloses a cell detection method based on YOLOv8, comprising the following steps:

[0068] Obtaining a test image data set, and preprocessing the data set, dividing the preprocessed image data into a training set and a test set;

[0069] Set a new loss function S-MPDIoU loss function to replace the original loss function, and integrate the improved MDA attention mechanism module into the backbone of the yolov8 network to build a target detection model based on the improved yolov8, and obtain an improved cell detection network model;

[0070] Inputting the training set into the cell detection network model for iterative training to obtain a target cell detection network model;

[0071] The test set is input into the target network cell detection network model for testing, and the cell detection result of the image to be tested is obtained through the tested target cell detection network model.

[0072] See also Figure 2, which is the improved yolov8 network framework of the present invention. YOLOv8 is mainly composed of three parts: the network backbone, the neck and the detection head. The three parts are connected in sequence. The backbone is responsible for extracting feature information, the neck is responsible for fusing feature information of different scales, and the detection head is responsible for processing the extracted features and completing related detection tasks. "CBS" represents the superposition of convolutional layer, batch normalization layer and Sigmoid activation function layer. "C2f" represents the C2f module in YOLOv8. In order to enhance the feature extraction of the input image, we introduced the MDA module in the network backbone to assist in feature processing and analysis.

[0073] In the prior art, the bounding box regression loss function is introduced as follows:

[0074] Early target detection tasks used l n -Regularized loss function, which is simple to implement but sensitive to the scale of the box. With the continuous development of related algorithms, the IoU-based bounding box loss function has gradually replaced l with its ability to reflect the scale difference between the real box and the predicted box. n -Regularized loss function has become the mainstream method for calculating bounding box loss. IoU is the intersection over union ratio of the predicted box and the true box. Its calculation formula is as follows, where B pred represents the prediction box, B gt Represents the real frame.

[0075]

[0076] The idea of ​​various bounding box loss functions derived from IoU is mainly to minimize the IoU loss between the predicted bounding box and the true bounding box, so that the network can learn how to better adjust the position and size of the predicted box, so as to achieve accurate matching with the target true bounding box. In recent years, there have been many studies on the improvement of the IoU loss function. Since IoU cannot be calculated when two predicted boxes do not overlap, in order to solve this problem, GIoU introduces the concept of minimum bounding box, which provides a more accurate distance metric when dealing with non-overlapping bounding boxes, thereby improving the positioning accuracy in target detection. Its loss function is shown in the following formula, where |C| represents the area of ​​the minimum bounding box surrounding the predicted box and the true box. Using GIoU, when the predicted box does not overlap with the true box, the predicted box can still move in the direction of the true box, successfully solving the problem that IoU does not exist when overlapping. However, if the predicted box and the true box have an overlapping relationship, GIoU degenerates into an ordinary IoU and loses its effectiveness.

[0077]

[0078] DIoU introduces distance loss, which makes the predicted box converge to the center of the real box under any circumstances, solving the problem of GIoU. Its calculation formula is shown as follows, where ρ 2 (B pred ,B gt ) represents the square of the Euclidean distance between the predicted box and the true box, C 2 Represents the square of the minimum diagonal length of the bounding box between the predicted box and the true box.

[0079]

[0080] However, DIoU only considers the distance between the predicted box and the real box. When the centers of the two overlap, DIoU will degenerate into IoU. Therefore, the CIoU loss function introduces a penalty term for the similarity of the aspect ratio of the predicted box and the real box on the basis of DIoU, thereby further improving the detection effect. Its formula is shown below, where V evaluates the consistency of the aspect ratio of the predicted box and the real box, and α represents its corresponding trade-off factor.

[0081]

[0082]

[0083]

[0084] Although CIoU introduces shape loss, its consideration of aspect ratio similarity is relative rather than absolute. Based on DIoU, EIoU introduces absolute consideration of regression box width and height, further improving the convergence speed. Its formula is shown below, where ρ 2 (w pred , w gt ), ρ 2 (h pred ,h gt ) represent the square of the width and height difference between the predicted box and the real box, respectively, and (w c ) 2 , (h c ) 2 The square of the width and height of the minimum bounding box representing the predicted box and the true box. This definition makes EIoU sensitive to the width and height of the regression box, avoiding the problem that the penalty term of CIoU is always 0 when the aspect ratio of the predicted box and the true box is the same.

[0085]

[0086] The SIoU loss function redefines the loss function and introduces angle loss, so that during the training process, the predicted box can be moved to the axis closest to the true box, thereby simplifying the training and avoiding the phenomenon that the predicted box wanders around the true box. The SIoU loss function consists of four parts: angle loss, distance loss, shape loss, and IoU loss. The formula is as follows, where Δ represents the distance loss and Ω represents the shape loss. Since the formula definition is relatively complex, it is not repeated here for the sake of simplicity. The problem with SIoU is that the calculation is complex, which makes the results difficult to interpret. In addition, multiple thresholds need to be specified, which makes model comparison difficult and lacks repeatability.

[0087]

[0088] MPDIoU combines the ideas of the above IoU strategies and solves the problem of limiting the convergence speed and accuracy when the predicted box and the real box have the same aspect ratio but different actual width and height by minimizing the vertex distance between the predicted box and the real box, thereby achieving better detection results. The formula is shown below, where w and h represent the square of the length of the diagonal line of the current detection feature map, Represents the square of the distance between the upper left corner and the lower right corner of the predicted box and the real box respectively.

[0089]

[0090] After a large number of simulation experiments, the present invention evaluates the performance of each of the above IoUs. Aiming at the problem that MPDIoU converges slowly in some cases, a scaling strategy is proposed and the penalty term is optimized, so that its convergence speed is further accelerated. The present invention names this new loss function S-MPDIoU. The specific calculation principle of S-MPDIoU is as follows Figure 3 shown.

[0091] Specifically, B pred represents the prediction box, B gt represents the real box, and α represents the angle between the predicted box and the real box coordinate center that is less than π / 2. Assume that the center coordinates of the predicted box and the real box are C h , C w Represents the width and height of the distance between the center of the predicted box and the center of the real box. Assume that the coordinates of the upper left corner and lower right corner of the predicted box and the real box are as well as The two blue lines d1 and d2 represent the distance between the upper left corner and the lower right corner of the predicted box and the real box, respectively. H and W represent the height and width of the predicted feature map, respectively. The orange line C represents the length of the baseline matched to the angle between the current predicted box and the real box. The calculation formulas for each variable are as follows.

[0092]

[0093]

[0094]

[0095]

[0096]

[0097] According to the definition of MPDIoU, C is the diagonal length of the feature map, that is, the baseline length, and its calculation formula is as follows:

[0098]

[0099] The C used in the present invention introduces consideration of the relative position of the predicted box and the real box. Different C values ​​are flexibly set according to the different positions of the predicted box and the real box, so that the convergence speed becomes faster than the fixed C value. The definition formula is shown in the following formula.

[0100] If α is less than π / 4:

[0101] w1=W,h1=W×tanα#(7)

[0102] If α is greater than π / 4:

[0103]

[0104] Combining w1 and h1 calculated above, the final calculated C is as follows:

[0105]

[0106] Although the definition of C is improved to speed up the convergence of the bounding box, when the distance between the predicted box and the real box is very close and does not overlap, the gradient contributed by the penalty term will drop significantly, making it very difficult for the predicted box to converge to the real box. In order to alleviate this problem, the present invention proposes a novel scale idea. By introducing a scaling factor in the loss function, a large gradient can still be maintained when the distance between the predicted box and the real box is close, thus overcoming the convergence bottleneck. The definition formula of scale is shown below, where downsample represents the downsampling ratio of the current detection feature map. For example, when the input original image size is 640*640, when the scale of the detection feature map is 80*80, downsample is 8 and scale is 3.

[0107] scale = log2(downsample) #(9)

[0108] Therefore, the final loss function is defined as follows, where IoU represents the intersection over union (IoU) of the predicted box and the true box.

[0109]

[0110] In summary, the present invention integrates the strategies and techniques in the above research results for cell detection tasks, and after a large number of experimental verifications, proposes a novel and effective IoU loss function S-MPDIoU. The calculation strategy of S-MPDIoU is mainly to calculate the distance between the upper left corner and the lower right corner vertices of the predicted box and the real box, and define baselines of different lengths according to the angle between the central coordinate points of the predicted box and the real box, so as to achieve the unified control of the network on the regression indicators such as the size and shape of the predicted box, avoid the errors and complexity caused by calculating each indicator separately, accelerate training and improve the detection effect.

[0111] In a specific embodiment, the attention mechanism is a key component to improve network performance, and its effectiveness has been fully demonstrated in a large number of visual tasks. There are two types of attention mechanisms widely used in machine vision research, namely the spatial attention mechanism and the channel attention mechanism, which focus on information in the channel and spatial dimensions respectively. The channel attention mechanism helps the model confirm the feature information of the detection target, and the spatial attention mechanism helps the model obtain the location information of the detection target. In recent years, the attention mechanism has developed rapidly. The SE attention mechanism completes the extraction of channel attention features through two steps of Squeeze and Excitation. The CBAM attention mechanism combines channel attention and spatial attention in sequence to improve the expressiveness of the model. The strategy of the GAM attention mechanism is similar to that of the CBAM, which also combines channel attention with spatial attention, but has more parameters to ensure accuracy. The SA attention mechanism uses a grouping method to reduce the number of parameters and splices and fuses features in the channel and spatial dimensions. The CA attention mechanism enhances the performance of the network by embedding position information into the channel attention. This mechanism solves the limitations caused by the traditional channel attention mechanism often ignoring long-distance dependencies. The ECA attention mechanism solves the computational overhead and redundancy introduced by the fully connected layer or complex model construction of the traditional channel attention mechanism through one-dimensional convolution. The EMA attention mechanism adopts a directional pooling strategy to integrate position information for channel attention feature extraction. At the same time, it adopts a cross-spatial multi-scale feature fusion strategy to enrich semantic information and improve detection effects.

[0112] However, conventional attention mechanisms often only focus on feature information in two dimensions, channel and space. This has led to many different design ideas, from conventional flat pooling to directional pooling to obtain features in two dimensions, width and height, thereby introducing position information in disguise, obtaining long-distance dependencies, and then cross-dimensional feature interactions, so that information between feature maps of different sizes can flow, thereby improving the overall feature representation capabilities, etc. However, the above-mentioned design methods often use channel grouping and channel flushing to save parameters, which not only loses a lot of feature information originally input to the feature map, but also the actual number of parameters is still high, and the complexity of the model architecture is therefore increased. Therefore, saving parameters by dimensionality reduction or grouping will greatly reduce the detection effect in most cases and is not applicable.

[0113] See also Figure 4 In a specific embodiment, the present invention integrates the research results of researchers in recent years and designs a novel and effective attention mechanism MDA for the data set of the present invention, that is, a multi-dimensional attention mechanism, which effectively improves the detection performance of the model while saving parameters. The schematic diagram is as follows Figure 4 As shown. The conventional channel attention mechanism is implemented by feature extraction in space, while spatial attention is implemented by feature extraction in channels, but in fact the width and height feature information between channels is also valuable. Compared with simply collecting features in space and channels, mastering the width and height dimensional feature information across channels further enriches and enhances the features obtained, so that each pixel in the feature map is closely related to the global situation, and is no longer dominated by a single dimension. The lymphocyte dataset used in the present invention has a small target volume and a large number, and the background is complex and interfered by many. Conventional attention can easily omit important features. Therefore, the use of multi-dimensional feature extraction can more effectively retain feature information and improve detection accuracy.

[0114] Assuming the input size is C×W×H, the input will first use average pooling along the spatial dimension to extract feature information. The obtained feature vector is roughly the same size as the conventional channel attention, which is C×1×1. The feature vector is subjected to adaptive one-dimensional convolution, and different channels interact with each other, further incorporating the correlation information between channels. After nonlinear processing by the Sigmoid activation function, it is assigned to the original input. Then, the width and height dimensions of the newly obtained feature map are respectively extracted using average pooling. The feature vector size obtained in the width dimension is 1×W×1, which means that the height information across channels is collected and reflected in the width dimension; the feature vector size obtained in the height dimension is 1×1×H, which means that the width information across channels is collected and reflected in the height dimension. The feature vectors of these two dimensions are then nonlinearly processed by the sigmoid activation function. Finally, the obtained multidimensional feature vector is assigned to the original input to obtain the final output. MDA adopts the above design strategy to fuse multi-dimensional features together, which has a gain effect on the detection of small and medium-sized targets. At the same time, the number of parameters is significantly lower than that of other attention mechanisms.

[0115] On the other hand, Embodiment 1 of the present invention further discloses an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements a cell detection method based on YOLOv8 when executing the computer program.

[0116] Specifically, the present invention adopts the following two improvements: 1. A novel and effective attention mechanism MDA is introduced in the backbone of the network. 2. A novel loss function called S-MPDIoU is designed to replace the original loss function. In order to verify the effectiveness of these improvement strategies, this embodiment uses the labial gland biopsy pathology images collected by the hospital to create a data set for lymphocytes, and uses this data set for model training. Compared with the original model, the recall rate of the improved model has increased by 9.1%, mAP.5 has increased by 3.2%, and mAP.95 has increased by 2%. Experimental results show that the improved model can accurately identify and locate lymphocytes in pathology images, and is also applicable to the detection of other types of cells. Based on this model, the present invention has built an auxiliary diagnosis system for Sjögren's syndrome. The experimental results show that the auxiliary system can effectively improve the diagnostic efficiency and provide a reference solution for the application of deep learning technology in the medical field.

[0117] Example 2

[0118] A cell detection method based on YOLOv8 and electronics provided by the present invention are used for simulation experiments.

[0119] Specifically, simulation experiment 1 is as follows:

[0120] In order to evaluate the convergence effect of S-MPDIoU compared with other IoU loss functions, this embodiment conducts a large number of simulation experiments. Since different IoU strategies are very sensitive to the adjustment of the learning rate, a small change will completely change the final convergence effect. Therefore, in order to more fairly evaluate the performance of different IoUs, this simulation experiment uses a fixed learning rate and sets its value to a small value of 0.02, which is sufficient to meet the convergence requirements of all IoUs.

[0121] Assume that the four parameters of the true box are (x gt ,y gt ,w gt ,h gt ), and the four parameters of the prediction box are (x, y, w, h), then the loss formula defined by the simulation experiment is as follows:

[0122] Loss = |x gt -x|+|y gt -y|+|w gt -w|+|h gt -h|#(11)

[0123] The initial number of training rounds in the experiment is set to 100,000. When the loss between the predicted box and the true box is less than 0.01, the predicted box is considered to have converged. Figure 5(a)-Figure 5(c) As shown in Figure 1. This experiment evaluates the convergence effect of different loss functions from three different directions. From Figure a to Figure c, the predicted box and the true box are in a horizontal position, a diagonal position, and a vertical position. The captions indicate the different IoUs and the number of rounds used for convergence. It should be noted that S-MPDIoU has converged in all three cases.

[0124] Please refer to Figure 5. The initial prediction box width and height are significantly larger than the real box. The convergence process of DIoU and CIoU has obvious shape changes. The convergence trend is to expand the width and height of the prediction box first, and then move toward the real box as a whole. When the prediction box surrounds the real box, the width and height are re-contracted to complete the convergence. EIoU and SIoU first compress the width and height of the prediction box, and when a certain scale of width and height converges first, they move toward the real box. However, the convergence speed of EIoU is significantly faster than that of SIoU, which may be due to the more complex definition of the penalty term of SIoU, making it converge more slowly. The convergence of GIoU is more difficult and unstable. In the case of diagonal lines, it is similar to DIoU, first expanding the width and height of the prediction box until it surrounds the real box and then shrinking it, but the convergence process is very slow. In the vertical or horizontal case, it will be similar to EIoU, first converging a certain scale of width and height, and then moving toward the real box. MPDIoU does not deform significantly in these three cases, and its convergence trend is to first reduce the distance between the predicted box and the true box, and then change the shape. However, since the distance between the predicted box and the true box is close and non-overlapping, the gradient contributed by the penalty term defined is very small, and IoU does not participate in the gradient calculation at this time, resulting in a very slow convergence process. The convergence trend of S-MPDIoU is the same as that of MPDIoU, which also moves the position of the predicted box first and then changes the shape of the predicted box. When the predicted box and the true box are on the diagonal line, the calculation of its penalty term is roughly the same as that of MPDIoU, but due to the introduction of scale, it can still retain a large gradient when the predicted box and the true box are close and non-overlapping, and the convergence speed is nearly 6 times faster. In the vertical and horizontal cases, since the reference line defined by the S-MPDIoU penalty term takes into account the angle between the coordinate centers of the predicted box and the true box, its convergence speed is further improved, nearly 10 times faster than MPDIoU.

[0125] From the simulation experiment, it can be seen that S-MPDIoU retains the advantages of MPDIoU, the shape will not mutate, there will be no drift during training, and each round converges along a fixed direction between the predicted box and the true box. By introducing scale and redefining the penalty term, it successfully overcomes the problem of MPDIoU's gradient dwindling when the predicted box and the true box are close and do not overlap, greatly improving its convergence speed.

[0126] Specifically, simulation experiment 2 is as follows:

[0127] In order to further evaluate the convergence performance of different IoU loss functions, another simulation experiment is conducted in this embodiment. Figure 6 (a) and Figure 6 (b) simulate feature maps of 20*20 and 40*40 sizes respectively. In the circle with a radius of 10 and a radius of 20, 50 prediction points are randomly generated. The prediction points are represented by blue, representing the coordinate center of the prediction box. In addition, 5 target points are randomly generated, represented by red, representing the coordinate center of the real box. Each point contains 7 prediction boxes with different aspect ratios, as shown in the red box in the figure. Their areas are all 1, and the aspect ratios are (7:1, 4:1, 2:1, 1:1, 1:2, 1:4, 1:7). In addition, each prediction box aspect ratio has an additional seven scales, which are (0.5, 0.67, 0.75, 1, 1.33, 1.5, 2). Therefore, the total number of regressions is 89500 = 50*7*7*5*7.

[0128] In order to reduce the complexity of the experiment and accelerate the convergence of different IoU, the simulation experiment sets the learning rate to a fixed value of 0.2. The prediction box calculation loss is considered to be converged when it is less than 0.5. The total number of training rounds is 10,000. The simulation experiment results are shown in Figure 2. Figure 6(a)-Figure 6(b) shown.

[0129] Figure 6(a) simulates the case where the size of the feature map to be detected is 20*20. Since the distance between the predicted box and the true box is relatively close, EIoU greatly adjusts the width and height of the predicted box in the early stage of training, causing its calculated loss to drop rapidly. However, in the later stage of training, due to the insufficient contribution of the penalty gradient for the movement of the predicted box, its convergence speed began to slow down, while S-MPDIoU maintained a stable convergence trend and finally converged earlier than EIoU.

[0130] Figure 6(b) simulates the case where the size of the detection feature map is 40*40. In this case, the distance between the predicted box and the true box is further increased, and the convergence difficulty of loss functions such as CIoU, DIoU, and GIoU, which are more dependent on IoU, increases significantly. In the early stage of training, these loss functions continue to expand the width and height of the prediction box, and the movement of the center point of the prediction box coordinates cannot make up for the loss caused by the shape change, resulting in an increase in the total loss. The loss function does not begin to converge until the deformation stops. However, due to the long distance between the predicted box and the true box and the insufficient gradient contributed by the distance penalty term, the convergence is very slow.

[0131] MPDIoU can only play its advantages when the predicted box is far away from the real box. The gradient of its penalty term contribution is significantly increased, allowing it to move quickly towards the real box in the early stage of training. However, when the predicted box is close to the real box, it still faces the problem of gradient reduction and the convergence speed slows down. However, the overall convergence speed is better than the other IoUs mentioned above. Although EIoU converges faster, it is also slowed down due to the distance problem. S-MPDIoU can always maintain a faster convergence speed and has a very obvious advantage.

[0132] Specifically, the target detection experiment is as follows:

[0133] (1) Experimental environment and parameter settings

[0134] The original YOLOv8 algorithm provides five models of different scales: N, S, M, L, and X. Although the structure of these five scale models remains unchanged, each scale model has different depths and widths, resulting in different sizes and complexities. In this embodiment, the ability of the YOLOv8n model to detect lymphocytes is tested and analyzed in the experiment. The platform used for model training in this experiment is Intel Core i9-13900 CPU and NVIDIA GTX40608G GPU. The software uses Windows system, Python 3.11, PyTorch 2.0.1 and Cuda11.8 deep learning framework.

[0135] The network is trained for a total of 100 rounds, with a batch size of 6. The input image size is 640 × 640. The initial learning rate is set to 0.001, and ADAM is used as the optimization algorithm. The weight decay coefficient is 0.005, the momentum is set to 0.937, and all data augmentation strategies are turned off.

[0136] 4.2.2 Experimental Dataset

[0137] The experimental dataset used in this embodiment is derived from the WSI of the labial gland biopsy specimen of Yantaishan Hospital. The use of this dataset has been approved by the hospital's Ethics Review Committee. Due to the large size of the WSI, it is not feasible to directly detect lymphocytes. Therefore, this embodiment divides the original WSI into block images of size 640×640 at the highest resolution, and creates a dataset containing 600 images under manual screening and annotation by professional pathologists. It should be noted that at a size of 640×640, the average number of lymphocytes contained in a single image is 30, and some images even exceed 70, so the manual labeling process is extremely cumbersome and time-consuming. Since the detection object of the detection task is only lymphocytes, and there are a large number of lymphocytes with unique and consistent characteristics in a single image. Therefore, this embodiment chooses to manually annotate 600 images and selects a relatively lightweight model for training to achieve a balance between model training and annotation complexity. The dataset is divided into training set, validation set, and test set in a ratio of 8:1:1. Examples of annotated images are shown in the figure. Figure 7 Shown, where green boxes indicate manually annotated lymphocytes.

[0138] (2) Experimental evaluation indicators

[0139] This example uses a set of standard metrics to evaluate the performance of the improved YOLOv8 in the lymphocyte detection task. The main metrics considered in this example are recall (R), precision (P), and mean average precision (mAP). Since the only object detected in this study is lymphocytes, these metrics can be expressed as follows:

[0140]

[0141]

[0142]

[0143] Among these evaluation indicators, TP refers to instances correctly predicted as positive samples, TN refers to instances correctly predicted as negative samples, FP refers to instances incorrectly predicted as positive samples, and FN refers to instances incorrectly predicted as negative samples.

[0144] (3) Experiment

[0145] This embodiment first evaluates the performance of various IoU loss functions on the data set of this embodiment, and the experimental results are shown in Table 1. As can be seen from Table 1, S-MPDIoU has the highest mAP among all IoU loss functions participating in the experiment, and its precision and recall rate are also very stable. Compared with the initial CIoU of v8, its detection accuracy is slightly reduced by 0.7%, but the recall rate is increased by 7.8%, mAP.5 is increased by 4.3%, and mAP.95 is increased by 2.1%. The experimental results fully demonstrate the effectiveness of S-MPDIoU.

[0146] Table 1 Comparison of experimental results at different IoUs

[0147]

[0148]

[0149] This embodiment also conducts an experiment to compare and analyze the detection performance of different attention mechanisms. The results are shown in Table 2. Compared with SE, CBAM, GAM, CA, ECA, EMA, and SA, the recall rate of the MDA module is slightly lower, but the precision and mAP are higher, and the parameters are relatively few. Based on the experimental results, this embodiment introduces multiple MDA modules in the backbone of the model to enhance and fuse features for subsequent detection, balancing the network detection accuracy and parameter efficiency.

[0150] Table 2 Comparison of the effects of different attention mechanisms

[0151]

[0152] The above improvement strategies are integrated into the original network framework, and ablation experiments are performed to evaluate the effectiveness of these strategies. The experimental results are shown in Table 3. As can be seen from Table 3, after incorporating all the above improvement strategies, the accuracy of the final model is 80%, indicating a relative decrease of 0.4%. However, its recall rate is 86.1%, a relative improvement of 9.1%. In addition, the model achieves 88.5% mAP.5, indicating a relative improvement of 3.2%, and 37.9% mAP.95, indicating a relative improvement of 2%. These results ultimately demonstrate the effectiveness of the cell detection improvement measures proposed in this embodiment.

[0153] Table 3 Ablation experiment results

[0154] SMPDIo MDA P / ％ R / ％ mAP.5 / % mAP.95 / % × × 80.4％ 77％ 85.3％ 35.9％ √ × 79.7％ 84.8％ 89％ 38％ × √ 80.9％ 83.3％ 88.9％ 37.8％ √ √ 80％ 86.1％ 88.5％ 37.9％

[0155] This embodiment also compares and analyzes the improved network with many state-of-the-art networks with similar parameters. The experimental results are shown in Table 4. The experimental results show that the enhanced YOLOv8 network proposed in this embodiment achieves a good balance between model detection accuracy and complexity.

[0156] Table 4 Model comparison test results

[0157]

[0158] Improve the detection effect of YOLOv8n Figure 8 As shown in the figure, the red box represents the lymphocytes detected by the network. As can be seen from the figure, the improved YOLOv8n model fully learns the characteristics of lymphocytes, complies with the discrimination criteria of lymphocytes, overcomes the interference caused by complex backgrounds, and basically achieves correct discrimination for suspicious similar cells such as epithelial cells and plasma cells, completing the detection and positioning task for lymphocytes.

[0159] (4) Construction of auxiliary diagnosis system

[0160] Finally, this embodiment uses this lymphocyte detection model to build an auxiliary diagnosis system for Sjögren's syndrome, and its flow chart is as follows: Fig. 9 First, the original WSI image is segmented and split into multiple patches, and then these patches are sent to the detection model one by one for lymphocyte detection. Patches with a lymphocyte count greater than the set threshold are marked in red and regarded as a suspicious lesion, thereby completing the task of auxiliary detection.

[0161] (5) Auxiliary diagnosis flow chart

[0162] As shown in Figure 10, Figure A represents the doctor's marking detection, and Figure B represents the improved YOLOv8n recognition-assisted detection. It is obvious that the lymphocyte detection model proposed in this embodiment effectively identifies and marks suspicious lesion areas, which roughly correspond to the areas annotated by the doctor, indicating that the model has a high accuracy in lesion identification and is helpful for pathological diagnosis to a certain extent.

[0163] (6) Summary

[0164] This embodiment carries the MDA attention mechanism on the network backbone of YOLOv8n, and introduces the S-MPDIoU loss function. Compared with the original model, the recall rate of the improved model has increased by 9.1%, mAP.5 has increased by 3.2%, and mAP.95 has increased by 2%, proving the effectiveness of the above-mentioned improvement measures. In recent years, with the development of artificial intelligence technology, its application in the medical field has also gradually increased. This embodiment uses deep learning methods to detect lymphocytes in pathological images, which has been proven to be feasible and effective, and provides a valuable reference case for the subsequent application of artificial intelligence technology in the medical field for auxiliary pathological diagnosis.

[0165] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0166] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be construed as limiting the scope of the present application. It should be noted that, for a person of ordinary skill in the art, several modifications and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A cell detection method based on YOLOv8, characterized in that: The following steps are involved: Obtaining a test image data set, and preprocessing the data set, dividing the preprocessed image data into a training set and a test set; A new loss function S-MPDIoU loss function is set to replace the original loss function, and the improved MDA attention mechanism module is integrated into the backbone of the yolov8 network to build an object detection model based on the improved yolov8, and obtain an improved cell detection network model; Inputting the training set into the cell detection network model for iterative training to obtain a target cell detection network model; Inputting the test set into the target cell detection network model for testing, and obtaining the cell detection result of the image to be tested through the tested target cell detection network model; The calculation steps of the new loss function S-MPDIoU are as follows: Calculate the width of the predicted box and the distance between the coordinate centers of the real box, and the height of the predicted box and the distance between the coordinate centers of the real box; Based on the height and width, calculate the angle between the predicted box and the center of the real box coordinates; Calculate the distance between the upper left corner vertex of the predicted box and the upper left corner vertex of the real box, and the distance between the lower right corner vertex of the predicted box and the lower right corner vertex of the real box; Calculate the baseline length based on the angle between the predicted box and the real box coordinate center; Introduce a scaling factor scale into the loss function; The new loss function S-MPDIoU is obtained based on the distance between the upper left corner vertex of the predicted box and the upper left corner vertex of the real box, the distance between the lower right corner vertex of the predicted box and the lower right corner vertex of the real box, the diagonal length of the feature map, and the scaling factor scale; The expression of the new loss function S-MPDIoU is: ； Where IoU represents the loss function, It is expressed as the distance between the upper left corner vertex of the predicted box and the upper left corner vertex of the real box. It is represented as the distance between the lower right corner vertex of the predicted box and the lower right corner vertex of the real box, and C is represented as the length of the baseline; The structure of the improved MDA attention mechanism module includes: one-dimensional convolution layer, average pooling layer and Sigmoid activation function. First, the channel-related feature statistics are obtained by performing pooling operation in the spatial dimension, and then the correlation information between the statistics is obtained through one-dimensional convolution, and then the Sigmoid activation function is used for nonlinear processing to complete the extraction of channel attention features; then, the average pooling operation is performed on the width and height dimensions respectively for feature extraction, and then the Sigmoid activation function is used for nonlinear processing to obtain the feature statistics of these two dimensions, and finally the feature information is fed back to the original input to achieve feature fusion and enhancement.

2. A cell detection method based on YOLOv8 according to claim 1, characterized in that, The expression for calculating the width between the centers of the predicted box and the real box coordinates and the height between the centers of the predicted box and the real box coordinates is: ； ； In the formula, Indicates the width of the distance between the predicted box and the real box coordinate center. It is expressed as the height between the center of the predicted box and the center of the real box coordinates. are the center coordinates of the predicted box and the true box respectively.

3. A cell detection method based on YOLOv8 according to claim 2, characterized in that, The expression for calculating the angle between the predicted box and the center of the real box coordinates based on the height and width is: ； In the formula, It is expressed as the angle between the center of the predicted box and the center of the ground-truth box coordinates.

4. A cell detection method based on YOLOv8 according to claim 1, characterized in that, The expression for calculating the distance between the upper left corner vertex of the prediction box and the upper left corner vertex of the real box, and the distance between the lower right corner vertex of the prediction box and the lower right corner vertex of the real box is: ； ； In the formula, It is expressed as the distance between the upper left corner vertex of the predicted box and the upper left corner vertex of the real box. It is expressed as the distance between the lower right corner vertex of the predicted box and the lower right corner vertex of the real box. as well as They are represented as the coordinates of the upper left and lower right corners of the predicted box and the true box, respectively.

5. A cell detection method based on YOLOv8 according to claim 1, characterized in that, The expression for calculating the baseline length based on the angle between the predicted box and the real box coordinate center is: If the angle between the predicted box and the real box coordinate center Less than : ； If the angle between the predicted box and the real box coordinate center Greater than : ； Combined with the above calculations , the final calculated baseline length C is as follows: ； Where H and W represent the height and width of the predicted feature map respectively; w1 and h1 represent the horizontal width and vertical height of the baseline respectively.

6. A cell detection method based on YOLOv8 according to claim 1, characterized in that, The calculation formula of the scaling factor scale is: ； In the formula, It represents the downsampling ratio of the current detection feature map.

7. An electronic device comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the YOLOv8-based cell detection method according to any one of claims 1 to 6 is implemented.

Images