A method for detecting aggregated platelets based on an improved YOLOX algorithm

Abstract

The present invention belongs to the technical field of image target detection, and discloses a method for detecting aggregated platelets based on an improved YOLOX algorithm. The method introduces an RFB module on a backbone feature extraction network, wherein the RFB structure is composed of four convolutional layer branches of different sizes and void rates and a residual edge, and the introduction method is behind the CspLayer layer of the CSPDarknet backbone feature extraction network. Then, in the feature fusion part, an adaptive weighted fusion is performed on the output of PANet, so that the features extracted from the three effective feature layers are fused with different weights before prediction, and more shallow information is introduced to increase the weight of the shallow network, and the weight of the deep network is relatively small, which can improve the accuracy of the network in detecting aggregated platelets.

Description

Technical Field

[0001] The present invention belongs to the technical field of image target detection, and in particular relates to a method for detecting aggregated platelets based on an improved YOLOX algorithm. Background Art

[0002] Routine blood test is a common physical examination item. It can be used to observe the changes in the number and morphological distribution of blood cells to judge the blood condition and disease. Among them, red blood cells can supply oxygen required by tissues and cells throughout the body, accounting for 40%-50% of the total number of blood cells; white blood cells can participate in cellular immunity and have immunomodulatory functions, accounting for 1% of the total number of blood cells; the main function of platelets is coagulation and hemostasis, repairing damaged blood vessels, and the number of platelets in the blood also accounts for a large proportion. One of the important physiological characteristics of platelets is the aggregation function, which refers to the adhesion between platelets and platelets, showing the characteristics of activated platelets interacting and aggregating into clusters. When the platelet aggregation rate is higher, platelets are prone to aggregation to form thrombi, which may also cause coronary artery spasm and cause myocardial microcirculation disorders. Therefore, the determination of platelet aggregation rate is of great significance for the clinical treatment of thrombotic diseases and the diagnosis of pre-thrombotic states. With the development of biology and medicine, the study of cell aggregation has become one of the important topics in cell biology. And the study of platelet adhesion and aggregation is expected to make new progress in this topic.

[0003] At present, the traditional principle of platelet aggregation detection is to add aggregating agents to platelet-rich plasma or whole blood and stir them continuously to produce platelet aggregation. In the 1960s, Born first used turbidimetry to determine platelet aggregation. The disadvantage of this method is that it is insensitive to the formation of platelet aggregates and can only detect large platelet aggregates; PRP of hyperlipidemia will affect the transmittance. Whole blood electrical impedance method, this method requires the electrodes to be cleaned after each measurement, and the wires connecting the electrodes need to be carefully placed and cannot be bent, making it difficult to meet the needs of clinical work. Moreover, most of these methods are physical and chemical experiments, with complex preparation processes, long time consumption, and large errors. The accuracy rate depends largely on the operating specifications and personal experience of the experimenters. Blood smears are currently a common way of clinical testing. By detecting the aggregation of platelets on blood smears, doctors and experts can quickly and accurately obtain the platelet aggregation of patients so that timely and effective treatment plans can be taken.

[0004] Object detection is an important component in the field of computer vision. Among them, the YOLOX algorithm, as an excellent single-target detection algorithm, can directly regress the category probability and position coordinates of the object, and the algorithm is fast. However, there are some inevitable problems when using YOLOX directly for target detection of aggregated platelets. In YOLOX, feature fusion is performed using the feature pyramid FPN, which alleviates the problem of information diffusion to a certain extent. However, directly fusing information of different densities will cause semantic conflicts, limit the expression of multi-scale features, and make small targets easily submerged in conflicting information. Since the size of platelets is less than one-tenth of the image size, they are small targets. Therefore, directly using YOLOX to detect aggregated platelets will result in a large degree of missed detection. Summary of the invention

[0005] In view of the above problems existing in the prior art, the present invention proposes a method for detecting aggregated platelets based on the improved YOLOX algorithm, which solves the problem that small targets cannot be detected and are easily missed in the detection of aggregated platelets.

[0006] The present invention adopts the following technical solutions:

[0007] A method for detecting aggregated platelets based on an improved YOLOX algorithm, the steps are as follows:

[0008] 1) A platelet dataset is collected and enhanced to obtain a platelet enhanced dataset; platelet images are collected using a biological microscope to screen out images in which both single platelets and aggregated platelets are clearly visible, and then data enhancement is performed to obtain an enhanced dataset.

[0009] 2) Collect two types of sample images of platelet aggregation and normal platelet distribution, adjust the sample images to 640×640 pixels suitable for model input, use labeling software LabelImg to label the object images, and store the labeling information in XML format;

[0010] 3) The platelet enhanced dataset was divided into training set, validation set, and test set in a ratio of 8:1:1;

[0011] 4) Input the training set into the YOLOX network, calculate the loss based on the network output value and the true value, and update the network parameters according to the gradient descent method;

[0012] 4.1) Input the images in the training set into the backbone network CSPDarknet of the YOLOX network for feature extraction to output three feature layers of different scales; the backbone network CSPDarkne is mainly composed of four Resblock body modules, the first three Resblock body modules are composed of three parts, the first part is convolution normalization plus activation function, the second part is CspLayer layer, and the third part is RFB module; the fourth Resblock body module is composed of three parts, the first part is convolution normalization plus activation function, the second part is SPBottleneck module, and the third part is CspLayer layer;

[0013] Among them, the RFB module is a multi-branch structure consisting of convolutional layers of different sizes and different dilation rates. The first convolution kernel size in the first branch is 1×1, the second convolution kernel is 3×3, and the dilation rate is 1. The first convolution kernel size in the second branch is 1×1, the second convolution kernel is 3×3, the third convolution kernel is 3×3, and the dilation rate is 3. The first convolution kernel in the third branch is 1×1, the second convolution kernel is 3×3, the third convolution kernel is 3×3, the fourth convolution kernel is 3×3, and the dilation rate is 5. The first convolution kernel in the fourth branch is 1×1, the second convolution kernel is 1×7, the third convolution kernel is 7×1, the fourth convolution kernel is 3×3, and the dilation rate is 7. Finally, the four branches are connected together using 1×1 convolution. The fifth branch is not processed and is directly stacked as the residual edge and the result of 1×1 convolution to obtain the final output.

[0014] 4.2) The feature layers of three different scales are fused through PANet to output three effective feature layers of the first, second and third scales from small to large, and the output of PANet is adaptively weighted fused (Adaptively Spatial Feature Fusion, ASFF) to obtain fused feature maps of three different scales;

[0015] 4.3) Adjust the number of channels of the fusion feature map to num_anchors*(5+num_classes) through YOLO Head to obtain the network output feature map of the training set, where YOLO Head represents the prediction head, num_anchors represents the number of prior boxes on each grid, and num_classes represents the number of categories;

[0016] 5) Calculate the validation set loss on the validation set through the YOLOX network with updated parameters, and determine whether the validation set loss converges;

[0017] 6) Repeat steps 4)-5) until the validation set loss calculated in step 5) converges to obtain the trained YOLOX network model;

[0018] 7) Obtaining new scanned images for detection: After obtaining a stable and efficient detection network in step 6), the automatic detection of aggregated platelets can be achieved by simply sending the scanned images to be detected into the detection network.

[0019] 7.1) Sort all adjusted test set candidate boxes according to their confidence scores;

[0020] 7.2) Use non-maximum suppression to remove redundant boxes from all adjusted test set candidate boxes to obtain the final predicted box.

[0021] Beneficial effects of the present invention:

[0022] (1) The present invention improves the YOLOX algorithm and introduces the RFB module on the backbone network. The RFB module mainly adds a hole convolution layer on the basis of Inception to effectively increase the receptive field. The introduction of the RFB module into YOLOX gives the network a more powerful representation ability, which can enable the network to obtain better detection speed and accuracy.

[0023] (2) For the feature fusion part, the pyramid feature fusion strategy of adaptive spatial feature fusion (ASFF) is added on the basis of the three effective feature layers of the original output, so that the features extracted from each feature layer are fused with different weights before prediction. For small platelet targets, more shallow information is introduced to increase the weight of the shallow network. This can greatly improve the accuracy of the network in platelet aggregation detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 This is a flow chart of aggregated platelet detection based on the improved YOLOX algorithm;

[0025] Figure 2 This is a schematic diagram of the improved network model structure based on YOLOX;

[0026] Figure 3 It is a schematic diagram of the Receptive Field Block module structure;

[0027] Figure 4 It is based on the detection accuracy of platelet aggregation based on the improved YOLOX network model. DETAILED DESCRIPTION

[0028] The specific implementation of the present invention is further described below in conjunction with the accompanying drawings and technical solutions.

[0029] In order to deepen the understanding of the present invention, the present invention will be further described in detail below with reference to the accompanying drawings. Figure 1 As shown in FIG. 1 , the image is a flow chart of the method. The method includes the following steps:

[0030] S1. collecting a platelet data set, and enhancing the platelet data set to obtain a platelet enhanced data set;

[0031] S2, dividing the platelet enhanced dataset into a training set, a validation set, and a test set according to a preset ratio;

[0032] S3. Send the training set to the YOLOX network, extract features through the backbone network, obtain feature layers of three different scales, and then send it to PANet for feature fusion to obtain effective feature layers of three different scales. Adaptively weighted fusion is performed on the effective feature layers to obtain fused feature maps of three different scales, which are sent to YOLO Head for classification and prediction. Calculate the loss based on the network output value and the true value, and update the network parameters according to the gradient descent method;

[0033] S4, training and verifying the detection network: send the training set to the detection network designed in S3 for training, and after the number of iterations is completed, send the test set to the detection network to analyze the network performance and adjust the network training parameters until the test set loss remains unchanged, and obtain the trained YOLOX network model;

[0034] S5. Obtain new images for detection: After obtaining a stable and efficient detection network in S4, the automatic detection of platelet aggregation can be achieved by simply sending the scanned images to be detected into the detection network.

[0035] Specifically:

[0036] Specifically in step S1: Use Nikon E100 biological microscope to take platelet pictures collected from the Second Affiliated Hospital of Dalian Medical University, and screen out platelet pictures that meet the requirements of clarity and size. Use Mosaic data to enhance the platelet data set. The Mosaic data enhancement is to splice any 4 pictures in the data set by random scaling, random cropping, and random arrangement to obtain new pictures and add them to the data set. Its advantage is that it can enrich the background of the detected object and expand the data set. And use LabelImg to annotate the aggregated platelets with the label juji.

[0037] In step S2, the image is adjusted to 300×300 in the network, and the data set is divided according to the ratio of (training set: validation set): test set = (9:1):1. And train.txt, val.txt, and test.txt files are generated to save the corresponding image names and the label positions and category information therein. The generated file format divides each image by line. Taking the train.txt file as an example, each line sequentially saves the path of the image divided into the training set, the position information of all the annotation boxes in the image (x1, y1, x2, y2) and the category information.

[0038] In step S3, specifically:

[0039] S3.1. The images in the training set are input into the YOLOX backbone network CSPDarknet for preliminary feature extraction. The input images are first feature extracted in CSPDarknet. The three feature layers extracted are the feature sets of the input images, with scales of 80×80×256, 40×40×512, and 20×20×1024 respectively.

[0040] The main body of the existing backbone feature extraction network CSPDarknet is composed of four Resblock body modules. The first three Resblock bodies are composed of two parts. The first part is convolution normalization plus activation function, and the second part is CspLayer layer. It is a large residual structure. Its advantages are easy to optimize and can improve accuracy by increasing the depth. The internal residual block uses jump connections to alleviate the gradient vanishing problem caused by increasing the depth in the deep neural network. The fourth Resblock body is composed of three parts. The first part is convolution normalization plus activation function, the second part is SPBottleneck, and the third part is CspLayer layer.

[0041] The present invention improves the backbone network by adding an RFB module after the CspLayer layer in the first three Resblock bodies of the backbone network. Figure 3As shown in the figure, RFB is a multi-branch structure consisting of convolutional layers of different sizes and different dilation rates, in which the first convolution kernel size in the first branch is 1×1, the second convolution kernel size is 3×3, and the dilation rate is 1; the first convolution kernel size in the second branch is 1×1, the second convolution kernel size is 3×3, the third convolution kernel size is 3×3, and the dilation rate is 3; the first convolution kernel in the third branch is 1×1, the second convolution kernel size is 3×3, the third convolution kernel size is 3×3, the fourth convolution kernel size is 3×3, and the dilation rate is 5; the first convolution kernel in the fourth branch is 1×1, the second convolution kernel size is 1×7, the third convolution kernel size is 7×1, the fourth convolution kernel size is 3×3, and the dilation rate is 7. Finally, the four branches are connected together using a 1×1 convolution, and the fifth branch is not processed and is directly stacked as a residual edge and the result of the 1×1 convolution to obtain the final output. RFB is a lightweight module that integrates the advantages of various detection algorithms. It combines the ideas of Inception and wormhole convolution to improve the feature extraction ability of the network by increasing the receptive field.

[0042] S3.2, feature fusion of three feature layers of different scales through feature pyramid, to output effective feature maps of the first, second and third scales from small to large, with scales of 13×13, 26×26, 52×52 respectively, and adaptively weighted fusion (Adaptively Spatial Feature Fusion, ASFF) is performed on the output of the feature pyramid, so that the features extracted from each feature layer are fused with different weights before prediction. For small platelet targets, more shallow information is introduced to increase the weight of the shallow network. This can greatly improve the accuracy of the network in platelet aggregation detection.

[0043] Specifically in step S4: the training is divided into two stages, the freezing and thawing stages. The preset training algorithm is the stochastic gradient descent SGD algorithm. The total number of iterations is 100 times. The freezing stage iterates 50 times, the learning rate is 0.001, and the thawing stage iterates 50 times, the learning rate is 0.0001. The trained model is tested on the test data set to obtain the detection accuracy of the model. When the detection accuracy does not meet the preset accuracy requirements, the model training method is redesigned and the number of model training iterations is increased until a model with detection accuracy that meets the preset accuracy requirements is obtained as the final platelet aggregation detection model.

[0044] Specifically in step S5: obtain a new scanned image for detection: after obtaining a stable and efficient detection network, just send the scanned image to be detected into the detection network, use non-maximum suppression to remove redundant frames, and obtain the final prediction frame.

[0045] In order to verify the performance of the method, the model was systematically evaluated by testing on the originally assigned test set. The experimental results are shown in the figure. The present invention can detect aggregated platelets. The accuracy of the network for aggregated platelets before the change was 88.09% at a confidence level of 0.5. After the change, the detection accuracy of aggregated platelets was 90.91%, an increase of 2.82%. In addition, the improved target detection model adds the RFB module, so the speed is faster than the traditional target detection model in detection.

Claims

1. A method for detecting aggregated platelets based on an improved YOLOX algorithm, characterized in that: Here are the steps: 1) collecting a platelet dataset, and enhancing the platelet dataset to obtain a platelet enhanced dataset; 2) Collect two types of sample images of platelet aggregation and normal platelet distribution, adjust the sample images to a size of 640×640 pixels suitable for model input, annotate the object images, and store the annotation information in XML format; 3) Divide the platelet enhanced dataset into a training set, a validation set, and a test set according to a preset ratio; 4) Input the training set into the YOLOX network, calculate the loss based on the network output value and the true value, and update the network parameters according to the gradient descent method; 4.1) Input the images in the training set into the backbone network CSPDarknet of the YOLOX network for feature extraction to output three feature layers of different scales; the backbone network CSPDarknet is mainly composed of four Resblock body modules, the first three Resblock body modules are composed of three parts, the first part is convolution normalization plus activation function, the second part is CspLayer layer, and the third part is RFB module; the fourth Resblock body module is composed of three parts, the first part is convolution normalization plus activation function, the second part is SPPBottleneck module, and the third part is CspLayer layer; 4.2) The feature layers of three different scales are fused through PANet to output three effective feature layers of the first, second and third scales from small to large, and the output of PANet is adaptively weighted fused to obtain fused feature maps of three different scales; 4.3) Adjust the number of channels of the fusion feature map to num_anchors*(5+num_classes) through YOLO Head to obtain the network output feature map of the training set, where YOLO Head represents the prediction head, num_anchors represents the number of prior boxes on each grid, and num_classes represents the number of categories; 5) Calculate the validation set loss on the validation set through the YOLOX network with updated parameters, and determine whether the validation set loss converges; 6) Repeat steps 4)-5) until the validation set loss calculated in step 5) converges to obtain the trained YOLOX network model; 7) Obtaining new scanned images for detection: After obtaining a stable and efficient detection network in step 6), the automatic detection of aggregated platelets can be achieved by simply sending the scanned images to be detected into the detection network.

2. The method for detecting aggregated platelets based on the improved YOLOX algorithm according to claim 1, characterized in that: The RFB module in step 4) is a multi-branch structure consisting of convolutional layers of different sizes and different dilation rates, where the first convolution kernel size in the first branch is 1×1, the second convolution kernel size is 3×3, and the dilation rate is 1; the first convolution kernel size in the second branch is 1×1, the second convolution kernel size is 3×3, the third convolution kernel size is 3×3, and the dilation rate is 3; the first convolution kernel in the third branch is 1×1, the second convolution kernel size is 3×3, the third convolution kernel size is 3×3, the fourth convolution kernel size is 3×3, and the dilation rate is 5; the first convolution kernel in the fourth branch is 1×1, the second convolution kernel size is 1×7, the third convolution kernel size is 7×1, the fourth convolution kernel size is 3×3, and the dilation rate is 7. Finally, the four branches are connected together using a 1×1 convolution, and the fifth branch is not processed and is directly stacked as a residual edge and the result of the 1×1 convolution to obtain the final output.

3. The method for detecting aggregated platelets based on the improved YOLOX algorithm according to claim 1, characterized in that: In step 1), a biological microscope is used to collect platelet images for the platelet dataset, and images in which both single platelets and aggregated platelets are clearly visible are screened out. Then, data enhancement is performed to obtain an enhanced dataset.

4. The method for detecting aggregated platelets based on the improved YOLOX algorithm according to claim 1, characterized in that: In step 3), the ratio of training set, validation set, and test set is 8:1:

1.

5. The method for detecting aggregated platelets based on the improved YOLOX algorithm according to claim 1, characterized in that: Specific implementation process in step 7): 7.1) Sort all adjusted test set candidate boxes according to their confidence scores; 7.2) Use non-maximum suppression to remove redundant boxes from all adjusted test set candidate boxes to obtain the final predicted box.

Images