Deep learning-based urine formed element detection method

Abstract

The application relates to a deep learning-based urine formed element detection method. First, in view of the problems that a urine formed element image has large background noise and weak edge features, a global maximum pooling feature aggregation channel is added on the basis of a global average pooling feature aggregation channel of an ECA-Net, more image edge information is reserved, an EECA-Net attention mechanism is further proposed, and the EECA-Net attention mechanism is added to a backbone network and a neck network, so that the extraction capability of the model for edge features is enhanced; second, considering that the parameter quantity of YOLOv5s is large, GSConv modules and VoV-GSCSP modules are used to perform lightweight processing on Conv modules of the backbone network and the neck network and C3 modules of the neck network, so that the model complexity is reduced. Experimental results show that the detection precision of the application is 1.7% higher than that of YOLOv5s, and the model parameter quantity is reduced by 15%.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of image processing, and particularly relates to a urine formed element detection method based on deep learning. BACKGROUND

[0002] Urine formed element examination is a very classic test project with a history of hundreds of years, and is the earliest clinical test technology. It, together with urine physics examination and urine chemistry examination, constitutes the whole content of urine routine analysis, and complements and verifies each other. However, it should be pointed out that formed element examination has more obvious application value for clinical doctors to understand the changes of each part of the urinary system, and to assist in positioning diagnosis, differential diagnosis and prognosis of urinary system diseases. However, at present, urine formed element detection still mainly relies on traditional manual microscopy, and the detection accuracy of urine formed elements depends on the clinical experience and professionalism of medical personnel. The efficiency is low, and the labor cost is high. At the same time, due to the image characteristics of formed elements, it is also easily affected by visual bias, equipment failure and other external factors.

[0003] In 1991, the American NCCLS proposed the recommended standard (GP-P) of "urine routine analysis", and in 2000, the Center for Clinical Laboratory of the Ministry of Health of China also proposed the standard document of "urine physics, chemistry and sediment analysis", so the research and development of urine formed element analysis instruments in China started in early 2000. The DiaSys urine formed element digital image shooting system appeared before 2000 opened up the precedent of digital image urine formed element analysis, and after 2002, domestic companies began to develop intelligent digital image urine formed element analysis systems with automatic recognition capability, opening up the leading position of this technology in the field of urine formed element analysis. In the following years, due to the rapid development of computer technology, digital image technology and other related technologies, instruments using digital image technology as the basic principle have entered an era of rapid development.

[0004] Before 2010, many urine formed element detection methods based on machine learning methods were developed. Ranzato et al. used Gaussian mixture generation model to classify each sample; Liang et al. used local context perception and SVM to construct a classifier, and the accuracy was improved to 93.72%; Liu et al. introduced genetic algorithm into BP neural network to optimize network weights and thresholds; Ji et al. proposed regional feature algorithm (AFA) to solve the shortcoming that CNN would weaken the area features of input images. Although traditional machine learning methods have achieved excellent performance, they can only detect limited types of urine formed elements.

[0005] With the development of deep learning, the performance of neural networks is further improved by the proposed convolutional neural network, recurrent neural network, etc., which makes it better cope with specific problems in specific fields and embodies its strong ability. Ji et al. proposed a semi-supervised learning-based urine sediment image classification method, and designed a reparameterization network-US-RepNet to extract complex features of urine sediment images. Zhang et al. used a pre-trained Faster R-CNN model to detect two types of cells, red blood cells and white blood cells, in urine sediment images, with an accuracy of 91.4%. Yan et al. used a bidirectional context propagation network BCPNet for urine sediment particle detection, which improved the positioning and classification ability of the model. Zhang et al. proposed a multi-view urine cell recognition method based on multi-view deep residual learning, and introduced a depth separable convolution to reduce network parameters. Ji et al. proposed a regional feature algorithm (AFA) to solve the problem that CNN can weaken the area features of input images. The network model trained with 300,000 urine sediment images can quickly and accurately identify 10 types of urine sediment images with an accuracy of 97%. Liang et al. based on SSD, PVANet and Faster R-CNN models, identified urine sediment particles in an end-to-end manner. Among the 7 types of urine sediment particles, the mAP reached 84.1%.

[0006] The deep learning-based method has good performance in urine formed element detection, but there are still problems in actual application scenarios: the low contrast and weak edge features of urine formed element images still make feature extraction relatively difficult, and the existing model has high complexity, and there is still room for improvement in model optimization.

[0007] The difficulties of urine formed element detection are: (1) The large image background noise and weak edge features make the segmentation and extraction of features more difficult. (2) The target objects in the urine formed element image are small compared to the picture. The model has high requirements for small-scale target detection capability. (3) The characteristics of different cell components are small, which increases the difficulty of recognition and classification. SUMMARY

[0008] The purpose of the present application is to solve the problem that the edge features of the urine formed element image sample are weak, and the model has weak extraction ability for the edge features. A deep learning-based urine formed element detection method is provided, which improves the urine formed element detection accuracy and reduces the model complexity.

[0009] To achieve the above object, the technical scheme of the present application is: a urine formed element detection method based on deep learning, first, on the basis of the global average pooling feature aggregation channel of ECA, a global maximum pooling feature aggregation channel is added, an EECA attention mechanism is proposed, and the EECA attention mechanism is added in the backbone network and the neck network to enhance the extraction ability of the model to edge features; secondly, on the basis of YOLOv5, the VoV-GSCSP module is used instead of the C3 module in the neck network, the GSConv module is used instead of the Conv module in the backbone network and the neck network, and the Slim-Neck structure is formed in the neck network.

[0010] In an embodiment of the present application, the backbone network uses CSP-Darknet53, and the GSConv module is used instead of the Conv module.

[0011] In an embodiment of the present application, in the neck network, the VoV-GSCSP and the GSConv designed by applying the one-shot aggregation method on the basis of the GSConv replace the C3 module and the Conv module in the neck network respectively, and are combined into the Slim-Neck structure.

[0012] In an embodiment of the present application, the EECA attention mechanism is obtained as follows:

[0013] ECA obtains features through global average pooling GAP, and generates channel weights through one-dimensional convolution with a size of k, wherein k is adaptively determined by the value of the channel number C; the calculation formula of k is as follows:

[0014]

[0015] Where |t| odd Indicates the nearest odd number to t; respectively, γ and b are set to 2 and 1; obviously, through the mapping relationship ψ, the high-dimensional channel has a larger range of interaction, and the low-dimensional channel has a smaller range of interaction through the use of nonlinear mapping; let the channel feature maps of different sizes obtain the cross-channel information interaction suitable for the channel size of the feature map;

[0016] On the basis of the global average pooling GAP feature aggregation channel of ECA, a global maximum pooling feature aggregation channel is added to retain more image edge texture information; specifically, different aggregation features are obtained by global average pooling and global maximum pooling on the input feature map with a size of HxWxC, one-dimensional convolution with a size of k is performed to generate channel weights with a size of 1x1xC, and after Sigmoid activation function, multiplication is performed, and then the original feature map is multiplied to obtain the output result.

[0017] In an embodiment of the present application, the method uses a loss function composed of three parts, as follows:

[0018] (1) Object Score loss

[0019] When detecting objects, the model uses a binary cross-entropy loss that can determine whether the target frame contains the object, where o is the label vector, t is the prediction vector, n is the total number of samples, and i is the sample label:

[0020]

[0021] (2) Class Probability Score loss

[0022] When detecting classes, the model uses a binary cross-entropy loss that can determine whether the target frame contains the correct class, as with Object Score, where o is the label vector, t is the prediction vector, n is the total number of samples, and i is the sample label:

[0023]

[0024] (3) Bounding Box loss

[0025] When detecting whether the predicted frame can compactly frame the target object, the model uses CIoU Loss:

[0026]

[0027]

[0028]

[0029] where IoU is the ratio of the overlapping area between the predicted frame and the real frame to the total area contained by the predicted frame and the real frame; D2 is the distance between the center of the predicted frame and the target frame, D c is the diagonal distance of the minimum circumscribed rectangle; w and h are the width and height of the predicted frame, respectively, and w gt and h gt are the width and height of the real frame, respectively.

[0030] Compared with the prior art, the present application has the following beneficial effects: by analyzing the existing problems of feature extraction, combining the characteristics of urine formed element images, and increasing the maximum pooling feature extraction channel based on the ECA attention mechanism, the attention mechanism of the present application enhances the feature extraction capability of weak edge feature images. The EECA attention mechanism is invented. In the prediction head module and the backbone network, the EECA attention mechanism module is added to optimize the model. The data of the comparative test shows that the EECA is the best in improving the model compared with other excellent attention mechanisms such as ECA, SE, CBAM and CA. In the backbone network, the Conv is replaced by the GSConv, and the GSConv and VoV-GSCSP are introduced in the neck network at the same time, forming the Slim-Neck structure, reducing the redundant repeated information, improving the attention mechanism effect, reducing the model complexity while maintaining the accuracy. BRIEF DESCRIPTION OF DRAWINGS

[0031] Figure 1 The method flowchart of the present application.

[0032] Figure 2 The backbone network: (left) the original backbone network structure; (right) the backbone network structure after optimization using GSConv and EECA attention mechanism.

[0033] Figure 3 GSConv module (DWConv is depthwise separable convolution).

[0034] Figure 4 VoV-GSCSP module.

[0035] Figure 5 ECA attention mechanism module.

[0036] Figure 6 EECA attention mechanism module.

[0037] Figure 7 Dataset1 each category part sample image and label number.

[0038] Figure 8 Dataset2 each category part sample image and label number.

[0039] Figure 9 Sample 1 and the corresponding label.

[0040] Figure 10 The detection result of the present application on sample 1.

[0041] Figure 11 Sample 2 and the corresponding label.

[0042] Figure 12 The detection results of the application on sample 2. DETAILED DESCRIPTION

[0043] The technical solutions of the application will be specifically described below with reference to the drawings.

[0044] As Figure 1 shown, after trying a plurality of existing attention mechanisms, the application improves the ECA attention mechanism, adds a maximum pooling feature extraction channel, and invents an EECA attention mechanism. Compared with the existing SE, CA, CBAM and ECA attention mechanisms, the EECA has obvious improvement in the feature extraction capability of urine components. Moreover, the existing model has high complexity, and there is still room for improvement in model optimization. In order to reduce the complexity of the model while maintaining the accuracy, the application uses a VoV-GSCSP module to replace the C3 module in the Neck network on the basis of YOLOv5, uses a GSConv module to replace the Conv module in the backbone network and the Neck network, and forms a Slim-Neck structure in the neck network. The GSconv makes the output of the depth separable convolution (DSC) operation as close as possible to the output of the standard convolution (SC), has less redundant repeated information, does not need to be compressed, and makes the attention module work better, reduces the calculation cost, and maintains the accuracy.

[0045] The method of the application is described in detail as follows:

[0046] 1 Network structure

[0047] 1.1 Backbone network

[0048] The application is a model optimized after modular modification on the basis of YOLOv5, and the backbone network uses a CSP-Darknet53. The backbone network structure is as shown in Figure 2 We introduce a new method-GSConv, and its structure is as shown in Figure 3 In order to speed up the calculation speed of prediction, the image in the CNN almost always has to experience the process of spatial information gradually transferring to the channel in the backbone. Each time of spatial compression and channel expansion will cause partial loss of semantic information. The dense convolution maximizes the hidden connection between each channel, but the sparse convolution completely cuts off these connections. The GSConv maximizes the preservation of these connections as much as possible, maximally reduces the negative influence of the defects of DSC on the model, and effectively utilizes the benefits of DSC.

[0049] 1.2 Neck network

[0050] In the neck network, we will replace the C3 and Conv modules of the original model with VoV-GSCSP and GSConv designed based on the one-shot aggregation method in the GSConv in the neck network, respectively, to form a Slim-Neck structure. The VoV-GSCSP structure is as shown in Figure 4 Experiments show that, at this stage, using GSConv to process the redundant repeated information of connected feature maps is less redundant and does not need to be compressed, and the attention module works better.

[0051] 1.3 Attention mechanism module

[0052] The edge features of urine formed elements images are weak, and the target objects only occupy a small part of the image area, so it is difficult to extract features, and a large amount of background information will have a negative impact on the detection accuracy of the model. Attention mechanism performs well for such problems. We integrate the ECA attention module in YOLOv5s.

[0053] ECA is an efficient channel attention mechanism for deep convolutional neural networks. As shown in Figure 5 , ECA obtains features through global average pooling (GAP) and generates channel weights through one-dimensional convolution of size k, where k is adaptively determined by the value of the number of channels C. The calculation formula of k is as follows:

[0054]

[0055] where |t| odd represents the nearest odd number to t. Set γ and b to 2 and 1, respectively. Obviously, through the mapping relationship ψ, high-dimensional channels have a larger range of interaction, while low-dimensional channels have a smaller range of interaction through the use of nonlinear mapping. Let the channel feature maps of different sizes obtain cross-channel information interaction suitable for the size of the feature map channel.

[0056] ECA-Net uses adaptive convolution to allow channels of different sizes to achieve different ranges of interaction. In feature extraction, due to limited neighborhood size and errors in convolutional layer parameters, problems such as increased variance in estimated values ​​and shifted estimated means arise. Average pooling focuses more on global information, thus reducing the increased variance in estimated values ​​caused by limited neighborhood size and preserving more image background information. Max pooling focuses more on prominent information, reducing the shift in estimated means caused by convolutional layer parameter errors and preserving more image texture information. Therefore, this invention adds a Global-max-pooling feature aggregation channel to the Global-average-pooling feature aggregation channel of ECA-Net to preserve more image edge texture information. Specifically, the input feature map of size H×W×C is processed through Global-average-pooling and Global-max-pooling to obtain different aggregated features. Fast one-dimensional convolutions of size k are then performed to generate channel weights of size 1×1×C. After passing through a Sigmoid activation function, these weights are multiplied, and then multiplied by the original feature map to obtain the output result. Its structure is as follows: Figure 6 As shown.

[0057] We incorporated the EECA module into both the prediction head module and the backbone network to optimize the model. Comparative experimental data demonstrate that EECA significantly enhances feature extraction capabilities in urine formed element images, offering a remarkable improvement over other excellent attention mechanisms such as ECA, SE, CBAM, and CA.

[0058] 1.4 Loss Function

[0059] The loss function of this invention consists of three parts.

[0060] 1.4.1 Object Score Loss

[0061] When detecting objects, the model uses binary cross-entropy loss to determine whether the target box contains an object, where o is the label vector, t is the predicted vector, n is the total number of samples, and i is the sample label.

[0062]

[0063] 1.4.2 Class Probability Score Loss

[0064] When detecting categories, similar to Object Score, the model uses binary cross-entropy loss to determine whether the target box contains the correct category, where o is the label vector, t is the predicted vector, n is the total number of samples, and i is the sample label:

[0065]

[0066] 1.4.3Bounding Box loss

[0067] When detecting whether the predicted bounding box can compactly frame the target object, the model uses the more superior CIOU Loss:

[0068]

[0069] wherein a is:

[0070]

[0071] v is:

[0072]

[0073] wherein, IoU is the ratio of the overlapping area between the predicted bounding box and the real bounding box to the total area contained by the predicted bounding box and the real bounding box; D2 is the distance between the center points of the predicted bounding box and the target box, D c is the diagonal distance of the minimum circumscribed rectangle; w and h are the width and height of the predicted bounding box, w gt and h gt are the width and height of the real bounding box.

[0074] 2. Implementation Examples

[0075] In order to evaluate the performance of the urine formed element detection algorithm, we use two urine formed element data sets for experiments (referred to as DataSet1 and DataSet2). DataSet1 is from the public data set of USE, mainly used for performance evaluation of module improvement. It is composed of 5646 images with a resolution of 800x600, containing 7 cell categories: Cast, Cryst, Epith, Epithn, Eryth, Leuko, and Mycete. Figure 7 The images of the 7 categories of DataSet1 and the number of labels for each category are shown. DataSet2 is from a self-built data set of an enterprise, mainly used for auxiliary verification. It contains 3200 urine sediment images with a size of 1024x1024, composed of 8 sediment categories: RBC, WBC, SQEP, CAOX, OCRY, BACI, FUNGI, and MUCS. The images in DataSet2 and the number of labels corresponding to each category are shown. Figure 8

[0076] mAP is the average of the APs of the model for multiple categories of cell detection. AP is the area under the curve of the accuracy Precision and recall:​

[0077]

[0078]

[0079] where r1, r2…r n are the Recall values corresponding to the first interpolation of the Precision interpolation segment in ascending order, and Precision and Recall are:

[0080]

[0081]

[0082] where TP represents True Positive, indicating a positive sample predicted as positive by the model, FP represents False Positive, indicating a negative sample predicted as positive by the model, and FN represents False Negative, indicating a positive sample predicted as negative by the model.

[0083] The higher the mAP value represents the better detection effect. The experimental environment of the present application is GNU / Linux operating system, and NVIDIA GEFORCE RTX 3080Ti graphics card is used for operation, the video memory is 12G, 128GB RAM, and the CPU processor is Intel(R) Xeon(R) Silver 4210 CPU@2.20GHz. All our models are based on Python 3.8 operating environment, Pytorch 1.12.0 deep learning framework, CUDA 11.4 and CUDNN 8.2.1 acceleration library.

[0084] 2.1 Qualitative comparison

[0085] In order to qualitatively compare the detection effect of our algorithm and other excellent baseline algorithms on the two urine formed element data sets, we carried out comparative experiments, and showed the effect images of our model for detecting the two data sets, and the detection results are as shown in Figures 9-12 Through comparison, it can be seen that our application can effectively and accurately detect most of the targets.

[0086] From Figures 9-12As can be seen from the urine formed elements image, the contrast and clarity of the image are low, the size of the target object is small compared to the picture, and the background information has a greater impact on the detection accuracy, which makes it more difficult to capture the characteristics between target objects. The characteristics of different cell components have small differences, increasing the difficulty of recognition and classification. Therefore, the sensitivity requirement of the target detection model is higher. Our model can effectively and accurately detect most of the targets. Although there are unavoidable misjudgments, it can provide pathological auxiliary analysis for the current clinical medical detection to some extent.

[0087] As shown in Table 1, the performance of the improved attention mechanism EECA and the current most advanced SE, CA, CBAM, ECA attention mechanism added to the YOLOv5s model in DataSet1 is compared, which illustrates the effectiveness of EECA. As shown in Tables 1 and 2, even compared with the current most advanced urine sediment detection model and target detection model, the performance of the algorithm we invented can be optimal, even far ahead of other models, which fully illustrates the effectiveness of our algorithm.

[0088] Table 1 Quantitative comparison with the most advanced attention mechanism on DataSet1

[0089]

[0090] Table 2 Quantitative comparison with the most advanced algorithm on DataSet1

[0091]

[0092] Table 3 Quantitative comparison with the most advanced algorithm on DataSet2

[0093]

[0094] 2.2 Quantitative comparison

[0095] In order to quantitatively compare the detection accuracy of the five methods (Faster R-CNN, SSD, YOLOv3, YOLOv5 and the algorithm of the application) and the models with different parameter scales, we carried out experiments on the USE public dataset and the enterprise dataset, and used mAP@.5 to quantitatively evaluate the detection results. Tables 1 and 2 show the quantitative evaluation results of the detection effect on the two datasets, and the bold data in each column is the optimal value. The data proves that the YOLOv5s-ES algorithm of the application has the best detection effect on urine formed elements and the least parameters. Compared with other models with larger parameter scale and larger calculation amount, the application still has superior performance. Therefore, in summary, the algorithm of the application has the best detection effect on urine formed elements.

[0096] The above is the preferred embodiment of the present application, any change made according to the technical solution of the present application, as long as the function generated does not exceed the scope of the technical solution of the present application, belongs to the protection scope of the present application.

Claims

1.A method for detecting formed elements in urine based on deep learning, characterized by, Firstly, based on the global average pooling feature aggregation channel of ECA, the global maximum pooling feature aggregation channel is added, and the EECA attention mechanism is proposed. The EECA attention mechanism is added to the backbone network and the neck network to enhance the model's ability to extract edge features. Secondly, based on YOLOv5, the VoV-GSCSP module is used instead of the C3 module in the neck network, and the GSConv module is used instead of the Conv module in the backbone network and the neck network, forming a Slim-Neck structure. The backbone network uses CSP-Darknet53, and the GSConv module is used instead of the Conv module. In the neck network, VoV-GSCSP and GSConv designed by applying one-shot aggregation method based on GSConv are used to replace C3 module and Conv module in the neck network, respectively, to form a Slim-Neck structure. The EECA attention mechanism is obtained as follows: ECA obtains features through global average pooling GAP, and generates channel weights through one-dimensional convolution of size k, where k is adaptively determined by the value of channel number C. The calculation formula of k is as follows: where |t| < 1 odd represents the odd number closest to t; γ and b are set to 2 and 1, respectively; obviously, through the mapping relationship ψ, the high-dimensional channel has a larger range of interaction, while the low-dimensional channel has a smaller range of interaction by using a nonlinear mapping; let the channel feature maps of different sizes obtain cross-channel information interaction suitable for the size of the feature map channel; On the basis of the global average pooling GAP feature aggregation channel of ECA, the global maximum pooling feature aggregation channel is added to retain more image edge texture information. Specifically, the input feature map of size HxWxC is obtained through global average pooling and global maximum pooling to obtain different aggregated features. Fast one-dimensional convolution of size k is performed to generate channel weights of size 1x1xC. After Sigmoid activation function, multiplication is performed, and then the original feature map is multiplied to obtain the output result. 2.The deep learning-based urine sediment detection method of claim 1, wherein, This method uses a loss function composed of three parts, as follows: (1) Object Score loss When detecting objects, the model uses a binary classification cross-entropy loss that can determine whether the target frame contains the object, where o is the label vector, t is the prediction vector, n is the total number of samples, and i is the sample label: (2) Class Probability Score loss When detecting classes, the model uses a binary classification cross-entropy loss that can determine whether the target frame contains the correct class, just like Object Score, where o is the label vector, t is the prediction vector, n is the total number of samples, and i is the sample label: (3) Bounding Box loss When detecting whether the predicted frame can compactly frame the target object, the model uses CIoU Loss: wherein, IoU is the ratio of the overlapping area between the predicted box and the real box to the total area contained by the predicted box and the real box; D2 is the distance between the center points of the predicted box and the target box, D c is the diagonal distance of the minimum circumscribed rectangle; w and h are the width and height of the predicted box, respectively, w gt and h gt are the width and height of the real box, respectively.

Images