A method and device for detecting urine formed components based on deep learning

Abstract

The present invention discloses a method and device for detecting urine formed components based on deep learning, using a single-stage target detection algorithm with faster detection speed and a network model with fewer parameters, and proposes a local context attention module and a global background suppression module to improve the detection effect in view of the global sparseness and local density of urine sediment images, so that the model focuses on the area containing the target, enhances the ability to distinguish targets in dense areas, and reduces attention to the background. The technical solution of the present invention can effectively detect urine formed components in urine sediment images, and greatly improves the detection of local dense distribution.

Description

Technical Field

[0001] The present application belongs to the field of medical image processing technology, and in particular, relates to a method and device for detecting urine formed components based on deep learning. Background Art

[0002] Object detection is one of the important research directions in the field of computer vision, and is widely used in the fields of natural images, remote sensing images, and medical images. In recent years, many scholars have carried out a lot of work in object detection and have made certain progress. General detection algorithms are designed for complex natural image detection. In order to cope with the diverse characteristics of the target and the complex background, the network models of these detection algorithms often have a large number of parameters. However, urine sediment images are different from natural images. Urine sediment images are obtained by photographing urine samples under a microscope, and the background of urine sample images contains almost no information. At the same time, the urine formed components in the sample are sparsely distributed, and there are occasional local aggregates of urine formed components, which means that the area of ​​urine formed components in the image accounts for a very small proportion of the image area. In addition, the types of urine formed components are much fewer than the types of targets in natural images. Natural images such as the VOC dataset have 20 types, the COCO dataset has 80 types, and ImageNet has hundreds of types, while urine formed components have only a few types. And the amount of urine formed component images available for training is also much smaller than that of natural image datasets. The general target detection algorithm network model is relatively large, and directly training the general target detection algorithm on the urine formed component dataset is prone to overfitting. In addition, the larger the network model, the more calculations are required, which will slow down the reasoning and cannot meet the requirements of real-time detection in practice. Summary of the invention

[0003] The purpose of this application is to provide a method and device for detecting urine formed components based on deep learning to overcome the technical problems raised in the above-mentioned background technology.

[0004] In order to achieve the above purpose, the technical solution of this application is as follows:

[0005] A method for detecting formed components of urine based on deep learning, constructing and training a deep learning model, wherein the deep learning model includes a multi-scale feature extraction module, a local context attention module, a global background suppression module, and an anchor frame-based detection module. The method for detecting formed components of urine based on deep learning includes:

[0006] The input urine sediment image is subjected to feature extraction by the multi-scale feature extraction module, and the last three layers of feature maps of different sizes are output by the multi-scale feature extraction module, which are arranged from small to large in size as the third layer feature map, the second layer feature map and the first layer feature map;

[0007] The extracted three-layer feature map is input into the local context attention module, and a multi-head self-attention mechanism operation with position coding information is added to the third-layer feature map, and the corresponding third-layer attention feature map is output. Then, the third-layer attention feature map is fused with the second-layer feature map, and after fusion, a multi-head self-attention mechanism operation with position coding information is added again, and the corresponding second-layer attention feature map is output. Then, the second-layer attention feature map is fused with the first-layer feature map, and the corresponding first-layer attention feature map is output.

[0008] The three-layer attention feature map output by the local context attention module is input into the global background suppression module, and the global background suppression process is performed on each layer of the attention feature map to obtain the corresponding three-layer feature map after global background suppression;

[0009] The feature map after three-layer global background suppression is input into the anchor frame-based detection module for detection, and the detection result is output.

[0010] Furthermore, the operation of adding a multi-head self-attention mechanism with position encoding information includes:

[0011] The input feature map is convolutionally encoded into three projections q, k, and v, and the relative position information R is encoded according to the width and height of the input feature map. h , R w ;

[0012] Perform matrix multiplication of q and k to obtain the contextual content information of the feature map, and R h , R w The relative position encoding of each point in the feature map is obtained by adding each element, and then matrix multiplication is performed with q to obtain the position information of each point;

[0013] The context content information is added to the position information of each point and processed with the softmax function to obtain the correlation degree information of each point, and then matrix multiplication is performed with v to obtain the attention feature map.

[0014] Furthermore, the global background suppression processing is performed on each layer of the attention feature map to obtain the corresponding feature map after global background suppression, including:

[0015] The current layer attention feature map to be processed is used as the target feature map, and the other two layers of attention feature maps are scaled to the target feature map. Figure 1 Sample size;

[0016] The three layers of feature maps after scaling are compressed into one channel using convolution respectively, and then the importance weight of each point in the feature map space of each layer is calculated by softmax;

[0017] Input the current layer attention feature map to be processed into the channel attention module to obtain the channel importance weight;

[0018] The scaled three-layer feature map is multiplied element-by-element by the importance weight of each point in the feature map space of the corresponding layer, and the elements are added element-by-element to obtain a feature map of the same size as the target feature map. Finally, it is multiplied by the channel importance weight to obtain the corresponding feature map after global background suppression.

[0019] Furthermore, the channel attention module performs the following operations:

[0020] A global average pooling operation is applied to the input feature map. The tensor obtained by the global average pooling operation is padded at both ends and then a 1D convolution with a step size of 1 is used.

[0021] Furthermore, the loss function of the deep learning model includes foreground and background loss, classification loss and frame regression loss, wherein the foreground and background loss and classification loss use a binary cross entropy loss function, and the frame regression loss uses an IoU loss.

[0022] The present application also proposes a urine formed component detection device based on deep learning, comprising a processor and a memory storing a plurality of computer instructions, wherein the computer instructions, when executed by the processor, implement the steps of the urine formed component detection method based on deep learning.

[0023] The present application proposes a method and device for detecting urine formed components based on deep learning, which uses a single-stage target detection algorithm with faster detection speed and a network model with fewer parameters. In view of the global sparseness and local density of urine sediment images, a local context attention module and a global background suppression module are proposed to improve the detection effect, so that the model focuses on the area containing the target, enhances the ability to distinguish targets in dense areas, and reduces attention to the background. The technical solution of the present application can effectively detect the formed components of urine in urine sediment images, and greatly improves the detection of local dense distribution. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 This is a flow chart of the urine formed component detection method based on deep learning in this application;

[0025] Figure 2 This is a schematic diagram of the deep learning model for this application;

[0026] Figure 3 Schematic diagram of the self-attention mechanism with relative position information;

[0027] Figure 4 It is a schematic diagram of the global background suppression module;

[0028] Figure 5 Network diagram for channel attention. DETAILED DESCRIPTION

[0029] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0030] In one embodiment, Figure 1 As shown, a method for detecting urine formed components based on deep learning is provided, comprising:

[0031] Step S1, input urine sediment image and extract features through multi-scale feature extraction module, output the last three layers of feature maps of different sizes of the multi-scale feature extraction module, which are arranged from small to large in size as the third layer feature map, the second layer feature map and the first layer feature map.

[0032] This application is a single-stage detection algorithm based on anchor boxes, such as Figure 2 As shown in the figure, the entire deep learning model consists of the following four parts: multi-scale feature extraction module, local context attention module, global background suppression module and anchor box based detection module.

[0033] Input urine sediment image, extract three layers of features of different scales from the multi-scale feature extraction module, which uses the single-stage target detection network YOLOv5. In the local context attention module, the overall structure follows the feature pyramid structure. Before feature fusion, a multi-head self-attention mechanism with position encoding information is added to the feature map. Each point in the feature map represents a region extracted from the image. Through this mechanism, the model focuses on the highly correlated area in the feature map. Then, in the global background suppression module, the importance weights of each point in the feature map are learned, so that the model pays more attention to the target area and less attention to the background. At the same time, a channel attention mechanism is added to learn the importance weights of different channels. The anchor-based detection module predicts the foreground and background information, position information, and classification information for each point in the processed feature map. Finally, for the foreground point, that is, the point containing the target, the center position and width and height of the target are calculated according to the width and height of the preset anchor box and the predicted position information, and visualized in the image together with the classification information with the highest confidence.

[0034] like Figure 2 As shown, input urine sediment image I 3×H×WFeatures are extracted by the multi-scale feature extraction module. The multi-scale feature extraction module uses the YOLOv5 network. Each layer of the module is composed of a different number of residual network blocks, which increases the depth of the network while avoiding the vanishing gradient. Different layers are separated by downsampling, which uses convolution instead of pooling to reduce the scale of the feature map tensor while increasing the number of channels. The feature map tensor processed by each layer has the same size. According to the size of the feature map tensor, the multi-scale feature extraction module is divided into 5 layers as a whole.

[0035] Output the feature maps of the last three layers of the multi-scale feature extraction module The width and height of each layer of feature maps are the width and height of the original image divided by the stride, and the sampling stride of the last three layers is stride∈{8,16,32}. According to the downsampling properties of the multi-scale feature extraction module, the width and height of the obtained three-layer feature map are half of the previous layer, and the number of channels is twice that of the previous layer.

[0036] Step S2, input the extracted three-layer feature map into the local context attention module, add a multi-head self-attention mechanism operation with position coding information to the third-layer feature map, output the corresponding third-layer attention feature map, and then fuse the third-layer attention feature map with the second-layer feature map. After fusion, add the multi-head self-attention mechanism operation with position coding information again, output the corresponding second-layer attention feature map, and then fuse the second-layer attention feature map with the first-layer feature map to output the corresponding first-layer attention feature map.

[0037] like Figure 2 As shown in the figure, the third-layer feature map F3 first adds a multi-head self-attention mechanism (MHSA) with position coding information to output the third-layer attention feature map F′3; the feature map F′3 is fused with the second-layer feature map F2, and after fusion, the multi-head self-attention mechanism with position coding information is added again to output the corresponding second-layer attention feature map F′2, and then the second-layer attention feature map F′2 is fused with the first-layer feature map F1, and finally the corresponding first-layer attention feature map F′1. When fusing two adjacent feature maps in pairs, the fusion process is to upsample the feature map with smaller width and height using nearest neighbor interpolation, and then stack it with the feature map of the target layer by channel, and then convolve to compress the number of channels to the same as the number of channels of the feature map of the target layer.

[0038] Before feature map fusion, in order to improve the model's ability to distinguish highly correlated regions in feature maps, use Figure 3The multi-head self-attention mechanism (MHSA) with relative position encoding information is shown, which enables the model to focus on highly correlated areas in the feature map. Since dense areas are often the same type of urine components, this operation can correspondingly enhance the resolution of local dense areas of the target.

[0039] In a specific embodiment, Figure 3 FIG. 1 is a simplified schematic diagram of a multi-head self-attention mechanism with position encoding information. The operation of adding a multi-head self-attention mechanism with position encoding information includes:

[0040] Step S2.1: Convolutionally encode the input feature map into three projections q, k, and v, and encode the relative position information R according to the width and height of the input feature map. h , R w .

[0041] The input feature map is encoded by 1×1 convolution into three projections of query, key, and value, denoted as q, k, and v, with the same size and number of channels as the input feature map. In addition, the relative position information R is encoded according to the width and height of the feature map. h ,R w .

[0042] Step S2.2: Perform matrix multiplication on q and k to obtain the contextual content information of the feature map, and convert R h , R w The relative position encoding of each point in the feature map is obtained by adding each element, and then matrix multiplication is performed with q to obtain the position information of each point.

[0043] The correlation of each point is calculated based on the information obtained. The level of correlation is calculated based on each position point. The context content information of the feature map is obtained by matrix multiplication based on q and k. h ,R w The relative position encoding of each point in the feature map is obtained by adding each element, and then matrix multiplication is performed with q to obtain the position information of each point.

[0044] Step S2.3, add the context content information to the position information of each point and process it with the softmax function to obtain the correlation degree information of each point, and perform matrix multiplication with v to obtain the attention feature map.

[0045] The context content information is added to the position information of each point and processed with the softmax function to obtain the correlation degree information of each point, and matrix multiplication is performed with v to obtain the self-attention weighted feature map (attention feature map):

[0046]

[0047] In this step, compared with the low-level feature map, the high-level feature has a larger receptive field, can include the entire target, and contains more semantic information, which is conducive to classification.

[0048] Step S3: input the three-layer attention feature map output by the local context attention module into the global background suppression module, perform global background suppression processing on each layer of the attention feature map, and obtain the corresponding feature map after global background suppression.

[0049] like Figure 2 ,The global background suppression module is placed after the global context attention module and is used to process the ,feature maps of each layer output by the global context attention module.

[0050] In a specific embodiment, Figure 4 As shown, the global background suppression module in this step performs global background suppression processing on each layer of attention feature map, including:

[0051] Step S3.1: Use the current layer attention feature map to be processed as the target feature map, and scale the other two layers of attention feature maps to the target feature map. Figure 1 Sample size.

[0052] The sizes and number of channels of the feature maps in the three branches are different, so they need to be processed separately. The width, height and number of channels of the other two layers of feature maps need to be scaled to the target features through convolution. Figure 1 Sample size.

[0053] If the side length ratio between the feature map and the target feature map is 1 / 2, one enlargement operation is performed, and if it is 1 / 4, two enlargement operations are performed. For the enlargement operation, the channel is first compressed by 1×1 convolution with a step size of 2, and then the feature map is enlarged using a bilinear interpolation scaling factor of 2. If the side length ratio between the feature map and the target feature map is 2, one reduction operation is performed, and if it is 4, two reduction operations are performed. For the reduction operation, 3×3 convolution with a step size of 2 is used for downsampling, reducing the size and increasing the number of channels.

[0054] Figure 4 The figure shows a schematic diagram of the feature map F2 branch in the global background suppression module, where F1 and F3 are enlarged and reduced respectively.

[0055] Step S3.2: The three layers of feature maps after scaling are compressed to one channel using convolution respectively, and then the importance weight of each point in the feature map space of each layer is calculated by softmax.

[0056] The width, height and number of channels of the three-layer feature map after scaling are the same as those of the target feature map. These three layers of feature maps are compressed to one channel using convolution. Two convolutions are performed to avoid information loss caused by reducing the number of channels too much at one time. First, a 3×3 convolution with a step size of 1 is used to obtain the feature map information after the compressed channel, and then a 3×3 convolution with a step size of 1 is used. Finally, a feature map with three layers and one channel is obtained. Where H and W are the width and height of the target feature map.

[0057] Then the three-layer tensor is calculated by softmax and the importance weight of each layer is recorded as α t ,β t ,γ t .

[0058]

[0059] After Softmax processing, the sum of the weights of the three layers of feature maps at each point is 1, and t represents the number of layers of the current attention feature map.

[0060] The relationship is as follows:

[0061]

[0062] Through this step, the importance weight of each point in the feature map space is obtained.

[0063] Step S3.3: Input the current layer attention feature map to be processed into the channel attention module to obtain the channel importance weight.

[0064] This step first copies the attention feature map of the current layer to be processed to avoid changing the original feature map during processing. The feature map is used to calculate the channel importance weight and the importance weight of each point in the feature map space of each layer.

[0065] After step S3.2, the importance weight of each point in the feature map space is obtained. In order to obtain the channel importance weight, channel attention is added, such as Figure 5 Shown is Figure 4 Detailed diagram of channel attention in

[15] , where a channel importance weight is learned based on the target layer feature map.

[0066] In a specific embodiment, the channel attention module performs the following operations:

[0067] A global average pooling operation is applied to the input feature map. The tensor obtained by the global average pooling operation is padded at both ends and then a 1D convolution with a step size of 1 is used.

[0068] That is, first use global average pooling on the H×W×C feature map to obtain a tensor of size 1×1×C, fill its ends with 0 so that it can maintain the same size as the original after convolution, and then use a one-dimensional convolution with a step size of 1 to obtain the channel importance weight CA (ChannelAttention), where the number of filled 0s is related to the convolution kernel length k of the one-dimensional convolution. Since the step size is 1, the number of filled 0s at both ends here can be recorded as (k-1) / 2.

[0069] Step S3.4, multiply the three layers of feature maps after scaling by the importance weights of each point in the feature map space of the corresponding layer element by element, and add them element by element to obtain a feature map of the same size as the target feature map, and finally multiply them by the channel importance weight to obtain the corresponding feature map after global background suppression.

[0070] like Figure 4 As shown in the right part, the three-layer weights obtained in step S3.2 are multiplied element-by-element with the scaled feature maps of each layer, and the elements are added element-by-element to obtain a feature map of the same size as the target feature map, and finally multiplied by the channel importance weight in step S3.3 to obtain the processed feature map.

[0071] It should be noted that the parameters in the network convolution in this application are learnable, and the parameters in the convolution kernel will be updated during the back propagation process during training. Since the background of the urine sediment image contains almost no information, the importance weight of the background is very small after learning, which will greatly suppress the background area and make the network model focus on the area with information, that is, the area with the target, thereby improving the performance of detecting the target.

[0072] Step S4: input the feature map after the three-layer global background suppression into the anchor frame-based detection module for detection, and output the detection result.

[0073] Specifically, the steps include:

[0074] Step S4.1: According to the weighted feature map obtained in the previous step, each point in the feature map is predicted by convolution to obtain the foreground and background confidence information, the center offset coordinates cx, cy of the urine formed components, the width and height scaling ratios h, w, and the classification information, a total of C confidences including each class, that is, Figure 2 The rightmost image is H×W×(5+C). The foreground is the target and the background is the background. The center coordinates are the position offset relative to the upper left corner of the grid point. The coordinates of the upper left corner of the grid point are calculated based on the position of the grid point. The number of grids in the horizontal and vertical directions is obtained by dividing the width and height of the image by the step length {8,16,32} of the sampling layer, starting from 0. The original image is pre-processed and filled to ensure that it is divisible. The side length of each grid is equal to the step length, so the coordinates of the upper left corner of the grid point are the number of horizontal grids × step length, and the number of vertical grids × step length, that is, (gridx ×stride,grid y × stride). After adding the offset, the coordinates of the center point relative to the upper left corner of the image are (grid x ×stride+2δ(cx)-0.5,grid y ×stride+2δ(cy)-0.5), where δ is the Sigmoid function, normalizing cx, cy to (0, 1), and the range of 2δ(cy)-0.5 is (-0.5, 1.5) so that the center of the prediction box can be offset to the adjacent grid point. The predicted width and height scaling ratio is normalized to (δ(h)+0.5, δ(w)+0.5), with a range of (0.5, 1.5), which can achieve a certain degree of scaling, and then multiplied by the width and height of the preset anchor box to obtain the width and height of the predicted anchor box. The width and height of the preset anchor box are preset relative to the image size of the input network. Three ratios of anchor boxes are preset for the feature maps of the three scales, for a total of 9 preset anchor boxes to detect urine formed components with different aspect ratios at multiple scales. Based on the above, the coordinates of the center point relative to the upper left corner of the grid are converted into the coordinates of the center point relative to the upper left corner of the image. The predicted width and height scaling ratios are applied to the corresponding scale anchor frame to obtain the width and height of the predicted target frame, and a total of 3 predicted frames are obtained at each point.

[0075] S4.2 predicts a large number of prediction boxes for each point according to 4.1, but there are not so many targets in the actual picture, and most of the prediction boxes are of low quality, that is, they do not surround the target well. The prediction results need to be post-processed to filter out targets with confidence exceeding the threshold. During training, this confidence is calculated based on the intersection of union (IoU) between the predicted box and the target's real annotated box. Finally, for the boxes exceeding the threshold, non-maximum suppression (NMS) is used to filter out the boxes with high IoU and relatively low confidence in the same category. For these prediction boxes with too high IoU, the predicted targets are likely to be the same, so they must be filtered out to leave the highest quality boxes. First, the coordinates of the upper left corner and the upper right corner of the prediction box are obtained, which are converted according to the center point coordinates and width and height calculated previously. The coordinates of the two corner points of the two prediction boxes are recorded as (x1, y1, x2, y2) and (x′1, y′1, x′2, y′2), respectively. The IoU calculation can be expressed by the following formula:

[0076] Intersection=[min(x2,x′2)-max(x1,x′1)]·[min(y2,y′2)-max(y1,y′1)]

[0077] Union=(x2-x1)·(y2-y1)+(x′2-x′1)·(y′2-y′1)-Intersection

[0078]

[0079] It is easy to understand that the entire deep learning model of the present application includes a multi-scale feature extraction module, a local context attention module, a global background suppression module and an anchor frame-based detection module, wherein the multi-scale feature extraction module and the anchor frame-based detection module are both target detection networks YOLOv5, which are relatively mature technologies and will not be repeated here.

[0080] The entire deep learning model needs to be trained before it can be actually applied. The training process is as follows:

[0081] S5.1. Preprocess the training data set. Each batch has 16 data. The data augmentation method is to select 4 images from a batch, randomly scale them and combine them to obtain new images, and then recalculate the labels and combine them.

[0082] S5.2, after the processing of steps S1-S4, the foreground and background loss L Obj and classification loss L Cls Using the binary cross entropy loss function, the bounding box regression L Box Use IoU loss.

[0083] Where L Obj The definition is as follows:

[0084]

[0085] in Indicates whether it is a target value, which is 0 or 1, and o is the probability range of predicting whether it is a target, which is (0,1).

[0086] Where L Cls The definition is as follows:

[0087]

[0088] in is the true category, c is the predicted category, and N is the total number of categories.

[0089] L Box The IOU loss function is defined as follows:

[0090] L Box =1-IoU

[0091] Here, IoU is the intersection over union (IoU) of the predicted box and the true box. The higher the IoU, the higher the box quality and the smaller the loss.

[0092] The final loss and importance are multiplied by different coefficients:

[0093] L=λ Obj LObj +λ Cls L Cls +λ Box L Box

[0094] where λ Obj =1,λ Cls =0.5,λ Box =0.05.

[0095] That is, the loss function of the deep learning model of the present application includes foreground and background loss, classification loss and frame regression loss, among which the foreground and background loss and classification loss use the binary cross entropy loss function, and the frame regression loss uses the IoU loss.

[0096] S5.4. During the training process, the batch size is 16, and the total number of batches is the number of images divided by the batch size. The training of all batches in the dataset is recorded as one epoch, and a total of 50 epochs are trained. The learning rate decreases from 0.1 to 0.002 with the epoch. The Stochastic Gradient Descent (SGD) algorithm is used to iterate the training for 50 epochs and save the results of each iteration.

[0097] In the actual application of this application, the image is input into the multi-scale feature extraction module for feature extraction, and feature maps of three scales of different depths are obtained. Before detection, the feature map is processed by the local context attention module and the global background suppression module. In the local context attention module, the feature pyramid network is used to fuse the features of two adjacent feature maps, and the high-level semantic information is passed to the low level. Before feature fusion, the self-attention mechanism with relative position information is used to focus on the highly correlated areas in the feature map, so as to improve the model's ability to distinguish dense areas in the picture. In the global background suppression module, the weight information of each position in each feature map is learned based on the information of the feature maps of three scales. And there is a channel attention mechanism to assign weights to the weight information of each channel to enhance the expressiveness of different channel features. During detection, the model predicts whether each position of the feature map is a target, the coordinates of the center point, the width and height scaling ratio, and the classification information. Finally, the model will output the information of the target, the border information of the target, and the classification information.

[0098] During the detection process, the image is input into the model, and after feature extraction and processing, each area in the image is predicted to obtain foreground and background information, classification information and regression information. The boxes with low confidence are screened out, and the rectangular box information surrounding the target in the original image is calculated based on the predicted information. Then, the repeated boxes predicting the same target are screened out, and finally the remaining high-quality rectangular boxes and classification information are visualized in the image.

[0099] The traditional method of analyzing and detecting urine sediment microscopic images is to manually analyze and count the patient's samples under a microscope by professional inspectors with the naked eye. This manual method is labor-intensive and requires a high level of professionalism from technicians. Computer-assisted detection can effectively reduce the workload and obtain the number of formed components in the prepared urine, while reducing the professional level requirements for technicians. Under the model detection trained by this algorithm, urine sediment images can be automatically and quickly detected and accurately identified, realizing the automation and intelligence of the medical microscopic image processing system.

[0100] The model can be deployed on the server, and the client uploads the image to the server for detection, and the detection results are returned. The model uses a multi-scale feature extraction module with the least parameters, with a simple structure and high reusability. It can also be deployed on the chip for offline detection.

[0101] In one embodiment, the present application also provides a urine formed component detection device based on deep learning, comprising a processor and a memory storing a plurality of computer instructions, wherein the computer instructions, when executed by the processor, implement the steps of the urine formed component detection method based on deep learning.

[0102] For the specific limitations of the urine formed component detection device based on deep learning, please refer to the limitations of the urine formed component detection method based on deep learning above, which will not be repeated here. The above-mentioned urine formed component detection device based on deep learning can be implemented in whole or in part through software, hardware and a combination thereof. It can be embedded in or independent of the processor in the computer device in hardware form, or it can be stored in the memory of the computer device in software form, so that the processor can call and execute the operations corresponding to the above modules.

[0103] The memory and the processor are electrically connected directly or indirectly to achieve data transmission or interaction. For example, these elements can be electrically connected to each other through one or more communication buses or signal lines. The memory stores a computer program that can be run on the processor, and the processor implements the network topology layout method in the embodiment of the present invention by running the computer program stored in the memory.

[0104] The memory may be, but is not limited to, a random access memory (RAM), a read only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable read-only memory (EEPROM), etc. The memory is used to store a program, and the processor executes the program after receiving an execution instruction.

[0105] The processor may be an integrated circuit chip with data processing capabilities. The above-mentioned processor may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc. The methods, steps and logic block diagrams disclosed in the embodiments of the present invention may be implemented or executed. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0106] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.

Claims

1. A method for detecting urine formed components based on deep learning, characterized in that: A deep learning model is constructed and trained, wherein the deep learning model includes a multi-scale feature extraction module, a local context attention module, a global background suppression module, and an anchor frame-based detection module. The deep learning-based urine formed component detection method includes: The input urine sediment image is subjected to feature extraction by the multi-scale feature extraction module, and the last three layers of feature maps of different sizes are output by the multi-scale feature extraction module, which are arranged from small to large in size as the third layer feature map, the second layer feature map and the first layer feature map; The extracted three-layer feature map is input into the local context attention module, and a multi-head self-attention mechanism operation with position coding information is added to the third-layer feature map, and the corresponding third-layer attention feature map is output. Then, the third-layer attention feature map is fused with the second-layer feature map, and after fusion, a multi-head self-attention mechanism operation with position coding information is added again, and the corresponding second-layer attention feature map is output. Then, the second-layer attention feature map is fused with the first-layer feature map, and the corresponding first-layer attention feature map is output. The three-layer attention feature map output by the local context attention module is input into the global background suppression module, and the global background suppression process is performed on each layer of the attention feature map to obtain the corresponding three-layer feature map after global background suppression; The feature map after the three-layer global background suppression is input into the anchor frame-based detection module for detection, and the detection result is output; The operation of adding a multi-head self-attention mechanism with position encoding information includes: The input feature map is convolutionally encoded into three projections q, k, and v, and the relative position information R is encoded according to the width and height of the input feature map. h , R w ; Perform matrix multiplication of q and k to obtain the contextual content information of the feature map, and R h , R w The relative position encoding of each point in the feature map is obtained by adding each element, and then matrix multiplication is performed with q to obtain the position information of each point; Add the context information to the position information of each point and process it with the softmax function to get the relevance information of each point, and perform matrix multiplication with v to get the attention feature map; The global background suppression process is performed on each layer of the attention feature map to obtain the corresponding feature map after global background suppression, including: The current layer attention feature map to be processed is used as the target feature map, and the other two layers of attention feature maps are scaled to the same size as the target feature map; The three layers of feature maps after scaling are compressed into one channel using convolution respectively, and then the importance weight of each point in the feature map space of each layer is calculated by softmax; Input the current layer attention feature map to be processed into the channel attention module to obtain the channel importance weight; The scaled three-layer feature map is multiplied element-by-element by the importance weight of each point in the feature map space of the corresponding layer, and the elements are added element-by-element to obtain a feature map of the same size as the target feature map. Finally, it is multiplied by the channel importance weight to obtain the corresponding feature map after global background suppression.

2. The method for detecting urine formed components based on deep learning according to claim 1, characterized in that: The channel attention module performs the following operations: A global average pooling operation is applied to the input feature map. The tensor obtained by the global average pooling operation is padded at both ends and then a 1D convolution with a step size of 1 is used.

3. The method for detecting urine formed components based on deep learning according to claim 1, characterized in that: The loss function of the deep learning model includes foreground and background loss, classification loss and frame regression loss, wherein the foreground and background loss and the classification loss use a binary cross entropy loss function, and the frame regression loss uses an IoU loss.

4. A urine formed component detection device based on deep learning, comprising a processor and a memory storing a plurality of computer instructions, characterized in that: When the computer instructions are executed by a processor, the steps of the method described in any one of claims 1 to 3 are implemented.

Images