A small-scale target detection method based on cross-domain few-shot learning
By employing a cross-domain few-shot learning method, combined with multi-scale feature extraction, background suppression, and feature space fine-tuning, the problem of insufficient accuracy in small-scale target detection is solved, enabling automated and intelligent detection of medical microscopic images.
Patent Information
- Application Number
- CN202211504235.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-11-28
- Publication Date
- 2025-12-02
- Estimated Expiration
- 2042-11-28
AI Technical Summary
Existing deep learning-based object detection algorithms suffer from insufficient accuracy in small-sample and small-scale object detection, especially in the field of medical imaging where data acquisition is difficult and there is a lack of sufficient labeled data.
We employ a cross-domain few-shot learning approach, which combines multi-scale feature extraction, background suppression, feature space fine-tuning, and multi-scale positive sample-assisted learning with the ResNet-101 network, utilizing stacked weights and self-attention mechanisms to improve detection accuracy.
It improves the accuracy of small-scale target detection, reduces the influence of image style and complex background, enhances the weight of target regions, improves the network's ability to recognize targets of different sizes, and realizes automated and intelligent detection of medical microscopic images.
Smart Images
Figure CN115797650B_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of small target detection technology, specifically relating to a small-scale target detection method based on cross-domain few-sample learning. Background Technology
[0002] Deep learning-based convolutional neural network (CNN) object detection algorithms have achieved great success in recent years. However, training general-purpose object detection algorithms to achieve excellent detection performance requires a large amount of labeled data, and obtaining this training data is extremely time-consuming and labor-intensive. Although these object detection algorithms perform well enough and are widely used in practice, they are still far inferior to the human visual system. The human visual system can quickly recognize a new object by establishing connections with existing knowledge. This is especially true in the field of medical imaging, where data acquisition is difficult and there is insufficient data available for training. Therefore, object detection algorithms based on small samples, mimicking the human process of learning new objects, have gradually attracted the attention of researchers. Summary of the Invention
[0003] The purpose of this invention is to provide a small-scale target detection method based on cross-domain few-sample learning, thereby improving the accuracy of small-scale target detection.
[0004] To achieve the above objectives, the technical solution adopted by the present invention is as follows:
[0005] A small-scale target detection method based on cross-domain few-shot learning, applied to the detection of formed elements in liquids, includes the following steps in the application phase:
[0006] Step 1, Multi-scale feature extraction: Input the image to be processed into the backbone network and output the feature maps of the last three layers of the backbone network;
[0007] Step 2: Background suppression;
[0008] Step 2.1: Scale the three feature maps to the size of the target features. Figure 1 To;
[0009] Step 2.2: Based on the scaled three-layer feature map, calculate the superposition weights at each pixel.
[0010] Step 2.3: Weight the three scaled feature maps according to the superposition weights to obtain the target feature map;
[0011] Step 2.4: Perform the first-stage prediction of the target feature map into foreground, background, and target bounding box using a fully connected layer;
[0012] Step 3: Fine-tuning the feature space;
[0013] Step 3.1: Expand the feature vectors corresponding to the target boxes in Step 2.4 by channel dimension, and use a fully connected layer to map the features on different channels to different vector spaces. Use dot product to measure the correlation between vectors and perform information fusion on the channel dimension to obtain a fused feature map.
[0014] Step 3.2: Based on the residual operation, add the feature vector corresponding to the target box in Step 2.4 to the feature vector of the fused feature map to obtain the fine-tuned target feature map;
[0015] Step 3.3: Predict the target category and location by fine-tuning the target feature map through a fully connected layer, i.e., perform the second-stage prediction, and output the final target category and location.
[0016] Several alternative methods are provided below, but they are not intended as additional limitations on the overall solution above. They are merely further additions or optimizations. Provided there are no technical or logical contradictions, each alternative method can be combined individually with respect to the overall solution above, or multiple alternative methods can be combined with each other.
[0017] Preferably, the backbone network is ResNet-101;
[0018] Output the feature maps F1, F2, and F3 of the last three layers of the backbone network. The width and height of each feature map are the width and height of the original input image to be processed divided by the sampling stride. The sampling stride of the last three layers is stride∈{8,16,32}.
[0019] Preferably, the calculation of the superposition weights at each pixel based on the scaled three-layer feature map includes:
[0020] The scaled three-layer feature maps are processed using 1×1 convolutions to obtain the control parameters for the coordinates of each layer. t is the identifier of the target feature map, 1≤i≤H, 1≤j≤W, H is the height of the target feature map, and W is the width of the target feature map;
[0021] Calculate the stacking weights at each pixel in each layer using control parameters:
[0022]
[0023]
[0024]
[0025] In the formula, This represents the superposition weight of pixel (i,j) on the first layer feature map after scaling. This represents the superposition weight of pixel (i,j) on the scaled second-layer feature map. This represents the superposition weight of pixel (i,j) on the scaled third-layer feature map. This represents the control parameters corresponding to pixel (i,j) on the first layer feature map after scaling. This represents the control parameters corresponding to pixel (i,j) on the scaled second-layer feature map. This represents the control parameters corresponding to pixel (i,j) on the third-layer feature map after scaling.
[0026] Preferably, the step of weighted summing of the scaled three-layer feature maps according to the superposition weights to obtain the target feature map includes:
[0027] The scaled feature maps of the three layers are as follows:
[0028] The target feature map F is obtained by weighted summation of pixels. t :
[0029]
[0030] In the formula, For the target feature map F t The pixel value of the upper pixel (i,j). The first layer feature map after scaling. The pixel value of the upper pixel (i,j). The second layer feature map after scaling. The pixel value of the upper pixel (i,j). This is the scaled-down third-layer feature map. The pixel value of the upper pixel (i,j).
[0031] Preferably, the first-stage prediction of the target feature map as foreground, background, and target bounding box through a fully connected layer includes:
[0032] For each pixel in the target feature map, anchor boxes with different scaling ratios and aspect ratios are preset. For each anchor box, foreground and background discrimination, offset prediction, scaling ratio prediction, and aspect ratio prediction are performed. The foreground represents the target, the background represents the background, and the offset represents the distance from the actual target center to the center of the preset anchor box.
[0033] The prediction results are post-processed to filter out targets that exceed the confidence threshold, and the non-maximum suppression method is used to filter anchor boxes to obtain the final anchor boxes as target boxes.
[0034] Preferably, the step of using dot product to measure the correlation between vectors and performing information fusion along the channel dimension to obtain a fused feature map includes:
[0035] The dot product is used to measure the correlation between vectors, and an attention map is generated.
[0036] Based on the attention map score, the feature vectors are weighted and information is fused along the channel dimension to obtain a fused feature map.
[0037] Preferably, the small-scale target detection method based on cross-domain few-shot learning further includes the following steps during the network training phase:
[0038] Step 4: Multi-scale positive sample-assisted learning;
[0039] Step 4.1: Extract the target region based on the label of the labeled sample image, scale the extracted image into three samples of different sizes, namely 1x, 2x, and 4x, to obtain the first sample image, the second sample image, and the third sample image. Input each sample image into the backbone network in Step 1.
[0040] Step 4.2: Take the feature map output by the third layer of the backbone network when the first sample image is input, the feature map output by the fourth layer of the backbone network when the second sample image is input, and the feature map output by the fifth layer of the backbone network when the third sample image is input. Input the three feature maps into the fully connected layer in step 2.4 for the first stage prediction.
[0041] Step 4.3: Take the feature map output by the first layer of the backbone network when the first sample image is input, the feature map output by the second layer of the backbone network when the second sample image is input, and the feature map output by the third layer of the backbone network when the third sample image is input. Input the three feature maps into the fully connected layer in step 3.3 for the second stage prediction.
[0042] Step 5: Calculate the first-stage prediction loss and the second-stage prediction loss according to steps 1 to 4 respectively. Update the network parameters of the fully connected layer for the first-stage prediction based on the first-stage prediction loss, and update the network parameters of the fully connected layer for the second-stage prediction based on the second-stage prediction loss, until the preset training conditions are met and the process ends.
[0043] Preferably, the calculation of the first-stage prediction loss and the second-stage prediction loss according to steps 1 to 4 includes:
[0044] The first-stage prediction loss includes two branches: the loss for foreground and background prediction, and the loss for target box prediction. The loss for foreground and background prediction is measured using the binary cross-entropy loss function, denoted as L. Bcls The loss for bounding box prediction is measured using a smoothed L1 function, denoted as L. Preg If the average is taken for each objective, then the first-stage prediction loss L RPN The definition is as follows:
[0045]
[0046] In the formula, Nobj M is the number of samples in the fully connected layer input during step 2.4 of the network training phase. obj Input the number of samples for the fully connected layer in step 4.2 of the network training phase. This represents the loss for predicting the foreground and background of the k-th sample. This represents the loss for predicting the bounding box of the k-th sample;
[0047] The second-stage prediction loss uses the cross-entropy loss function to measure the accuracy of target classification in both branches, denoted as L. Kcls The accuracy of the predicted bounding box is measured using the IoU loss, denoted as . Second stage prediction loss L RoI Defined as:
[0048]
[0049] In the formula, N RoI M is the number of samples in the fully connected layer input during step 3.3 of the network training phase. RoI Input the number of samples for the fully connected layer in step 4.3 of the network training phase. This represents the cross-entropy loss of the k-th sample. Let λ represent the IoU loss of the k-th sample, where λ is a preset fixed value.
[0050] This invention provides a small-scale target detection method based on cross-domain few-shot learning. It uses background information suppression to reduce the influence of image style and complex backgrounds, lowering the weight of non-target information regions while enhancing the weight of target regions. During feature space fine-tuning, a self-attention mechanism is used in the channel dimension to make the feature representations of the same target more aggregated and the differences between different targets more significant. Furthermore, a multi-scale positive sample-assisted learning branch is added to strengthen the network's ability to recognize targets of different sizes. Attached Figure Description
[0051] Figure 1 This is a flowchart of the small-scale target detection method based on cross-domain few-shot learning in the application stage of the present invention;
[0052] Figure 2 This is a flowchart of the background suppression part of the present invention;
[0053] Figure 3 This is a flowchart of the feature space fine-tuning part of the present invention;
[0054] Figure 4 This is a flowchart of the small-scale target detection method based on cross-domain few-shot learning in the network training phase of this invention;
[0055] Figure 5 This is a schematic diagram of the cross-domain few-sample joint training of the present invention;
[0056] Figure 6 This is a framework diagram of the small-scale target detection method based on cross-domain few-shot learning in the network training stage of this invention. Detailed Implementation
[0057] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0058] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used herein in the description of the invention is for the purpose of describing particular embodiments only and is not intended to limit the invention.
[0059] To overcome the shortcomings of existing technologies for small sample detection, this embodiment provides a small-scale target detection method based on cross-domain small sample learning. This method has a network training stage and an application stage. First, the prediction head is adjusted in the network training stage, and then it is applied in practice in the application stage.
[0060] like Figure 1 As shown, in the application stage, this embodiment takes the detection of formed elements in urine as an example for illustration, and specifically includes the following steps:
[0061] S1. Multi-scale feature extraction.
[0062] S1.1 Input a urine sediment image and extract features using a backbone network. The backbone network consists of a series of residual network blocks of varying numbers. Each residual network block successively performs downsampling operations on the urine sediment image, thereby outputting feature maps of different resolutions. The backbone network uses convolutional computation for downsampling, and each downsampled image halves its width and height while doubling the number of channels. Therefore, as the network deepens, the tensor sizes H and W gradually decrease, while the number of channels C increases accordingly. In this embodiment, the backbone network used is ResNet-101. In other embodiments, the backbone network can be other choices, such as ResNet-50, ResNet-34, DenseNet, etc.
[0063] S1.2 Output the feature maps of the last three layers of the backbone network. The width and height of each feature map layer are the original image width and height divided by the sampling stride. The sampling stride of the last three layers is stride∈{8,16,32}.
[0064] S2, background suppression, such as Figure 2 As shown.
[0065] S2.1 Scale the three feature maps to the size of the target features. Figure 1 To.
[0066] Background suppression integrates the feature maps output from step 1 and assigns weights to each pixel, which are learned by the network. First, three layers of feature maps are extracted and copied, scaling feature maps of different sizes to the same size. For small feature maps, a magnification operation is performed: first, 1×1 convolutions are used to compress the channels, then bilinear interpolation is used to magnify the feature map. For large feature maps, a downsampling operation is performed: 3×3 convolutions with a stride of 2 are used for downsampling, reducing the size and increasing the number of channels.
[0067] The scaled feature map is denoted as... t is the identifier of the target feature map, used for ease of description. This is the first layer feature map. For the second layer feature map, This is the feature map for the third layer. It should be noted that the first, second, and third layers here are used to distinguish the order of the layers defined in the description, and do not represent the number of layers in the network. According to the correspondence, the first layer here corresponds to the third-to-last layer of the backbone network, the second layer here corresponds to the second-to-last layer of the backbone network, and the third layer here corresponds to the last layer of the backbone network.
[0068] S2.2 Calculate the superposition weights on each pixel based on the scaled three-layer feature map.
[0069] The three-layer feature map after scaling has the same width, height, and number of channels as the target feature map. A 1×1 convolution is then applied to these three feature maps to obtain the control parameters for the corresponding coordinates of each layer. 1≤i≤H, 1≤j≤W, where H is the height of the target feature map and W is the width of the target feature map.
[0070] Calculate the stacking weights at each pixel in each layer using control parameters:
[0071]
[0072] In the formula, This represents the first layer feature map after scaling. The superposition weight of the pixel (i,j) This represents the scaled second-layer feature map. The superposition weight of the pixel (i,j) This represents the scaled feature map of the third layer. The superposition weight of the pixel (i,j) This represents the first layer feature map after scaling. The control parameters corresponding to the upper pixel (i,j) This represents the scaled second-layer feature map. The control parameters corresponding to the upper pixel (i,j) This represents the scaled feature map of the third layer. The control parameters corresponding to the upper pixel (i,j).
[0073] The weights of the three feature maps at each pixel sum to 1, as shown below:
[0074]
[0075] S2.3. The scaled three-layer feature maps are weighted and summed according to the superposition weights to obtain the target feature map.
[0076] In this embodiment, the three-layer feature maps after scaling are weighted and summed according to the importance weight of each layer to obtain the target feature map F. t Let the value of the corresponding coordinate be... but:
[0077]
[0078] In the formula, For the target feature map F t The pixel value of the upper pixel (i,j). The first layer feature map after scaling. The pixel value of the upper pixel (i,j). The second layer feature map after scaling. The pixel value of the upper pixel (i,j). This is the scaled-down third-layer feature map. The pixel value of the upper pixel (i,j).
[0079] S2.4. The first stage of prediction is performed on the target feature map as foreground, background and target box through a fully connected layer.
[0080] In this embodiment, after performing the operations corresponding to steps 2.1 to 2.3 on the last three layers output from step 1, the obtained fusion features are integrated and connected: F t =Concat(F1,F2,F3)#(4) to obtain the target feature map, and perform the first stage prediction based on the target feature map. The specific prediction process of the first stage in this embodiment is as follows:
[0081] S2.4.1. For each pixel in the target feature map, preset anchor boxes with different scaling ratios and aspect ratios. For each anchor box, perform foreground / background discrimination, offset prediction, scaling ratio prediction, and aspect ratio prediction. Here, the foreground represents the target, and the background represents the background; the offset represents the distance from the actual target center to the center of the preset anchor box.
[0082] S2.4.2 Post-process the prediction results, filter out targets with confidence scores exceeding the threshold, and use non-maximum suppression (NMS) to remove boxes with excessively high Intersection of Union (IoU) and low scores in the same category. These boxes are likely to point to the same target. Keep the best one to obtain the final anchor box as the target box.
[0083] S3, Feature space fine-tuning, such as Figure 3 As shown.
[0084] S3.1 Expand the feature vectors corresponding to the target boxes in step 2.4 according to the channel dimension, and use a fully connected layer to map the features on different channels to different vector spaces. Use dot product to measure the correlation between vectors and perform information fusion on the channel dimension to obtain a fused feature map.
[0085] Feature space fine-tuning aims to enable the model to quickly adapt to uncommon application scenarios and notice the necessary features. First, the feature vectors corresponding to the predicted locations (i.e., the bounding boxes) in step 2 are expanded along the channel dimension, and the ROI features in each channel are denoted as... Where C i Let the i-th channel be represented by a fully connected layer that maps the features to different vector spaces, denoted as Q, K, V:
[0086]
[0087] In the formula, Q, K, and V correspond to Query, key, and value, respectively. They represent the attention mechanism in deep learning, which originally intended to establish key-value pairs. The similarity between the query and the key was calculated as the weight, and the value was weighted to obtain the output. FC() represents a fully connected layer.
[0088] The dot product is used to measure the correlation between vectors, generating an attention map M. s :
[0089] M S =QK T #(6)
[0090] In the formula, K T It is the transpose of K.
[0091] Based on the attention map scores, feature vectors are weighted to achieve information fusion along the channel dimension. Vectors with higher scores store more similar information and are more likely to represent the same category or target. This operation makes intra-class representations more aggregated and inter-class differences more obvious. Let E be the fused feature map, then:
[0092] E = Softmax(M S )×V#(7)
[0093] In the formula, Softmax() is the activation function.
[0094] S3.2. Based on the residual operation, add the feature vector corresponding to the target box in step 2.4 to the feature vector of the fused feature map to obtain the fine-tuned target feature map, which is the adjusted ROI feature.
[0095] In this embodiment, to ensure the model's ability to detect known categories, a residual operation is used to add the results before fine-tuning to the features after fine-tuning:
[0096]
[0097] In the formula, To fine-tune the features in the i-th channel of the target feature map, To fuse the features on the i-th channel of the feature map.
[0098] Step 3.3: Predict the category of the target through the fully connected layer and readjust the position of the target box, which is the second stage of prediction, to achieve more refined detection results.
[0099] In this embodiment, the small-scale target detection method based on cross-domain few-sample learning inputs an image into the model, extracts and processes features, and then predicts each region in the image to obtain foreground and background information, classification information, and regression information. Boxes with low confidence are filtered out, and the bounding boxes surrounding the target in the original image are calculated based on the predicted information. Then, boxes that predict the same target repeatedly are filtered out. Finally, the remaining high-quality bounding boxes and classification information are visualized in the image.
[0100] like Figure 4 As shown, this embodiment uses urine formed element detection as an example to illustrate the network training phase, specifically including the following steps:
[0101] S1. Multi-scale feature extraction: Input labeled sample images into the backbone network and output feature maps of the last three layers of the backbone network.
[0102] S2, Background Suppression.
[0103] S2.1 Scale the three feature maps to the size of the target features. Figure 1 To.
[0104] S2.2 Calculate the superposition weights on each pixel based on the scaled three-layer feature map.
[0105] S2.3. The scaled three-layer feature maps are weighted and summed according to the superposition weights to obtain the target feature map.
[0106] S2.4. The first stage of prediction is performed on the target feature map as foreground, background and target box through a fully connected layer.
[0107] S3, Feature Space Fine-tuning.
[0108] S3.1 Expand the feature vectors corresponding to the target boxes in S2.4 according to the channel dimension, and use a fully connected layer to map the features on different channels to different vector spaces. Use dot product to measure the correlation between vectors and perform information fusion on the channel dimension to obtain a fused feature map.
[0109] S3.2. Based on the residual operation, add the feature vector corresponding to the target box in S2.4 to the feature vector of the fused feature map to obtain the fine-tuned target feature map.
[0110] S3.3. The target category and location are predicted by fine-tuning the target feature map through a fully connected layer, which is the second stage of prediction, and the final target category and location are output.
[0111] S4, Multi-scale positive sample-assisted learning.
[0112] S4.1. Based on the labels of the labeled sample images, the target region is cropped. The cropped image is then scaled into three samples of different sizes: 1x, 2x, and 4x, resulting in the first sample image, the second sample image, and the third sample image. Each sample image is then input into the backbone network in S1 to obtain five feature maps at different scales. Feature maps from specific layers are then selected based on the sample size to form the second branch.
[0113] S4.2 Since the width and height are halved layer by layer during feature extraction, the feature map output by the third layer of the backbone network when the first sample image is input, the feature map output by the fourth layer of the backbone network when the second sample image is input, and the feature map output by the fifth layer of the backbone network when the third sample image is input are taken. These three feature maps are then input into the fully connected layer in S2.4 for the first stage prediction.
[0114] S4.3. Take the feature map output by the first layer of the backbone network when the first sample image is input, the feature map output by the second layer of the backbone network when the second sample image is input, and the feature map output by the third layer of the backbone network when the third sample image is input. Input the three feature maps into the fully connected layer in S3.3 for the second stage prediction.
[0115] It should be noted that the first, second, third, fourth, and fifth layers in steps S4.2 and S4.3 represent the number of layers in the backbone network. The FPN network output is generally five layers.
[0116] S5. Calculate the first-stage prediction loss and the second-stage prediction loss according to S1 to S4 respectively. Update the network parameters of the fully connected layer for the first-stage prediction based on the first-stage prediction loss. Update the network parameters of the fully connected layer for the second-stage prediction based on the second-stage prediction loss. Continue until the preset training conditions are met.
[0117] like Figure 5 As shown, during the network training phase, steps S1 to S3 serve as the first branch, and the labeled sample images input to the first branch serve as the source domain samples. Step S4 serves as the second branch, and the scaled target region output by the second branch serves as the target domain sample. The second branch uses positive samples to enhance the network's ability to identify targets of different sizes. The network update training is completed through loss calculation of the two branches.
[0118] For specific limitations on steps S1 to S3 in the network training phase, please refer to the limitations on steps S1 to S3 in the application phase described above. This embodiment will not repeat them here. The specific training process in the network training phase is as follows:
[0119] 1) such as Figure 6 As shown, experiments were first conducted on the base class datasets VOC and COCO, which have a large number of samples, and then the model was fine-tuned on a new class dataset with a small number of samples, namely the urine sediment dataset.
[0120] 2) After executing steps S1-S5 of the network training phase, calculate the prediction loss for the first stage and the prediction loss for the second stage. The prediction loss for the first stage includes the loss for both branches in predicting foreground and background images, as well as the loss in predicting the target box. The loss in predicting foreground and background images is measured using the binary cross-entropy loss function, denoted as L. Bcls The loss for bounding box prediction is measured using a smoothed L1 function, denoted as L. Preg If the average is taken for each objective, then the first-stage prediction loss L RPN The definition is as follows:
[0121]
[0122] In the formula, N obj M is the number of samples in the fully connected layer input during step 2.4 of the network training phase. obj Input the number of samples for the fully connected layer in step 4.2 of the network training phase. This represents the loss for predicting the foreground and background of the k-th sample. This represents the loss for predicting the bounding box of the k-th sample;
[0123] The second-stage prediction loss uses the cross-entropy loss function to measure the accuracy of target classification in both branches, denoted as L. Kcls The accuracy of the predicted bounding box is measured using the IoU loss, denoted as L. Rreg Second stage predicted loss L RoI Defined as:
[0124]
[0125] In the formula, N RoI M is the number of samples in the fully connected layer input during step 3.3 of the network training phase. RoI Input the number of samples for the fully connected layer in step 4.3 of the network training phase. This represents the cross-entropy loss of the k-th sample. This represents the IoU loss of the k-th sample, where λ is a preset fixed value (e.g., 0.1).
[0126] 3) During training, the input image size was adjusted so that the shorter side was 800 pixels and the longer side was less than 1333 pixels, while maintaining the aspect ratio. Weights pre-trained on ImageNet were used in initialization, with a batch size of 4. Training was performed on two GPUs, running the SGD optimizer with momentum of 0.9 and parameter decay of 0.0001. For basic training on VOC, the model was trained for 240k, 8k, and 4k iterations with learning rates of 0.005, 0.0005, and 0.00005, respectively. Training was performed on the basic COCO class for 56k, 14k, and 10k iterations. For fine-tuning on the urine sediment dataset, the model was trained for 1300, 400, and 300 iterations with learning rates of 0.005, 0.0005, and 0.00005, respectively.
[0127] Traditional methods for analyzing and detecting urine sediment microscopic images involve manual analysis and counting of patient samples by professional technicians under a microscope. This manual approach is labor-intensive and requires a high level of expertise from the technicians. Computer-aided detection can effectively reduce the workload, obtain accurate quantities of formed elements in urine, and lower the required level of expertise. The model trained using the method of this invention can automatically and quickly detect and accurately identify urine sediment images, achieving automation and intelligence in the medical microscopic image processing system. This method effectively alleviates the problem of insufficient samples in urine sediment detection scenarios and saves on image annotation workload. By jointly using base class and target class samples, the model is easier to train and does not require the creation of additional datasets.
[0128] The method of this invention can be deployed on a server, where clients upload images for detection and receive the results. This invention uses a backbone network with minimal parameters, resulting in a simple structure and high reusability. It can also be deployed on a chip for offline detection.
[0129] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.
[0130] The embodiments described above are merely illustrative of several implementations of the present invention, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these modifications and improvements all fall within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the appended claims.
Claims
1. A small-scale target detection method based on cross-domain few-sample learning, applied to the detection of formed elements in liquids, characterized in that, The small-scale target detection method based on cross-domain few-shot learning includes the following steps in the application phase: Step 1, Multi-scale feature extraction: Input the image to be processed into the backbone network and output the feature maps of the last three layers of the backbone network; Step 2: Background suppression; Step 2.1: Scale the three feature maps to the same size as the target feature map; Step 2.2: Based on the scaled three-layer feature map, calculate the superposition weights at each pixel. Step 2.3: Weight the three scaled feature maps according to the superposition weights to obtain the target feature map; Step 2.4: Perform the first-stage prediction of the target feature map into foreground, background, and target bounding box using a fully connected layer; Step 3: Fine-tuning the feature space; Step 3.1: Expand the feature vectors corresponding to the target boxes in Step 2.4 by channel dimension, and use a fully connected layer to map the features on different channels to different vector spaces. Use dot product to measure the correlation between vectors and perform information fusion on the channel dimension to obtain a fused feature map. Step 3.2: Based on the residual operation, add the feature vector corresponding to the target box in Step 2.4 to the feature vector of the fused feature map to obtain the fine-tuned target feature map; Step 3.3: Predict the target category and location by fine-tuning the target feature map through a fully connected layer, i.e., perform the second-stage prediction, and output the final target category and location.
2. The small-scale target detection method based on cross-domain few-shot learning as described in claim 1, characterized in that, The backbone network is ResNet-101; Output the feature maps F1, F2, and F3 of the last three layers of the backbone network. The width and height of each feature map are the width and height of the original input image to be processed divided by the sampling stride. The sampling stride of the last three layers is stride∈{8,16,32}.
3. The small-scale target detection method based on cross-domain few-shot learning as described in claim 1, characterized in that, The calculation of the superposition weights at each pixel based on the scaled three-layer feature map includes: The scaled three-layer feature maps are processed using 1×1 convolutions to obtain the control parameters for the coordinates of each layer. t is the identifier of the target feature map, 1≤i≤H, 1≤j≤W, H is the height of the target feature map, and W is the width of the target feature map; Calculate the stacking weights at each pixel in each layer using control parameters: In the formula, This represents the superposition weight of pixel (i,j) on the first layer feature map after scaling. This represents the superposition weight of pixel (i,j) on the scaled second-layer feature map. This represents the superposition weight of pixel (i,j) on the scaled third-layer feature map. This represents the control parameters corresponding to pixel (i,j) on the first layer feature map after scaling. This represents the control parameters corresponding to pixel (i,j) on the scaled second-layer feature map. This represents the control parameters corresponding to pixel (i,j) on the third-layer feature map after scaling.
4. The small-scale target detection method based on cross-domain few-shot learning as described in claim 3, characterized in that, The step of weighted summing of the scaled three-layer feature maps according to the superposition weights to obtain the target feature map includes: The scaled feature maps of the three layers are as follows: The target feature map F is obtained by weighted summation of pixels. t : In the formula, For the target feature map F t The pixel value of the upper pixel (i,j). The first layer feature map after scaling. The pixel value of the upper pixel (i,j). The second layer feature map after scaling. The pixel value of the upper pixel (i,j). This is the scaled-down third-layer feature map. The pixel value of the upper pixel (i,j).
5. The small-scale target detection method based on cross-domain few-shot learning as described in claim 1, characterized in that, The first-stage prediction of the target feature map into foreground, background, and target bounding box through a fully connected layer includes: For each pixel in the target feature map, anchor boxes with different scaling ratios and aspect ratios are preset. For each anchor box, foreground and background discrimination, offset prediction, scaling ratio prediction, and aspect ratio prediction are performed. The foreground represents the target, the background represents the background, and the offset represents the distance from the actual target center to the center of the preset anchor box. The prediction results are post-processed to filter out targets that exceed the confidence threshold, and the non-maximum suppression method is used to filter anchor boxes to obtain the final anchor boxes as target boxes.
6. The small-scale target detection method based on cross-domain few-shot learning as described in claim 1, characterized in that, The process of using dot product to measure the correlation between vectors and fusing information along the channel dimension to obtain a fused feature map includes: The dot product is used to measure the correlation between vectors, and an attention map is generated. Based on the attention map score, the feature vectors are weighted and information is fused along the channel dimension to obtain a fused feature map.
7. The small-scale target detection method based on cross-domain few-shot learning as described in claim 1, characterized in that, The small-scale target detection method based on cross-domain few-shot learning also includes the following steps during the network training phase: Step 4: Multi-scale positive sample-assisted learning; Step 4.1: Extract the target region based on the label of the labeled sample image, scale the extracted image into three samples of different sizes, namely 1x, 2x, and 4x, to obtain the first sample image, the second sample image, and the third sample image. Input each sample image into the backbone network in Step 1. Step 4.2: Take the feature map output by the third layer of the backbone network when the first sample image is input, the feature map output by the fourth layer of the backbone network when the second sample image is input, and the feature map output by the fifth layer of the backbone network when the third sample image is input. Input the three feature maps into the fully connected layer in step 2.4 for the first stage prediction. Step 4.3: Take the feature map output by the first layer of the backbone network when the first sample image is input, the feature map output by the second layer of the backbone network when the second sample image is input, and the feature map output by the third layer of the backbone network when the third sample image is input. Input the three feature maps into the fully connected layer in step 3.3 for the second stage prediction. Step 5: Calculate the first-stage prediction loss and the second-stage prediction loss according to steps 1 to 4 respectively. Update the network parameters of the fully connected layer for the first-stage prediction based on the first-stage prediction loss, and update the network parameters of the fully connected layer for the second-stage prediction based on the second-stage prediction loss, until the preset training conditions are met and the process ends.
8. The small-scale target detection method based on cross-domain few-shot learning as described in claim 7, characterized in that, The calculation of the first-stage prediction loss and the second-stage prediction loss according to steps 1 to 4 respectively includes: The first-stage prediction loss includes two branches: the loss for foreground and background prediction, and the loss for target box prediction. The loss for foreground and background prediction is measured using the binary cross-entropy loss function, denoted as L. Bcls The loss for bounding box prediction is measured using a smoothed L1 function, denoted as L. Preg If the average is taken for each objective, then the first-stage prediction loss L RPN The definition is as follows: In the formula, N obj M is the number of samples in the fully connected layer input during step 2.4 of the network training phase. obj Input the number of samples for the fully connected layer in step 4.2 of the network training phase. This represents the loss for predicting the foreground and background of the k-th sample. This represents the loss for predicting the bounding box of the k-th sample; The second-stage prediction loss uses the cross-entropy loss function to measure the accuracy of target classification in both branches, denoted as L. Kcls The accuracy of the predicted bounding box is measured using the IoU loss, denoted as L. Rreg The second stage predicts the loss L. RoI Defined as: In the formula, N RoI M is the number of samples in the fully connected layer input during step 3.3 of the network training phase. RoI Input the number of samples for the fully connected layer in step 4.3 of the network training phase. This represents the cross-entropy loss of the k-th sample. Let λ represent the IoU loss of the k-th sample, where λ is a preset fixed value.