Small sample target detection method and device
Patent Information
- Application Number
- CN202410537281.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-04-30
- Publication Date
- 2026-10-16
- Estimated Expiration
- 2044-04-30
AI Technical Summary
目前常规的目标检测领域的研究已趋于成熟，但存在一个严重的问题是高度依赖于大规模的标注数据集，目标检测的性能受到数据集质量和大小影响
[0092] Compared with existing technologies, the small-sample target detection method of the present invention has the following advantages:
Smart Images
Figure CN118262206B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of target detection in computer vision, and in particular to a method and apparatus for small-sample target detection. Background Technology
[0002] Object detection is a hot topic in computer vision, and its task is to locate objects of interest in an image and identify their category. As one of the fundamental tasks of computer vision, object detection has wide applications, playing a crucial role in areas such as face recognition, agricultural pest and disease identification, and autonomous driving. Existing object detection methods typically require a large number of labeled images for model training, which demands significant human and material resources, and often yields poor results without access to a large number of labeled samples.
[0003] In recent years, the field of deep learning has developed rapidly, enabling object detection on new samples by training neural network models on existing samples. While research in conventional object detection has matured, a serious problem remains: it heavily relies on large-scale labeled datasets. The performance of object detection is significantly affected by the quality and size of the dataset. Due to the inherent long-tail distribution of data in the real world, the number of samples for most targets is relatively small, such as many rare diseases. In real-world scenarios, it is difficult to obtain a sufficiently large number of high-quality labeled samples. When the retrieved information is insufficient to fit a large number of parameters, overfitting often occurs, leading to poor generalization and reduced accuracy in object detection. Summary of the Invention
[0004] Therefore, the purpose of this invention is to provide a method and apparatus for small sample target detection.
[0005] A small-sample target detection method includes the following steps:
[0006] S1: Extract features from the sample image to obtain several feature maps;
[0007] S2: Perform feature fusion on the feature maps to obtain several fused feature maps;
[0008] S3: Perform target detection on the fused feature map and output the detection results;
[0009] Step S1 includes the following sub-steps:
[0010] S11: Extract features from the sample image to obtain the first feature map;
[0011] S12: Extract features from the first feature map to obtain the second feature map;
[0012] S13: Extract features from the second feature map by combining self-attention mechanism and hybrid dilated convolution to obtain the third feature map.
[0013] The few-sample object detection method described in this invention combines a self-attention mechanism and hybrid dilated convolution during feature extraction. The self-attention mechanism adjusts the weights of regions on the feature map, focusing on more important information during feature extraction and achieving a wider focus on important regions. The hybrid dilated convolution can obtain different receptive field sizes, thereby obtaining feature information at different scales. Therefore, this method can handle both local semantic information and consider global semantic dependencies, effectively extracting more feature information from sample images, thus improving the performance of few-sample object detection tasks.
[0014] Further, step S13 includes the following sub-steps:
[0015] S131: Perform convolution processing on the second feature map to obtain a convolutional feature map;
[0016] S132: Perform convolution, self-attention mechanism and weighted combination processing on the convolutional feature map to obtain the SAC feature map;
[0017] S133: Perform convolution, pooling, and concatenation on the SAC feature map to obtain the SPPF feature map;
[0018] S134: Perform convolution and concatenation on the SPPF feature map to obtain the third feature map.
[0019] Further, step S132 includes the following sub-steps:
[0020] S1321: Convolutional feature maps are processed by three 1*1 convolutional kernels to obtain three intermediate feature groups, each with N feature maps;
[0021] S1322A: Perform self-attention mechanism calculation on the intermediate feature group to obtain several self-attention feature maps;
[0022] S1322B: Perform hybrid dilated convolution processing on the intermediate feature group to obtain several dilated convolution feature maps;
[0023] S1323: Weight the self-attention feature map and the dilated convolution feature map to obtain the SAC feature map.
[0024] Further, step S134 includes the following sub-steps:
[0025] S1341: Convolve the SPPF feature map with two convolution kernels respectively to obtain the first convolution map and the second convolution map;
[0026] S1342: Perform hybrid dilated convolution on the second convolutional image and average pool the result to obtain a dilated convolutional image;
[0027] S1343: The first convolutional image and the dilated convolutional image are concatenated and convolved to obtain the third feature map.
[0028] Further, step S2 includes the following sub-steps:
[0029] S21: Perform convolution and upsampling on the third feature map, then concatenate and convolve it with the second feature map to obtain the concatenated feature map;
[0030] S22: Perform convolution and upsampling on the concatenated feature map, then concatenate and convolve it with the first feature map to obtain the first fused feature map;
[0031] S23: Perform convolution processing on the first fused feature map, and then concatenate and convolve it with the concatenated feature map to obtain the second fused feature map;
[0032] S24: Perform convolution processing on the second fused feature map, and then concatenate and convolve it with the third feature map that has undergone convolution processing to obtain the third fused feature map.
[0033] Furthermore, before the second feature map and the first feature map are spliced, they are first processed by patch merging and self-attention mechanism.
[0034] Furthermore, this invention also provides a small-sample target detection device, comprising a backbone network, a neck network, and a head network. The backbone network is used to extract features from sample images to obtain several feature maps; the neck network is used to fuse features from the feature images to obtain several fused feature maps; and the head network is used to perform target detection on the fused feature images and output the detection results. The backbone network includes...
[0035] The first feature extraction unit is used to extract features from the sample image to obtain the first feature map;
[0036] The second feature extraction unit is used to extract features from the first feature map to obtain the second feature map;
[0037] The third feature extraction unit is used to extract features from the second feature map to obtain the third feature map.
[0038] Furthermore, the third feature extraction unit includes
[0039] The CBS module is used to perform convolution processing on the second feature map to obtain a convolutional feature map;
[0040] The SAC module is used to perform convolution, self-attention mechanism and weighted combination processing on the convolutional feature map to obtain the SAC feature map.
[0041] The SPPF module is used to perform convolution, pooling, and concatenation on the SAC feature map to obtain the SPPF feature map.
[0042] The C3Dconv module is used to perform convolution and concatenation on SPPF feature maps to obtain a third feature map.
[0043] Furthermore, the SAC module includes
[0044] The feature regrouping module is used to perform convolution processing on the convolutional feature map using three 1*1 convolution kernels to obtain three intermediate feature groups, each with N feature maps.
[0045] The self-attention module is used to perform self-attention mechanism calculations on the intermediate feature group to obtain several self-attention feature maps.
[0046] The hybrid dilated convolution module is used to perform hybrid dilated convolution processing on the intermediate feature group to obtain several dilated convolution feature maps;
[0047] The weighted combination module is used to perform weighted processing on the self-attention feature map and the dilated convolution feature map to obtain the SAC feature map.
[0048] Furthermore, the C3DConv module includes
[0049] The first convolutional kernel is used to perform convolution processing on the SPPF feature map to obtain the first convolutional map;
[0050] The second convolutional kernel is used to perform convolution processing on the SPPF feature map to obtain the second convolutional map;
[0051] A hybrid dilated convolution kernel is used to perform hybrid dilated convolution on the second convolutional image, and the result is then averaged to obtain a dilated convolutional image.
[0052] The third convolution kernel is used to concatenate and convolve the first convolutional image and the dilated convolutional image to obtain the third feature map.
[0053] To better understand and implement this invention, the following detailed description is provided in conjunction with the accompanying drawings. Attached Figure Description
[0054] Figure 1 This is a schematic diagram of a small sample target detection method;
[0055] Figure 2 Here is a flowchart of a small-sample target detection method;
[0056] Figure 3This is a schematic diagram of a small sample target detection model;
[0057] Figure 4 A schematic diagram of the SAC module;
[0058] Figure 5 This is a schematic diagram of the C3DConv module. Detailed Implementation
[0059] Existing object detection techniques often suffer from overfitting when the sample size is insufficient to fit a large number of parameters, resulting in poor generalization. Few-shot learning, on the other hand, can achieve rapid classification and recognition of new object classes with a small number of labeled samples, reducing reliance on large-scale labeled data. Therefore, this invention proposes a few-shot object detection method, which improves upon few-shot learning and object detection methods by combining them to address the problem of limited labeled data in real-life scenarios. This allows for the detection of targets of interest with only a small amount of labeled information. Please refer to [link to relevant documentation]. Figure 1 and 2 The small sample target detection method described in this invention combines self-attention mechanism and hybrid dilated convolution, enabling the method to process both local semantic information and consider global semantic dependencies, effectively extracting more feature information from sample images and improving the detection efficiency of small samples.
[0060] Please see Figure 3 Based on the aforementioned method, this invention also provides a small-sample target detection device, employing a network model based on an improved Yolov5 architecture, including a backbone network, a neck network, and a head network. The backbone network is used for feature extraction from the image, obtaining multi-level feature maps; the neck network is used for feature fusion of the multi-level feature maps, obtaining a fused feature map; and the head network is used for detecting the fused feature map. This invention enhances the feature extraction capability of the backbone network by adding a SAC module and a C3DConv module, enabling the extraction of feature information at more scales. The SAC module combines a self-attention mechanism and hybrid dilated convolution. The self-attention mechanism adjusts the weights of regions on the feature map, making the model focus more on important regions; the hybrid dilated convolution, by setting multiple dilation kernels with different dilation rates, has different receptive field sizes, enabling the acquisition of multi-scale image information.
[0061] The backbone network is used to perform step S1: extracting features from the sample image to obtain several feature maps.
[0062] Specifically, the backbone network includes a first feature extraction unit, a second feature extraction unit, and a third feature extraction unit connected in sequence.
[0063] The first feature extraction unit includes a Focus module, a first CBS module, a CSP1_1 module, a second CBS module, and a first CSP1_3 module connected in sequence, and is used to perform step S11: extract features from the sample image to obtain a first feature map.
[0064] The second feature extraction unit includes a third CBS module and a second CSP1_3 module connected in sequence, used to perform step S12: extract features from the first feature map to obtain a second feature map.
[0065] The third feature extraction unit includes a fourth CBS module, a SAC module, an SPPF module, and a C3DConv module connected in sequence, and is used to perform step S13: extract features from the second feature map to obtain the third feature map.
[0066] Specifically, it includes the following sub-steps:
[0067] S131: The fourth CBS module performs convolution processing on the second feature map to obtain a convolutional feature map.
[0068] S132: The SAC module performs convolution, self-attention mechanism and weighted combination processing on the convolution feature map to obtain the SAC feature map.
[0069] For details, please refer to Figure 4 The SAC module includes a feature reorganization module, a self-attention module, a hybrid dilated convolution module, and a weighted combination module. The feature reorganization module is used to perform step S1321: convolution processing the convolutional feature map using three 1*1 convolution kernels to obtain three intermediate feature groups, each with N feature maps.
[0070] The self-attention sub-module is used to perform step S1322A: perform self-attention mechanism calculation on the intermediate feature group to obtain several self-attention feature maps.
[0071] Specifically, the intermediate feature groups are first reassembled to obtain N rearranged feature groups, each containing 3 features. The 3 features in each group come from the 3 1*1 convolutional kernels. Then, the 3 features in each group are treated as query, key, and value, respectively, and a self-attention mechanism is used to calculate N self-attention feature maps.
[0072] The hybrid dilated convolution module is used to perform step S1322B: perform hybrid dilated convolution processing on the intermediate feature group to obtain several dilated convolution feature maps.
[0073] Specifically, three dilated convolution kernels with dilation coefficients of 1, 2, and 5 are set to perform dilated convolution processing on the intermediate feature group to obtain multi-scale features. Then, average pooling is performed on the multi-scale features to finally obtain N dilated convolution feature maps.
[0074] The weighted combination module is used to perform step S1323: weighting the self-attention feature map and the dilated convolution feature map to obtain the SAC feature map.
[0075] Specifically, the N self-attention feature maps and the N dilated convolution feature maps are concatenated (Concat) to obtain the self-attention feature F. att and dilated convolution feature F HDConv And according to F out =αF att +βF HDConv The SAC feature map is calculated, where α and β are variable parameters.
[0076] S133: The SPPF module performs convolution, pooling, and concatenation on the SAC feature map to obtain the SPPF feature map.
[0077] S134: The C3DConv module performs convolution and concatenation on the SPPF feature map to obtain the third feature map.
[0078] For details, please refer to Figure 5 The C3DConv module includes three 1*1 convolution kernels and one hybrid dilated convolution kernel. The three 1*1 convolution kernels are the first convolution kernel, the second convolution kernel, and the third convolution kernel. The hybrid dilated convolution kernel includes three dilated convolution kernels with dilation coefficients of 1, 2, and 5, respectively.
[0079] The first and second convolution kernels are used to perform step S1341: convolution processing on the SPPF feature maps respectively to obtain the first convolution map and the second convolution map.
[0080] The hybrid dilated convolution kernel is used to perform step S1342: perform hybrid dilated convolution processing on the second convolution map, and perform average pooling on the result to obtain a dilated convolution map.
[0081] The third convolution kernel is used to perform step S1343: concatenating and convolving the first convolutional image and the dilated convolutional image to obtain the third feature map.
[0082] After obtaining the first, second, and third feature maps through the above steps, they are input into the neck network. The neck network is used to perform step S2: feature fusion of the feature maps to obtain several fused feature maps.
[0083] Specifically, the neck network includes a splicing unit, a first fusion unit, a second fusion unit, and a third fusion unit.
[0084] The splicing unit is used to perform step S21: convolution and upsampling processing on the third feature map, and then splicing and convolution processing with the second feature map to obtain a spliced feature map.
[0085] The first fusion unit is used to perform step S22: convolution and upsampling processing on the spliced feature map, and then splicing and convolution processing with the first feature map to obtain the first fused feature map.
[0086] Preferably, before the second feature map and the first feature map are stitched together, they are first processed by patch merging and self-attention mechanism. The effect of patch merging is similar to pooling, which can reduce the resolution of the feature map and adjust the number of channels, while not losing information compared to pooling.
[0087] The second fusion unit is used to perform step S23: convolution processing on the first fusion feature map, and then concatenating and convolution processing with the spliced feature map to obtain the second fusion feature map.
[0088] The third fusion unit is used to perform step S24: convolution processing on the second fusion feature map, and then concatenating and convolution processing with the third feature map that has undergone convolution processing to obtain the third fusion feature map.
[0089] After obtaining the fused feature map, it is input into the head network, which is used to perform step S3: target detection on the fused feature map and output the detection result.
[0090] Specifically, the head network is equipped with three detection heads, which perform target detection on the first, second and third fused feature maps respectively, and output the final detection result after weighted combination of the three detection results.
[0091] Preferably, the model further includes a data augmentation network to augment the small sample data. In this embodiment, the data augmentation network uses the TokenMix data augmentation method, which offers better consistency accuracy gains compared to commonly used methods such as Mosaic and Cutmix in the prior art.
[0092] Compared with existing technologies, the small-sample target detection method of the present invention has the following advantages:
[0093] (1) The Yolov5 network is improved by setting up the SAC module and C3DConv module in the backbone network, which combines hybrid dilated convolution and self-attention mechanism, thereby improving the feature extraction capability of the image and enabling the extraction of multi-scale feature information.
[0094] (2) By setting adaptive feature fusion, the amount of computation is effectively reduced, the feature fusion speed is improved, the overall running efficiency is improved, and the detection speed is accelerated.
[0095] (3) By introducing the TokenMix data augmentation method, data augmentation is performed on small sample data to obtain more sample data, which effectively improves the detection capability for small samples.
[0096] The embodiments described above are merely examples of several implementations of the present invention, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and the present invention also intends to include these modifications and variations.
Claims
1. A method for small-sample target detection, comprising the following steps: S1: Extract features from the sample image to obtain several feature maps; S2: Perform feature fusion on the feature maps to obtain several fused feature maps; S3: Perform target detection on the fused feature map and output the detection results; The feature is that step S1 includes the following sub-steps: S11: Extract features from the sample image to obtain the first feature map; S12: Extract features from the first feature map to obtain the second feature map; S13: Extract features from the second feature map using a self-attention mechanism and hybrid dilated convolution to obtain the third feature map, including the following sub-steps: S131: Perform convolution processing on the second feature map to obtain a convolutional feature map; S132: Perform convolution, self-attention mechanism and weighted combination processing on the convolutional feature map to obtain the SAC feature map; S133: Perform convolution, pooling, and concatenation on the SAC feature map to obtain the SPPF feature map; S134: Perform convolution and concatenation on the SPPF feature map to obtain the third feature map, including the following sub-steps: S1341: Convolve the SPPF feature map with two convolution kernels respectively to obtain the first convolution map and the second convolution map; S1342: Perform hybrid dilated convolution on the second convolutional image and average pool the result to obtain a dilated convolutional image; S1343: Concatenate and convolve the first convolutional image and the dilated convolutional image to obtain the third feature map; Step S2 includes the following sub-steps: S21: Perform convolution and upsampling on the third feature map, then concatenate and convolve it with the second feature map to obtain the concatenated feature map; S22: Perform convolution and upsampling on the concatenated feature map, then concatenate and convolve it with the first feature map to obtain the first fused feature map; S23: Perform convolution processing on the first fused feature map, and then concatenate and convolve it with the concatenated feature map to obtain the second fused feature map; S24: Perform convolution processing on the second fused feature map, and then concatenate and convolve it with the third feature map that has undergone convolution processing to obtain the third fused feature map.
2. The small sample target detection method according to claim 1, characterized in that: Step S132 includes the following sub-steps: S1321: Through 3 ones One convolution kernel is used to perform convolution processing on the convolution feature map to obtain three intermediate feature groups, each with N feature maps; S1322A: Perform self-attention mechanism calculation on the intermediate feature group to obtain several self-attention feature maps; S1322B: Perform hybrid dilated convolution processing on the intermediate feature group to obtain several dilated convolution feature maps; S1323: Weight the self-attention feature map and the dilated convolution feature map to obtain the SAC feature map.
3. The small sample target detection method according to claim 1, characterized in that: Before the second feature map and the first feature map are spliced, they are first processed by patch merging and self-attention mechanism.
4. A small-sample target detection device, comprising a backbone network, a neck network, and a head network, wherein the backbone network is used to extract features from sample images to obtain several feature maps; the neck network is used to fuse features from the feature images to obtain several fused feature maps; and the head network is used to perform target detection on the fused feature images and output detection results; characterized in that: The backbone network includes The first feature extraction unit is used to extract features from the sample image to obtain a first feature map; The second feature extraction unit is used to extract features from the first feature map to obtain the second feature map; A third feature extraction unit is used to extract features from the second feature map to obtain a third feature map; the third feature extraction unit includes: The CBS module is used to perform convolution processing on the second feature map to obtain a convolutional feature map; The SAC module is used to perform convolution, self-attention mechanism and weighted combination processing on the convolutional feature map to obtain the SAC feature map. The SPPF module is used to perform convolution, pooling, and concatenation on the SAC feature map to obtain the SPPF feature map. The C3Dconv module is used to perform convolution and concatenation on SPPF feature maps to obtain a third feature map, including: The first convolutional kernel is used to perform convolution processing on the SPPF feature map to obtain the first convolutional map; The second convolutional kernel is used to perform convolution processing on the SPPF feature map to obtain the second convolutional map; A hybrid dilated convolution kernel is used to perform hybrid dilated convolution on the second convolutional image, and the result is then averaged to obtain a dilated convolutional image. The third convolution kernel is used to concatenate and convolve the first convolutional image and the dilated convolutional image to obtain the third feature map.
5. The small sample target detection device according to claim 4, characterized in that: The SAC module includes Feature reorganization module, used to reorganize components through 3 1 One convolution kernel is used to perform convolution processing on the convolution feature map to obtain three intermediate feature groups, each with N feature maps; The self-attention module is used to perform self-attention mechanism calculations on the intermediate feature group to obtain several self-attention feature maps. The hybrid dilated convolution module is used to perform hybrid dilated convolution processing on the intermediate feature group to obtain several dilated convolution feature maps; The weighted combination module is used to perform weighted processing on the self-attention feature map and the dilated convolution feature map to obtain the SAC feature map.