Small sample target detection method with high-efficiency scale perception

Abstract

The invention relates to an efficient scale perception small sample target detection method comprising the following steps: obtaining a data set used for model training, and dividing the data set into a base class data set and a new class data set; constructing small sample data sets with different sample numbers from the new class data set; the method comprises the following steps: constructing a small sample target detection model of high-efficiency scale perception, wherein the model comprises a feature extraction network reconstructed by adopting a high-efficiency scale perception module ESA, two gradient decoupling layers GDL, a candidate box generation network RPN introducing an enhanced triple attention mechanism ETA, a region convolutional network RCNN, and a classifier and a regression device introducing a prototype calibration module PCB; training the small sample target detection model by adopting a two-stage fine tuning method based on a transfer learning strategy; migrating priori knowledge extracted from base class data to new class target detection through two-stage training; and performing small sample target detection through the trained small sample target detection model. The method is beneficial to improving the precision of small target detection under the conditions of complex background and scale change.

Description

Technical Field

[0001] The present invention relates to the technical field of target detection, and in particular to a small sample target detection method with high efficiency and scale perception. Background Art

[0002] Small sample target detection is an important research direction in the field of computer vision. It aims to solve the problem of how to train a detection model with good generalization ability under the condition of limited labeled samples. In practical applications, it is often difficult and costly to obtain a large amount of high-quality labeled data, especially in some specific fields such as military remote sensing detection, disease diagnosis, and defective product detection in industrial production. Therefore, the research on small sample target detection technology is of great significance to promote technological progress in these fields.

[0003] At present, the research on small sample object detection mainly focuses on the following aspects: meta-learning-based methods, transfer learning-based methods, data enhancement-based methods, and metric learning-based methods. These methods have their own advantages and disadvantages and are suitable for different application scenarios.

[0004] Most current research methods use a transfer learning-based approach that adopts a "pre-training-fine-tuning" training paradigm, directly training on the dataset, pre-training on a base class dataset with a large number of annotations to obtain a base class detection model, and fine-tuning on a small sample dataset to generalize to new classes. The advantage of this method is that it can make full use of the rich features that the pre-trained model has learned, so that it can quickly adapt to new tasks in the fine-tuning stage, even with only a small amount of labeled data. This not only reduces the annotation cost, but also speeds up the training of the model. In addition, the generalization ability of the pre-trained model also lays the foundation for the performance of the fine-tuned model, allowing it to maintain a high detection accuracy when facing new categories.

[0005] However, small sample object detection based on transfer learning also faces some challenges. When the new class is very different from the base class, simple fine-tuning may not be enough to effectively transfer knowledge, resulting in unsatisfactory detection performance. In addition, overfitting is prone to occur during fine-tuning, that is, the model over-relies on limited labeled data and ignores the generalization ability of unseen samples. The DeFRCN algorithm proposed by Qiao et al. analyzed the contradictions of traditional Faster R-CNN from the perspective of multi-task and multi-stage, and extended Faster R-CNN by introducing a gradient decoupling layer for multi-level decoupling and a prototypical calibration block for multi-task decoupling. However, the ability of the DeFRCN algorithm to cope with changes in target scale and complex background interference still needs to be improved, especially in the absence of data sets. It is difficult to effectively obtain and utilize the spatial information and perceptual context information of feature maps of different scales, which greatly reduces the detection performance of the model. Summary of the invention

[0006] The purpose of the present invention is to provide a small sample target detection method with high efficiency and scale perception, which is beneficial to improving the accuracy of small target detection under complex background and scale change conditions.

[0007] In order to achieve the above object, the technical solution adopted by the present invention is: an efficient scale-aware small sample target detection method, comprising the following steps:

[0008] Step 1: Obtain the dataset for model training, divide the dataset into a base class dataset and a new class dataset according to the small sample detection task; construct small sample datasets with different sample numbers from the new class dataset;

[0009] Step 2: Build an efficient scale-aware small sample target detection model, which includes a feature extraction network reconstructed by an efficient scale-aware module ESA, two gradient decoupling layers GDL, a candidate box generation network RPN with an enhanced triple attention mechanism ETA, a regional convolutional network RCNN, and a classifier and regressor with a prototype calibration module PCB.

[0010] Step 3: Use a two-stage fine-tuning method based on a transfer learning strategy to train the small sample target detection model; input the sample data in the base class data set into the pre-trained feature extraction network for basic stage training to obtain the basic model and save the basic model parameters; then, set the number of classification categories of the detection head from the number of base class categories to the number of new class categories, and according to different small sample detection task settings, use small sample data sets with different sample numbers to train the basic model to fine-tune the classifier and regressor to obtain a trained small sample target detection model; thus, the prior knowledge extracted from the base class data is transferred to the new class target detection through two-stage training;

[0011] Step 4: Perform small sample target detection using the trained small sample target detection model.

[0012] Furthermore, the PASCAL VOC dataset is obtained as the dataset for model training; according to the definition of the small sample detection task, the dataset is divided into a base class dataset and a new class dataset in a ratio of 3:1; from the new class dataset, i pictures are selected from each category to construct a small sample dataset; i takes different values ​​to obtain small sample datasets with different sample numbers.

[0013] Furthermore, the implementation method of reconstructing the feature extraction network using the efficient scale perception module ESA is as follows:

[0014] The feature extraction network adopts the ResNet-101 convolutional neural network, and the ESA module is introduced into several residual blocks in the res4 stage of ResNet-101. The 3×3 convolutional layers of several residual blocks in the res4 stage are replaced by the ESA module. When the feature map is input into the residual block fused with the ESA module, it first enters the convolution layer with a convolution kernel of 1×1, and the output feature map then enters the ESA module. The feature map processed by the ESA module then enters the convolution layer with a convolution kernel of 1×1 to obtain the final output of the residual block.

[0015] Furthermore, the ESA module includes a context conversion module, an offset convolution based on 1×1 convolution, and a deformable convolution based on 3×3 convolution. After the feature map is input into the ESA module, it is divided into two parallel branches. In the offset learning branch, the input feature map first obtains global and local context information through the context conversion module, and then extracts the offset in the horizontal and vertical directions through the offset convolution. Finally, the input feature map and the learned offset are simultaneously input into the deformable convolution to achieve efficient dynamic sampling convolution operation.

[0016] Furthermore, two GDL modules are inserted after the feature extraction network, and are connected to the RPN network and the RCNN network respectively through the two GDL modules. The feature forward operation and gradient backward operation are redefined through the GDL modules, and a learnable affine transformation is performed on the forward feature mapping. At the same time, the gradient flowing to the feature extraction network is restricted in the back propagation, so as to achieve effective decoupling of RPN and RCNN.

[0017] Furthermore, in the RPN network, an enhanced triple attention mechanism is introduced to improve the quality of generated candidate boxes; the GDL module outputs a feature map, which is processed by the enhanced triple attention mechanism to obtain an enhanced feature map and input into the RPN network;

[0018] The enhanced triple attention mechanism divides the input feature map into three branches, each of which processes the feature interaction between different dimensions; the first branch focuses on the interaction between channels and heights, the second branch focuses on the interaction between channels and widths, and the third branch focuses on the spatial dimension; considering the impact of factors such as background noise in the image and variable target scale on the detection accuracy of the model, deformable residual blocks are inserted before the input and after the output to further enhance the feature representation of the target-related areas.

[0019] Furthermore, the RPN network uses the enhanced feature maps and anchor boxes to generate a series of category-independent region proposals for fitting the true bounding box; then, the RCNN network extracts features from the region proposals and inputs them into the classifier and regressor to obtain the bounding box coordinates and category scores predicted by the model.

[0020] Furthermore, the specific implementation method of the enhanced triple attention mechanism is:

[0021] (1) After the feature map is input into the enhanced triple attention mechanism, the feature map is first processed by the first deformable residual block and then divided into three branches;

[0022] (2) In the first and second branches, the input tensor is rotated to change its dimensional arrangement so that attention can be allocated according to the importance of features in different dimensions. The rotated tensor is then reduced in dimension through the Z-Pool operation. After the Z-Pool operation, the tensor is reduced in dimension through a 7×7 convolution layer, and then the corresponding attention weights are generated through batch normalization and sigmoid activation function. The generated attention weights are then multiplied by the rotated input tensor, and then the rotation operation is performed.

[0023] In the third branch, the Channel Pool operation is performed on the input tensor. After the Channel Pool operation, the tensor is reduced in dimension through a 7×7 convolution layer, and then the corresponding attention weights are generated through batch normalization and sigmoid activation function. Then the generated attention weights are multiplied with the original input tensor.

[0024] (3) The outputs of the three branches are averaged and then processed by the second deformable residual block to form the final output.

[0025] Furthermore, the PCB module is an offline module that uses a strong classifier based on SENet to correct the category scores output by the RCNN network.

[0026] Furthermore, in the model training phase, the back propagation algorithm is used to update the model parameters, and the loss function L of the model is:

[0027] L=L cls +L RPN +L reg

[0028] Among them, L cls is the Faster R-CNN loss, L RPN is the RPN classification loss, L reg is the bounding box regression loss; L cls and L RPN Using cross entropy loss, L reg GIoU loss is used.

[0029] Compared with the prior art, the present invention has the following beneficial effects:

[0030] 1. The efficient scale-aware small sample target detection method proposed in the present invention reconstructs the feature extraction network by adopting the ESA module, so that the network can adaptively learn targets of different shapes and scales, thereby improving the multi-scale feature representation capability of the small sample target detection model.

[0031] 2. The efficient scale-aware small sample target detection method proposed in the present invention introduces an enhanced triple attention mechanism in RPN to obtain more representative features, improve the quality of candidate regions generated by RPN, and improve the accuracy of target detection tasks in small sample scenarios.

[0032] 3. The efficient scale-aware small sample target detection method proposed in the present invention uses a strong classifier based on SENet to correct the classification prediction score of the small sample target detection model, which significantly improves the detection accuracy of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1is a schematic diagram of the implementation principle of a small sample target detection model in an embodiment of the present invention;

[0034] Figure 2 is a schematic diagram of the structure of a context conversion module in an embodiment of the present invention;

[0035] Figure 3 is a schematic diagram of the structure of the enhanced triple attention mechanism in an embodiment of the present invention;

[0036] Figure 4 is a schematic structural diagram of a gradient decoupling layer in an embodiment of the present invention;

[0037] Figure 5 Schematic diagram of the structure of the prototype calibration module in the embodiment of the present invention. DETAILED DESCRIPTION

[0038] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0039] It should be noted that the following detailed descriptions are exemplary and are intended to provide further explanation of the present application. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present application belongs.

[0040] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit the exemplary embodiments according to the present application. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprise" and / or "include" are used in this specification, it indicates the presence of features, steps, operations, devices, components and / or combinations thereof.

[0041] This embodiment provides an efficient scale-aware small sample target detection method, including the following steps:

[0042] Step 1: Obtain the dataset for model training, divide the dataset into a base class dataset and a new class dataset according to the small sample detection task; construct small sample datasets with different sample numbers from the new class dataset.

[0043] Step 2: Build an efficient and scale-aware small sample target detection model, such as Figure 1 As shown in the figure, the model includes a feature extraction network reconstructed by an efficient scale-aware module ESA, two gradient decoupling layers GDL, a candidate box generation network RPN introducing an enhanced triple attention mechanism ETA, a regional convolutional network RCNN, and a classifier and regressor introducing a prototype calibration module PCB.

[0044] Step 3: Use a two-stage fine-tuning method based on a transfer learning strategy to train the small sample target detection model; use the feature extraction network pre-trained on ImageNet as the pre-training model, input the sample data in the base class dataset into the pre-training model for basic stage training, obtain the basic model and save the basic model parameters; then, set the number of classification categories of the detection head from the number of base class categories to the number of new class categories, and according to different small sample detection task settings, use small sample datasets with different sample numbers to train the basic model to fine-tune the classifier and regressor of the last layer to obtain a trained small sample target detection model.

[0045] Step 4: Perform small sample target detection using the trained small sample target detection model.

[0046] In this embodiment, the PASCAL VOC dataset is obtained as the dataset for model training; the dataset is divided into a base class dataset and a new class dataset in a ratio of 3:1 according to the definition of the small sample detection task; from the new class dataset, i pictures are selected for each category to construct a small sample dataset; i takes different values ​​(in this embodiment, i takes 1, 2, 3, 5, 10) to obtain small sample datasets with different sample numbers. In order to evaluate the robustness and generalization of the model, different division methods can be used to divide the base class dataset and the new class dataset, and then construct different small sample datasets.

[0047] In this embodiment, the small sample target detection model uses Faster R-CNN as the basic detection framework, and adopts transfer learning to transfer the prior knowledge extracted from the base class data to the new class target detection through two-stage training. The feature extraction network is ESA-ResNet-101 pre-trained on ImageNet and combined with the ESA module.

[0048] The implementation method of reconstructing the feature extraction network using the efficient scale perception module ESA is as follows:

[0049] The feature extraction network adopts the ResNet-101 convolutional neural network, and the ESA module is introduced into several residual blocks in the res4 stage of ResNet-101. The 3×3 convolutional layers of several residual blocks in the res4 stage are replaced by the ESA module. When the feature map is input into the residual block fused with the ESA module, it first enters the convolution layer with a convolution kernel of 1×1, and the output feature map then enters the ESA module. The feature map processed by the ESA module then enters the convolution layer with a convolution kernel of 1×1 to obtain the final output of the residual block.

[0050] Specifically, the ESA module includes a context conversion module, an offset convolution based on 1×1 convolution, and a deformable convolution based on 3×3 convolution. After the feature map is input into the ESA module, it is divided into two parallel branches. In the offset learning branch, the input feature map first obtains global and local context information through the context conversion module, and then extracts the offset in the horizontal and vertical directions through the offset convolution. Finally, the input feature map and the learned offset are simultaneously input into the deformable convolution to achieve efficient dynamic sampling convolution operation.

[0051] Two GDL modules are inserted after the feature extraction network, and are connected to the RPN network and RCNN network respectively through the two GDL modules. The feature forward operation and gradient backward operation are redefined through the GDL modules, and a learnable affine transformation is performed on the forward feature mapping. At the same time, the gradient flowing to the feature extraction network is restricted in the back propagation, so as to achieve effective decoupling of RPN and RCNN.

[0052] In the RPN network, an enhanced triple attention mechanism is introduced to improve the quality of generated candidate boxes; the GDL module outputs a feature map, which is processed by the enhanced triple attention mechanism to obtain an enhanced feature map and input into the RPN network. The RPN network uses the enhanced feature map and anchor box to generate a series of category-independent region proposals for fitting the real bounding box; then, the RCNN network extracts features from the region proposals, inputs the classifier and regressor, obtains the bounding box coordinates and category scores predicted by the model, and then corrects the category scores through the PCB module. The overall improvement improves the detection performance and generalization of the model.

[0053] The enhanced triple attention mechanism divides the input feature map into three branches, each of which processes the feature interaction between different dimensions; the first branch focuses on the interaction between channels and heights, the second branch focuses on the interaction between channels and widths, and the third branch focuses on the spatial dimension; considering the impact of factors such as background noise in the image and variable target scales on the model detection accuracy, deformable residual blocks are inserted before the input and after the output to further enhance the feature representation of the target-related area. The specific implementation method of the enhanced triple attention mechanism is:

[0054] (1) After the feature map is input into the enhanced triple attention mechanism, the feature map is first processed by the first deformable residual block and then divided into three branches.

[0055] (2) In the first branch and the second branch, a rotation operation is performed on the input tensor to change the arrangement of its dimensions so that attention can be allocated according to the importance of features in different dimensions; the rotated tensor is then subjected to dimensionality reduction through a Z-Pool operation; after the Z-Pool operation, the tensor is subjected to dimensionality reduction through a 7×7 convolutional layer, and then the corresponding attention weights are generated through batch normalization and a sigmoid activation function; the generated attention weights are then multiplied by the rotated input tensor, and then a rotation operation is performed.

[0056] In the third branch, a Channel Pool operation is performed on the input tensor; after the Channel Pool operation, the tensor is reduced in dimension through a 7×7 convolution layer, and then the corresponding attention weights are generated through batch normalization and Sigmoid activation function; then the generated attention weights are multiplied with the original input tensor.

[0057] (3) The outputs of the three branches are averaged and then processed by the second deformable residual block to form the final output.

[0058] The PCB module is an offline module that uses a strong classifier based on SENet to correct the category scores output by the RCNN network. The use of a strong classifier based on SENet makes the features extracted by the network more representative, so that a more reliable matching score is obtained by calculating the cosine similarity between the prototype features obtained from the support set and the feature vector of the detected target to effectively correct the classification score output by the classifier. As an offline module, the PCB module can correct the final predicted category score in the inference stage without additional training, effectively alleviating the potential contradiction between classification and localization tasks, thereby improving the performance of small sample target detectors.

[0059] In the model training stage, the back propagation algorithm is used to update the model parameters, and the loss function L of the model is:

[0060] L=L cls +L RPN +L reg

[0061] Among them, L cls is the Faster R-CNN loss, L RPN is the RPN classification loss, L reg is the bounding box regression loss; L cls and L RPN Using cross entropy loss, L reg GIoU loss is used.

[0062] The implementation of related modules is further explained below.

[0063] The ESA module is built on the basis of Deformable Convolution Network (DCN). In order to cope with the irrelevant contextual information that may be brought in complex scenarios, such as the detection of small targets and objects with blurred boundaries, the learned sampling offset may not be accurate enough, and thus be interfered by background features, losing the key features of small targets, resulting in a decrease in the model detection effect. A context conversion module is introduced in the offset learning branch to focus on key features. Combined with the powerful context modeling capability of this module, a more efficient feature representation is provided for convolution offset learning.

[0064] Ordinary convolution can be described by the following formula:

[0065]

[0066] Among them, p0 represents any point in the input feature map, p n represents the relative position of each element in the convolution kernel, w(p0) represents the weight of the corresponding position of the convolution kernel, and x(p0+p n ) represents p0+p on the input feature map n The pixel value at position y(p0) represents the pixel value at position p0 on the output feature map.

[0067] Deformable convolution introduces an offset based on ordinary convolution to achieve dynamic convolution:

[0068]

[0069] Among them, Δp n is the offset, p n Represents the relative position of each element in the convolution kernel, w(p n ) represents the weight of the corresponding position of the convolution kernel, x(p0+p n +Δp n ) represents p0+p on the input feature map n +Δp n The pixel value at position y′(p0) represents the pixel value at position p0 on the output feature map, which is obtained by convolving the weight with the input feature map.

[0070] Δp n It contains the offsets in both horizontal and vertical directions, which are learned through a serial context transition module (CoT) and a 3×3 convolution. The CoT module is as follows Figure 2 As shown, the specific implementation steps are as follows.

[0071] For a feature map input X of size H×W×C, where H represents height, W represents width, and C represents the number of channels. The CoT module first contextually encodes all neighboring keys through a 3x3 convolution operation to obtain a static context representation of the input. This process can be expressed as follows:

[0072] K 1 =Conv 3×3 (X)

[0073] Among them, K 1 is the key feature after convolutional encoding, which reflects the static context information between local neighboring keys.

[0074] Next, the CoT module converts the encoded key K 1 Connected with the input query Q, two consecutive 1×1 convolution operations are performed to learn the dynamic multi-head attention matrix:

[0075]

[0076] Among them, W θ and They are 1×1 convolution kernels with ReLU activation function and without activation function, respectively. This process utilizes the static context K 1 To enhance self-attention learning.

[0077] The feature map is calculated based on the learned attention matrix A and the aggregated values ​​V. The specific formula is as follows:

[0078]

[0079] in, represents a local matrix multiplication operation, which is used to compute the relationship between each query and its corresponding key, k 2 It is a dynamic context representation used to capture the dynamic interactions between input features.

[0080] Finally, the static context K 1 and dynamic context K 2 By fusing the local and global context information, we can obtain output features that have both local and global context information, thereby improving the visual representation ability of the detection model.

[0081] For the enhanced triple attention mechanism introduced in RPN, such as Figure 3As shown in the figure, the specific implementation steps are as follows. For a feature map input X of size H×W×C, where H represents height, W represents width, and C represents the number of channels. First, the rotation operation is used to realize the interaction of features between different dimensions. The first branch rotates 90° counterclockwise along the H axis to obtain X1, whose shape is W×H×C; the second branch rotates 90° counterclockwise along the W axis to obtain X2, whose shape is H×C×W; the third branch uses the original input features.

[0082] Next, each branch performs feature dimensionality reduction through the Z-Pool operation, which concatenates the features after maximum pooling and average pooling along a specific dimension. For example, for the first branch X1, the Z-Pool operation can be expressed as:

[0083] Z-Pool(X1)=[max(X1),avg(X1)]

[0084] Among them, max and avg represent maximum pooling and average pooling operations respectively.

[0085] The output of the Z-Pool operation then passes through a 7x7 convolutional layer and a batch normalization layer, which can be expressed as:

[0086] Y i =BatchNorm(Conv(Z-Pool(X i )))

[0087] Among them, X i Represents the output of the i-th branch after rotation, i∈{1, 2, 3}.

[0088] The output Y of each branch i Generate the attention weight W through a Sigmoid activation layer i :

[0089] W i =σ(Y i )

[0090] Among them, σ represents the Sigmoid activation function.

[0091] The generated attention weights act on the corresponding rotation input X i Then rotate the output back to its original shape:

[0092] X i =Rotate(X i *W i ,90°)

[0093] Finally, the output features of the three branches after applying attention are fused through mean calculation to obtain the final output Y:

[0094]

[0095] like Figure 4 As shown in the figure, the GDL module redefines the feature forward operation and gradient backward operation, uses a learnable affine transformation for forward feature mapping, and restricts the gradient flowing to the feature extraction network in the back propagation, effectively decoupling the RPN and RCNN. The specific calculation is as follows:

[0096] G (A,λ)(X)＝A(X)

[0097]

[0098] Among them, A is the affine transformation layer, λ is a decoupling coefficient, is the Jacobian matrix of the affine layer.

[0099] like Figure 5 As shown in the figure, the core components of PCB include a strong classifier based on ImageNet pre-training, a RoIAlign layer, and a prototype library. In N-way K-shot small sample detection, PCB first inputs the support set into the strong classifier for feature extraction, then inputs the extracted features and image annotation boxes into the RoIAlign layer to generate category prototype features, and collects the prototype features of each category in the support set to build a prototype library. The prototype pc of each category is composed of the average value of the feature vectors of all samples in its support set, and its calculation formula is as follows:

[0100]

[0101] Among them, (x i ,y i ) represents the feature vector x of each sample belonging to category c in the support set S i and the corresponding label y i , S c is the set of samples belonging to category c in the support set. The support set with the final number of categories M is converted into the prototype library

[0102] For each detected object proposal c i is the predicted category, s i is the score of the predicted category, b i is the predicted bounding box coordinate. PCB first uses RoIAlign to extract b i The eigenvector x i , then calculate x i Corresponding category prototype features The cosine similarity between

[0103]

[0104] Finally, the cosine similarity The corrected classification score s is obtained by weighted fusion with the detector prediction category score obtained by training * as follows:

[0105]

[0106] Among them, α is the correction adjustment factor. By changing α, the weight of the category classification score output by the model classifier and the calibration category score output by the PCB can be adjusted.

[0107] Compared with the ResNet101 classifier in the original PCB, SENet154 has a deep network structure and attention-guided feature learning that strengthens the network's feature extraction capabilities, helping to better capture the distinguishing features of categories in a few-sample learning scenario, allowing the model to focus more on features that are more critical to classification decisions, thereby correcting outliers in the prediction scores and improving the model's confidence scores for correct category predictions. In addition, as an offline module, the PCB module can correct the final predicted category scores in the inference phase without additional training, effectively alleviating the potential contradiction between classification and positioning tasks, thereby improving the performance of small-sample target detectors.

[0108] This embodiment also provides an efficient scale-aware small sample target detection system, including a memory, a processor, and computer program instructions stored in the memory and capable of being executed by the processor. When the processor executes the computer program instructions, the above method can be implemented.

[0109] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0110] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0111] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0112] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0113] The above is only a preferred embodiment of the present invention, and does not limit the present invention in other forms. Any technician familiar with the profession may use the above disclosed technical content to change or modify it into an equivalent embodiment with equivalent changes. However, any simple modification, equivalent change and modification made to the above embodiment according to the technical essence of the present invention without departing from the technical solution of the present invention still belongs to the protection scope of the technical solution of the present invention.

Claims

1. An efficient scale-aware small sample target detection method, characterized in that: The steps include: Step 1: Obtain the dataset for model training, divide the dataset into a base class dataset and a new class dataset according to the small sample detection task; construct small sample datasets with different sample numbers from the new class dataset; Step 2: Build an efficient scale-aware small sample target detection model, which includes a feature extraction network reconstructed by an efficient scale-aware module ESA, two gradient decoupling layers GDL, a candidate box generation network RPN with an enhanced triple attention mechanism ETA, a regional convolutional network RCNN, and a classifier and regressor with a prototype calibration module PCB. Step 3: Use a two-stage fine-tuning method based on a transfer learning strategy to train the small sample target detection model; input the sample data in the base class data set into the pre-trained feature extraction network for basic stage training to obtain the basic model and save the basic model parameters; then, set the number of classification categories of the detection head from the number of base class categories to the number of new class categories, and according to different small sample detection task settings, use small sample data sets with different sample numbers to train the basic model to fine-tune the classifier and regressor to obtain a trained small sample target detection model; thus, the prior knowledge extracted from the base class data is transferred to the new class target detection through two-stage training; Step 4: Perform small sample target detection using the trained small sample target detection model.

2. The efficient scale-aware small sample target detection method according to claim 1, characterized in that: Obtain the PASCAL VOC dataset as the dataset for model training; divide the dataset into a base class dataset and a new class dataset in a ratio of 3:1 according to the definition of the small sample detection task; From the new class dataset, i pictures are selected from each category to construct a small sample dataset; i takes different values ​​to obtain small sample datasets with different sample numbers.

3. The efficient scale-aware small sample target detection method according to claim 1, characterized in that: The implementation method of reconstructing the feature extraction network using the efficient scale perception module ESA is as follows: The feature extraction network adopts the ResNet-101 convolutional neural network, and the ESA module is introduced into several residual blocks in the res4 stage of ResNet-101. The 3×3 convolutional layers of several residual blocks in the res4 stage are replaced by the ESA module. When the feature map is input into the residual block fused with the ESA module, it first enters the convolution layer with a convolution kernel of 1×1, and the output feature map then enters the ESA module. The feature map processed by the ESA module then enters the convolution layer with a convolution kernel of 1×1 to obtain the final output of the residual block.

4. The efficient scale-aware small sample target detection method according to claim 1, characterized in that: The ESA module includes a context conversion module, an offset convolution based on 1×1 convolution, and a deformable convolution based on 3×3 convolution. After the feature map is input into the ESA module, it is divided into two parallel branches. In the offset learning branch, the input feature map first obtains global and local context information through the context conversion module, and then extracts the offset in the horizontal and vertical directions through the offset convolution. Finally, the input feature map and the learned offset are simultaneously input into the deformable convolution to achieve efficient dynamic sampling convolution operation.

5. The efficient scale-aware small sample target detection method according to claim 1, characterized in that: Two GDL modules are inserted after the feature extraction network, and are connected to the RPN network and RCNN network respectively through the two GDL modules. The feature forward operation and gradient backward operation are redefined through the GDL modules, and a learnable affine transformation is performed on the forward feature mapping. At the same time, the gradient flowing to the feature extraction network is restricted in the back propagation, so as to achieve effective decoupling of RPN and RCNN.

6. The efficient scale-aware small sample target detection method according to claim 1, characterized in that: In the RPN network, an enhanced triple attention mechanism is introduced to improve the quality of generated candidate boxes; the GDL module outputs a feature map, which is processed by the enhanced triple attention mechanism to obtain an enhanced feature map and input into the RPN network; The enhanced triple attention mechanism divides the input feature map into three branches, each of which processes the feature interaction between different dimensions; the first branch focuses on the interaction between channels and heights, the second branch focuses on the interaction between channels and widths, and the third branch focuses on the spatial dimension; considering the impact of factors such as background noise in the image and variable target scale on the detection accuracy of the model, deformable residual blocks are inserted before the input and after the output to further enhance the feature representation of the target-related areas.

7. The efficient scale-aware small sample target detection method according to claim 6, characterized in that: The RPN network uses the enhanced feature maps and anchor boxes to generate a series of category-independent region proposals for fitting the true bounding box. Then, the RCNN network extracts features from the region proposals and inputs them into the classifier and regressor to obtain the bounding box coordinates and category scores predicted by the model.

8. The efficient scale-aware small sample target detection method according to claim 6, characterized in that: The specific implementation method of the enhanced triple attention mechanism is: (1) After the feature map is input into the enhanced triple attention mechanism, the feature map is first processed by the first deformable residual block and then divided into three branches; (2) In the first and second branches, the input tensor is rotated to change its dimensional arrangement so that attention can be allocated according to the importance of features in different dimensions. The rotated tensor is then reduced in dimension through the Z-Pool operation. After the Z-Pool operation, the tensor is reduced in dimension through a 7×7 convolution layer, and then the corresponding attention weights are generated through batch normalization and sigmoid activation function. The generated attention weights are then multiplied by the rotated input tensor, and then the rotation operation is performed. In the third branch, the Channel Pool operation is performed on the input tensor. After the Channel Pool operation, the tensor is reduced in dimension through a 7×7 convolution layer, and then the corresponding attention weights are generated through batch normalization and sigmoid activation function. Then the generated attention weights are multiplied with the original input tensor. (3) The outputs of the three branches are averaged and then processed by the second deformable residual block to form the final output.

9. The efficient scale-aware small sample target detection method according to claim 1, characterized in that: The PCB module is an offline module that uses a strong classifier based on SENet to correct the category scores output by the RCNN network.

10. The efficient scale-aware small sample target detection method according to claim 1, characterized in that: In the model training stage, the back propagation algorithm is used to update the model parameters, and the loss function L of the model is: L=L cls +L RPN +L reg Among them, L cls is the Faster R-CNN loss, L RPN is the RPN classification loss, L reg is the bounding box regression loss; L cls and L RPN Using cross entropy loss, L reg GIoU loss is used.

Images