An efficient scale-aware small sample target detection method

Abstract

The application relates to a high-efficiency scale perception small sample target detection method, which comprises the following steps: acquiring a data set for model training and dividing the data set into a base class data set and a new class data set; constructing small sample data sets with different sample quantities from the new class data set; constructing a high-efficiency scale perception small sample target detection model, which comprises a feature extraction network reconstructed by using an efficient scale perception module ESA, two gradient decoupling layers GDL, a candidate box generation network RPN introducing a reinforced triple attention mechanism ETA, a region convolution network RCNN, a classifier and a regressor introducing a prototype calibration module PCB; training the small sample target detection model by adopting a two-stage fine-tuning method based on a migration learning strategy; transferring priori knowledge extracted from the base class data to new class target detection through two-stage training; and performing small sample target detection through the trained small sample target detection model. The method is beneficial to improving the precision of small target detection under complex background and scale change.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of target detection, and particularly relates to a high-efficiency scale perception small sample target detection method. BACKGROUND

[0002] Small sample target detection is an important research direction in the field of computer vision, which aims to solve the problem of how to train a detection model with good generalization ability under the condition of limited labeled samples. In practical applications, it is often difficult and costly to obtain a large amount of high-quality labeled data, especially in some specific fields such as military remote sensing detection, disease diagnosis and industrial production of defective product detection. Therefore, the research of small sample target detection technology has important significance for promoting the technological progress of these fields.

[0003] Currently, the research of small sample target detection mainly focuses on the following aspects: meta-learning based methods, transfer learning based methods, data augmentation based methods and metric learning based methods. These methods have their own advantages and disadvantages, and are suitable for different application scenarios.

[0004] Most of the current research methods use transfer learning based methods to adopt the training paradigm of "pre-training-fine-tuning", which directly trains on the dataset. A base class detection model is pre-trained on a large annotated base dataset, and then fine-tuned on a small sample dataset to generalize to new classes. The advantage of this method is that it can fully utilize the rich features learned by the pre-trained model, so as to quickly adapt to new tasks in the fine-tuning stage even with only a small amount of labeled data. This not only reduces the labeling cost, but also speeds up the training of the model. In addition, the generalization ability of the pre-trained model also lays the foundation for the performance of the fine-tuned model, so that it can maintain high detection accuracy when facing new classes.

[0005] However, small sample target detection based on transfer learning also faces some challenges. When the new class is significantly different from the base class, simple fine-tuning may not be enough to effectively transfer knowledge, resulting in unsatisfactory detection performance. In addition, overfitting phenomenon is prone to occur during the fine-tuning process, that is, the model over-reliance on limited labeled data, while ignoring the generalization ability to unseen samples. The DeFRCN algorithm proposed by Qiao et al. analyzes the contradictions existing in the traditional Faster R-CNN from multi-task and multi-stage, through the introduction of gradient decoupling layer (Gradient Decoupling Layer) for multi-level decoupling, and the introduction of prototype calibration block (Prototypical Calibration Block) for multi-task decoupling, thereby expanding the Faster R-CNN. But the ability of DeFRCN algorithm to deal with target scale change and complex background interference still needs to be improved, especially in the case of lack of data set, there is a problem that it is difficult to effectively obtain and utilize the spatial information and perceptual context information of feature maps of different scales, which greatly reduces the detection performance of the model. SUMMARY

[0006] The purpose of the present application is to provide an efficient scale perception small sample target detection method, which is beneficial to improve the precision of small target detection under complex background and scale change.

[0007] In order to achieve the above purpose, the technical scheme adopted by the present application is: an efficient scale perception small sample target detection method, comprising the following steps:

[0008] Step 1: Obtain a data set for model training, divide the data set into base class data set and new class data set according to the small sample detection task; construct small sample data sets with different sample quantities from the new class data set;

[0009] Step 2: Construct an efficient scale perception small sample target detection model, which includes a feature extraction network reconstructed by an efficient scale perception module ESA, two gradient decoupling layers GDL, a candidate box generation network RPN introducing a reinforced triple attention mechanism ETA, a region convolutional network RCNN, a classifier and a regressor introducing a prototype calibration module PCB;

[0010] Step 3: training the small sample target detection model by using a two-stage fine-tuning method based on a transfer learning strategy; inputting sample data in a base class data set into a pre-trained feature extraction network for basic stage training to obtain a base model and save the base model parameters; then, setting the number of classification categories of the detection head from the number of base class categories to the number of new class categories, and according to different small sample detection task settings, using small sample data sets with different sample numbers to train the base model to fine-tune the classifier and the regressor to obtain a trained small sample target detection model; thereby transferring the prior knowledge extracted from the base class data to the target detection of the new class through two-stage training;

[0011] Step 4: performing small sample target detection by using the trained small sample target detection model.

[0012] Further, the PASCAL VOC data set is obtained as the data set for model training; the data set is divided into a base class data set and a new class data set according to a 3:1 ratio according to the definition of the small sample detection task; i images are selected from each category in the new class data set to construct a small sample data set; i takes different values to obtain small sample data sets with different sample numbers.

[0013] Further, the implementation method of the feature extraction network using the efficient scale perception module ESA is as follows:

[0014] The feature extraction network uses a ResNet-101 convolutional neural network, and the ESA module is introduced into a plurality of residual blocks in the res4 stage of the ResNet-101. The 3x3 convolutional layer in the plurality of residual blocks in the res4 stage is replaced by the ESA module. When the feature map is input into the residual block with the ESA module, it is first input into the convolutional layer with a 1x1 convolutional kernel, and the output feature map is then input into the ESA module. The feature map processed by the ESA module is then input into the convolutional layer with a 1x1 convolutional kernel to obtain the final output of the residual block.

[0015] Further, the ESA module includes a context conversion module, an offset convolution based on a 1x1 convolution, and a deformable convolution based on a 3x3 convolution. After the feature map is input into the ESA module, it is divided into two parallel branches. In the offset learning branch, the input feature map first obtains global and local context information through the context conversion module, then extracts the offset in the horizontal and vertical directions through the offset convolution, and finally the input feature map and the learned offset are simultaneously input into the deformable convolution to realize the efficient dynamic sampling convolution operation.

[0016] Further, two GDL modules are inserted after the feature extraction network and connected with the RPN network and the RCNN network respectively through the two GDL modules, the forward feature mapping and the gradient backward operation are redefined through the GDL modules, the learnable affine transformation is performed when the forward feature mapping is performed, and the gradient flowing to the feature extraction network is limited in the backward propagation, so that the RPN and the RCNN are effectively decoupled.

[0017] Further, in the RPN network, a reinforced triple attention mechanism is introduced to improve the quality of the generated candidate frame; the feature map output by the GDL module is processed through the reinforced triple attention mechanism to obtain a reinforced feature map, which is input into the RPN network.

[0018] The reinforced triple attention mechanism divides the input feature map into three branches, each branch processing the feature interaction between different dimensions; the first branch focuses on the interaction between the channel and the height, the second branch focuses on the interaction between the channel and the width, and the third branch focuses on the spatial dimension; considering the influence of factors such as background noise and target scale variation of the image on the detection accuracy of the model, a deformable residual block is inserted before input and after output to further strengthen the feature representation of the target related region.

[0019] Further, the RPN network generates a series of class-independent region proposals for fitting the real bounding box using the reinforced feature map and the anchor frame; then, the RCNN network inputs the region proposals after feature extraction into the classifier and the regressor to obtain the bounding box coordinates and the class score predicted by the model.

[0020] Further, the specific implementation method of the reinforced triple attention mechanism is as follows:

[0021] (1) After the feature map is input into the reinforced triple attention mechanism, the feature map is first processed through the first deformable residual block, and then divided into three branches;

[0022] (2) In the first branch and the second branch, a rotation operation is performed on the input tensor to change the arrangement of its dimensions, so as to allocate attention according to the importance of the features in different dimensions; then, the Z-Pool operation is performed on the rotated tensor to reduce its dimension; after the Z-Pool operation, a 7x7 convolution layer is used to reduce the dimension of the tensor, and then the batch normalization and the Sigmoid activation function are used to generate the corresponding attention weight; then, the generated attention weight is multiplied by the rotated input tensor, and then the rotation operation is performed;

[0023] In the third branch, the Channel Pool operation is performed on the input tensor; after the Channel Pool operation, the tensor is reduced in dimension through a 7*7 convolution layer, and then the corresponding attention weight is generated through batch normalization and Sigmoid activation function; then the generated attention weight is multiplied with the original input tensor;

[0024] (3) The outputs of the three branches are averaged, and then processed through the second deformable residual block to form the final output.

[0025] Further, the PCB module is an offline module, and a strong classifier based on SENet is used to correct the class score output by the RCNN network.

[0026] Further, in the model training stage, the back propagation algorithm is used to update the model parameters, and the loss function L of the model is:

[0027] L = L cls + L RPN + L reg

[0028] Wherein, L cls is the Faster R-CNN loss, L RPN is the RPN classification loss, L reg is the boundary box regression loss; L cls and L RPN use cross-entropy loss, and L reg uses GIoU loss.

[0029] Compared with the prior art, the present application has the following beneficial effects:

[0030] 1、The efficient scale perception small sample target detection method proposed in the present application reconstructs the feature extraction network by using the ESA module, so that the network can adaptively learn targets of different shapes and scales, and the multi-scale feature representation capability of the small sample target detection model is improved.

[0031] 2、The efficient scale perception small sample target detection method proposed in the present application introduces a reinforced triple attention mechanism in the RPN to obtain more representative features, improves the quality of the candidate regions generated by the RPN, and improves the accuracy of the target detection task in the small sample scene.

[0032] 3、The efficient scale perception small sample target detection method proposed in the present application uses a strong classifier based on SENet to correct the classification prediction score of the small sample target detection model, which significantly improves the detection accuracy of the model. BRIEF DESCRIPTION OF DRAWINGS

[0033] Figure 1This is a schematic diagram illustrating the implementation principle of the small sample target detection model in this embodiment of the invention;

[0034] Figure 2 This is a schematic diagram of the context conversion module in an embodiment of the present invention;

[0035] Figure 3 This is a schematic diagram of the enhanced triple attention mechanism in an embodiment of the present invention;

[0036] Figure 4 This is a schematic diagram of the gradient decoupling layer in an embodiment of the present invention;

[0037] Figure 5 This is a schematic diagram of the prototype calibration module in an embodiment of the present invention. Detailed Implementation

[0038] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0039] It should be noted that the following detailed descriptions are exemplary and intended to provide further explanation of this application. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains.

[0040] It should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the exemplary embodiments according to this application. As used herein, the singular form is intended to include the plural form as well, unless the context clearly indicates otherwise. Furthermore, it should be understood that when the terms "comprising" and / or "including" are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof.

[0041] This embodiment provides an efficient scale-aware small-sample target detection method, including the following steps:

[0042] Step 1: Obtain the dataset for model training, and divide the dataset into base class dataset and new class dataset according to the few-shot detection task; construct few-shot datasets with different numbers of samples from the new class dataset.

[0043] Step 2: Construct an efficient scale-aware small-sample target detection model, such as... Figure 1 As shown, the model includes a feature extraction network reconstructed using an efficient scale-aware module (ESA), two gradient decoupling layers (GDL), a candidate box generation network (RPN) with an enhanced triple attention mechanism (ETA), a region convolutional network (RCNN), and a classifier and regressor with a prototype calibration module (PCB).

[0044] Step 3: training the small sample target detection model by using a two-stage fine-tuning method based on a transfer learning strategy; using a feature extraction network pre-trained on ImageNet as a pre-trained model, inputting sample data in a base class data set into the pre-trained model for basic stage training, obtaining a base model and saving the base model parameters; then, setting the number of classification categories of the detection head from the base class category number to the new class category number, training the base model by using small sample data sets with different sample numbers according to different small sample detection task settings to fine-tune the classifier and the regressor in the last layer, and obtaining the trained small sample target detection model.

[0045] Step 4: performing small sample target detection by using the trained small sample target detection model.

[0046] In this embodiment, the PASCAL VOC data set is obtained as a data set for model training; the data set is divided into a base class data set and a new class data set according to a 3:1 ratio according to the definition of a small sample detection task; i pictures are selected from each category in the new class data set to construct a small sample data set; i takes different values (in this embodiment, i takes 1, 2, 3, 5, and 10), and small sample data sets with different sample numbers are obtained. In order to evaluate the robustness and generalization of the model, different division methods can be used to divide the base class data set and the new class data set, and then different small sample data sets are constructed.

[0047] In this embodiment, the small sample target detection model is based on the Faster R-CNN detection framework, and the prior knowledge extracted from the base class data is transferred to the target detection of the new class by two-stage training in the manner of transfer learning. The feature extraction network is ESA-ResNet-101 pre-trained on ImageNet and combined with an ESA module.

[0048] The implementation method of the feature extraction network using the efficient scale perception module ESA is as follows:

[0049] The feature extraction network uses a ResNet-101 convolutional neural network, and the ESA module is introduced into a plurality of residual blocks in the res4 stage of the ResNet-101. The 3x3 convolutional layer in the plurality of residual blocks in the res4 stage is replaced by the ESA module. When the feature map is input into the residual block with the ESA module, it is first input into the convolutional layer with a 1x1 convolutional kernel, and the output feature map is then input into the ESA module. The feature map processed by the ESA module is then input into the convolutional layer with a 1x1 convolutional kernel to obtain the final output of the residual block.

[0050] Specifically, the ESA module includes a context conversion module, an offset convolution implemented based on a 1x1 convolution, and a deformable convolution implemented based on a 3x3 convolution. After the feature map is input into the ESA module, the feature map is divided into two parallel branches. In the offset learning branch, the input feature map first obtains global and local context information through the context conversion module, then extracts offsets in the horizontal and vertical directions through the offset convolution, and finally the input feature map and the learned offsets are simultaneously input into the deformable convolution to realize efficient dynamic sampling convolution operation.

[0051] Two GDL modules are inserted after the feature extraction network and connected with the RPN network and the RCNN network through the two GDL modules. The GDL modules redefine the forward feature mapping and the gradient backward operation. The forward feature mapping is performed through the learnable affine transformation, and the gradient flowing to the feature extraction network is limited in the backward propagation, thereby effectively decoupling the RPN and the RCNN.

[0052] In the RPN network, a reinforced triple attention mechanism is introduced to improve the quality of the generated candidate frame. The GDL module outputs a feature map, which is processed by the reinforced triple attention mechanism to obtain a reinforced feature map, which is input into the RPN network. The RPN network generates a series of class-independent region proposals for fitting the real boundary frame by using the reinforced feature map and the anchor frame. Then, the RCNN network extracts features from the region proposals, inputs the features into the classifier and regressor, obtains the model predicted boundary frame coordinates and class score, and corrects the class score through the PCB module. The overall improvement improves the detection performance and generalization of the model.

[0053] The reinforced triple attention mechanism divides the input feature map into three branches, and each branch processes the feature interaction between different dimensions. The first branch focuses on the interaction between the channel and the height, the second branch focuses on the interaction between the channel and the width, and the third branch focuses on the spatial dimension. Considering the influence of factors such as background noise and target scale variation of the image on the detection accuracy of the model, a deformable residual block is inserted before input and after output to further strengthen the feature representation of the target related region. The specific implementation method of the reinforced triple attention mechanism is as follows:

[0054] (1) After the feature map is input into the reinforced triple attention mechanism, the feature map is first processed through the first deformable residual block, and then divided into three branches.

[0055] (2) In the first branch and the second branch, a rotation operation is performed on the input tensor to change the arrangement of its dimensions, so as to allocate attention according to the importance of features in different dimensions; then the rotated tensor is processed by Z-Pool operation for dimension reduction; after the Z-Pool operation, the tensor is processed by a 7x7 convolution layer for dimension reduction, and then the batch normalization and Sigmoid activation function are used to generate the corresponding attention weight; then the generated attention weight is multiplied with the input tensor after rotation, and then the rotation operation is performed.

[0056] In the third branch, a Channel Pool operation is performed on the input tensor; after the Channel Pool operation, the tensor is processed by a 7x7 convolution layer for dimension reduction, and then the batch normalization and Sigmoid activation function are used to generate the corresponding attention weight; then the generated attention weight is multiplied with the original input tensor.

[0057] (3) The outputs of the three branches are averaged, and then processed by the second deformable residual block to form the final output.

[0058] The PCB module is an offline module, which uses a SENet-based strong classifier to correct the class score output by the RCNN network. The SENet-based strong classifier makes the features extracted by the network more representative, so that a more reliable matching score is obtained by calculating the cosine similarity between the prototype features obtained from the support set and the feature vector of the detection target, thereby strongly correcting the classification score output by the classifier. The PCB module is an offline module, which can correct the final prediction class score in the inference stage without additional training, effectively alleviating the potential contradiction between classification and positioning tasks, and thereby improving the performance of the small sample target detector.

[0059] In the model training stage, the back propagation algorithm is used to update the model parameters, and the loss function L of the model is:

[0060] L=L cls +L RPN +L reg

[0061] Wherein, L cls is the Faster R-CNN loss, L RPN is the RPN classification loss, and L reg is the bounding box regression loss; L cls and L RPN use cross-entropy loss, and L reg uses GIoU loss.

[0062] The implementation of the related modules is further described below.

[0063] The ESA module is constructed based on a deformable convolution (DCN). In order to cope with irrelevant context information that may be brought by the ESA module in a complex scene, for example, for small target and boundary fuzzy object detection, the learned sampling offset may not be accurate enough, so as to be disturbed by background features, lose the key features of small targets, and cause the model detection effect to decrease. A context conversion module is introduced in the offset learning branch to focus on key features, and the powerful context modeling capability of the module provides more efficient feature representation for convolution offset learning.

[0064] The ordinary convolution can be described by the following formula:

[0065]

[0066] wherein p0 represents any point in the input feature map, p n represents the relative position of each element in the convolution kernel, w(p0) represents the weight of the corresponding position of the convolution kernel, x(p0+p n ) represents the pixel value at the p0+p n position on the input feature map, and y(p0) represents the pixel value at the p0 position on the output feature map.

[0067] The deformable convolution introduces an offset to realize dynamic convolution on the basis of the ordinary convolution:

[0068]

[0069] wherein Δp n is the offset, p n represents the relative position of each element in the convolution kernel, w(p n ) represents the weight of the corresponding position of the convolution kernel, x(p0+p n + Δp n ) represents the pixel value at the p0+p n + Δp n position on the input feature map, and y'(p0) represents the pixel value at the p0 position on the output feature map, which is obtained by convolution of the weight and the input feature map.

[0070] Δp n contains the offset in the horizontal and vertical directions, which is learned through a serial context conversion module (CoT) and a 3x3 convolution. The CoT module is as shown in Figure 2 , and the specific implementation steps are as follows.

[0071] For an input X with a size of HxWxC, where H represents the height, W represents the width, and C represents the number of channels. The CoT module first encodes all the neighboring keys by a 3x3 convolution operation to obtain the static context representation of the input. This process can be represented by the following formula:

[0072] K 1 =Conv 3×3 (X)

[0073] where K 1 is the key feature after convolution encoding, which reflects the static context information between the local neighboring keys.

[0074] Next, the CoT module connects the encoded key K 1 with the input query (Q) to learn a dynamic multi-head attention matrix through two consecutive 1x1 convolution operations:

[0075]

[0076] where W θ and are 1x1 convolution kernels with ReLU activation function and no activation function, respectively. This process utilizes the static context K 1 to strengthen the self-attention learning.

[0077] According to the learned attention matrix A and the aggregation of all values V, the feature mapping is calculated, and the specific formula is as follows:

[0078]

[0079] where represents a local matrix multiplication operation for calculating the relationship between each query and its corresponding key, and k 2 is the dynamic context representation for capturing the dynamic interaction between input features.

[0080] Finally, the static context K 1 and the dynamic context K 2 are fused to obtain output features that have both local and global context information, improving the visual representation ability of the detection model.

[0081] For the reinforced triple attention mechanism introduced in the RPN, such as Figure 3As shown, the specific implementation steps are as follows. For an input X of a feature map of size HxWxC, where H represents height, W represents width, and C represents the number of channels. First, the feature interaction between different dimensions is realized through a rotation operation. The first branch is rotated counterclockwise by 90° along the H axis to obtain X1, which has a shape of WxHxC; the second branch is rotated counterclockwise by 90° along the W axis to obtain X2, which has a shape of HxCxW; and the third branch uses the original input feature.

[0082] Next, each branch performs feature dimension reduction through a Z-Pool operation. The Z-Pool operation concatenates the features after maximum pooling and average pooling along a specific dimension. For example, for the first branch X1, the Z-Pool operation can be represented as:

[0083] Z-Pool(X1) = [max(X1), avg(X1)]

[0084] where max and avg represent the maximum pooling and average pooling operations, respectively.

[0085] The output obtained by the Z-Pool operation is then passed through a 7x7 convolution layer and a batch normalization layer, which can be represented as:

[0086] Y i = BatchNorm(Conv(Z-Pool(X i )))

[0087] where X i i represents the output of the i-th branch after rotation, i ∈ {1, 2, 3}.

[0088] The output Y i of each branch is passed through a Sigmoid activation layer to generate attention weights W i :

[0089] W i = σ(Y i )

[0090] where σ represents the Sigmoid activation function.

[0091] The generated attention weights act on the corresponding rotated input X i , and the output is rotated back to the original shape:

[0092] X i = Rotate(X i *W i , 90°)

[0093] Finally, the output features of the three branches after applying attention are fused through mean calculation to obtain the final output Y:

[0094]

[0095] like Figure 4 As shown, the GDL module redefines the feature forward operation and gradient backward operation, using a learnable affine transformation for the forward feature mapping, and restricting the gradient flowing to the feature extraction network during backpropagation, effectively decoupling RPN and RCNN. The specific calculations are as follows:

[0096] G (A,λ)(X)＝A(X)

[0097]

[0098] Where A is the affine transformation layer, and λ is a decoupling coefficient. It is the Jacobian matrix of the affine layer.

[0099] like Figure 5 As shown, the core components of PCB include a strong classifier pre-trained on ImageNet, a RoIAlign layer, and a prototype library. In N-way K-shot few-shot detection, PCB first inputs the support set into the strong classifier for feature extraction. Then, the extracted features and image bounding boxes are input into the RoIAlign layer to generate category prototype features. The prototype features of each category in the support set are collected to construct the prototype library. The prototype pc for each category is composed of the average of the feature vectors of all samples in its support set, and its calculation formula is as follows:

[0100]

[0101] Among them, (x i ,y i Let x represent the feature vector x of each sample belonging to class c in the support set S. i and the corresponding label y i S c It supports the set of samples belonging to category c. The final support set with M categories was converted into a prototype library.

[0102] For each detected object proposal c i It is the predicted category, s i It is the score of the predicted category, b i These are the predicted bounding box coordinates. The PCB first uses RoIAlign to extract the bounding box coordinates. i eigenvector x i Then calculate x i Corresponding category prototype features Cosine similarity between

[0103]

[0104] Finally, the obtained cosine similarity The corrected classification score s is obtained by weighted fusion with the predicted class scores of the detector obtained during training. * as follows:

[0105]

[0106] Here, α is the calibration adjustment factor, which can be changed to adjust the weight of the class classification score output by the model classifier and the calibration class score output by the PCB.

[0107] Compared to the ResNet101 classifier in the original PCB, SENet154's deep network structure and attention-guided feature learning enhance the network's feature extraction capabilities. This helps to better capture discriminative features of categories in few-shot learning scenarios, allowing the model to focus more on features crucial for classification decisions. This corrects outliers in predicted scores and improves the model's confidence score for correct category predictions. Furthermore, as an offline module, the PCB module can correct the final predicted category score during the inference stage without additional training, effectively mitigating the potential conflict between classification and localization tasks, thereby improving the performance of few-shot object detectors.

[0108] This embodiment also provides a high-efficiency scale-aware small-sample target detection system, including a memory, a processor, and computer program instructions stored in the memory and executable by the processor. When the processor executes the computer program instructions, it can implement the above-described method.

[0109] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0110] The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart block or blocks. Figure 1 one or more flow or blocks Figure 1 one or more flow or blocks

[0111] The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart block or blocks. Figure 1 one or more flow or blocks Figure 1 one or more flow or blocks

[0112] The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart block or blocks. Figure 1 one or more flow or blocks Figure 1 one or more flow or blocks

[0113] The above description is only preferred embodiments of the present application, and is not intended to limit the present application to other forms described above. Any person skilled in the art can make modifications or improvements to the above-mentioned disclosed technical content without departing from the technical scope of the present application. Any simple modification, equivalent change and modification made according to the technical essence of the present application to the above-mentioned embodiments still belongs to the protection scope of the present application.

Claims

1. A highly efficient scale-aware small-sample target detection method, characterized in that, Includes the following steps: Step 1: Obtain the dataset for model training, and divide the dataset into a base class dataset and a new class dataset according to the few-shot detection task; construct few-shot datasets with different numbers of samples from the new class dataset; Step 2: Construct an efficient scale-aware small sample target detection model, which includes a feature extraction network reconstructed using an efficient scale-aware module ESA, two gradient decoupling layers GDL, a candidate box generation network RPN with an enhanced triple attention mechanism ETA, a region convolutional network RCNN, and a classifier and regressor with a prototype calibration module PCB. Step 3: A two-stage fine-tuning method based on transfer learning is used to train the few-shot object detection model. Sample data from the base class dataset is input into the pre-trained feature extraction network for basic training, resulting in a basic model whose parameters are saved. Then, the number of categories in the detection head is changed from the number of base class categories to the number of new class categories. Depending on the specific few-shot detection task, different numbers of samples from the few-shot dataset are used to train the basic model, fine-tuning the classifier and regressor to obtain a trained few-shot object detection model. Thus, through two-stage training, the prior knowledge extracted from the base class data is transferred to the detection of new class objects. Step 4: Perform few-shot object detection using the trained few-shot object detection model; The implementation method of reconstructing the feature extraction network using the efficient scale-aware module ESA is as follows: The feature extraction network uses a ResNet-101 convolutional neural network. The ESA module is introduced into several residual blocks in the res4 stage of ResNet-101. The 3×3 convolutional layers of several residual blocks in the res4 stage are replaced with the ESA module. When the feature map is input into the residual block fused with the ESA module, it first enters a convolutional layer with a 1×1 kernel. The output feature map then enters the ESA module. The feature map after being processed by the ESA module then enters a convolutional layer with a 1×1 kernel to obtain the final output of the residual block. The ESA module includes a context transformation module, offset convolution based on 1×1 convolution, and deformable convolution based on 3×3 convolution. After the feature map is input into the ESA module, it is divided into two parallel branches. In the offset learning branch, the input feature map first obtains global and local context information through the context transformation module, then extracts the offsets in the horizontal and vertical directions through offset convolution, and finally the input feature map and the learned offsets are simultaneously input into the deformable convolution to achieve efficient dynamic sampling convolution operation. In the RPN network, an enhanced triple attention mechanism is introduced to improve the quality of generated candidate boxes; the feature map output by the GDL module is processed by the enhanced triple attention mechanism to obtain an enhanced feature map, which is then input into the RPN network. The enhanced triple attention mechanism divides the input feature map into three branches, each branch handling feature interactions between different dimensions; the first branch focuses on the interaction between channels and height, the second branch focuses on the interaction between channels and width, and the third branch focuses on spatial dimensions; considering the impact of background noise and variable target scale on the model's detection accuracy, deformable residual blocks are inserted before input and after output to further enhance the feature representation of the target-related region; The specific implementation method of the enhanced triple attention mechanism is as follows: (1) After the feature map input is enhanced by the triple attention mechanism, the feature map is first processed by the first deformable residual block and then divided into three branches; (2) In the first and second branches, the input tensor is rotated to change its dimensionality arrangement so that attention is allocated according to the importance of features in different dimensions; then the rotated tensor is reduced in dimension by Z-Pool operation; after Z-Pool operation, the tensor is reduced in dimension by a 7×7 convolutional layer, and then the corresponding attention weights are generated by batch normalization and sigmoid activation function; then the generated attention weights are multiplied by the rotated input tensor, and then the rotation operation is performed. In the third branch, a Channel Pool operation is performed on the input tensor. After the Channel Pool operation, the tensor is reduced in dimensionality by a 7×7 convolutional layer, and then the corresponding attention weights are generated by batch normalization and the Sigmoid activation function. Finally, the generated attention weights are multiplied by the original input tensor. (3) The outputs of the three branches are averaged and then processed by the second deformable residual block to form the final output.

2. The efficient scale-aware small-sample target detection method according to claim 1, characterized in that, Obtain the PASCAL VOC dataset as the training dataset for the model; according to the definition of few-shot detection task, divide the dataset into base class dataset and new class dataset in a 3:1 ratio; From the new class dataset, select i images for each class to construct a small sample dataset; take different values ​​for i to obtain small sample datasets with different sample sizes.

3. The efficient scale-aware small-sample target detection method according to claim 1, characterized in that, Two GDL modules are inserted after the feature extraction network and connected to the RPN and RCNN networks respectively. The feature forward operation and gradient backward operation are redefined through the GDL modules. Learnable affine transformation is performed during the forward feature mapping, and the gradient flowing to the feature extraction network is restricted during backpropagation, thus achieving effective decoupling of RPN and RCNN.

4. The efficient scale-aware small-sample target detection method according to claim 1, characterized in that, The RPN network uses enhanced feature maps and anchor boxes to generate a series of class-independent region proposals to fit the true bounding boxes. Then, the RCNN network extracts features from the region proposals and inputs them into the classifier and regressor to obtain the bounding box coordinates and class scores predicted by the model.

5. The efficient scale-aware small-sample target detection method according to claim 1, characterized in that, The PCB module is an offline module that uses a strong classifier based on SENet to correct the class scores output by the RCNN network.

6. The efficient scale-aware small-sample target detection method according to claim 1, characterized in that, During the model training phase, the backpropagation algorithm is used to update the model parameters, and the model's loss function L is: L=L cls +L RPN +L reg Among them, L cls For Faster R-CNN loss, L RPN For RPN classification loss, L reg For bounding box regression loss; L cls and L RPN Using cross-entropy loss, L reg GIoU loss is used.

Images