Small-sample target detection method based on coordinate attention group optimization

Abstract

The present invention discloses a small sample target detection method based on coordinate attention group optimization. The method of the present invention first samples the image data samples to obtain a support set and a query set, and uses a deep visual feature extractor to obtain a support feature map set and a query feature map set; then the support feature map set and the initialized target parameter matrix set are input into a coordinate attention guidance module to obtain a target fusion matrix set; then a group optimization module is constructed, and the support feature map set is used to obtain an updated support target class vector set through the target true bounding box and label propagation algorithm; finally, the above-obtained set is input into a query target prediction module to obtain the bounding box and category of the query set sample target. The method of the present invention uses the coordinate attention mechanism to enable the detector to adaptively focus on the area where the target is located in the query set, and at the same time obtains a discriminative support target class vector through the group optimization module, thereby improving the performance of small sample target detection.

Description

Technical Field

[0001] The present invention belongs to the field of computer technology, in particular to the field of computer vision technology, and relates to target detection in small sample scenarios, and specifically to a small sample target detection method based on coordinate attention group optimization. Background Art

[0002] Object detection is a basic computer vision task that aims to locate and identify people or objects of interest from images. Traditional methods obtain target features through sliding window mechanisms and manual feature extraction algorithms such as Scale-Invariant Feature Transform (SIFT) and Histogram of Oriented Gradient (HOG), and then use support vector machines to predict target categories. In the past decade, deep learning methods based on convolutional neural networks have been widely used in the field of object detection, significantly improving the performance of object detection. However, deep learning methods usually require a large amount of labeled data to train models, which makes it difficult to directly apply to scenarios with scarce data. At the same time, manually annotating the bounding boxes and categories of objects in images requires a lot of labor costs, which greatly limits the promotion and application of deep learning methods in the industry. To solve the above problems of data shortage and labor costs, researchers proposed the task of small sample object detection. The small sample object detection model aims to detect similar objects in unlabeled images based on a small number of labeled images. This method can be applied to scenarios such as industrial product defect detection and rare animal detection, while reducing the cost of manually annotated data. For example, industrial product defect detection usually requires a large number of defective product samples to train the model. However, the proportion of defective products in actual production scenarios is very small, and it is difficult to collect enough qualified samples, resulting in insufficient model training and thus model overfitting. The small sample target detection model can detect defects on the product surface through a small number of defective samples, avoiding the above problems.

[0003] At present, most small sample target detection methods use the two-stage faster region convolution neural network Faster-RCNN (Faster Region-Convolution Neural Network) model as the detection framework, and use meta-learning to train. During data preprocessing, the small sample method divides the data set into a training set and a test set according to the target category in the image, that is, the target categories in the training set and the test set have no intersection. Among them, the target category in the training set is called the base class, and the target category in the test set is called the novel class. During training, at each iteration optimization, the model randomly selects some samples from the training set containing the base class target as the support set, and then randomly selects some remaining samples containing the same type of target as the query set. Among them, the support set is a set of samples with known annotations, and the model needs to use a small number of support samples to detect the target in the query sample. The support set and the query set constitute a data set in a small sample target detection task. These methods usually adopt a dual-branch structure, and the support set and the query set are input into the backbone network with shared parameters to obtain the support feature map and the query feature map. Subsequently, the query feature map is converted into the query target feature map through the candidate box output by the Region Proposal Network (RPN), while the support feature map is converted into the support target class feature vector through the global pooling operation. Finally, the support target class feature vector and the query target feature map are fused through the attention module to obtain the fused feature map, and then the fused feature map is input into the detector to obtain the target bounding box and category of the query set. During training, the model learns task-independent prior knowledge based on the small sample target detection task dataset generated in each iteration, so that it can quickly adapt to new detection tasks in practical applications. After training, the model detects similar targets in other unlabeled samples through a small number of labeled samples containing new class targets.

[0004] The shortcomings of the above methods are mainly manifested in three aspects: 1) Using a two-stage detection framework, the positioning of new class targets depends on the generated candidate boxes, and the model is only trained on base class samples. If the new class is very different from the base class, it is difficult to output a suitable candidate box to locate the new class target; 2) The use of global pooling operations makes the model easy to lose the spatial position characteristics of the target, which is not conducive to the accurate positioning of the target; 3) Each sample in the support set is treated as an independent individual, and the similarity relationship between different targets in the support set in the specific detection task is not fully considered, which is easy to cause misclassification when the targets of different categories are similar. In summary, in order to alleviate the problem that the existing methods are difficult to capture the spatial position characteristics of the support targets and the similarity of targets of different categories, resulting in inaccurate classification, there is an urgent need for a method that does not rely on candidate boxes but locates the targets of the query set according to the spatial position of the targets in the support set, and uses the similarity relationship between the targets in the support set to identify the targets in the query set. Summary of the invention

[0005] The purpose of the present invention is to provide a small sample target detection method based on coordinate attention group optimization in view of the shortcomings of the prior art. The method of the present invention does not need to generate candidate boxes in advance, but captures the key spatial position features of the support set targets from the horizontal and vertical directions of the feature map through coordinate attention, guides the detector to focus on the area containing the target in the query set, thereby improving the accuracy of target positioning; at the same time, a group optimization module is used to update the support target class vector, and the similarity relationship between different targets in the support set is used to obtain more discriminative class features, thereby improving the accuracy of target recognition.

[0006] The method of the present invention first obtains an image data set containing target bounding boxes and category annotations, and then performs the following operations:

[0007] Step (1) sampling the image data set to obtain a support set and a query set, inputting the two into a deep visual feature extraction module, and outputting a support feature map set and a query feature map set;

[0008] Step (2) constructs a coordinate attention guidance module, the input of which is a randomly initialized target parameter matrix set and a support feature map set, and the output is a target fusion matrix set;

[0009] Step (3) constructing a group optimization module, the input of which is the support feature map set and the bounding box annotation corresponding to the support set, and the output is the support target class vector set;

[0010] Step (4) constructing a query target prediction module, the input of which is a query feature map set, a target fusion matrix set and a support target class vector set, and the output is a predicted query target bounding box and class probability;

[0011] Step (5) uses a stochastic gradient descent algorithm to optimize a small sample target detection model consisting of a coordinate attention guidance module, a group optimization module, and a query target prediction module, and obtains the target bounding box and category of the image in the query set through steps (1) to (4) for the new support set and query set.

[0012] Further, step (1) is specifically:

[0013] (1-1) First, scale the images in the dataset to the same size and perform random sampling without replacement to obtain the support set. and queryset in, is the field of real numbers, N s represents the number of image samples in the support set, represents the i-th support image sample, N q represents the number of image samples in the query set, represents the j-th query image sample, H represents the image height, W represents the image width, and 3 represents the number of RGB channels;

[0014] Each supporting image sample With annotation Among them, Φ i express The number of targets in, c i,φ ∈{1,…,C} represents The category of the φth target in , express The four-dimensional vector consisting of the coordinates of the upper left corner and lower right corner of the φth target bounding box in the support set The CPC contains CK targets, CK = C × K, that is, C represents the number of target categories, and K represents the number of targets in each category;

[0015] (1-2) Construct a deep visual feature extraction module consisting of a deep convolutional network and a two-dimensional convolutional layer, where the deep convolutional network is a 50-layer residual network ResNet-50 pre-trained on the ImageNet dataset, and the convolution kernel size of the two-dimensional convolutional layer is 1×1;

[0016] (1-3) will support the set and queryset Input into the deep visual feature extraction module to obtain a set of supporting feature maps and query feature graph collection in, represents the i-th support feature map, represents the j-th query feature map, H′, W′ and 256 represent the height, width and number of channels of a single feature map respectively.

[0017] Furthermore, step (2) is specifically:

[0018] (2-1) Random initialization to obtain the target parameter matrix set A j represents the target parameter matrix corresponding to the j-th query image sample, M represents the number of targets to be detected in a single query image sample, and 256 represents the row vector dimension in a single target parameter matrix; constructs a coordinate attention guidance module, the coordinate attention guidance module consists of a coordinate attention submodule and a cross attention submodule, wherein the coordinate attention submodule calculates the weight of each coordinate position from the horizontal and vertical directions of the feature map to obtain a set of spatial position attention feature maps, and the cross attention submodule is used to fuse the target parameter matrix set with the set of spatial position attention feature maps;

[0019] (2-2) For the i-th support feature map Perform average pooling operations along the horizontal coordinate direction and the vertical coordinate direction to obtain the horizontal feature map and vertical feature map Representing a tensor The value at the coordinate position (h′, w′) in the μth channel, 1≤μ≤256, 1≤h′≤H′, 1≤w′≤W′, then: The value at the coordinate position (h′,1) in the μth channel Tensor The value at the coordinate position (1,w′) in the μth channel

[0020] (2-3) The horizontal feature map and vertical feature map Input the two-dimensional convolution layer and activation function layer in sequence to obtain the horizontal weight feature map And vertical weight feature map Among them, Conv 1 (·) represents a two-dimensional convolutional layer with a kernel size of 1×1, and σ(·) represents the Sigmoid activation function;

[0021] (2-4) According to the support feature map Horizontal weight feature map And vertical weight feature map Calculate the spatial position attention feature map Tensor The value at coordinate (h′, w′) in the μth channel Representing a tensor The value at the coordinate position (h′,1) in the μth channel, Representing a tensor The value at the coordinate position (1, w′) in the μth channel;

[0022] (2-5) Support feature map set All feature maps in Execute steps (2-2) to (2-4) to obtain a set of spatial position attention feature maps Will gather All feature maps in Expand and concatenate according to the spatial dimension to obtain the spatial target position feature

[0023] (2-6) Each target parameter matrix A j and spatial target location characteristics Input into the cross attention submodule to obtain the target fusion matrix set Softmax(·) is a normalized exponential function, and the superscript T indicates the transposition operation.

[0024] Furthermore, step (3) is specifically:

[0025] (3-1) Construct a group optimization module. A group is a set of support targets belonging to the same category. The support target probability matrix and similarity matrix are calculated based on the support target class labels and the obtained support target vector set, and the group propagation matrix is ​​obtained through the support target probability matrix and the similarity matrix.

[0026] (3-2) Set the support feature map obtained in step (1-3) And each support image sample obtained in step (1-1) Corresponding annotation Each bounding box b in i,φ Input into the region of interest pooling layer to obtain a set of supporting target feature maps The region of interest pooling layer represents the maximum pooling operation of the feature map in the corresponding target bounding box area. c,k Represents the feature map of the kth target in the cth class in the set;

[0027] (3-3) Support target feature map set Through convolution and global average pooling operations in sequence, we get the set of support target vectors. Among them, the feature vector of the kth target in the cth class in the set GAP(·) represents the global average pooling operation in the feature map space dimension, Conv 2(·) represents a 2D convolutional layer with a kernel size of 3×3;

[0028] (3-4) Calculate the set of support target class vectors The support target probability matrix is ​​obtained by supporting the target class vector set. Among them, the support target class vector of the cth class in the set The value of the u-th row and v-th column of the matrix P represents the probability that the u-th support vector belongs to the v-th class. 1≤u≤CK, 1≤v≤C, u=(c-1)×K+k, Representing a collection The u-th support target vector in exp(·) represents the exponential function with the natural constant e as the base, dist(·,·) represents the Euclidean distance function, then

[0029] (3-5) Use the Gaussian kernel function to calculate the similarity matrix between support target vectors Construct the group propagation matrix through the similarity matrix Z and the support target probability matrix P Among them, the value of the uth row and wth column of the matrix Z represents the support target vector and Similarity between 1≤w≤CK, the hyperparameter γ in the Gaussian kernel function is greater than 0, Representing a collection The w-th supporting target vector in , the group propagation matrix Λ=ZP, the value Λ in the u-th row and v-th column of Λ u,v represents the support target vector The probability of belonging to the vth class is obtained by weighted summation of similarities between samples;

[0030] (3-6) Initialize the iteration upper limit value Ψ, iteratively optimize the support target probability matrix P through the group propagation matrix Λ, the similarity matrix Z and the label propagation algorithm, until the number of iterations reaches the upper limit value Ψ, and obtain the updated support target probability matrix P (Ψ) , at the initial iteration P (0) =P,Λ (0) =Λ, the label propagation algorithm iteration is 100≤ψ≤120, and Indicates The support target probability matrix and group propagation matrix at the iteration, and Representation Matrix The value and matrix of row u and column v The value of row u and column v, and Representation Matrix The value of the uth row and vth column and the value of the uth row and cth column;

[0031] (3-7) Using the updated support target probability matrix P (Ψ) and the set of support target vectors Calculate the new set of support target class vectors Among them, the support target class vector of the cth class in the set P (Ψ) represents the target probability matrix after the Ψth iteration, is the matrix P (Ψ) The value of the uth row and cth column indicates the uth support target vector after the update The probability of belonging to class c.

[0032] Furthermore, step (4) is specifically:

[0033] (4-1) Constructing a query target prediction module, which consists of a converter submodule, a target classification function, and a bounding box prediction submodule;

[0034] Query the feature graph collection Each feature map is expanded along the spatial dimension to obtain the query feature matrix set Calculate the position encoding matrix of the query sample The value of the κth row and ωth column of the matrix G 1≤κ≤H′×W′, 1≤ω≤256, mod means remainder operation;

[0035] (4-2) Set the query feature matrix And the position encoding matrix E is input into the encoder of the converter submodule to obtain the query target encoding feature set The encoder consists of an attention layer and two fully connected layers. The j-th query target encodes features represents the element-by-element addition operation, and FFN(·) represents a feed-forward neural network consisting of two fully connected layers;

[0036] (4-3) Set the query target encoding feature set And the target fusion matrix set obtained in step (2-6) Input into the decoder of the converter submodule to obtain the query target decoding feature set The decoder consists of two attention layers and two fully connected layers. The j-th query target decoding feature express The intermediate result matrix obtained by inputting into the first attention layer,

[0037] (4-4) Decoding feature sets by querying the target And the set of support target class vectors obtained in steps (3-6) Calculate the target prediction category probability of the query set, the probability that the mth target in the jth query sample belongs to the cth category 1≤m≤M, Representation Matrix The vector of the mth row;

[0038] (4-5) Decode the query target feature set Input to the bounding box prediction submodule to obtain the predicted target bounding box set of the query set The bounding box prediction submodule is a multilayer perceptron consisting of three fully connected layers. The predicted bounding box matrix of the jth query sample is The mth row represents the predicted bounding box of the mth target in the jth query sample, is the coordinate of the upper left corner of the bounding box, The coordinates of the lower right corner of the bounding box.

[0039] Furthermore, step (5) is specifically:

[0040] (5-1) Predicting category probabilities through the target of the query set And the cross entropy loss function calculates the target classification loss y j,m,c ∈{0,1} indicates the true label value that the mth target in the jth query sample belongs to the cth class;

[0041] (5-2) Predicted target bounding box set through query set Calculating bounding box loss B j,m represents the true bounding box of the mth target in the jth query sample, Represents the intersection-over-union ratio of the true bounding box and the predicted bounding box;

[0042] (5-3) According to the target classification loss and bounding box loss Get the total loss The small sample target detection model composed of the coordinate attention guidance module, the group optimization module and the query target prediction module is optimized by using the stochastic gradient descent algorithm, and the model is iteratively trained until convergence to obtain the optimized small sample target detection model;

[0043] (5-4) Sampling the new image dataset to obtain the support set and queryset Input the optimized small sample target detection model and execute steps (1) to (4) in sequence to output the query set The target category probability of the image sample in and bounding box collection The target class index with the maximum probability is selected as the predicted class.

[0044] The small sample target detection method based on coordinate attention group optimization proposed in the present invention has the following characteristics: 1) the converter module is directly used to detect the targets in the query set without relying on the generated candidate boxes; 2) the coordinate attention is used to capture the spatial position features of the targets in the support set from the two spatial directions of the feature map, and the target spatial features are integrated into the target parameter matrix, so that the detector can adaptively focus on the area containing the targets in the query set according to the support set; 3) the label propagation algorithm is used in the group optimization module to update the support target class vector, which makes full use of the similarity relationship between different targets in the support set, so that the updated support target class vector is more discriminative in the embedding space.

[0045] The present invention is applicable to target detection tasks in a small sample environment, and has the following beneficial effects: 1) directly predicting the bounding box and category of the query target through the converter module, avoiding inaccurate candidate boxes generated when the new class differs greatly from the base class; 2) the coordinate attention can capture the key spatial position features of the support target, guiding the detector to dynamically adjust the focus on the relevant areas in the query sample according to the current target spatial position features, effectively improving the accuracy of query target positioning; 3) the group optimization module fully utilizes the similarity relationship between different targets in the support set to update the support target class vector, and obtains more discriminative support target class features, which helps the model distinguish different categories of targets, thereby improving the classification accuracy of the query target. The coordinate attention mechanism and group optimization mechanism of the present invention significantly improve the performance of the small sample target detection model, and can be applied to practical application fields such as industrial product defect detection and rare animal detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 is a flow chart of the method of the present invention. DETAILED DESCRIPTION

[0047] Small sample target detection methods based on coordinate attention group optimization, such as Figure 1As shown in the figure, firstly, the image dataset is sampled to obtain the support set and query set, and the support feature map set and query feature map set are obtained by using the deep visual feature extractor; then, the support feature map set and the initialized target parameter matrix set are input into the coordinate attention guidance module to obtain the target fusion matrix set; then, the group optimization module is constructed, and the support feature map set is updated by the target true bounding box and label propagation algorithm to obtain the support target class vector set; finally, the query feature map set, the target fusion matrix set and the support target class vector set are input into the query target prediction module to obtain the bounding box and category of the sample target in the query set. This method uses coordinate attention to capture the spatial position features of the target in the support set, so that the detector can adaptively focus on the area containing the target in the query set, improve the accuracy of query target positioning, and obtain the discriminative support target class vector through the group optimization module, thereby improving the classification accuracy of the query target.

[0048] This method first obtains a set of image data containing target bounding boxes and category annotations, and then performs the following operations:

[0049] Step (1) samples the image data set to obtain a support set and a query set, inputs the two into a deep visual feature extraction module, and outputs a support feature map set and a query feature map set; specifically:

[0050] (1-1) First, scale the images in the dataset to the same size and perform random sampling without replacement to obtain the support set. and queryset in, is the domain of real numbers, s represents "support", N s represents the number of image samples in the support set, represents the i-th support image sample, q represents the “query”, N q represents the number of image samples in the query set, represents the j-th query image sample, H represents the image height, W represents the image width, and 3 represents the number of RGB channels;

[0051] Each supporting image sample With annotation Among them, Φ i express The number of targets in, c i,φ ∈{1,…,C} represents The category of the φth target in , express The four-dimensional vector consisting of the coordinates of the upper left corner and lower right corner of the φth target bounding box in the support set The CPC contains CK targets, CK = C × K, that is, C represents the number of target categories, and K represents the number of targets in each category;

[0052] (1-2) Construct a deep visual feature extraction module consisting of a deep convolutional network and a two-dimensional convolutional layer, where the deep convolutional network is a 50-layer residual network ResNet-50 (ResidualNetwork) pre-trained on the ImageNet dataset, and the convolution kernel size of the two-dimensional convolutional layer is 1×1;

[0053] (1-3) will support the set and queryset Input into the deep visual feature extraction module to obtain a set of supporting feature maps and query feature graph collection in, represents the i-th support feature map, represents the j-th query feature map, H′, W′ and 256 represent the height, width and number of channels of a single feature map respectively.

[0054] Step (2) constructs a coordinate attention guidance module, with the input being a randomly initialized target parameter matrix set and a support feature map set, and the output being a target fusion matrix set; specifically:

[0055] (2-1) Random initialization to obtain the target parameter matrix set Among them A j represents the target parameter matrix corresponding to the j-th query image sample, M represents the number of targets to be detected in a single query image sample, and 256 represents the row vector dimension in a single target parameter matrix; constructs a coordinate attention guidance module, the coordinate attention guidance module consists of a coordinate attention submodule and a cross attention submodule, wherein the coordinate attention submodule calculates the weight of each coordinate position from the horizontal and vertical directions of the feature map, so that the weight of the position area containing the target feature is larger, thereby obtaining a set of spatial position attention feature maps, and the cross attention submodule is used to fuse the target parameter matrix set with the spatial position attention feature map set;

[0056] (2-2) For the i-th support feature map Perform average pooling operations along the horizontal coordinate direction and the vertical coordinate direction to obtain the horizontal feature map and vertical feature map Hor means "horizontal" and ver means "vertical". Representing a tensor The value at the coordinate position (h′, w′) in the μth channel, 1≤μ≤256, 1≤h′≤H′, 1≤w′≤W′, then: The value at the coordinate position (h′,1) in the μth channel Tensor The value at the coordinate position (1,w′) in the μth channel

[0057] (2-3) The horizontal feature map and vertical feature map Input the two-dimensional convolution layer and activation function layer in sequence to obtain the horizontal weight feature map And vertical weight feature map Among them, Conv 1 (·) represents a two-dimensional convolutional layer with a kernel size of 1×1, and σ(·) represents the Sigmoid activation function;

[0058] (2-4) According to the support feature map Horizontal weight feature map And vertical weight feature map Calculate the spatial position attention feature map Tensor The value at coordinate (h′, w′) in the μth channel Representing a tensor The value at the coordinate position (h′,1) in the μth channel, Representing a tensor The value at the coordinate position (1, w′) in the μth channel;

[0059] (2-5) Support feature map set All feature maps in Execute steps (2-2) to (2-4) to obtain a set of spatial position attention feature maps Will gather All feature maps in Expand and concatenate according to the spatial dimension to obtain the spatial target position feature

[0060] (2-6) Set the target parameter matrix Each target parameter matrix A j and spatial target location characteristics Input into the cross attention submodule to obtain the target fusion matrix set Softmax(·) is a normalized exponential function, and the superscript T indicates the transposition operation.

[0061] Step (3) constructs a group optimization module, the input is the support feature map set and the bounding box annotation corresponding to the support set, and the output is the support target class vector set; specifically:

[0062] (3-1) Construct a group optimization module. A group is a set of support targets belonging to the same category. The support target probability matrix and similarity matrix are calculated based on the support target class labels and the obtained support target vector set, and the group propagation matrix is ​​obtained through the support target probability matrix and the similarity matrix.

[0063] (3-2) Set the support feature map obtained in step (1-3) And each support image sample obtained in step (1-1) Corresponding annotation Each bounding box b in i,φ Input to the Regions Of Interest Pooling (ROIPooling) layer to obtain a set of supporting target feature maps The region of interest pooling layer represents the maximum pooling operation of the feature map in the corresponding target bounding box area. c,k Represents the feature map of the kth target in the cth class in the set;

[0064] (3-3) Support target feature map set Through convolution and global average pooling operations in sequence, we get the set of support target vectors. Among them, the feature vector of the kth target in the cth class in the set GAP(·) represents the global average pooling operation in the feature map space dimension, Conv 2 (·) represents a 2D convolutional layer with a kernel size of 3×3;

[0065] (3-4) Calculate the set of support target class vectors The support target probability matrix is ​​obtained by the support target class vector set. Among them, the support target class vector of the cth class in the set The value of the u-th row and v-th column of the matrix P represents the probability that the u-th support vector belongs to the v-th class. 1≤u≤CK, 1≤v≤C, u=(c-1)×K+k, Representing a collection The u-th support target vector in exp(·) represents the exponential function with the natural constant e as the base, dist(·,·) represents the Euclidean distance function, then

[0066] (3-5) Use the Gaussian kernel function to calculate the similarity matrix between support target vectors Construct the group propagation matrix through the similarity matrix Z and the support target probability matrix P Among them, the value of the uth row and wth column of the matrix Z represents the support target vector and Similarity between 1≤w≤CK, the hyperparameter γ in the Gaussian kernel function is greater than 0, Representing a collection The w-th supporting target vector in , the group propagation matrix Λ=ZP, the value Λ in the u-th row and v-th column of Λ u,v represents the support target vector The probability of belonging to the vth class is obtained by weighted summation of similarities between samples;

[0067] (3-6) Initialize the iteration upper limit value Ψ, iteratively optimize the support target probability matrix P through the group propagation matrix Λ, the similarity matrix Z and the label propagation algorithm, until the number of iterations reaches the upper limit value Ψ, and obtain the updated support target probability matrix P (Ψ) , at the initial iteration P (0) =P,Λ (0) =Λ, the label propagation algorithm iteration is The value range of Ψ is 100≤ψ≤120. and Indicates The support target probability matrix and group propagation matrix at the iteration, and Representation Matrix The value and matrix of row u and column v The value of row u and column v, and Representation Matrix The value of the uth row and vth column and the value of the uth row and cth column;

[0068] (3-7) Using the updated support target probability matrix P (Ψ) and the set of support target vectors Calculate the new set of support target class vectors Among them, the support target class vector of the cth class in the set P (Ψ) represents the target probability matrix after the Ψth iteration, is the matrix P (Ψ) The value of the uth row and cth column indicates the uth support target vector after the update The probability of belonging to class c.

[0069] Step (4) constructs a query target prediction module, the input of which is a query feature map set, a target fusion matrix set, and a support target class vector set, and the output is a predicted query target bounding box and class probability; specifically:

[0070] (4-1) Construct a query target prediction module, which consists of a transformer submodule, a target classification function, and a bounding box prediction submodule;

[0071] Query the feature graph collection Each feature map is expanded along the spatial dimension to obtain the query feature matrix set Calculate the position encoding matrix of the query sample The value of the κth row and ωth column of the matrix G 1≤κ≤H′×W′, 1≤ω≤256, mod means remainder operation;

[0072] (4-2) Set the query feature matrix And the position encoding matrix E is input into the encoder of the converter submodule to obtain the query target encoding feature set The encoder consists of an attention layer and two fully connected layers. The j-th query target encodes features represents the element-by-element addition operation, and FFN(·) represents a feed-forward neural network consisting of two fully connected layers;

[0073] (4-3) Set the query target encoding feature set And the target fusion matrix set obtained in step (2-6) Input into the decoder of the converter submodule to obtain the query target decoding feature set The decoder consists of two attention layers and two fully connected layers. The j-th query target decoding feature express The intermediate result matrix obtained by inputting into the first attention layer,

[0074] (4-4) Decoding feature sets by querying the target And the set of support target class vectors obtained in steps (3-6) Calculate the target prediction category probability of the query set, the probability that the mth target in the jth query sample belongs to the cth category 1≤m≤M, Representation Matrix The vector of the mth row;

[0075] (4-5) Decode the query target feature set Input to the bounding box prediction submodule to obtain the predicted target bounding box set of the query set The bounding box prediction submodule is a multilayer perceptron consisting of three fully connected layers. The predicted bounding box matrix of the jth query sample is The mth row represents the predicted bounding box of the mth target in the jth query sample, is the coordinate of the upper left corner of the bounding box, The coordinates of the lower right corner of the bounding box.

[0076] Step (5) uses a stochastic gradient descent algorithm to optimize a small sample target detection model consisting of a coordinate attention guidance module, a group optimization module, and a query target prediction module, and obtains the target bounding box and category of the image in the query set through steps (1) to (4) for the new support set and query set; specifically:

[0077] (5-1) Predicting category probabilities through the target of the query set And the cross entropy loss function calculates the target classification loss y j,m,c ∈{0,1} indicates the true label value that the mth target in the jth query sample belongs to the cth class;

[0078] (5-2) Predicted target bounding box set through query set Calculating bounding box loss B j,m represents the true bounding box of the mth target in the jth query sample, Represents the intersection-over-union ratio of the true bounding box and the predicted bounding box;

[0079] (5-3) According to the target classification loss and bounding box loss Get the total loss The small sample target detection model composed of the coordinate attention guidance module, the group optimization module and the query target prediction module is optimized by using the stochastic gradient descent algorithm, and the model is iteratively trained until convergence to obtain the optimized small sample target detection model;

[0080] (5-4) Sampling the new image dataset to obtain the support set and queryset Input the optimized small sample target detection model and execute steps (1) to (4) in sequence to output the query set The target category probability of the image sample in and bounding box collection The target class index with the maximum probability is selected as the predicted class.

[0081] The contents described in this embodiment are merely an enumeration of the implementation forms of the inventive concept. The protection scope of the present invention should not be regarded as being limited to the specific forms described in the embodiment. The protection scope of the present invention also extends to the equivalent technical means that can be conceived by those skilled in the art based on the inventive concept.

Claims

1. A small sample target detection method based on coordinate attention group optimization, characterized in that: First, obtain a set of image data containing target bounding boxes and category annotations, and then perform the following operations: Step (1) sampling the image data set to obtain a support set and a query set, inputting the two into a deep visual feature extraction module, and outputting a support feature map set and a query feature map set; Step (2) constructs a coordinate attention guidance module, the input of which is a randomly initialized target parameter matrix set and a support feature map set, and the output is a target fusion matrix set; Step (3) constructing a group optimization module, the input of which is the support feature map set and the bounding box annotation corresponding to the support set, and the output is the support target class vector set; Step (4) constructing a query target prediction module, the input of which is a query feature map set, a target fusion matrix set and a support target class vector set, and the output is a predicted query target bounding box and class probability; Step (5) uses a stochastic gradient descent algorithm to optimize a small sample target detection model consisting of a coordinate attention guidance module, a group optimization module, and a query target prediction module, and obtains the target bounding box and category of the image in the query set through steps (1) to (4) for the new support set and query set.

2. The small sample target detection method based on coordinate attention group optimization as claimed in claim 1, characterized in that: Step (1) specifically includes: (1-1) First, scale the images in the dataset to the same size and perform random sampling without replacement to obtain the support set. and queryset in, is the field of real numbers, N s represents the number of image samples in the support set, represents the i-th support image sample, N q represents the number of image samples in the query set, represents the j-th query image sample, H represents the image height, W represents the image width, and 3 represents the number of RGB channels; Each supporting image sample With annotation Among them, Φ i express The number of targets in, c i,φ ∈{1,…,C} represents The category of the φth target in , express The four-dimensional vector consisting of the coordinates of the upper left corner and lower right corner of the φth target bounding box in the support set The total number of targets in the CCP is CK, CK = C × K, that is, C represents the number of target categories, and K represents the number of targets in each category; (1-2) Construct a deep visual feature extraction module consisting of a deep convolutional network and a two-dimensional convolutional layer, where the deep convolutional network is a 50-layer residual network ResNet-50 pre-trained on the ImageNet dataset, and the convolution kernel size of the two-dimensional convolutional layer is 1×1; (1-3) will support the set and queryset Input into the deep visual feature extraction module to obtain a set of supporting feature maps and query feature graph collection in, represents the i-th support feature map, represents the j-th query feature map, H′, W′ and 256 represent the height, width and number of channels of a single feature map respectively.

3. The small sample target detection method based on coordinate attention group optimization as claimed in claim 2, characterized in that: Step (2) is specifically: (2-1) Random initialization to obtain the target parameter matrix set A j represents the target parameter matrix corresponding to the j-th query image sample, M represents the number of targets to be detected in a single query image sample, and 256 represents the row vector dimension in a single target parameter matrix; constructs a coordinate attention guidance module, the coordinate attention guidance module consists of a coordinate attention submodule and a cross attention submodule, wherein the coordinate attention submodule calculates the weight of each coordinate position from the horizontal and vertical directions of the feature map to obtain a set of spatial position attention feature maps, and the cross attention submodule is used to fuse the target parameter matrix set with the set of spatial position attention feature maps; (2-2) For the i-th support feature map Perform average pooling operations along the horizontal coordinate direction and the vertical coordinate direction to obtain the horizontal feature map and vertical feature map Representing a tensor The value at the coordinate position (h′, w′) in the μth channel, 1≤μ≤256, 1≤h′≤H′, 1≤w′≤W′, then: The value at the coordinate position (h′,1) in the μth channel Tensor The value at the coordinate position (1,w′) in the μth channel (2-3) The horizontal feature map and vertical feature map Input the two-dimensional convolution layer and activation function layer in sequence to obtain the horizontal weight feature map And vertical weight feature map Where Conv1(·) represents a two-dimensional convolutional layer with a convolution kernel size of 1×1, and σ(·) represents the Sigmoid activation function; (2-4) According to the support feature map Horizontal weight feature map And vertical weight feature map Calculate the spatial position attention feature map Tensor The value at coordinate (h′, w′) in the μth channel Representing a tensor The value at the coordinate position (h′,1) in the μth channel, Representing a tensor The value at the coordinate position (1, w′) in the μth channel; (2-5) Support feature map set All feature maps in Execute steps (2-2) to (2-4) to obtain a set of spatial position attention feature maps Will gather All feature maps in Expand and concatenate according to the spatial dimension to obtain the spatial target position feature (2-6) Each target parameter matrix A j and spatial target location characteristics Input into the cross attention submodule to obtain the target fusion matrix set Softmax(·) is a normalized exponential function, and the superscript T indicates the transposition operation.

4. The small sample target detection method based on coordinate attention group optimization as claimed in claim 3, characterized in that: Step (3) is specifically: (3-1) Construct a group optimization module. A group is a set of support targets belonging to the same category. The support target probability matrix and similarity matrix are calculated based on the support target class labels and the obtained support target vector set, and the group propagation matrix is ​​obtained through the support target probability matrix and the similarity matrix. (3-2) Set the support feature map obtained in step (1-3) And each support image sample obtained in step (1-1) Corresponding annotation Each bounding box b in i,φ Input into the region of interest pooling layer to obtain a set of supporting target feature maps The region of interest pooling layer represents the maximum pooling operation of the feature map in the corresponding target bounding box area. c,k Represents the feature map of the kth target in the cth class in the set; (3-3) Support target feature map set Through convolution and global average pooling operations in sequence, we get the set of support target vectors. Among them, the feature vector of the kth target in the cth class in the set GAP(·) represents the global average pooling operation in the spatial dimension of the feature map, and Conv2(·) represents a two-dimensional convolutional layer with a convolution kernel size of 3×3; (3-4) Calculate the set of support target class vectors The support target probability matrix is ​​obtained by supporting the target class vector set. Among them, the support target class vector of the cth class in the set The value of the u-th row and v-th column of the matrix P represents the probability that the u-th support vector belongs to the v-th class. 1≤u≤CK, 1≤v≤C, u=(c-1)×K+k, Representing a collection The u-th support target vector in exp(·) represents the exponential function with the natural constant e as the base, dist(·,·) represents the Euclidean distance function, then (3-5) Use the Gaussian kernel function to calculate the similarity matrix between support target vectors Construct the group propagation matrix through the similarity matrix Z and the support target probability matrix P Among them, the value of the uth row and wth column of the matrix Z represents the support target vector and Similarity between 1≤w≤CK, the hyperparameter γ in the Gaussian kernel function is greater than 0, Representing a collection The w-th supporting target vector in , the group propagation matrix Λ=ZP, the value Λ in the u-th row and v-th column of Λ u,v represents the support target vector The probability of belonging to the vth class is obtained by weighted summation of similarities between samples; (3-6) Initialize the iteration upper limit value Ψ, iteratively optimize the support target probability matrix P through the group propagation matrix Λ, the similarity matrix Z and the label propagation algorithm, until the number of iterations reaches the upper limit value Ψ, and obtain the updated support target probability matrix P (Ψ) , at the initial iteration P (0) =P,Λ (0) =Λ, the label propagation algorithm iteration is 100≤ψ≤120, and Indicates The support target probability matrix and group propagation matrix at the iteration, and Representation Matrix The value and matrix of row u and column v The value of row u and column v, and Representation Matrix The value of the uth row and vth column and the value of the uth row and cth column; (3-7) Using the updated support target probability matrix P (Ψ) and the set of support target vectors Calculate the new set of support target class vectors Among them, the support target class vector of the cth class in the set P (Ψ) represents the target probability matrix after the Ψth iteration, is the matrix P (Ψ) The value of the uth row and cth column indicates the uth support target vector after the update The probability of belonging to class c.

5. The small sample target detection method based on coordinate attention group optimization as claimed in claim 4, characterized in that: Step (4) is specifically: (4-1) Constructing a query target prediction module, which consists of a converter submodule, a target classification function, and a bounding box prediction submodule; Query the feature graph collection Each feature map is expanded along the spatial dimension to obtain the query feature matrix set Calculate the position encoding matrix of the query sample The value of the κth row and ωth column of the matrix G 1≤κ≤H′×W′, 1≤ω≤256, mod means remainder operation; (4-2) Set the query feature matrix And the position encoding matrix E is input into the encoder of the converter submodule to obtain the query target encoding feature set The encoder consists of an attention layer and two fully connected layers. The j-th query target encodes features represents the element-by-element addition operation, and FFN(·) represents a feed-forward neural network consisting of two fully connected layers; (4-3) Set the query target encoding feature set And the target fusion matrix set obtained in step (2-6) Input into the decoder of the converter submodule to obtain the query target decoding feature set The decoder consists of two attention layers and two fully connected layers. The j-th query target decoding feature express The intermediate result matrix obtained by inputting into the first attention layer, (4-4) Decoding feature sets by querying the target And the set of support target class vectors obtained in steps (3-6) Calculate the target prediction category probability of the query set, the probability that the mth target in the jth query sample belongs to the cth category 1≤m≤M, Representation Matrix The vector of the mth row; (4-5) Decode the query target feature set Input to the bounding box prediction submodule to obtain the predicted target bounding box set of the query set The bounding box prediction submodule is a multilayer perceptron consisting of three fully connected layers. The predicted bounding box matrix of the jth query sample is The mth row represents the predicted bounding box of the mth target in the jth query sample, is the coordinate of the upper left corner of the bounding box, The coordinates of the lower right corner of the bounding box.

6. The small sample target detection method based on coordinate attention group optimization as claimed in claim 5, characterized in that: Step (5) is specifically: (5-1) Predicting category probabilities through the target of the query set And the cross entropy loss function calculates the target classification loss y j,m,c ∈{0,1} indicates the true label value that the mth target in the jth query sample belongs to the cth class; (5-2) Predicted target bounding box set through query set Calculating bounding box loss B j,m represents the true bounding box of the mth target in the jth query sample, Represents the intersection-over-union ratio of the true bounding box and the predicted bounding box; (5-3) According to the target classification loss and bounding box loss Get the total loss The small sample target detection model composed of the coordinate attention guidance module, the group optimization module and the query target prediction module is optimized by using the stochastic gradient descent algorithm, and the model is iteratively trained until convergence to obtain the optimized small sample target detection model; (5-4) Sampling the new image dataset to obtain the support set and queryset Input the optimized small sample target detection model and execute steps (1) to (4) in sequence to output the query set The target category probability of the image sample in and bounding box collection The target class index with the maximum probability is selected as the predicted class.

Images