Small sample target detection method based on variational coding
Through the small sample object detection method of variational encoding, the feature extraction is enhanced by using Soft-Res2Net and the convolutional attention module, combined with the variational autoencoder and RPN network, the low accuracy rate caused by the small sample object detection is solved. Problem, improving detection accuracy and generalization capabilities.
Patent Information
- Application Number
- CN202411069080.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-06
- Publication Date
- 2024-11-26
AI Technical Summary
In small sample object detection, due to the small number of training samples, the model is prone to overfitting, resulting in poor performance on unseen data, difficult to identify new targets, and low accuracy.
Using a small sample object detection method based on variational encoding, feature extraction is enhanced through Soft-Res2Net feature extraction network and convolutional attention module, combining variational autoencoder and RPN network, and using feature aggregator and classifier regressor for targeting. Detect and optimize the network model through multi-dimensional loss function.
It improves the accuracy and generalization ability of detection, overcomes the limitations of traditional methods in new object detection and background suppression, and improves the stability and performance of the model.
Smart Images

Figure CN119027647A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to relevant fields of computer vision, and in particular to a small sample target detection method based on variational coding. Background Art
[0002] In recent years, the field of computer vision research has attracted increasing attention, and deep convolutional neural networks have made significant progress in the field of object detection, but most of these methods rely on a large amount of annotated data. When annotated data becomes scarce, deep neural networks may face serious overfitting problems, resulting in limited generalization ability. In the real world, many object categories have only a small number of examples, or bounding box annotations are difficult to obtain, such as medical data. Therefore, few-shot learning has emerged to train models with limited examples. Most existing few-shot learning works focus on image classification problems, and only a few focus on few-shot object detection problems. The few-shot object detection task requires not only predicting the category of the object, but also accurately locating the object's location, which is more complex than few-shot classification.
[0003] Traditional object detection relies on manually designed feature extraction algorithms. In the object detection process, a sliding window strategy is usually adopted to extract features on windows of different positions and sizes in the image, and each window is classified by a trained classifier to determine whether there is a target object. Although traditional methods are effective in specific situations, they usually require expert knowledge to design features and their performance is limited in complex environments.
[0004] With the development of deep convolutional neural networks, deep learning-based object detection methods are becoming the mainstream for solving computer vision problems. Popular object detection networks can be roughly divided into two categories: one is a single-stage detection network structure, whose basic principle is to use a convolutional neural network to tightly sample the input image, use multiple scales and aspect ratios when sampling, and then perform classification and position regression analysis on each sample. The other is a two-stage detection network structure, whose basic principle is to extract features from the input image to obtain a feature map, and then generate a candidate box after the feature map is convolved, and the candidate box is sent to the classifier and regressor for target classification and position regression, and finally completes the target detection.
[0005] Small-sample object detection provides a systematic framework, which uses a large amount of base class data as support samples for training. After training, predictions can be made on query samples with a small amount of new class data. The detection process is modeled as episodes. In each episode, K photos of support samples of a certain category and the corresponding calibration boxes are provided. It is required to locate the positions of all objects of this category in another different query sample. This setting is called K-shot setting. K here is generally small, ranging from 1 to 30.
[0006] However, the core problem of small sample learning is that the available labeled data is very limited, which limits the model from learning enough features and patterns. Due to the small number of training samples, the model can easily overfit these samples, resulting in poor performance on unseen data, and may not be able to learn sufficiently generalized features from limited data and have difficulty recognizing new targets. Summary of the invention
[0007] The purpose of the embodiment of the present invention is to provide a small sample target detection method based on variational coding to alleviate the technical problem existing in the prior art that the small sample target detection accuracy is low due to the small number of training samples.
[0008] To solve the above technical problems, the embodiments of the present invention provide the following technical solutions:
[0009] A first aspect of the present invention provides a small sample target detection method based on variational coding, comprising the following steps:
[0010] S10. Divide the input image into a support set image and a query set image; the support set image is an image including annotated target samples, and the query set image is an image to be detected; an image including unlabeled target samples;
[0011] S20. The Soft-Res2Net feature extraction network is used to enhance the learning ability of the network by further introducing residual connections inside the residual block, so that the network can capture features at different scales, obtain initial support features for support set images, obtain initial query features for query set images, and improve the richness of feature representation through multi-scale feature fusion;
[0012] S30. A convolutional attention module is used, and the convolutional attention module includes a channel attention module and a spatial attention module; the initial support feature input in step S20 is passed through the channel attention module to obtain a weighted result, and then passed through the spatial attention module to obtain a target support feature, and the initial query feature is passed through the channel attention module to obtain a weighted result, and then passed through the spatial attention module to obtain a target query feature;
[0013] S40. Extracting features using a variational autoencoder according to the target support features obtained in step S30, wherein the variational autoencoder includes a variational feature encoder and a variational feature decoder, wherein the variational feature encoder outputs variational features; and the variational feature decoder outputs reconstructed features;
[0014] S50. Aggregate the target query feature and the variational feature through a feature aggregator, calculate the average value of all feature vectors by average aggregation, and obtain the aggregated feature;
[0015] S60. Input the aggregated features obtained in S50 into the RPN network to obtain the region candidate frame. After being sorted and filtered by the RPN network, the anchor frame is used as the region candidate frame set, and then input into the RoI pooling layer to align the sizes of the region candidate frames of different sizes. After further extracting the features of the region candidate frame through the two-layer convolutional neural network of the RoI feature extractor, the features are sent to the classifier to predict the category and the regressor to predict the precise position of the bounding box.
[0016] S70. Classify and regress the final obtained region candidate frame to select the final target;
[0017] S80. Guide the network model learning by calculating the root mean square loss, KL divergence loss, consistency loss, classification loss and regression loss during training.
[0018] In some modified implementations of the first aspect of the present invention, in step S20, the following steps are included:
[0019] S21. After convolving the input support set image and query set image using 1×1 convolution, split them into multiple corresponding feature subsets;
[0020] S22. In the feature subset, except X1, each X i There is a corresponding 3×3 convolution, with K i Indicates; with y i K i Output of; feature subset X i With K i-1 The outputs are added and then fed into K i middle;
[0021] S23. The output of the Soft-Res2Net feature extraction network using soft pooling contains different numbers and combinations of receptive field sizes. For the input feature X = [x ij ], where i represents the row index and j represents the column index. The output of soft pooling is Y = [y ij ]; its expression is:
[0022]
[0023] in, m and n Traverse all positions of the input features, a is an adjustable parameter.
[0024] In some modified implementations of the first aspect of the present invention, in step S30, the following steps are included:
[0025] S31. In the channel attention module, average pooling and maximum pooling are used to calculate weights, and their expressions are as follows:
[0026] M c (x) = σ(MLP(Avgpool c (x))+MLP(Maxpool c (x)));
[0027] Among them, MLP is fully connected, Avgpool c (x) is average pooling, Maxpool c (x) is the maximum pooling, c is the cth channel, σ is the sigmoid activation function; M c (x) represents the weight obtained by channel c;
[0028] Multiply the weights and the features of each input channel to get the channel attention feature, which is expressed as follows:
[0029] X c =x c ×M c (x)
[0030] Where M c (x) is the channel attention feature weight obtained by the c channel, x c is the c-channel input feature, X c It is the feature after adding channel attention to the c channel.
[0031] S32. In the spatial attention module, its expression is as follows:
[0032] M s (x) = σ(F(concat(Avgpool s (x),(Maxpool s (x))));
[0033] Where s represents all feature points in different channels of the same pixel, Avgpool s (x) is the average pooling of all channels, Maxpool s(x) is the maximum pooling of all channels, concat is the cascade operation, F is a 7×7 convolution layer, and the feature convolution of the two cascaded channels is restored to the original number of channels; finally, the sigmoid activation function is used to process the result;
[0034] Multiply the weights and the features of each input channel to get the spatial attention feature, which is expressed as follows:
[0035] X s =x c ×M s (x)
[0036] Where M s (x) is the spatial attention feature weight obtained by the c channel, x c is the c-channel input feature, X s It is the feature after adding spatial attention to the c channel;
[0037] In some modified implementations of the first aspect of the present invention, in step S40, the following steps are included:
[0038] S41. In the variational feature encoder, the input data x is mapped to the distribution parameter of the variational feature z, expressed as:
[0039] μ(x)=f μ (x;θ μ )
[0040] σ 2 (x) = f σ (x;θ σ )
[0041] In the formula, f μ and f σ is the function implemented by the neural network, θ μ and θ σ are the corresponding network parameters;
[0042] Using the reparameterization technique, the variational features z are sampled from the distribution parameters;
[0043] z=μ(x)+ε·σ(x)
[0044] Where ε~N(0,I) is a random variable sampled from a standard normal distribution;
[0045] S42. In the variational feature decoder, the variational feature z is mapped back to the data space to generate reconstructed data That is, reconstruction features.
[0046] In some modified implementations of the first aspect of the present invention, in step S50, the target query feature and the variational feature are aggregated together, and the expression is as follows:
[0047] Q ij =A(Q i ,z j )=Q i ·sig(z j );
[0048] In the formula, ⊙ represents channel multiplication, and sig is the abbreviation of sigmoid operation; during the training phase, a target support feature S is randomly selected j Aggregate; in the test phase, take the average of the K target support features of class j And estimate the distribution N(μ j ,σ j ).
[0049] In some modified implementations of the first aspect of the present invention, in step S70, the following steps are included:
[0050] S71. Calculate the feature vector x and the weight vector w for each category j The cosine similarity of is expressed as:
[0051]
[0052] S72. The probability P(y=j|x) belonging to each category is calculated based on the pre-defined softmax function, and the expression is:
[0053]
[0054] Where α is a scaling parameter used to control the difficulty of classification. is the cosine similarity, represents the unstandardized probability of category j, and C is the total number of categories;
[0055] S73. Predict the position and size of the target bounding box using smoothing The loss function is used to measure the regression error, smoothing function The expression is:
[0056]
[0057] Where x represents the residual in the loss function, that is, the difference between the predicted value and the true value.
[0058] In some modified implementations of the first aspect of the present invention, in step S80, the total loss function used in the training process As shown below:
[0059]
[0060] in, is the root mean square loss, is the KL divergence loss, is the consistency loss, is the classification loss, is the regression loss, α is the weight coefficient, and α defaults to 2.5×10 -4 ;
[0061] The steps include:
[0062] S81. Calculate the root mean square loss and compare the target support feature S with the reconstructed feature S′ to obtain the root mean square loss The expression is as follows:
[0063]
[0064] S82. Calculate the KL divergence loss to measure the difference between the latent variable distribution q(z|x) obtained by the encoder and the prior distribution p(z); the expression of KL divergence is as follows:
[0065]
[0066] Where each term corresponds to a dimension j in the latent space, J is the total number of dimensions in the latent space; q(z|x) is the latent variable distribution; p(z) is the prior distribution and follows the normal distribution N(0,I); D KL represents divergence; is a constant factor used to simplify calculations; 1 is the entropy of the standard normal distribution; is the logarithm of the standard deviation of the encoder output in the jth dimension, which represents the uncertainty of the encoder output distribution; is the square of the mean, indicating the center position of the encoder output distribution; is the square of the standard deviation, which indicates the width of the encoder output distribution;
[0067] S83. Calculate the consistency loss to make the distribution of the target support features consistent with the reconstructed features. The expression is as follows:
[0068]
[0069] in, is the cross entropy loss; S′ is the reconstructed feature; is a classification function, which indicates the probability that S′ belongs to different categories; c is the actual category label;
[0070] S84. Calculate classification loss The expression is as follows:
[0071]
[0072] Where j is the category index, C is the total number of categories, including background or negative samples, and y j is the unique hot encoding of the true label. If sample x belongs to category j, then y j =1, otherwise y j =0; P(y=j|x) is the probability of predicting that sample x belongs to category j, which is the output category probability distribution; The natural function logarithm converts the probability into logarithmic probability, which is convenient for calculating the loss;
[0073] S85. Calculate regression loss The expression is as follows:
[0074]
[0075] Where b is the true bounding box, is the predicted bounding box, b i is the value of the i-th dimension of the true bounding box, To predict the value of the i-th dimension of the bounding box, i∈{x,y,w,h} means that i can take any value of x, y, w or h, corresponding to the center point coordinates and size of the bounding box respectively. For smooth The loss function is used to measure the regression error. By minimizing this loss, the model can learn how to predict the location and size of the target more accurately.
[0076] The present invention is a small sample target detection method based on variational coding. In the small sample target detection scenario, the present invention uses a two-stage detection network structure that requires a candidate box, and improves the accuracy and generalization ability of detection through a series of technical strategies. The method first divides the input image, adopts a weight-sharing Soft-Res2Net feature extraction network, captures multi-scale features through residual connections, and extracts features. On this basis, convolutional attention is introduced to obtain more accurate target features. Further, the expression ability of features is enhanced by a variational autoencoder. Subsequently, the aggregated features are obtained using a feature aggregator. Finally, after the RPN network, a refined regional candidate box is generated, and features are extracted by a RoI pooling layer and a feature extractor, and the classifier and regressor are combined to predict the category of the target and the precise position of the bounding box. The method optimizes the learning process of the network model by calculating a multi-dimensional loss function, and further improves the stability and performance of the model. In addition, the model of the present invention also has the potential to improve detection performance, which to a certain extent overcomes the limitations of traditional methods in new object detection and background suppression. BRIEF DESCRIPTION OF THE DRAWINGS
[0077] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required to be used in the embodiments are briefly introduced below.
[0078] Figure 1 A flowchart of a small sample target detection method based on variational coding provided by the present invention;
[0079] Figure 2 A network structure diagram of a small sample target detection method based on variational coding provided by the present invention;
[0080] Figure 3 A schematic diagram of the network structure of the Soft-Res2Net module in a small sample target detection method based on variational coding provided by the present invention;
[0081] Figure 4 A schematic diagram of a convolutional attention module in a small sample target detection method based on variational coding provided by the present invention;
[0082] Figure 5 A schematic diagram of the structure of a channel attention module and a spatial attention module in a small sample target detection method based on variational coding provided by the present invention;
[0083] Figure 6 A schematic diagram of the network structure of a variational autoencoder module in a small sample target detection method based on variational coding provided by the present invention;
[0084] Figure 7 A flowchart for generating RPN region candidate boxes in a small sample target detection method based on variational coding provided by the present invention. DETAILED DESCRIPTION
[0085] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0086] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention.
[0087] Embodiment 1
[0088] like Figure 1 As shown in FIG. 1 , a small sample target detection method based on variational coding specifically includes the following steps:
[0089] Step 1: Input image: Divide the input image into support set images and query set images, where the support set images are images that include labeled target samples, and are input in the form of N types of targets, each type of target has K images, where the samples have been labeled. The query set images include images of unlabeled target samples, which are used for queries in training, but are consistent with the support set category space;
[0090] Step 2: Feature extraction: Use Soft-Res2Net feature extraction network, such as Figure 3 As shown in Figure 1. First, the input support set image and query set image are convolved using a 1×1 convolution, and then divided into multiple feature subsets. Except for X1, each X i There is a corresponding 3×3 convolution, with K i Indicates. Use y i K i The output of feature subset X i With K i-1 The outputs are added and then fed into K i Each time a feature segmentation passes through a 3×3 convolution, the output may have a larger receptive field than the feature segmentation. Due to the combinatorial explosion, the output of the Soft-Res2Net feature extraction network contains different numbers and combinations of receptive field sizes. In the Soft-Res2Net feature extraction network, a soft pooling method is used. Soft pooling is a learnable pooling method that weights each position of the input feature by a trainable weight vector and applies a nonlinear activation function to generate the final pooling output. For the input feature X = [x ij ], where i represents the row index and j represents the column index. The output of soft pooling is Y = [y ij ]The calculation method is shown in formula (1):
[0091]
[0092] Where m and n traverse all positions of the input features, and a is an adjustable parameter.
[0093] By introducing the exponential function To assign different weights to different input values, larger values will receive higher weights.
[0094] Step 3: Attention mechanism: Convolutional attention (CBAM) module diagram, as shown Figure 4 The initial support features and initial query features obtained after feature extraction are sent to the convolutional attention module, first through the channel attention module to obtain the weighted result, and then through the spatial attention module to obtain the final result after weighting again.
[0095] The specific operations of the channel attention module and the spatial attention module are as follows: Figure 5 As shown. The channel attention module calculates the value of the input feature on the same channel as a weight. When calculating the weight in this way, two methods are used, namely average pooling (Average Pool) and maximum pooling (Max Pool). Average pooling calculates the average value of the input data with a length of H and a width of W for each channel, and obtains a 1×1 output. Maximum pooling calculates the maximum value of the input data with a length of H and a width of W for each channel, and obtains a 1×1 output. The maximum value weight and average value weight obtained by the two methods are respectively sent to the corresponding features of the fully connected (MLP) module output, and the results of the two outputs are added. The obtained weight evaluates the features in the channel from the two perspectives of average value and maximum value. The weight value is sent to the sigmoid activation function to obtain the weight of each channel. The channel attention feature weight is expressed as shown in formula (2):
[0096] M c (x) = σ(MLP(Avgpool c (x))+MLP(Maxpool c (x))) (2)
[0097] In the formula, MLP is fully connected, Avgpool c (x) is average pooling, Maxpool c (x) is the maximum pooling, c is the cth channel, and σ is the sigmoid activation function. c (x) represents the weight obtained by the c channel.
[0098] Multiply the weights and the features of each input channel to obtain the channel attention features, as shown in formula (3):
[0099] X c =x c ×M c (x) (3)
[0100] Where M c (x) is the channel attention feature weight obtained by the c channel, x c is the c-channel input feature, X c It is the feature after adding channel attention to the c channel, that is, the channel attention feature.
[0101] The input of the spatial attention module is the channel attention feature X c, this feature channel is c, length and width are H and W. Spatial attention uses maximum pooling and average pooling for features in different channels, that is, the c×H×W feature X_c is divided into H×W c×1×1 small blocks in length and width, and the maximum value and average value are calculated for each small block to obtain two 1×1×1 features. The two 1×1×1 features are merged in length and width to obtain two 1×H×W features, and the two features are cascaded to obtain 2×H×W features. Then, a convolution kernel with an output channel of c is used to convert the feature with channel 2 into a feature with channel c, and then the sigmoid activation function is used for activation to obtain the spatial attention feature weight. The spatial attention feature weight is expressed as formula (4):
[0102] M s (x) = σ(F(concat(Avgpool s (x),(Maxpool s (x)))) (4)
[0103] Where s represents all feature points in different channels of the same pixel, Avgpool s (x) is the average pooling of all channels, Maxpool s (x) is the maximum pooling of all channels, concat is the cascade operation, F is a 7×7 convolution layer, which performs feature convolution on the two cascaded channels and restores the original number of channels. Finally, the sigmoid activation function is used to process the result.
[0104] Multiply the weights and the features of each input channel to obtain the spatial attention feature, as shown in formula (5):
[0105] X s =x c ×M s (x) (5)
[0106] Where M s (x) is the spatial attention feature weight obtained by the c channel, x c is the c-channel input feature, X s It is the feature after the c channel adds spatial attention, that is, the spatial attention feature.
[0107] Attention features are inferred in the channel dimension and spatial dimension, and global information is learned to selectively strengthen certain features and suppress certain less useful features.
[0108] Step 4: Variational feature extraction: Variational Autoencoder (VAE) is a generative model, and its operation process mainly includes the following steps: First, the variational feature encoder part. Map the input data x to the distribution parameters of the variational feature z. Usually, it is assumed that the variational feature z follows a normal distribution, that is, z~N(μ,σ 2 ). The encoder outputs the mean μ(x) and variance σ of the latent variable 2 (x), as shown in formula (6) and formula (7):
[0109] μ(x)=f μ (x;θ μ ) (6)
[0110] σ 2 (x) = f σ (x;θ σ ) (7)
[0111] In the formula, f μ and f σ is the function implemented by the neural network, θ μ and θ σ is the corresponding network parameter. In order to sample the variational feature z from this distribution, use the reparameterization trick, as shown in formula (8):
[0112] z=μ(x)+ε·σ(x) (8)
[0113] Where ε~N(0,I) is a random variable sampled from a standard normal distribution.
[0114] Next, there is the variational feature decoder part. The decoder maps the variational feature z back to the data space to generate reconstructed data That is, reconstruction features.
[0115] Step 5: Feature aggregation: Use average aggregation to calculate the average value of all feature vectors. Figure 2 As shown, the target query feature is aggregated with the variational feature. Given the target query feature Q of class i, i and the target support feature S of class j j , first estimate its distribution N(μ j ,σ j ) and sample the variational features z j =μ j +σ j , and then fuse them together, as shown in formula (9):
[0116]
[0117] In the formula, ⊙ represents channel multiplication, and sig is the abbreviation of sigmoid operation. During the training phase, a target support feature S is randomly selected j Aggregate; in the test phase, take the average of the K target support features of class j And estimate the distribution N(μ j ,σ j ).
[0118] Step 6: Obtaining region candidate boxes: Obtaining region candidate boxes is divided into three parts: region proposal network (RPN), RoI pooling, and RoI feature extractor.
[0119] Region Proposal Network (RPN): At each position in the feature map, it constructs and generates anchor boxes of various scales and aspect ratios. These anchor boxes are like detection points scattered on the image, covering various areas that may contain target objects. Figure 7 As shown in the figure, first, a sliding window is used on the feature map with each pixel as the center to obtain anchor frames of different sizes. Each pixel will obtain 9 anchor frames of different sizes and aspect ratios. Finally, a total of K anchor frames are obtained on the feature map; then, these K anchor frames will be subjected to convolution operations for feature alignment. Secondly, the prediction branch generates 2K prediction scores for the K anchor frames, representing the probability that the corresponding anchor frames belong to the foreground and background; the regression branch generates 4K prediction coordinates for the K anchor frames, representing the coordinate offset of the corresponding anchor frames. Finally, the proposal box screening module further modifies the anchor frames to generate the final set of regional candidate frames. Among them, Proposal selection will sort the anchor frames according to the probability that the anchor frames belong to the foreground category, and select some anchor frames with higher probabilities to be sent to the next stage; Proposal location modifies the anchor frame position according to the predicted anchor frame coordinate position offset, and deletes the anchor frames outside the original image range.
[0120] RoI pooling: The anchor boxes after RPN sorting and screening are used as the region candidate box set and input into the RoI pooling layer. First, each candidate region is mapped to the corresponding feature map. Secondly, the candidate region is divided into a fixed number of sub-regions. For each sub-region, the RoI pooling layer performs a maximum pooling operation and selects the maximum value from the sub-region as the feature value of the sub-region. Finally, it is converted into a feature map with a fixed size.
[0121] RoI feature extractor: takes the fixed-size feature map from the RoI pooling layer as input. The features are processed through a series of convolutional layers to extract high-level features, which may contain information such as the shape, texture, and pattern of the target. Due to the large number of anchor boxes and the possibility of a large amount of overlap, the non-maximum suppression (NMS) algorithm is used to remove redundant and less important regions based on the confidence and overlap of the candidate regions, and only retain the most representative and unique region candidate boxes.
[0122] Finally, the RoI feature extractor outputs a set of feature vectors, which are fed into the classifier to predict the category and the regressor to predict the precise position of the bounding box.
[0123] Step 7: Classification and regression: Classify and regress the obtained region candidate boxes to screen out the final target.
[0124] For classification tasks, a cosine softmax-based classifier is used to learn separable features, reduce intra-class variance, and improve the detection accuracy of new classes. First, the feature vector x and the weight vector w of each class are calculated. j The cosine similarity of is shown in formula (10):
[0125]
[0126] Secondly, the probability P(y=j|x) belonging to each category is calculated based on the pre-softmax function, as shown in formula (11):
[0127]
[0128] Where α is a scaling parameter used to control the difficulty of classification. is the cosine similarity, represents the unnormalized probability of category j, and C is the total number of categories.
[0129] For regression tasks, predict the location and size of the target bounding box using smoothing The loss function is used to measure the regression error, smoothing function The definition is shown in formula (12):
[0130]
[0131] Where x represents the residual in the loss function, that is, the difference between the predicted value and the true value.
[0132] Step 8: Loss calculation and training:
[0133] In variational feature extraction, the goal is to minimize the reconstruction error loss and the KL divergence (Kullback-LeiblerDivergence) loss, and at the same time, obtain the consistency loss.
[0134] Comparing the target support feature S with the reconstructed feature S′, the root mean square loss is obtained, as shown in formula (13):
[0135]
[0136] KL divergence is used to regularize the latent space and measure the difference between the latent variable distribution q(z|x) obtained by the encoder and the prior distribution p(z). The calculation formula of KL divergence is shown in formula (14):
[0137]
[0138] Where each term corresponds to a dimension j in the latent space, J is the total number of dimensions in the latent space; q(z|x) is the latent variable distribution; p(z) is the prior distribution and follows the normal distribution N(0,I);
[0139] D KL represents divergence; is a constant factor used to simplify calculations; 1 is the entropy of the standard normal distribution; is the logarithm of the standard deviation of the encoder output in the jth dimension, which represents the uncertainty of the encoder output distribution; is the square of the mean, indicating the center position of the encoder output distribution;
[0140] is the square of the standard deviation and represents the width of the encoder output distribution.
[0141] The consistency loss indicates that the distribution of the target support features is consistent with the reconstructed features, as shown in formula (15):
[0142]
[0143] In the formula, is the cross entropy loss; S′ is the reconstructed feature; is a classification function, which indicates the probability that S′ belongs to different categories; c is the actual category label.
[0144] During classification and regression, classification loss and regression loss are obtained.
[0145] Classification Loss As shown in formula (16):
[0146]
[0147] Where j is the category index, C is the total number of categories, including background or negative samples, and y j is the unique hot encoding of the true label. If sample x belongs to category j, then y j =1, otherwise y j = 0. P(y=j|x) is the probability that the predicted sample x belongs to category j, which is the output category probability distribution. The natural function logarithm converts the probability into logarithmic probability, which is convenient for calculating the loss.
[0148] Regression Loss As shown in formula (17):
[0149]
[0150] Where b is the true bounding box, is the predicted bounding box, b i is the value of the i-th dimension of the true bounding box, To predict the value of the i-th dimension of the bounding box, i∈{x,y,w,h} means that i can take any value of x, y, w or h, corresponding to the center point coordinates and size of the bounding box respectively. For smooth The loss function is used to measure the regression error. By minimizing this loss, the model can learn how to predict the location and size of the target more accurately.
[0151] The total loss function used during training As shown in formula (18):
[0152]
[0153] in, is the root mean square loss, is the KL divergence loss, is the consistency loss, is the classification loss, is the regression loss, α is the weight coefficient (α defaults to 2.5×10 -4 ).
[0154] The network model learning is guided by calculating the root mean square loss, KL divergence loss, consistency loss, classification loss and regression loss during the training process.
[0155] The above are only specific embodiments of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art who is familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.
Claims
1. A small sample target detection method based on variational coding, characterized in that: The method comprises the following steps: S10. Divide the input image into a support set image and a query set image; the support set image is an image including annotated target samples; the query set image is an image to be detected, including an image of an unlabeled target sample; S20. The Soft-Res2Net feature extraction network is used to enhance the learning ability of the network by further introducing residual connections inside the residual block, so that the network can capture features at different scales, obtain initial support features for support set images, obtain initial query features for query set images, and improve the richness of feature representation through multi-scale feature fusion; S30. A convolutional attention module is used, and the convolutional attention module includes a channel attention module and a spatial attention module; After the initial support features input in step S20 are passed through the channel attention module to obtain weighted results, they are passed through the spatial attention module to obtain target support features. After the initial query features are passed through the channel attention module to obtain weighted results, they are passed through the spatial attention module to obtain target query features. S40. Extracting features using a variational autoencoder according to the target support features obtained in step S30, wherein the variational autoencoder includes a variational feature encoder and a variational feature decoder, and the variational feature encoder outputs variational features; The variational feature decoder outputs the reconstructed features; S50. Aggregate the target query feature and the variational feature through a feature aggregator, calculate the average value of all feature vectors by average aggregation, and obtain the aggregated feature; S60. Input the aggregated features obtained in S50 into the RPN network to obtain the region candidate box. After the anchor boxes are sorted and filtered by the RPN network as the region candidate box set, they are input into the RoI pooling layer to align the sizes of the region candidate boxes of different sizes. After the features of the region candidate boxes are further extracted by the two-layer convolutional neural network of the RoI feature extractor, they are sent to the classifier to predict the category and the regressor to predict the precise position of the bounding box. S70. Classify and regress the final obtained region candidate frame to select the final target; S80. Guide the network model learning by calculating the root mean square loss, KL divergence loss, consistency loss, classification loss and regression loss during training.
2. The small sample target detection method based on variational coding according to claim 1, characterized in that: In step S20, the following steps are included: S21. After convolving the input support set image and query set image using 1×1 convolution, split them into multiple corresponding feature subsets; S22. In the feature subset, except X1, each X i There is a corresponding 3×3 convolution, with K i Indicates; with y i K i Output of; feature subset X i With K i-1 The outputs are added and then fed into K i middle; S23. The output of the Soft-Res2Net feature extraction network using soft pooling contains different numbers and combinations of receptive field sizes. For the input feature X = [x ij ], where i represents the row index and j represents the column index. The output of soft pooling is Y = [y ij ]; Its expression is: Among them, m and n traverse all positions of the input features, and a is an adjustable parameter.
3. The small sample target detection method based on variational coding according to claim 1, characterized in that: In step S30, the following steps are included: S31. In the channel attention module, average pooling and maximum pooling are used to calculate weights, and their expressions are as follows: M c (x)=σ(MLP(Avgpool c (x))+MLP(Maxpool c (x))); Among them, MLP is fully connected, Avgpool c (x) is average pooling, Maxpool c (x) is the maximum pooling, c is the cth channel, σ is the sigmoid activation function; M c (x) represents the weight obtained by channel c; Multiply the weights and the features of each input channel to get the channel attention feature, which is expressed as follows: X c =x c ×M c (x); Where M c (x) is the channel attention feature weight obtained by the c channel, x c is the c-channel input feature, X c It is the feature after adding channel attention to the c channel; S32. In the spatial attention module, its expression is as follows: M s (x)=σ(F(concat(Avgpool s (x),(Maxpool s (x)))); Where s represents all feature points in different channels of the same pixel, Avgpool s (x) is the average pooling of all channels, Maxpool s (x) is the maximum pooling of all channels, concat is the cascade operation, F is a 7×7 convolution layer, and the feature convolution of the two cascaded channels is restored to the original number of channels; finally, the sigmoid activation function is used to process the result; Multiply the weights and the features of each input channel to get the spatial attention feature, which is expressed as follows: X s =x c ×M s (x); Where M s (x) is the spatial attention feature weight obtained by the c channel, x c is the c-channel input feature, X s It is the feature after adding spatial attention to the c channel.
4. The small sample target detection method based on variational coding according to claim 1 is characterized in that: In step S40, the following steps are included: S41. In the variational feature encoder, the input data x is mapped to the distribution parameter of the variational feature z, expressed as: μ(x)=f μ (x;θ μ ) s 2 (x)=f σ (x;θ σ ) In the formula, f μ and f σ is the function implemented by the neural network, θ μ and θ σ are the corresponding network parameters; Using the reparameterization technique, the variational features z are sampled from the distribution parameters; z=μ(x)+ε·σ(x) Where ε~N(0,I) is a random variable sampled from a standard normal distribution; S42. In the variational feature decoder, the variational feature z is mapped back to the data space to generate reconstructed data That is, reconstruction features.
5. According to the small sample target detection method based on variational coding according to claim 4, it is characterized in that: In step S50, the target query feature and the variational feature are aggregated together, and the expression is as follows: Q ij =A(Q i ,z j )=Q i ·sig(z j ); In the formula, ⊙ represents channel multiplication, and sig is the abbreviation of sigmoid operation; during the training phase, a target support feature S is randomly selected j Aggregate; in the test phase, take the average of the K target support features of class j And estimate the distribution N(μ j ,σ j ).
6. The small sample target detection method based on variational coding according to claim 1, characterized in that: In step S70, the following steps are included: S71. Calculate the feature vector x and the weight vector w for each category j The cosine similarity of is expressed as: S72. The probability P(y=j|x) belonging to each category is calculated based on the pre-defined softmax function, and the expression is: Where α is a scaling parameter used to control the difficulty of classification. is the cosine similarity, represents the unstandardized probability of category j, and C is the total number of categories; S73. Predict the position and size of the target bounding box using smoothing The loss function is used to measure the regression error, smoothing function The expression is: Where x represents the residual in the loss function, that is, the difference between the predicted value and the true value.
7. The small sample target detection method based on variational coding according to claim 1, characterized in that: In step S80, the total loss function used in the training process is As shown below: in, is the root mean square loss, is the KL divergence loss, is the consistency loss, is the classification loss, is the regression loss, α is the weight coefficient, and α defaults to 2.5×10 -4 ; The steps include: S81. Calculate the root mean square loss and compare the target support feature S with the reconstructed feature S′ to obtain the root mean square loss The expression is as follows: S82. Calculate the KL divergence loss to measure the difference between the latent variable distribution q(z|x) obtained by the encoder and the prior distribution p(z); the expression of KL divergence is as follows: Where each term corresponds to a dimension j in the latent space, J is the total number of dimensions in the latent space; q(z|x) is the latent variable distribution; p(z) is the prior distribution and follows the normal distribution N(0,I); D KL represents divergence; is a constant factor used to simplify calculations; 1 is the entropy of the standard normal distribution; is the logarithm of the standard deviation of the encoder output in the jth dimension, which represents the uncertainty of the encoder output distribution; is the square of the mean, indicating the center position of the encoder output distribution; is the square of the standard deviation, which indicates the width of the encoder output distribution; S83. Calculate the consistency loss to make the distribution of the target support features consistent with the reconstructed features. The expression is as follows: In the formula, is the cross entropy loss; S′ is the reconstructed feature; is a classification function, which indicates the probability that S′ belongs to different categories; c is the actual category label; S84. Calculate classification loss The expression is as follows: Where j is the category index, C is the total number of categories, including background or negative samples, and y j is the unique hot encoding of the true label. If sample x belongs to category j, then y j =1, otherwise y j =0; P(y=j|x) is the probability of predicting that sample x belongs to category j, which is the output category probability distribution; The natural function logarithm converts the probability into logarithmic probability, which is convenient for calculating the loss; S85. Calculate regression loss The expression is as follows: Where b is the true bounding box, is the predicted bounding box, b i is the value of the i-th dimension of the true bounding box, To predict the value of the i-th dimension of the bounding box, i∈{x,y,w,h} means that i can take any value of x, y, w or h, corresponding to the center point coordinates and size of the bounding box respectively. For smooth The loss function is used to measure the regression error. By minimizing this loss, the model can learn how to predict the location and size of the target more accurately.
Citation Information
Cited By
Meta-DETR small sample rotating target detection method based on directed candidate box generation mechanism
CN120070860A
