Small sample target detection method and system

Abstract

The invention relates to a small sample target detection method and system, and the method comprises the steps: obtaining a plurality of types of small sample images; the obtained small sample images are input into an improved Faster-Rcnn model, the improved Faster-Rcnn model is obtained by improving the Faster-Rcnn model, and the improvement method comprises the following steps: enhancing the perception ability of the Faster-Rcnn model for different scale targets through a hierarchical sampling method; the recognition capability of the Faster-Rcnn model for a new category in a small sample image is enhanced through ternary category division and comparative learning; the discrimination capability of a classifier of the Faster-Rcnn model for different categories in the small sample image is enhanced through a simplex equiangular frame; and through the improved Faster-Rcnn model, different types of targets in the image are detected. The improved model is high in detection capability.

Description

Technical Field

[0001] The present invention relates to the technical field of target detection, and in particular to a small sample target detection method and system. Background Art

[0002] As a core task in the field of computer vision, object detection aims to identify and locate objects of interest from images or videos. With the development of deep learning technology, especially the widespread application of convolutional neural networks, object detection technology has made significant progress. However, traditional object detection models usually require a large amount of labeled data to train, which is often costly and time-consuming in practical applications. Especially in industrial or medical environments, such labeled data is more difficult to collect.

[0003] To solve this problem, researchers began to explore small sample target detection technology. The small sample problem refers to the problem that some categories in the data set have insufficient labeled samples, which makes it impossible to build a large training set for supervised machine learning. Small sample target detection technology aims to use a small number of labeled samples to train the model so that it can detect and identify objects of new categories. Research in this field mainly focuses on how to transfer knowledge learned from basic categories to new categories through technologies such as transfer learning and meta-learning.

[0004] At present, small sample learning technology mainly focuses on methods such as model fine-tuning, data enhancement, and transfer learning. Model fine-tuning technology pre-trains the model on a large amount of data and then fine-tunes it on a small number of samples, but it may encounter overfitting problems. Data enhancement methods use auxiliary data or information to expand the target data set to help the model better extract features. Transfer learning methods transfer learned knowledge to new fields without requiring a strong correlation between the source domain and the target domain. It can be further subdivided into metric learning, meta-learning, and graph neural network methods.

[0005] The application of domain transfer technology in small-sample learning enhances the cross-domain capability of the model by reducing the domain gap between the visible and unseen classes. Meta-learning, especially its application in small-sample object detection, trains the model by simulating a series of small-sample tasks so that it can quickly adapt to new tasks. Metric learning methods map the target region features of the query image and the support set image features to the same embedding space to achieve detection through the idea of ​​learning comparison. Data augmentation technology improves sample diversity and feature robustness by enhancing the sample space and feature space. Recently, the Transformer model has also begun to be applied to small-sample object detection, using a multi-head attention mechanism to associate support and query image features.

[0006] In addition to traditional transfer learning and meta-learning techniques, some emerging methods have also begun to attract attention. These methods attempt to address the challenges of small sample learning from different angles, such as increasing sample diversity by generating synthetic data through generative adversarial networks, or using attention mechanisms and feature fusion techniques to improve the model's ability to capture key information. In addition, some studies also focus on exploring the intrinsic relationships between categories, revealing the connections between different categories by constructing category maps, thereby helping the model better understand and generalize the features of new categories.

[0007] Furthermore, researchers are also exploring how to use natural language processing technology to enhance small sample object detection. For example, by combining image descriptions and visual features, the model can understand the image content more comprehensively, so that it can generalize more flexibly when faced with new categories. This method can not only improve the generalization ability of the model, but also reduce the dependence on a large amount of labeled data to a certain extent.

[0008] In addition, with the deepening of the research on the interpretability of deep learning models, researchers are also trying to improve the reliability and transparency of small sample object detection by visualizing and explaining the decision-making process of the model. This can not only help users better understand the behavior of the model, but also provide guidance for further optimization of the model.

[0009] These methods can alleviate the overfitting and data scarcity problems in small sample learning to a certain extent by utilizing a large amount of existing data or unlabeled data. However, when dealing with imbalanced categories and uneven sample distribution, existing algorithms often suffer from model overfitting, which leads to a decrease in model generalization ability and a serious reduction in accuracy. Therefore, these methods do not fundamentally solve the core problem of small sample learning. Summary of the invention

[0010] To this end, the technical problem to be solved by the present invention is to overcome the problem that small sample learning in the prior art still suffers from model overfitting, resulting in poor model generalization ability, which in turn reduces the model prediction accuracy.

[0011] In order to solve the above technical problems, the present invention provides a small sample target detection method, comprising:

[0012] Step S1: obtaining a small sample image with multiple categories;

[0013] Step S2: Input the acquired small sample image into the improved Faster-Rcnn model, where:

[0014] The improved Faster-Rcnn model is obtained by improving the Faster-Rcnn model, and the improvement method includes:

[0015] The Faster-Rcnn model's ability to perceive objects of different scales is enhanced through a hierarchical sampling method;

[0016] Enhance the Faster-RCNN model's ability to recognize new categories in small sample images through ternary category division and contrastive learning;

[0017] The Faster-Rcnn model classifier is enhanced to distinguish different categories in small sample images through a simplex equiangular framework;

[0018] Step S3: Detecting targets of different categories in the image through the improved Faster-Rcnn model.

[0019] In one embodiment of the present invention, in step S2, the perception capability of the Faster-Rcnn model for targets of different scales is enhanced by a hierarchical sampling method, and the method includes:

[0020] For the feature map extracted by the Faster-Rcnn model, an adaptive sampling method is used to dynamically adjust the number and position of sampling points according to the importance and distribution of the features. The formula is:

[0021]

[0022] Among them, f is a function that dynamically adjusts the sampling points according to the feature importance, I feature is the original feature map, ε is the scaling factor used to control the total number of sampling points; It is the coefficient that controls the influence of gradient, which is used to adjust the influence of gradient on sampling density; grad(I feature ) is the feature map I feature The gradient is used to represent the rate of change of features in the feature map;

[0023] Normalize the result of function f to ensure that the total number of sampling points does not exceed the preset maximum value N max , the normalized formula is:

[0024]

[0025] The round function is used to round to the nearest integer, max(f(I feature )) indicates that f is in the entire feature map I feature The maximum value on is used to normalize the output of f to ensure that the sampling point does not exceed the preset maximum value N max ;

[0026] A hierarchical sampling method is introduced in the Faster-Rcnn model: for targets below a preset size, the model performs dense sampling on feature maps with a resolution greater than a preset resolution to capture detail information; for targets above a preset size, the model performs sparse sampling on feature maps with a resolution less than a preset resolution to obtain overall structural information. The hierarchical sampling method is described by the resolution adjustment function R, which is:

[0027]

[0028] Among them, S size Represents the size of the target in the feature map, c and d are the first and second adjustment parameters, c controls the base, d is used to control the speed of exponential growth, and e is the base of the natural logarithm;

[0029] A multi-scale fusion mechanism is introduced in the Faster-Rcnn model to fuse feature maps of different resolutions to generate a comprehensive feature representation:

[0030] I combined =F fusion (I low-res ,I high-res )=Upsample(I high-res )+Conv(I low-res )

[0031] Among them, F fusion is a fusion function, Upsample represents upsampling, Conv represents a convolution operation, I low-res and I high-res Represent feature maps with resolutions lower than the preset resolution and higher than the preset resolution, respectively. combined It is the comprehensive feature representation after fusion.

[0032] In one embodiment of the present invention, in step S2, the recognition capability of the Faster-Rcnn model for new categories in small sample images is enhanced by ternary category division and contrastive learning, and the method includes:

[0033] When training the Faster-Rcnn model, a ternary classification loss and a contrast loss are constructed, where:

[0034] The formula for the three-element classification loss is:

[0035]

[0036] Among them, L ternary is the triple classification loss, N represents the total number of anchor points considered, is the objectness score of the anchor point predicted by the model; is the true object label of the i-th generation anchor point, when When it is 1, it indicates a known foreground object. When it is 0, it means background. When it is 2, it indicates a potential new category object; represents the foreground loss function, represents the background loss function, The loss function for representing novel classes usually uses the cross entropy loss function;

[0037] The formula for the contrast loss is:

[0038]

[0039] Among them, L contrastive is the contrast loss, N obj is the number of proposals with the same objectness label, N prop represents the total number of all proposals, I is the indicator function, if the condition is true, I = 1, otherwise I = 0; z i is the representation of the i-th feature proposal, obj gt,t is the ground-truth objectness label of the i-th proposal; It is a standardized feature representation used to calculate the cosine distance;

[0040] Combining the ternary classification loss and the contrast loss yields:

[0041] L total1 =L ternary +μL contrastive Among them, L total1 It is the combined loss of the three-element classification loss and the contrastive learning loss, and μ is the weight parameter used to balance the two loss terms.

[0042] In one embodiment of the present invention, in step S2, the classifier of the Faster-Rcnn model is enhanced in its ability to distinguish between different categories in a small sample image by using a simplex equiangular framework, and the method includes:

[0043] Before model training, a set of maximized and equivalently separated weight vectors are calculated and used as fixed centers of all categories in small sample images, where:

[0044] The weight vector is calculated as follows: first, the initialization vector w1 is set to the unit vector e1; then, the vector is constructed, for i=2,...,N c , choose vector w i with w1,...,w i-1 Orthogonal, and try to stay away from the existing vector; then adjust the vector, for each vector w i and wj , if the angle between them is not the largest By rotating w i To increase the vector w i and w j The angle between them, while maintaining the vector w i is a unit vector; repeat the adjustment until the angles between all vectors are as close as possible

[0045] In order to enhance the intra-class compactness, an adaptive class-specific boundary is introduced, which is used to push the feature vector toward the corresponding class center w i near;

[0046] Calculate the class-specific boundary offset using the formula:

[0047]

[0048] Among them, m c is the class-specific boundary offset for class c, p c is the instance frequency of category c, which indicates the proportion of the number of instances of category c to the total number of instances in the training data; p bg is the estimated probability of the background class, which indicates the proportion of the background class or non-target class in all classes; c b is the set of known foreground object categories, c n is a new category set; background is the background;

[0049] According to the class-specific boundary offset m c Construct a simplex equiangular frame loss function, the formula is:

[0050]

[0051] Among them, L intra is the simplex equiangular frame loss, y c is the indicator function of the true label of sample x belonging to category c; v c is the original prediction value of sample x belonging to category c; m c is the class-specific boundary offset of class c; c' is other classes that are not class c; C is the total class set.

[0052] In one embodiment of the present invention, after adjusting the vector w i and w j After the angle between them, we also need to continue to optimize the vector w i and w j Get the simplex equiangular frame weight matrix W * , and the optimized simplex equiangular frame weight matrix W *The weight parameters applied to the Faster-Rcnn model, where the simplex equiangular frame weight matrix W * The formula is:

[0053]

[0054] Among them, W * is the simplex equiangular frame weight matrix, argmax is the maximum index function, W is the weight set of the classifier, N c is the number of categories, w i and w j are the feature vectors representing the i-th and j-th categories respectively, d(w i -w j ) 2 Represents vector w i and w j The square of the distance between .

[0055] In one embodiment of the present invention, the step S2 further includes:

[0056] The class-specific boundaries are then adjusted through the self-distillation mechanism to force the feature points to be more closely distributed around the class center. In the self-distillation process, a teacher model is first used to generate soft labels, which are used to reflect the model's prediction confidence for the input features. Then a student model with the same architecture as the teacher model but different parameters is used to learn the soft labels. The goal of the student model is to adjust its own parameters by imitating the predictions of the teacher model, thereby achieving adaptive adjustment of the class-specific boundaries.

[0057] Construct the adaptive boundary distillation objective loss function, the formula is:

[0058]

[0059] Among them, L adapt is the adaptive boundary distillation objective loss, is the predicted probability of category c for the teacher model, y c is a binary variable whose true label of sample x belongs to category c. If x really belongs to category c, then y c =1, otherwise y c =0; is the predicted value of the student model on category c, is the adaptive boundary offset learned by the student model in category c.

[0060] In one embodiment of the present invention, it also includes a combined loss L according to the triple classification loss and the contrastive learning loss. total1 , simple equiangular frame loss Lintra , Adaptive boundary distillation target loss L adapt , to construct the total loss function, the formula is:

[0061] L tota2 =α(L total1 )+β(L intra )+λ(L adapt )

[0062] Among them, L total1 It is the combined loss of the triple classification loss and the contrastive learning loss; L intra is the simple equiangular frame loss; L adapt is the adaptive boundary distillation target loss; α, β, and λ are weight parameters used to balance the three losses;

[0063] Through the total loss function L total2 To train the Faster-Rcnn model.

[0064] In order to solve the above technical problems, the present invention provides a small sample target detection system, comprising:

[0065] Acquisition module: used to obtain small sample images with multiple categories;

[0066] Input module: used to input the acquired small sample image into the improved Faster-Rcnn model, where:

[0067] The improved Faster-Rcnn model is obtained by improving the Faster-Rcnn model, and the improvement method includes:

[0068] The Faster-Rcnn model's ability to perceive objects of different scales is enhanced through a hierarchical sampling method;

[0069] Enhance the Faster-RCNN model's ability to recognize new categories in small sample images through ternary category division and contrastive learning;

[0070] The Faster-Rcnn model classifier is enhanced to distinguish different categories in small sample images through a simplex equiangular framework;

[0071] Detection module: used to detect targets of different categories in the image through the improved Faster-Rcnn model.

[0072] To solve the above technical problems, the present invention provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above small sample target detection method when executing the computer program.

[0073] In order to solve the above technical problems, the present invention provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of the above small sample target detection method are implemented.

[0074] The above technical solution of the present invention has the following advantages compared with the prior art:

[0075] The present invention adopts a hierarchical sampling method to enhance the model's perception of objects of different sizes; adopts contrastive learning technology and combines it with a ternary classification method to further enhance the model's recognition ability of new categories; uses a simplex equiangular framework for geometric perception and designs a model classifier, making the model's discrimination effect between different categories more significant;

[0076] The improved Faster-Rcnn model of the present invention shows stronger robustness when facing class imbalance or uneven sample distribution, reducing the risk of overfitting; it not only improves the detection accuracy, but also improves the ability to recognize new categories in the case of small samples;

[0077] The present invention can also use transfer learning technology to enable the model to quickly adjust and adapt to new tasks after receiving a small number of samples of new categories, significantly reducing the time and resources required to retrain the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0078] In order to make the contents of the present invention more clearly understood, the present invention is further described in detail below based on specific embodiments of the present invention in conjunction with the accompanying drawings.

[0079] Figure 1 It is a flow chart of the method of the present invention. DETAILED DESCRIPTION

[0080] The present invention is further described below in conjunction with the accompanying drawings and specific embodiments so that those skilled in the art can better understand the present invention and implement it, but the embodiments are not intended to limit the present invention.

[0081] Embodiment 1

[0082] Reference Figure 1 As shown, the present invention relates to a small sample target detection method, comprising:

[0083] Step S1: obtaining a small sample image with multiple categories;

[0084] Step S2: Input the acquired small sample image into the improved Faster-Rcnn model, where:

[0085] The improved Faster-Rcnn model is obtained by improving the Faster-Rcnn model, and the improvement method includes:

[0086] The Faster-Rcnn model's ability to perceive objects of different scales is enhanced through a hierarchical sampling method;

[0087] Enhance the Faster-RCNN model's ability to recognize new categories in small sample images through ternary category division and contrastive learning;

[0088] The Faster-Rcnn model classifier is enhanced to distinguish different categories in small sample images through a simplex equiangular framework;

[0089] Step S3: Detecting targets of different categories in the image through the improved Faster-Rcnn model.

[0090] The following is a detailed introduction to this embodiment:

[0091] Step 1: Read small sample data of multiple categories, including their category information and the location information of the geometric box; perform data cleaning on image data with missing or incomplete information; then divide the cleaned image data into training and test sets; perform preprocessing operations on the divided training set images as input for the next stage network model.

[0092] Step 2: Use hierarchical sampling method to enhance the model's perception ability for objects of different sizes;

[0093] The feature map of the image is extracted through the feature extraction network (Conv layers) in the Faster-Rcnn model. Since the feature extraction network is a prior art, this embodiment will not be repeated. On the feature map generated by each image, this embodiment adopts an adaptive sampling method to dynamically adjust the number and position of sampling points according to the importance and distribution of the features. The calculation formula is as follows:

[0094]

[0095] Among them, f is a function that dynamically adjusts the sampling points according to the feature importance, I feature is the original feature map, ε is the scaling factor used to control the total number of sampling points; It is the coefficient that controls the influence of gradient, which is used to adjust the influence of gradient on sampling density; grad(I feature ) is the feature map I featureThe gradient of , which represents the rate of change of the feature in the feature map. The purpose of this formula is to increase the number of sampling points in areas with large gradients (i.e., significant feature changes), and decrease the number of sampling points in areas with small gradients (i.e., gentle feature changes). By adjusting ε and The value of , this embodiment can control the density and distribution of sampling.

[0096] In addition, in order to ensure that the number of sampling points is within an acceptable range, the result of f in this embodiment can be normalized to ensure that the total number of sampling points does not exceed the preset maximum value N max , normalization can be achieved by:

[0097]

[0098] The round function is used to round to the nearest integer, max(f(I feature )) represents the maximum value of f in the entire feature map, which is used to normalize the output of f to ensure that the sampling point does not exceed the preset maximum value N max .

[0099] In order to make the model focus more on those areas that are critical to target recognition, while ignoring those areas with low information content. This embodiment adopts a hierarchical sampling method. For small-sized targets, the model will perform dense sampling on the feature map with higher resolution to capture detailed information; while for large-sized targets, it will perform sparse sampling on the feature map with lower resolution to obtain overall structural information. This strategy not only improves the generalization ability of the model, but also reduces the consumption of computing resources. This strategy can be described by the resolution adjustment function R:

[0100]

[0101] Among them, S size Represents the size of the target in the feature map, c and d are adjustment parameters, c controls the base, d controls the speed of exponential growth, and e is the base of the natural logarithm;

[0102] Finally, a multi-scale fusion mechanism is introduced to effectively integrate feature maps of different resolutions to generate a comprehensive feature representation. This fusion can be achieved through the fusion function F fusion To achieve:

[0103] I combined =F fusion (I low-res ,I high-res )=Upsample(I high-res )+Conv(I low-res )

[0104] Among them, F fusion is a fusion function, Upsample represents upsampling, Conv represents a convolution operation, I low-res and I high-res Represent the low-resolution and high-resolution feature maps, I combined is the comprehensive feature representation after fusion. This fusion not only enhances the model's ability to perceive objects of different sizes, but also improves its robustness in complex scenes. In this way, the model can more effectively handle objects of different sizes and complexities while optimizing the use of computing resources. It should be noted that I combined It is equivalent to the feature maps extracted by the feature extraction network (Conv layers).

[0105] The hierarchical sampling method replaces the original feature pyramid pooling (FPN) in Faster-RCNN. The purpose is that the multi-scale feature representation constructed by FPN is used to enhance the model's detection ability for objects of different sizes under a large amount of image data. In small sample learning, the hierarchical sampling method adopted can better dynamically adjust the sampling points according to the importance of the features in a small amount of data, so as to improve the model's learning ability for key features.

[0106] Step 3: Use contrastive learning technology and ternary classification to further improve the model's ability to recognize new categories;

[0107] An innovative ternary classification mechanism is introduced to improve the Region Proposal Network (RPN) in Faster-Rcnn for object detection. Based on the traditional binary classification (foreground and background), this embodiment adds a third category in the fine-tuning stage to distinguish potential new category objects. This ternary classification mechanism enables the model of this embodiment to not only recognize known foreground and background, but also recognize and utilize new category objects that are unlabeled or rare in the training data.

[0108] In the method of this embodiment, each proposal of the RPN layer is classified into one of the following three categories:

[0109] 1. Known foreground object (Foreground Known);

[0110] 2. Background;

[0111] 3. Potential new category objects (Nove l);

[0112] For each proposal, this embodiment defines a three-element classification loss, which is calculated as follows:

[0113]

[0114] Among them, L ternary is the triple classification loss, N represents the total number of anchor points considered, is the objectness score of the anchor point predicted by the model; is the true objectness label of the i-th generation anchor, which can be 1 (known foreground object), 0 (background) or 2 (potential new category object); represents the foreground loss function, represents the background loss function, The loss function for representing novel classes usually uses the cross entropy loss function;

[0115] On the basis of ternary classification, this embodiment uses contrastive learning technology to enhance the model's ability to distinguish new category features. Specifically, this embodiment designs a contrast loss to encourage the model to get closer to the positive samples in the feature space and push away the negative samples. The addition of contrastive learning to ternary classification not only helps the model distinguish known foreground and background, but also enhances the recognition of potential new category objects.

[0116] Contrasting loss function, the calculation formula is as follows:

[0117]

[0118] Among them, L contrastive is the contrast loss, N obj is the number of proposals with the same objectness label, N prop represents the total number of all proposals, I is the indicator function, if the condition is true, I = 1, otherwise I = 0; z i is the representation of the i-th feature proposal, obj gt,t is the ground-truth objectness label of the i-th proposal, is the standardized feature representation used to calculate the cosine distance.

[0119] This loss function encourages the model to bring proposals with the same objectness label closer together, while pushing proposals with different objectness labels farther apart. By minimizing this loss function, the feature representations learned by the model will be able to better distinguish different objectness categories.

[0120] Combining the contrast loss and the three-element classification loss, the calculation formula is as follows:

[0121] L total1 =L ternary +μL contrastive

[0122] Among them, L total1 is the total loss with two losses, L ternary is the triple classification loss, L contrastive is the contrastive loss and μ is the weight parameter used to balance the two loss terms.

[0123] Step 4: Use the simplex equirectangular framework (ETF) for geometric perception and design the classifier of the Faster-Rcnn model to make the model more effective in distinguishing different categories.

[0124] The weight vector initialized by the simplex equiangular framework maximizes the minimum distance between category centers, thereby creating a clear boundary for each category in the feature space to enhance the classifier's ability to distinguish features of different categories.

[0125] Specifically, a simplex equiangular frame is a special set of vectors in which the vectors are not only of equal length, but also of equal angles between each other. This structure mathematically guarantees the maximum equidistant separation between vectors. This embodiment utilizes the properties of a simplex equiangular frame to provide a fixed and equidistant initialization method for the weight vectors of the classifier. The core advantage of this method is that it can ensure the initial separation between the category centers in the feature space, thereby providing a strong starting point for the model to learn and distinguish different categories more effectively.

[0126] The initial separation of the class centers is crucial to improving classification performance. If the class centers are too close in feature space, the model may face difficulty in distinguishing them in the early stages of training, resulting in inefficient learning. By using the properties of the simplex equiangular frame, this embodiment can set a clear and balanced geometric structure for the class centers before training begins, which helps the model converge faster and improves its final classification accuracy.

[0127] The initialization process can directly calculate a set of maximized and equivalently separated weight vectors through mathematical formulas. These vectors serve as fixed centers for all categories in the small sample image.

[0128] The construction method of mathematical formula calculation is to first initialize the vector w1 to the unit vector e1; then construct the subsequent vectors, for i = 2, ..., N c , choose vector w i with w1,...,w i-1 Orthogonal, and as far away from the existing vector as possible; then adjust the vector, for each vector w i and w j If the angle between them is not the largest By rotating w iTo increase this angle while keeping the vector w i is a unit vector. Then repeat the adjustment until all vectors meet the condition, that is, the angle between them is as close as possible.

[0129] To enhance intra-class compactness, this embodiment introduces adaptive class-specific boundaries that push feature vectors toward their corresponding class centers w i near.

[0130] Calculate the adaptive boundary offset, the calculation formula is as follows:

[0131]

[0132] Among them, m c is the class-specific boundary offset for class c, p c is the instance frequency of category c, which indicates the proportion of the number of instances of category c to the total number of instances in the training data; p bg is the estimated probability of the background class, which indicates the proportion of the background class or non-target class in all classes; c b is the base class (known foreground object) set, c n is a new category set and background is the background.

[0133] Then, according to the adaptive boundary offset calculation, the simplex equiangular frame loss function calculation is improved, and the calculation formula is as follows:

[0134]

[0135] Among them, L intra is the simplex equiangular frame loss, y c is the indicator function of the true label of sample x belonging to category c; v c is the original prediction that sample x belongs to category c; m c is the class-specific boundary offset of class c, c' is other classes that are not class c; C is the total class set.

[0136] It is worth mentioning that after adjusting the vector w i and w j After the angle between them, we also need to continue to optimize the vector w i and w j Get the simplex equiangular frame weight matrix W * , and the optimized simplex equiangular frame weight matrix W * The weight parameters applied to the Faster-Rcnn model, where the simplex equiangular frame weight matrix W * The formula is:

[0137]

[0138] Among them, W * is the simplex equiangular frame weight matrix, argmax is the maximum index function, W is the weight set of the classifier, N c is the number of categories, w i and w j are the eigenvectors representing the i-th and j-th classes respectively, d(w i -w j ) 2 Represents the squared distance between two vectors.

[0139] After enhancing the classifier of the Faster-Rcnn model through the simplex isometric framework, the following steps are also included: In order to further improve the performance of the model, the present embodiment adjusts the class-specific boundaries through a self-distillation mechanism to cause the feature points to be more closely distributed around the category center. During the self-distillation process, the present embodiment first uses a teacher model to generate soft labels, which reflect the model's prediction confidence for the input features. Then, the present embodiment uses a student model with the same architecture as the teacher model but different parameters to learn these soft labels. The goal of the student model is to adjust its own parameters by imitating the predictions of the teacher model, thereby achieving adaptive adjustment of the class-specific boundaries.

[0140] The adaptive boundary distillation objective loss function is calculated as follows:

[0141]

[0142] Among them, L adapt is the adaptive boundary distillation objective loss, is the predicted probability of category c for the teacher model; y c is a binary variable whose true label of sample x belongs to category c (if x actually belongs to category c, then y c =1; otherwise y c =0), is the predicted value of the student model on category c; is the adaptive boundary shift learned by the student model for category c.

[0143] Combining the above three loss functions to construct the total loss function L total2 , through the total loss function L total2 To train the model Faster-Rcnn model, the total loss function L total2 The calculation formula is:

[0144] L tota2 =α(L total1 )+β(L intra )+λ(Ladapt )

[0145] Among them, L total1 It is the combined loss of the triple classification loss and the contrastive learning loss; L intra is the simple equiangular frame loss; L adapt is the adaptive boundary loss; where α, β, and λ are weight parameters used to balance the three losses.

[0146] Step 5: Use transfer learning technology to transfer the knowledge learned from the basic categories to the new categories, so that the model can quickly adapt to new tasks and achieve effective target detection tasks even with only small sample data.

[0147] The model of this embodiment is pre-trained on the basis of a large amount of data. Through the pre-training process, the model can learn rich feature representations and general pattern recognition capabilities.

[0148] Fine-tune on small sample data of new categories and adjust the parameters of the pre-trained model to adapt to the new task. Through feature alignment technology, the feature space of the base category is mapped to the feature space of the new category to reduce the distribution difference between categories. Ensure that the model can maintain high performance on new and old categories, especially on small sample data of new categories. Finally, the model outputs the category probability and bounding box coordinates of each candidate region to complete the small sample object detection task.

[0149] Embodiment 2

[0150] This embodiment provides a small sample target detection system, including:

[0151] Acquisition module: used to obtain small sample images with multiple categories;

[0152] Input module: used to input the acquired small sample image into the improved Faster-Rcnn model, where:

[0153] The improved Faster-Rcnn model is obtained by improving the Faster-Rcnn model, and the improvement method includes:

[0154] The Faster-Rcnn model's ability to perceive objects of different scales is enhanced through a hierarchical sampling method;

[0155] Enhance the Faster-RCNN model's ability to recognize new categories in small sample images through ternary category division and contrastive learning;

[0156] The Faster-Rcnn model classifier is enhanced to distinguish different categories in small sample images through a simplex equiangular framework;

[0157] Detection module: used to detect targets of different categories in the image through the improved Faster-Rcnn model.

[0158] Embodiment 3

[0159] This embodiment provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps of the small sample target detection method described in Embodiment 1 are implemented.

[0160] Embodiment 4

[0161] This embodiment provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of the small sample target detection method described in the first embodiment are implemented.

[0162] Those skilled in the art will appreciate that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can adopt the form of complete hardware embodiments, complete software embodiments, or embodiments in combination with software and hardware. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code. The scheme in the embodiments of the present application can be implemented in various computer languages, for example, object-oriented programming language Java and literal scripting language JavaScript, etc.

[0163] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0164] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1A function specified in one or more boxes.

[0165] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0166] Although the preferred embodiments of the present application have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications falling within the scope of the present application.

[0167] Obviously, the above embodiments are merely examples for clear explanation and are not intended to limit the implementation methods. For those skilled in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all the implementation methods here. The obvious changes or modifications derived from these are still within the protection scope of the invention.

Claims

1. A small sample target detection method, characterized by: include: Step S1: obtaining a small sample image with multiple categories; Step S2: Input the acquired small sample image into the improved Faster-Rcnn model, where: The improved Faster-Rcnn model is obtained by improving the Faster-Rcnn model, and the improvement method includes: The Faster-Rcnn model's ability to perceive objects of different scales is enhanced through a hierarchical sampling method; Enhance the Faster-RCNN model's ability to recognize new categories in small sample images through ternary category division and contrastive learning; The Faster-Rcnn model classifier is enhanced to distinguish different categories in small sample images through a simplex equiangular framework. Step S3: Detecting different categories of objects in the image through the improved Faster-Rcnn model.

2. The small sample target detection method according to claim 1, characterized in that: In step S2, the perception capability of the Faster-Rcnn model for targets of different scales is enhanced by a hierarchical sampling method, and the method includes: For the feature map extracted by the Faster-Rcnn model, an adaptive sampling method is used to dynamically adjust the number and position of sampling points according to the importance and distribution of the features. The formula is: Among them, f is a function that dynamically adjusts the sampling points according to the feature importance, I feature is the original feature map, ε is the scaling factor used to control the total number of sampling points; It is the coefficient that controls the influence of gradient, which is used to adjust the influence of gradient on sampling density; grad(I feature ) is the feature map I feature The gradient is used to represent the rate of change of features in the feature map; Normalize the result of function f to ensure that the total number of sampling points does not exceed the preset maximum value N max , the normalized formula is: The round function is used to round to the nearest integer, max(f(I feature )) indicates that f is in the entire feature map I feature The maximum value on is used to normalize the output of f to ensure that the sampling point does not exceed the preset maximum value N max ; A hierarchical sampling method is introduced in the Faster-Rcnn model: for targets below a preset size, the model performs dense sampling on feature maps with a resolution greater than a preset resolution to capture detail information; for targets above a preset size, the model performs sparse sampling on feature maps with a resolution less than a preset resolution to obtain overall structural information. The hierarchical sampling method is described by the resolution adjustment function R, which is: Among them, S size Represents the size of the target in the feature map, c and d are the first and second adjustment parameters, c controls the base, d is used to control the speed of exponential growth, and e is the base of the natural logarithm; A multi-scale fusion mechanism is introduced in the Faster-Rcnn model to fuse feature maps of different resolutions to generate a comprehensive feature representation: I combined =F fusion (I low-res ,I high-res )=Upsample(I high-res )+Conv(I low-res ) Among them, F fusion is a fusion function, Upsample represents upsampling, Conv represents a convolution operation, I low-res and I high-res Represent feature maps with resolutions lower than the preset resolution and higher than the preset resolution, respectively. combined It is the comprehensive feature representation after fusion.

3. The small sample target detection method according to claim 1, characterized in that: In step S2, the recognition capability of the Faster-Rcnn model for new categories in small sample images is enhanced by ternary category division and contrastive learning, and the method includes: When training the Faster-Rcnn model, a ternary classification loss and a contrast loss are constructed, where: The formula for the three-element classification loss is: Among them, L ternary is the triple classification loss, N represents the total number of anchor points considered, is the objectness score of the anchor point predicted by the model; is the true object label of the i-th generation anchor point, when When it is 1, it indicates a known foreground object. When it is 0, it means background. When it is 2, it indicates a potential new category object; represents the foreground loss function, represents the background loss function, The loss function for representing novel classes usually uses the cross entropy loss function; The formula for the contrast loss is: Among them, L contrastive is the contrast loss, N obj is the number of proposals with the same objectness label, N prop represents the total number of all proposals, I is the indicator function, if the condition is true, I = 1, otherwise I = 0; z i is the representation of the i-th feature proposal, obj gt,t is the ground-truth objectness label of the i-th proposal; It is a standardized feature representation used to calculate the cosine distance; Combining the ternary classification loss and the contrast loss yields: L total1 =L temary +μL contrastive Among them, L total1 It is the combined loss of the three-element classification loss and the contrastive learning loss, and μ is the weight parameter used to balance the two loss terms.

4. The small sample target detection method according to claim 3, characterized in that: In step S2, the classifier of the Faster-Rcnn model is enhanced in its ability to distinguish between different categories in a small sample image by using a simplex equiangular framework, and the method includes: Before model training, a set of maximized and equivalently separated weight vectors are calculated and used as fixed centers of all categories in small sample images, where: The weight vector is calculated as follows: first, the initialization vector w1 is set to the unit vector e1; then, the vector is constructed, for i=2,...,N c , choose vector w i with w1,...,w i-1 Orthogonal, and try to stay away from the existing vector; then adjust the vector, for each vector w i and w j , if the angle between them is not the largest By rotating w i To increase the vector w i and w j The angle between them, while maintaining the vector w i is a unit vector; repeat the adjustment until the angles between all vectors are as close as possible In order to enhance the intra-class compactness, an adaptive class-specific boundary is introduced, which is used to push the feature vector toward the corresponding class center w i near; Calculate the class-specific boundary offset using the formula: Among them, m c is the class-specific boundary offset for class c, p c is the instance frequency of category c, which indicates the proportion of the number of instances of category c to the total number of instances in the training data; p bg is the estimated probability of the background class, which indicates the proportion of the background class or non-target class in all classes; c b is the set of known foreground object categories, c n is a new category set; background is the background; According to the class-specific boundary offset m c Construct a simplex equiangular frame loss function, the formula is: Among them, L intra is the simplex equiangular frame loss, y c is the indicator function of the true label of sample x belonging to category c; v c is the original prediction value of sample x belonging to category c; m c is the class-specific boundary offset of class c; c' is other classes that are not class c; C is the total class set.

5. The small sample target detection method according to claim 4, characterized in that: After adjusting the vector w i and w j After the angle between them, we also need to continue to optimize the vector w i and w j Get the simplex equiangular frame weight matrix W * , and the optimized simplex equiangular frame weight matrix W * The weight parameters applied to the Faster-Rcnn model, where the simplex equiangular frame weight matrix W * The formula is: Among them, W * is the simplex equiangular frame weight matrix, argmax is the maximum index function, W is the weight set of the classifier, N c is the number of categories, w i and w j are the feature vectors representing the i-th and j-th categories respectively, d(w i -w j ) 2 Represents vector w i and w j The square of the distance between .

6. The small sample target detection method according to claim 4, characterized in that: After the step S2 enhances the classifier of the Faster-Rcnn model by the simplex isometric framework, the following steps are further included: The class-specific boundaries are then adjusted through the self-distillation mechanism to force the feature points to be more closely distributed around the class center. In the self-distillation process, a teacher model is first used to generate soft labels, which are used to reflect the model's prediction confidence for the input features. Then a student model with the same architecture as the teacher model but different parameters is used to learn the soft labels. The goal of the student model is to adjust its own parameters by imitating the predictions of the teacher model, thereby achieving adaptive adjustment of the class-specific boundaries. Construct the adaptive boundary distillation objective loss function, the formula is: Among them, L adapt is the adaptive boundary distillation objective loss, is the predicted probability of category c for the teacher model, y c is a binary variable whose true label of sample x belongs to category c. If x really belongs to category c, then y c =1, otherwise y c =0; is the predicted value of the student model on category c, is the adaptive boundary offset learned by the student model in category c.

7. The small sample target detection method according to claim 6, characterized in that: It also includes the combined loss L based on the triple classification loss and the contrastive learning loss total1 , simple equiangular frame loss L intra , Adaptive boundary distillation target loss L adapt , to construct the total loss function, the formula is: L tota2 =α(L total1 )+β(L intra )+λ(L adapt ) Among them, L total1 It is the combined loss of the triple classification loss and the contrastive learning loss; L intra is the simple equiangular frame loss; L adapt is the adaptive boundary distillation target loss; α, β, and λ are weight parameters used to balance the three losses; Through the total loss function L total2 To train the Faster-Rcnn model.

8. A small sample target detection system, characterized by: include: Acquisition module: used to obtain small sample images with multiple categories; Input module: used to input the acquired small sample image into the improved Faster-Rcnn model, where: The improved Faster-Rcnn model is obtained by improving the Faster-Rcnn model, and the improvement method includes: The Faster-Rcnn model's ability to perceive objects of different scales is enhanced through a hierarchical sampling method; Enhance the Faster-RCNN model's ability to recognize new categories in small sample images through ternary category division and contrastive learning; The Faster-Rcnn model classifier is enhanced to distinguish different categories in small sample images through a simplex equiangular framework. Detection module: used to detect targets of different categories in the image through the improved Faster-Rcnn model.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the steps of the small sample target detection method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the small sample target detection method according to any one of claims 1 to 7 are implemented.

Images