Small sample target detection method, system and device

Abstract

The present application discloses a small sample target detection method, system and device, which relates to the field of target detection. The method includes: dividing the data set according to the meta-learning data organization form to obtain support and query set images; using twin networks to process the support and query set images to obtain support and query set features; obtaining the first and second category prototypes and region of interest features based on the support and query set features; inputting the category prototypes and query set features into a multi-semantic category-aware region proposal network to obtain target prediction position information; using a region of interest alignment network to obtain the region of interest features of the query set; inputting the region of interest features of the query set and the second category prototype into a multi-granularity matching network of bidirectional feature alignment to obtain category scores and prediction box offsets, and then obtaining target detection results. The present application can significantly improve the recall rate of new category targets and improve the performance of small sample target detection rate.

Description

Technical Field

[0001] The present application relates to the field of target detection, and in particular to a small sample target detection method, system and device. Background Art

[0002] As one of the most basic and important tasks in computer vision, object detection has been developing rapidly in recent years. Object detection algorithms based on deep learning frameworks can achieve high detection accuracy after model training. But the only drawback is that the training of these object detection models relies on a large amount of instance-level annotation information. However, in reality, it may be difficult to sample due to the rarity of the target, or environmental and human conditions, making it impossible to obtain enough training data. Therefore, how to accurately identify and locate the target using a small number of image samples has become a very practical issue.

[0003] At present, the methods for small sample object detection mainly include meta-learning and transfer learning paradigms. There are fundamental differences in the basic principles of the two paradigms. In the paradigm of transfer learning, the image samples of the dataset are divided into two categories: base class and new class. In model training, a large amount of base class data is first used for training, and then the knowledge learned from the base class data, such as network parameters, feature representation, etc., is applied to the training process of the small sample new class to achieve fine-tuning of the model. The meta-learning paradigm is based on tasks, and each task consists of support set images and query set images. The purpose is to find the target object in the query set image that belongs to the support set category. Meta-learning is a training method for learning specific tasks from a deeper level, rather than learning the characteristic attributes of a specific target category. Therefore, in terms of the degree of dependence on the dataset, the transfer learning dataset requires that the new class and the base class targets have a close correlation, while meta-learning has a looser requirement for the correlation between the new class and the base class data.

[0004] Most of the current meta-learning-based Few-Shot Object Detection (FSOD) methods are based on a two-stage detection framework. The RoI (Region of Interest) dataset includes a large number of labeled base class target images and a small number of labeled new class target images. FSOD uses a sufficient number of base class target images to construct specific learning tasks (usually called N way K shot tasks), and then trains the model to be able to learn such tasks, thereby generalizing to the recognition of new classes of targets. It does not learn the specific feature attributes of these targets for specific targets like transfer learning or other general target detection algorithms.

[0005] In the specific content of constructing the N way K shot task, it is necessary to divide the base class samples and the mixed samples of the base class and the new class into the support set and the query set. Multiple batches of data can be obtained by sampling from them. The training process for each batch of data is called an episode. Each episode includes several support set images and query set images of the base class in the training. The key to training is how to effectively use the knowledge of the support set category prototype to help the model accurately identify the target in the query set. In the first stage of the two-stage detection method, the role of the support set is mainly reflected in helping the model to better determine the target in the query set and perform rough positioning. Among them, the Region Proposal Network (RPN) is the most important part. RPN is usually considered to be class-independent in general target detection methods. It deals with a simple two-class classification problem, namely foreground and background. Unlike general detection algorithms, the class-independence of RPN in small sample target detection has certain limitations: that is, the training of FSOD based on meta-learning is driven by tasks, and the object categories detected by each task may be different. In meta-learning, the target category in the current episode is different from the target category in the next episode, that is, the object regarded as the target in this training may be regarded as the background in the next training, which requires the RPN network to maintain relevance to the support set category. Some methods have improved RPN by constructing category matching features and combining feature fusion. However, since the reweighting term is a global pooling performed on the support features, candidate regions with high intersection-over-union (IoU) are discarded, and there is a problem of low recall rate of new class targets, which limits its performance. The proposal (proposal) generated by RPN, that is, the target, may have a deviation from the actual position of the target, which will cause feature misalignment between the RoI and the support set category prototype. There is a method to reduce this deviation by unidirectional feature alignment of the support set category prototype with the query set RoI, but this unidirectional feature alignment has limited improvement capabilities. In addition, the previous FSOD method is based on constructing the region of interest (RoI) feature of a specific class for the classification and regression of the proposal. Among them, most methods do not consider the feature misalignment problem caused by spatial position in the process of matching RoI and category prototype. Although a few methods try to solve this problem, they use simple one-way correction, which has limited improvement ability and still has the problem of reduced detection performance. Summary of the invention

[0006] The purpose of this application is to provide a small sample target detection method, system and device, which can significantly improve the recall rate of new category targets and improve the performance of small sample target detection rate, thereby solving the problems of low recall rate of new category targets and decreased detection performance of current algorithms.

[0007] To achieve the above objectives, this application provides the following solutions:

[0008] In a first aspect, the present application provides a small sample target detection method, comprising:

[0009] Acquire a small sample data set of the target to be detected; the small sample data set includes base class targets and new class targets; the number of the base class targets is greater than the number of the new class targets;

[0010] Dividing the small sample data set according to the data organization form of meta-learning to obtain support set images and query set images corresponding to the task;

[0011] Using a weight-sharing twin network to extract features from the support set image and the query set image to obtain support set features and query set features;

[0012] Performing average calculation on the support set features to obtain a first category prototype;

[0013] Constructing a multi-semantic category-aware region proposal network, and inputting the first category prototype and the query set features into the multi-semantic category-aware region proposal network to obtain target prediction position information;

[0014] Using a region of interest alignment network to obtain a region of interest feature of a query set based on the target predicted position information and the query set features;

[0015] Using a region of interest alignment network to obtain region of interest features of a support set based on the support set features and the support set image;

[0016] averaging the features of the region of interest of the support set to obtain a prototype of the second category;

[0017] Constructing a multi-granularity matching network for bidirectional feature alignment, and inputting the region of interest features of the query set and the second category prototype into the multi-granularity matching network for bidirectional feature alignment to obtain a category score and a prediction box offset;

[0018] An object detection result is obtained based on the category score and the prediction box offset.

[0019] In a second aspect, the present application provides a small sample target detection system comprising:

[0020] A data set acquisition module is used to acquire a small sample data set of the target to be detected; the small sample data set includes base class targets and new class targets; the number of the base class targets is greater than the number of the new class targets;

[0021] A data set partitioning module, used to partition the small sample data set according to the data organization form of meta-learning to obtain support set images and query set images corresponding to the task;

[0022] A feature extraction module, used to extract features from the support set image and the query set image using a weight-sharing twin network to obtain support set features and query set features;

[0023] A first average value calculation module, used for performing average calculation on the support set features to obtain a first category prototype;

[0024] A network construction module is used to construct a multi-semantic category-aware region proposal network, and input the first category prototype and the query set features into the multi-semantic category-aware region proposal network to obtain target prediction position information;

[0025] A first region feature extraction module, configured to obtain the region of interest features of the query set based on the target predicted position information and the query set features by using a region of interest alignment network;

[0026] A second region feature extraction module is used to obtain the region of interest features of the support set based on the support set features and the support set image by using a region of interest alignment network;

[0027] A second average value calculation module is used to average the features of the region of interest of the support set to obtain a second category prototype;

[0028] A category score-offset determination module is used to construct a multi-granularity matching network for bidirectional feature alignment, and input the region of interest features of the query set and the second category prototype into the multi-granularity matching network for bidirectional feature alignment to obtain a category score and a prediction box offset;

[0029] The target detection module is used to obtain the target detection result based on the category score and the prediction box offset.

[0030] In a third aspect, the present application provides a computer device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of any of the small sample target detection methods described above.

[0031] According to the specific embodiments provided in this application, this application has the following technical effects:

[0032] This application provides a small sample target detection method, system and device. By constructing a multi-semantic class-aware region proposal network (MC-RPN), it can fully mine the spatial information of supporting features and integrate multiple semantic descriptions of similar targets, thereby significantly improving the recall rate of new category target proposals and improving the detection performance of the model. At the same time, by constructing a multi-granularity matching network (Multi-granularity Fusion Head with Bidirectional Feature Alignment, MFHw / BFA) with bidirectional feature alignment, it can effectively solve the problem of feature misalignment and realize cross-matching of semantics at different levels. By introducing MC-RPN and MFHw / BFA, the purpose of improving the small sample target detection rate and target detection performance can be achieved. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0034] Figure 1 A schematic diagram of a flow chart of a small sample target detection method provided in an embodiment of the present application;

[0035] Figure 2 An overall framework diagram of a small sample target detection method provided in an embodiment of the present application;

[0036] Figure 3 A schematic diagram of the structure of a multi-semantic category-aware region proposal network provided in an embodiment of the present application;

[0037] Figure 4 A schematic diagram of the structure of a multi-granularity matching network for bidirectional feature alignment provided in an embodiment of the present application;

[0038] Figure 5 A schematic diagram of the structure of a multi-granularity fusion module provided in one embodiment of the present application. DETAILED DESCRIPTION

[0039] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0040] This application aims to design a unique fusion method from a multi-semantic perspective, and by making full use of the spatial relationship between the two, the query set can better refer to the feature information of the support set, so as to improve the recall rate of new types of targets in the first stage of target detection through the improvement of RPN, thereby achieving the purpose of improving the performance of small sample target detection. At the same time, this application solves the problem of feature misalignment by designing RoI and category prototypes.

[0041] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, the present application is further described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0042] In an exemplary embodiment, the present application provides a small sample target detection method, which is executed by a computer device, specifically, it can be executed by a computer device such as a terminal or a server alone, or it can be executed by a terminal and a server together. In the embodiment of the present application, the method is applied to a server as an example for description. Figure 1 As shown, the method includes:

[0043] Step 100: Obtain a small sample data set of the target to be detected. This small sample data set contains a large number of base class targets and a small number of new class targets.

[0044] Step 101: Divide the small sample data set according to the data organization form required for meta-learning to obtain support set images and query set images corresponding to the task.

[0045] Step 102: Use a weight-sharing twin network to extract features from the support set image and the query set image to obtain support set features and query set features.

[0046] Step 103: average the support set features to obtain the first category prototype. The calculation formula of the first category prototype is:

[0047] (1)

[0048] In the formula, represents the support set, c represents the support set A category of the target object to be tested in represents the first category prototype of category c, K represents the number of shots in K shots, represents the nth image of category c, Represents the calculation results obtained by the twin network.

[0049] Step 104: construct a multi-semantic category-aware region proposal network, and input the first category prototype and query set features into the multi-semantic category-aware region proposal network to obtain target prediction position information.

[0050] Step 105: Using the region of interest alignment network to obtain the region of interest features of the query set based on the target prediction position information and the query set features.

[0051] Step 106: Using a region of interest alignment network to obtain the region of interest features of the support set based on the support set features and the support set image.

[0052] Step 107: Calculate the average of the features of the region of interest in the support set to obtain the prototype of the second category.

[0053] Step 108: construct a multi-granularity matching network for bidirectional feature alignment, and input the region of interest features and the second category prototype of the query set into the multi-granularity matching network for bidirectional feature alignment to obtain the category score and the prediction box offset.

[0054] Step 109: Obtain the target detection result based on the category score and the predicted box offset.

[0055] In another exemplary embodiment of the present application, the small sample target detection method provided by the present application can be divided into three components: the first part is a twin network with shared weights, which is used to extract features from the support set and query set images, wherein the support set images are obtained by cropping the annotation information of the existing data set and only contain part of the instance content of the target. The query set image is a normal image to be tested with background information, and the main function of this part is to extract features of the query set and support set images. The second part is MC-RPN. This network can cleverly utilize the spatial information of the category prototype and query set features, and capture the unique expression of the target features to be tested from multiple semantic dimensions through a series of semantic matching networks with the same structure but independent weights. Afterwards, these multi-semantic features are fused to achieve deep integration of these multi-semantic expressions, and a simple nonlinear head structure is used to accurately evaluate the similarity between the dense anchor box in the query set and the small sample new class features, thereby generating high-quality, category-specific candidate region proposals for small sample new class targets. The third part is MFHw / BFA. This network obtains the semantic similarity between RoI and category prototype by constructing the correlation matrix between the two, and performs accurate bidirectional feature alignment between the features of the region of interest and the category prototype. The aligned features are matched and fused at multiple granularity levels, enabling the model to more comprehensively learn the relationship between the query set and the support set objects, thereby improving the detection accuracy of the target.

[0056] In another exemplary embodiment of the present application, the motivation for improving MC-RPN lies in the limitations of the class-independent binary classification theory of ordinary RPN under the meta-learning framework. Since the meta-learning method emphasizes task-driven learning, the object categories that need to be detected in different tasks are different. Objects that are regarded as backgrounds in the current task may need to be regarded as targets in subsequent tasks. This requires a close connection between the construction of RPN and the support set categories to ensure that the model can effectively acquire meta-knowledge. Ignoring this may mistakenly identify high IoU candidate areas that should belong to new categories as backgrounds, resulting in errors. Therefore, it is crucial to improve RPN. Among the improvement strategies of RPN, a widely used method is to use the information of the support set to fuse and enhance the features of the query image, thereby learning potential meta-knowledge from more refined feature descriptions and ultimately revealing the location information of the object. However, this fusion is difficult because only the key instance parts in the image are retained in the support set, while the query set retains background information in addition to the target, which results in different image sizes for the support set and the query set, and the feature maps of the two cannot be directly fused through operations such as splicing or subtraction. Although there is a method to adjust the support set category prototype to After resizing, it is multiplied by the query set channel to enhance the features of the target image. However, this method has two major problems: (1) Directly weighting the query features according to the spatial pooling results of the support set category prototypes may lead to false detection due to semantic feature mismatch. (2) The pooling operation ignores the spatial information of the category prototype, resulting in the meta-knowledge obtained during the training process being not comprehensive and sufficient. This application solves the above two major problems by constructing MC-RPN. The core idea is to enable MC-RPN to fully utilize the spatial information of the support set features by fusing the support set features from multiple semantic perspectives. In the process of fusion of specific semantic perspectives, by identifying the common high-level semantic contextual information between the support set and the query set, and enhancing the query set features based on this, the purpose of effectively increasing class-aware information is achieved, while filtering out other redundant information, thereby generating more targeted suggestions for the support set. Specifically, MC-RPN consists of a multi-semantic fusion network and a nonlinear prediction network. Its structure is as follows: Figure 3 Based on this, the implementation process of the above step 104 can be described as:

[0057] 1) Construct a multi-semantic fusion network and a nonlinear prediction network to obtain a multi-semantic category-aware region proposal network. The multi-semantic fusion network includes multiple semantic matching networks with the same structure but different weights and a multi-head feature fusion module.

[0058] 2) Input the first category prototype and query set features into a multi-semantic fusion network to obtain fusion enhanced features of multiple semantic structures. For example, input the category prototype and query set features into three semantic matching networks with the same structure but different weights and a multi-head feature fusion module. In actual application, the implementation process of step 2) includes:

[0059] 2-1) Using multiple semantic matching networks with the same structure but different weights to obtain enhanced feature graphs of multiple semantic structures based on the first category prototype and query set features. The process of obtaining an enhanced feature graph based on the first category prototype and query set features by each semantic matching network includes:

[0060] 2-11) Unify the semantics of the first category prototype and the query set features to obtain a semantically unified result. That is, determine a common semantic context for the category prototype and the query set to unify the semantics of the category prototype and the query set features.

[0061] 2-12) Couple the query set with the semantically unified result to obtain the query set matching block. That is, couple the query set with the determined common semantic context.

[0062] Based on the description of the above steps 2-11) to 2-12), the specific calculation process can be expressed as:

[0063] (2)

[0064] (3)

[0065] (4)

[0066] In the formula, and They represent the semantic matching modules that unify the first category prototype and query set features, represents a semantically coupled module, represents average pooling, represents the rectified linear unit, Represents a query set feature. Represents the index number of each semantic matching network. , , Both said The convolutional layer, , , Both said All convolutional layers are used to adjust the semantic information so that the support set and the query set match at a higher semantic level.

[0067] 2-13) Sum the matching and coupling results to incorporate common semantic information into the query set, which is:

[0068] (5)

[0069] In the formula, Indicates i The output of the semantic matching network, Represents the query set features output through the semantic coupling module, and Respectively represent the query set features and the output of the first category prototype through the semantic matching module, represents the dot product, for The convolutional layer.

[0070] 2-2) The multi-head feature fusion module is used to fuse the enhanced feature maps of multiple semantic structures to obtain the fusion enhanced features of multiple semantic structures. Among them, the multi-head feature fusion module is used to integrate the enhanced features under multiple semantics, which can more clearly and comprehensively represent the features of the query set, including:

[0071] (6)

[0072] In the formula, Represents the fusion enhanced feature map under multiple semantics, , , Represent the outputs of three semantic matching networks respectively. Indicates a connection operation. is a The convolutional layer.

[0073] 3) Input the fused enhanced features into the nonlinear prediction network to obtain the target prediction position information. The fused enhanced features are used to generate anchor points and calculate the category scores and position offsets. When predicting the scores, this application designs a simple and efficient nonlinear detection head to compensate for the loss of support set feature information caused by previous matching. The suggestions generated using MC-RPN can achieve high recall rates for new categories, thus benefiting the detection network.

[0074] In another exemplary embodiment of the present application, by adopting MC-RPN, the possible location of the target (target predicted location information), namely the proposal, can be obtained. However, since the proposal given by MC-RPN inevitably deviates from the actual location of the target, there will be feature misalignment between the category prototype and the RoI feature. This will have a serious adverse effect on subsequent classification and regression. At the same time, the current FSOD method only focuses on the semantic relationship at a fixed level, while ignoring the semantic relevance of different granularities. The above two reasons will cause a decrease in detection accuracy. Therefore, the present application constructs the following Figure 4 The MFHw / BFA shown in the figure can solve this problem. Based on this, the implementation process of the above step 108 of the present application can be described as:

[0075] 1. In the multi-granularity matching network of bidirectional feature alignment, the features of the region of interest of the second category prototype and the query set are encoded along the channel direction to obtain the first codeword sequence and the second codeword sequence The code elements in the two codeword sequences are the values ​​of each spatial position, and the codeword length is the number of channels C.

[0076] 2. The first codeword sequence and the second codeword sequence Perform layer normalization processing respectively, and stretch the codeword sequence after layer normalization processing to a set length (for example ), get the second category prototype codeword sequence and query set RoI codeword sequence. Among them, the first codeword sequence and the second codeword sequence The purpose of performing layer normalization separately is to balance the size of the codeword. Based on this, the specific calculation formula is:

[0077] (7)

[0078] (8)

[0079] In the formula, Represents the first codeword sequence The The code word Code elements, Represents the second codeword sequence The kth code element of the nth code word in . represents the mean operation, represents the variance operation, is a covariate that maintains numerical stability. and are learnable affine parameters used to scale and normalize data for subsequent calculations. and They are and The result after layer normalization.

[0080] 3. The code value of each position in the normalized second category prototype code word sequence The codeword value at each position in the query set RoI codeword sequence Perform inner product operation, using Indicates that the size is The cross-correlation matrix The calculation formula is:

[0081] (9)

[0082] Cross-correlation matrix It can reflect two semantic features (i.e., category prototype codeword and category prototype codewords ) at corresponding spatial positions.

[0083] 4. In order to prepare for the subsequent weight adjustment, the elements in the cross-correlation matrix are adjusted column by column and row by row. Operation, get the first vector sequence and the second vector sequence The calculation formula is as follows:

[0084] (10)

[0085] (11)

[0086] In the formula, is the cross-correlation matrix In coordinates The value at . The first vector sequence In coordinates The value at The second vector sequence In coordinates The value at . The first vector sequence To proceed line by line The result of the calculation. The second vector sequence To do it column by column The result of the calculation.

[0087] 5. In the multi-granularity matching network of bidirectional feature alignment, use Convolution changes the organizational structure of the first category prototype codeword sequence and the second category prototype codeword sequence at the channel level and straightens them to obtain the first organizational structure feature and second organizational structure characteristics .in, , Indicates size The real matrix of is:

[0088] (12)

[0089] In the formula, Indicates a straightening operation. and Both The convolutional layer, represents the second category prototype, Represents the region of interest features of the query set.

[0090] 6. Use the first vector sequence and the second vector sequence Adjust the first organizational structure characteristics respectively and second organizational structure characteristics , so that after the channel level matching in step 5, the original features are further matched in the spatial structure direction to obtain the first matching feature and the second matching feature The specific calculation is achieved through the Einstein summation convention, and the formula is as follows:

[0091] (13)

[0092] In the formula, Indicates the second matching feature The feature of the mth channel, Indicates the first matching feature The characteristics of the nth channel, Indicates the second organizational structure feature The characteristics of the nth channel, Indicates the first organizational structure feature The features of the mth channel.

[0093] 7. For the second vector sequence and the first vector sequence The vectors in are accumulated and normalized horizontally and vertically respectively to obtain the first attention mask and the second attention mask .

[0094] 8. Use the First Attention Mask and the second attention mask For the first matching feature and the second matching feature Perform mask processing to obtain the first feature alignment compensation Alignment compensation with the second feature .in:

[0095] (14)

[0096] (15)

[0097] In the formula, Indicates the first feature alignment compensation The characteristics of the nth channel, Represents the first attention mask The nth mask value of represents normalization, Indicates the second feature alignment compensation The feature of the mth channel, Represents the second attention mask The mth mask value of .

[0098] 9. Align the first feature to the compensation amount Alignment compensation with the second feature The first characteristic coefficient and the second characteristic coefficient Multiply and add to the second category prototype and the region of interest features of the query set Add element by element to get the aligned second category prototype And the region of interest features of the aligned query set ,have:

[0099] (16)

[0100] (17)

[0101] 10. A multi-granularity fusion module is used to obtain the category score and prediction box offset based on the aligned second category prototype and the aligned region of interest features of the query set.

[0102] In another exemplary embodiment of the present application, in order to further improve the detection accuracy, the detection head of the multi-granularity fusion module used can be composed of four parts: coarse-grained matching, medium-grained matching, fine-grained matching and synthetic matching, so as to improve the detection performance through diversified matching of high-level and low-level features. The specific structure is as follows: Figure 5 As shown. Among them, coarse-grained matching is a way to implement high-level feature matching, specifically, it is to pool and connect two input features and then perform convolution. Medium-grained matching is to match between channel-level semantics, which is achieved by channel-by-channel convolution of two features. Fine-grained matching is to match between low-level semantics, specifically, it is to directly splice two features in the channel dimension and then perform convolution. Synthetic matching is achieved by matching the high-level features of the support set features with the low-level features of the RoI features, specifically, it is to perform dot multiplication of the pooled results of the support set with the RoI features. Based on this, the implementation process of the above step 10 of the present application may include:

[0103] Step 1) performs coarse-grained matching on the aligned second category prototypes and the aligned region of interest features of the query set to obtain the first category score and the first position offset.

[0104] Step 2) Perform medium-granularity matching on the aligned second category prototypes and the aligned region of interest features of the query set to obtain the second category score and the second position offset.

[0105] Step 3) Perform fine-grained matching on the aligned second category prototypes and the aligned region of interest features of the query set to obtain the third category score and the third position offset.

[0106] Step 4), perform synthetic matching on the aligned second category prototypes and the aligned region of interest features of the query set to obtain a fourth category score and a fourth position offset.

[0107] Step 5), based on the first category score, the first position offset, the second category score, the second position offset, the third category score, the third position offset, the fourth category score and the fourth position offset, obtain the category score and the prediction box offset.

[0108] The calculation process of the above-mentioned different granularity matching detection heads is expressed as:

[0109] (18)

[0110] In the formula, Represents channel-wise convolution. , , and Representing four and a A neural network consisting of convolutional layers, ReLU layers and fully connected layers. represents the category score, Indicates the position offset, represents the first category score, Indicates the first position offset, represents the second category score, Indicates the second position offset, represents the third category score, Indicates the third position offset, represents the fourth category score, Indicates the fourth position offset, Indicates a connection operation. Represents a pooling operation.

[0111] Based on the above description, coarse-grained matching is achieved between high-level features. Specifically, the two input features are first pooled and concatenated. Then, they are processed through two (2048, 2048) fully connected layers and ReLU layers, and finally sent to the (2048, 2) and (2048, 4) fully connected layers to calculate the class score and position offset. Medium-grained matching is to match the corresponding features at the channel level.

[0112] Specifically, it is achieved by convolving the two features channel by channel. The results of the channel by channel convolution are directly input into the fully connected layers of (2048, 2) and (2048, 4) to calculate the category score and position offset. For fine-grained matching between low-level semantics, specifically, the two features are directly concatenated in the channel dimension. Then, through two The convolutional layer and pooling layer of the support set realize cross-channel low-level feature cross-matching, and finally send it to the fully connected layer of (2048, 2) and (2048, 4) for prediction. The synthetic matching is completed by matching the high-level features of the support set features with the low-level features of the RoI features. Specifically, the pooling result of the support set is dot-multiplied with the RoI feature. The result of the dot-multiplication is passed through two and a The image is processed by the convolutional layer and then sent to the fully connected layer for prediction.

[0113] The above four parts can obtain their own category scores and position offsets respectively, and the sum of these values ​​​​of the four parts is the final prediction result. Classify and use position offset And anchor information for regression.

[0114] In summary, based on the meta-learning paradigm, the method provided in this application introduces two networks, MC-RPN and MFHw / BFA. First, it uses the spatial information of the features to construct multi-semantic representations and designs a new RPN network with attention to new types of targets. Second, a new network is designed to align the category prototypes and proposal features of the support set to improve the problem of feature misalignment when matching the two. At the same time, the aligned features are matched at multiple granularities to maximize the correlation between the RoI features and the category prototypes at the object-wise level. Ultimately, the model can achieve significant performance improvements in multiple key indicators, achieving the effect of improving the detection rate of small sample targets.

[0115] In another exemplary embodiment of the present application, the present application proposes a small sample target detection method based on meta-learning, and its overall framework is as follows: Figure 2 As shown in the figure, this method mainly includes two core modules: MC-RPN and MFHw / BFA. The two modules work together to effectively improve the detection performance of small sample targets.

[0116] Specifically, the training of this method is divided into two stages: meta-training stage and fine-tuning stage. The meta-training stage is mainly to learn the meta-knowledge of a large number of base class objects, where the data required for each episode includes support images and query images. All support and query together constitute the meta-training set H train , and H train All instances in the image are derived from the base class target images. In the fine-tuning phase after meta-training, based on the rich meta-knowledge of the base class targets previously acquired, new target images to be tested are added to further learn the knowledge of the new class targets. The fine-tuning training set H constructed in this phase is finetune It not only includes instances from base class targets, but also samples from new categories.

[0117] In each episode of training, from H train (Meta-training phase) or H finetune(fine-tuning stage), and the sampling results are input into the weight-sharing twin backbone network, which can choose to use simple or complex feature extraction networks according to the actual computing power conditions, such as ResNet, DenseNet, MobileNet, etc. In this embodiment, ResNet101 is selected as the twin backbone network. After the support set features and query set features are obtained respectively through the twin backbone network, the support set features are averaged to obtain the category prototype, and the category prototype and query set features are input into MC-RPN together. MC-RPN can calculate the possible location of the target, and then obtain the region of interest features of the query set through the region of interest alignment network RoI-Align, and input the features and the category prototype of the support set into the multi-granularity fusion detection head of the bidirectional feature alignment for feature alignment, fusion and detection, and obtain the final category score logits and prediction box offset delta. In the inference stage, the category score and prediction box offset can be used to obtain the final detection result.

[0118] Based on the above description, the small sample target detection method provided by this application may include the following steps in actual application:

[0119] S1. Dataset division. The dataset is divided into base class and new class image data. According to the requirements of meta-learning training, samples are taken from the two types of data to form the support images and query images required for each episode. In the meta-training stage, all the support and query images used together constitute the meta-training set H train In the meta-fine-tuning stage, the support and query used together constitute the fine-tuning training set H finetune .

[0120] S2. Construct a network structure. Based on the two-stage target detection framework, extract the features of the support set and query set images through the feature extraction network, and add a multi-semantic class-aware region recommendation network after the feature extraction network to generate high-quality, specific category candidate regions for small sample new class targets. At the same time, a multi-granularity fusion detection head with a bidirectional feature alignment function is added after the multi-semantic class-aware region recommendation network to solve the adverse effects of feature misalignment.

[0121] S3. Use the meta-training set H train Meta-train the model.

[0122] S4. Use the meta-fine-tuning set H finetune Meta-fine-tune the model. In step S4, the process of meta-fine-tuning is basically the same as the process of meta-training above, except that during meta-fine-tuning, the support set and query set used for training are from the H finetune , and the learning rate is reduced during the meta-fine-tuning phase.

[0123] S5, input the image to be tested of the small sample target, use the model fine-tuned in step S4 to perform inference, and output the detection result. That is, the image to be tested is sent to the trained network, and the target detection result is obtained through inference.

[0124] Further, in step S1, the meta-training set H train contains only instances of the base class target, while the meta-fine-tuning set H finetune It includes not only base class objects, but also instances of new class objects. In step S2, the following steps are specifically included:

[0125] S21: Construct a feature extraction network, which is a weight-shared twin network used to extract features from the support set and query set images. The specific network structure is not limited, and a suitable backbone network can be selected according to the hardware resource configuration, such as ResNet, DenseNet, MobileNet, etc.

[0126] S22: Construct a multi-semantic class-aware region recommendation network. This network uses the spatial information of the category prototype and query set features to capture the feature representation of the target from multiple semantic dimensions through several semantic matching networks with the same structure but independent weights. And through multi-head feature fusion and a small nonlinear network, it accurately evaluates the similarity between the dense anchor boxes in the query set and the features of the small sample new class, thereby generating high-quality, specific category candidate regions for the small sample new class targets.

[0127] S23: Construct MFHw / BFA. The network consists of two modules, namely the bidirectional feature alignment module and the multi-granularity fusion module. The bidirectional feature alignment module obtains the semantic similarity between the RoI and the category prototype by constructing the cross-correlation matrix between the two, and performs accurate bidirectional feature alignment between the features of the region of interest and the category prototype. The multi-granularity fusion module matches and fuses the aligned features at multiple granularity levels, so that the model can learn the relationship between the query set and the support set objects more comprehensively, thereby improving the detection accuracy of the target.

[0128] Further, in step S3, the meta-training set H is used train The process of meta-training the model is as follows:

[0129] S31: According to the N-way K-shot task in the meta-learning paradigm, an episode training method is adopted to train H in each episode. trainSampling is performed. For each type of target, K images are extracted as support set images according to the set shot value, and one image that is not repeated in the support set is extracted as the query set image. The extracted support set and query set images are used as input for each episode. The support set images only contain part of the instance content of the target, which can be obtained by cropping the original image according to the annotation information, while the query set images are normal images to be tested with background information.

[0130] S32: Input the support set and query set images into the twin network, and obtain the support set feature map and the query set feature map after calculation through the twin network.

[0131] S33: average the K support set feature maps to obtain the support set category prototype P1, input P1 and the query set feature map into the multi-semantic class-aware region recommendation network, and obtain the query image suggestion box proposal. The specific processing process of the multi-semantic class-aware region recommendation network is as follows:

[0132] S331: The category prototype P1 of the support set and the query set features are first input into several parallel feature enhancement networks. These feature enhancement networks have the same structure but different weights. The processing of each feature enhancement network includes two steps: first, through the semantic matching module and A common semantic context is determined for the category prototype and query set to unify the semantics of the category prototype and query set features. The query set is coupled with the determined common semantic context. The specific calculation formula is shown in the above formula (2) to formula (4).

[0133] S332: Using the coupled semantic features, perform element summation on the matching blocks of the query set, and incorporate the information of the support set into the query set. The calculation formula is shown in the above formula (5).

[0134] S333: After obtaining the enhanced feature map with semantic structure, a multi-head feature fusion module is used to further integrate the enhanced features under multiple semantics. The calculation formula is shown in formula (6).

[0135] S334: The enhanced features are used to generate anchor points and calculate the category score and position offset. A nonlinear prediction network is used to calculate the prediction score, and the calculation formula is:

[0136] (19)

[0137] In the formula, represents the category score of the multi-semantic category-aware region proposal network, represents the position offset of the multi-semantic category-aware region proposal network, Indicates a The convolutional layer, , , , , It means 5 different Convolutional layers, represents the Relu function, Represents the enhanced features after fusion of multiple semantics, Represents the category prototypes of the support set.

[0138] S34: Input the obtained suggestion box and the feature map of the query image into the region of interest alignment (RoI Align) network to obtain the query set region of interest feature RoI of fixed size.

[0139] S35: Send the features of the support set obtained in step S32 to the RoI Align network and perform average calculation again to obtain the second category prototype P2 of the support set.

[0140] S36: Send the second category prototype P2 and the query set RoI obtained in step S34 to MFHw / BFA for feature alignment and multi-granularity fusion prediction to obtain the prediction result. In this step, the multi-granularity fusion detection head for bidirectional feature alignment consists of two modules, namely, a bidirectional feature alignment module and a multi-granularity fusion module. Based on this structure, its specific implementation process is as follows:

[0141] S361: Encode the category prototype P2 and the query set RoI along the channel direction to obtain the first codeword sequence and the second codeword sequence .

[0142] S362: The first codeword sequence and the second codeword sequence Perform layer normalization and straighten it to a length of The specific calculation formulas are shown in the above formulas (7) and (8).

[0143] S363: Perform an inner product operation on the category prototype codeword and the RoI codeword at each position in the feature map space to obtain a cross-correlation matrix, the calculation formula of which is shown in the above formula (9).

[0144] S364: Perform softmax operations on the elements in the cross-correlation matrix column by column and row by row to obtain a first vector sequence and the second vector sequence , and its calculation formula is shown in the above formula (10) and formula (11).

[0145] S365: In addition to spatial structure, use Convolution changes the structural organization of support set features and RoI features at the channel level and straightens them to obtain the first organizational structure feature and second organizational structure characteristics , the specific calculation formula is shown in the above formula (12).

[0146] S366: Through the first vector sequence Adjustment in two dimensions: channel and spatial structure and , thus obtaining the first matching feature and the second matching feature , the specific calculation formula is shown in the above formula (13).

[0147] S367: Yes and The vectors in are accumulated and normalized horizontally and vertically to obtain the attention mask of each unit in the codeword, and multiplied by the matching feature to obtain the feature alignment compensation, as shown in the above formulas (14) and (15).

[0148] S368: The characteristic compensation amount and Respectively with the learnable feature coefficients and Multiply and and Add element by element to achieve bidirectional feature alignment and obtain the aligned features and .

[0149] S369: The feature-aligned category prototype and query set RoI are sent to the multi-granularity matching detection module to obtain the detection results at four different granularities, and they are added together to obtain the total category score and position offset. The calculation formula is shown in the above formula (18).

[0150] S37: Compare the predicted results with the true labels, calculate the classification loss, positioning loss and RPN loss, and perform backpropagation to update the network weights.

[0151] Based on the above description, the network module given in the method provided in this application can also be applied to two-stage small sample target detection methods in other meta-learning paradigms, and is not limited to the backbone network structure, while improving the detection performance of small sample targets.

[0152] In another exemplary embodiment of the present application, for the usability and effectiveness of the present application, the present application uses two widely used data sets in the FSOD field, namely, the PASCAL VOC few-shot data set (few-shot VOC) and the COCO few-shot data set (few-shot COCO). These two data sets are formed by re-dividing and organizing the sampling results of the VOC2007+2012 and COCO data sets. To ensure a fair comparison with other algorithms, this embodiment uses the same small number of images for verification.

[0153] The hardware and software environment used in the embodiment is as follows: Faster RCNN is selected as the network infrastructure, and the pre-trained ResNet-101 is used as the twin feature extraction network. The training is started using the SGD optimizer with a warm-up phase of 1000 times, the input batch size is 8, and the initial learning rate is set to 0.002 (few-shot coco dataset) and 0.001 (few-shot voc dataset). In terms of hardware, two Nvidia A100 image processors with 80G video memory are used in the training, and the software environment includes a computer vision library based on PyTorch 1.10 and a detectron 2 target detection framework.

[0154] Under the above settings, this method is compared with the current advanced small sample target detection methods. The results are shown in Tables 1 and 2.

[0155] Table 1 Comparison table of other advanced algorithms under Few-Shot VOC dataset

[0156]

[0157] The two sets of comparative experiments in Table 1 and Table 2 use the same data splitting format and training images. In Table 2, since some comparative methods only conduct experiments under certain fixed sample numbers, and do not conduct complete experiments on 1-shot, 2-shot, 5-shot, and 10-shot, the parts that are not tested are indicated by "-" in Table 2. From the experimental results, it can be seen that after meta-training and meta-fine-tuning, our method has significantly improved the accuracy of small sample target detection, especially for larger sample numbers (such as 5-shot, 10-shot, and 30-shot).

[0158] Table 2 Comparison table of Few-Shot COCO dataset with other advanced algorithms

[0159]

[0160] Based on the above description, compared with the known small sample target detection methods, the small sample target detection method provided by this application has the following advantages:

[0161] In this application, by constructing MC-RPN, a multi-semantic fusion idea is proposed. Through a multi-head structure, a series of semantic matching networks with the same structure but independent weights are used to capture the unique expression of RoI from multiple semantic dimensions, and then a multi-head fusion network is used to deeply integrate these multi-semantic expressions. Finally, a simple nonlinear head structure is used to accurately evaluate the similarity between the dense anchor boxes in the query set and the features of the new class of small samples. In this multi-semantic fusion framework, since the weights of each semantic matching network are independent, it means that different convolution kernels are used to calculate the same category prototype and query features. This method can effectively utilize the spatial information of the features and achieve deep feature fusion. It can generate high-quality, specific category candidate region proposals for small sample new class targets, thereby improving the recall rate of the model for the proposal box of the new class target in the first stage, and then improving the detection performance.

[0162] In this application, considering that the support set category prototype and the query set RoI use a twin convolutional neural network, the spatial positions of the feature maps of the two should be consistent. Therefore, the network encodes the two along the channel direction, and obtains the mutual correlation matrix of the two by means of inner product. The mutual correlation matrix is ​​respectively subjected to Einstein summation operation with the category prototype and the query set RoI, and reweighted to obtain the respective calibration offsets. The offset is applied to the original category prototype and RoI features, which can work together from two directions to better align the features. Compared with the existing one-way feature alignment method, the present application can better eliminate the influence of feature misalignment, thereby having a better detection effect. On this basis, by constructing MFHw / BFA, the aligned category prototype and RoI features can continue to realize cross-matching of semantics at different levels, so as to make better use of the support set information to assist in the detection of targets in the query set.

[0163] In summary, by introducing the two networks of MC-RPN and MFHw / BFA, the method provided in this application can achieve significant performance improvement in multiple key indicators, thereby achieving the effect of improving the detection rate of small sample targets.

[0164] In an exemplary embodiment, a small sample target detection system is provided, comprising:

[0165] The data set acquisition module is used to acquire a small sample data set of the target to be detected. The small sample data set includes base class targets and new class targets. The number of base class targets is greater than the number of new class targets.

[0166] The data set partitioning module is used to partition the small sample data set according to the data organization form of meta-learning to obtain support set images and query set images corresponding to the task.

[0167] The feature extraction module is used to extract features from the support set images and the query set images using a weight-sharing twin network to obtain support set features and query set features.

[0168] The first average value calculation module is used to perform average calculation on the support set features to obtain a first category prototype.

[0169] The network construction module is used to construct a multi-semantic category-aware region proposal network, and input the first category prototype and query set features into the multi-semantic category-aware region proposal network to obtain target prediction position information.

[0170] The first region feature extraction module is used to obtain the region of interest features of the query set based on the target predicted position information and the query set features by using the region of interest alignment network.

[0171] The second region feature extraction module is used to obtain the region of interest features of the support set based on the support set features and the support set image by using the region of interest alignment network.

[0172] The second average value calculation module is used to perform average calculation on the features of the region of interest of the support set to obtain a second category prototype.

[0173] The category score-offset determination module is used to construct a multi-granularity matching network for bidirectional feature alignment, and input the region of interest features and the second category prototype of the query set into the multi-granularity matching network for bidirectional feature alignment to obtain the category score and the prediction box offset.

[0174] The object detection module is used to obtain object detection results based on category scores and prediction box offsets.

[0175] In an exemplary embodiment, a computer device is provided, which may be a server or a terminal, and includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the small sample target detection method provided in the present application.

[0176] The technical features of the above embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0177] This article uses specific examples to illustrate the principles and implementation methods of this application. The description of the above embodiments is only used to help understand the method and core ideas of this application. At the same time, for those skilled in the art, according to the ideas of this application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting this application.

Claims

1. A small sample target detection method, characterized in that: The small sample target detection method comprises: Acquire a small sample data set of the target to be detected; the small sample data set includes base class targets and new class targets; the number of the base class targets is greater than the number of the new class targets; Dividing the small sample data set according to the data organization form of meta-learning to obtain support set images and query set images corresponding to the task; Using a weight-shared twin network to extract features from the support set image and the query set image to obtain support set features and query set features; Performing average calculation on the support set features to obtain a first category prototype; Constructing a multi-semantic category-aware region proposal network, and inputting the first category prototype and the query set features into the multi-semantic category-aware region proposal network to obtain target prediction position information; wherein, constructing a multi-semantic fusion network and a nonlinear prediction network to obtain a multi-semantic category-aware region proposal network; the multi-semantic fusion network includes multiple semantic matching networks with the same structure but different weights and a multi-head feature fusion module; Using a region of interest alignment network to obtain a region of interest feature of a query set based on the target predicted position information and the query set features; Using a region of interest alignment network to obtain region of interest features of a support set based on the support set features and the support set image; averaging the features of the region of interest of the support set to obtain a prototype of the second category; A multi-granularity matching network for bidirectional feature alignment is constructed, and the region of interest features of the query set and the second category prototype are input into the multi-granularity matching network for bidirectional feature alignment to obtain a category score and a prediction box offset; wherein, in the multi-granularity matching network for bidirectional feature alignment, the second category prototype and the region of interest features of the query set are respectively encoded along the channel direction to obtain a first codeword sequence and a second codeword sequence; the first codeword sequence and the second codeword sequence are respectively subjected to layer normalization processing, and the layer-normalized codeword sequences are straightened to a set length to obtain a first category prototype codeword sequence and a second category prototype codeword sequence; an inner product operation is performed on the category prototype codeword at each position in the first category prototype codeword sequence and the category prototype codeword at each position in the second category prototype codeword sequence to obtain a cross-correlation matrix; An object detection result is obtained based on the category score and the prediction box offset.

2. The small sample target detection method according to claim 1, characterized in that: Constructing a multi-semantic category-aware region proposal network, and inputting the first category prototype and the query set features into the multi-semantic category-aware region proposal network to obtain target prediction position information, including: Inputting the first category prototype and the query set features into the multi-semantic fusion network to obtain fusion enhanced features of multiple semantic structures; The fused enhanced features are input into the nonlinear prediction network to obtain the target prediction position information.

3. The small sample target detection method according to claim 2, characterized in that: Inputting the first category prototype and the query set features into the multi-semantic fusion network to obtain fusion enhanced features of multiple semantic structures, including: Using a plurality of semantic matching networks with the same structure but different weights to obtain enhanced feature graphs of a plurality of semantic structures based on the first category prototype and the query set features; The multi-head feature fusion module is used to fuse enhanced feature maps of multiple semantic structures to obtain fused enhanced features of multiple semantic structures.

4. The small sample target detection method according to claim 3, characterized in that: The process of obtaining the enhanced feature graph based on the first category prototype and the query set features by each semantic matching network includes: Unifying the semantics of the first category prototype and the query set features to obtain a semantically unified result; The query set is coupled with the semantically unified result to obtain the query set matching block; The query set matching blocks are element-summed to obtain the enhanced feature map.

5. The small sample target detection method according to claim 1, characterized in that: Inputting the region of interest features of the query set and the second category prototype into the multi-granularity matching network of the bidirectional feature alignment to obtain the category score and the prediction box offset, including: Performing a softmax operation on the elements in the cross-correlation matrix in a column-by-column and row-by-row manner to obtain a first vector sequence and a second vector sequence; In the multi-granularity matching network of the bidirectional feature alignment, along the channel direction, the Convolution respectively changes the organizational structure of the first category prototype codeword sequence and the second category prototype codeword sequence, and straightens them to obtain a first organizational structure feature and a second organizational structure feature; Using the first vector sequence to adjust the first tissue structure feature and the second tissue structure feature to obtain a first matching feature and a second matching feature; Performing horizontal and vertical accumulation and normalization processing on the vectors in the first vector sequence and the second vector sequence to obtain a first attention mask and a second attention mask for each unit in the codeword; Using the first attention mask and the second attention mask to perform mask processing on the first matching feature and the second matching feature respectively, to obtain a first feature alignment compensation amount and a second feature alignment compensation amount; The first feature alignment compensation amount and the second feature alignment compensation amount are respectively multiplied by the first feature coefficient and the second feature coefficient, and are added element by element to the second category prototype and the region of interest feature of the query set to obtain an aligned second category prototype and an aligned region of interest feature of the query set; A multi-granularity fusion module is used to obtain the category score and the prediction box offset based on the aligned second category prototype and the aligned region of interest features of the query set.

6. The small sample target detection method according to claim 5, characterized in that: The multi-granularity fusion module is used to obtain the category score and the prediction box offset based on the aligned second category prototype and the aligned region of interest features of the query set, including: The aligned second category prototypes and the aligned query set region of interest features are coarse-grained matched to obtain a first category score and a first position offset; the coarse-grained matching refers to the aligned second category prototypes and the aligned query set region of interest features being convolved after pooling and concatenation to achieve feature matching; Performing medium-granularity matching on the aligned second category prototypes and the features of the region of interest of the aligned query set to obtain a second category score and a second position offset; the medium-granularity matching refers to achieving feature matching on the aligned second category prototypes and the features of the region of interest of the aligned query set by channel-by-channel convolution; Fine-grained matching is performed on the features of the regions of interest of the aligned second category prototypes and the aligned query set to obtain a third category score and a third position offset; the fine-grained matching refers to achieving feature matching by directly concatenating the aligned second category prototypes and the aligned query set in the channel dimension and then convolving them; Performing synthetic matching on the aligned second category prototype and the region of interest features of the aligned query set to obtain a fourth category score and a fourth position offset; the synthetic matching refers to matching the high-level features of the aligned second category prototype with the low-level features of the region of interest features of the aligned query set; the high-level features refer to features obtained after further processing on the aligned second category prototype, and the low-level features refer to unprocessed aligned query set region of interest features; The category score and the prediction box offset are obtained based on the first category score, the first position offset, the second category score, the second position offset, the third category score, the third position offset, the fourth category score and the fourth position offset.

7. The small sample target detection method according to claim 6, characterized in that: The multi-granularity fusion module is expressed as: ； In the formula, represents the dot product, Represents channel-by-channel convolution; , , and Representing four and a A neural network consisting of convolutional layers, ReLU layers and fully connected layers; represents the category score, Indicates the position offset, represents the first category score, Indicates the first position offset, represents the second category score, Indicates the second position offset, represents the third category score, Indicates the third position offset, represents the fourth category score, Indicates the fourth position offset, represents the aligned second category prototype, represents the region of interest features of the aligned query set, Indicates a connection operation. Represents a pooling operation.

8. A small sample target detection system, characterized in that: The small sample target detection system is used to implement the small sample target detection method according to any one of claims 1 to 7; the system comprises: A data set acquisition module is used to acquire a small sample data set of the target to be detected; the small sample data set includes base class targets and new class targets; the number of the base class targets is greater than the number of the new class targets; A data set partitioning module, used to partition the small sample data set according to the data organization form of meta-learning to obtain support set images and query set images corresponding to the task; A feature extraction module, used to extract features from the support set image and the query set image using a weight-sharing twin network to obtain support set features and query set features; A first average value calculation module, used for performing average calculation on the support set features to obtain a first category prototype; A network construction module is used to construct a multi-semantic category-aware region proposal network, and input the first category prototype and the query set features into the multi-semantic category-aware region proposal network to obtain target prediction position information; A first region feature extraction module, configured to obtain the region of interest features of the query set based on the target predicted position information and the query set features by using a region of interest alignment network; A second region feature extraction module, configured to obtain the region of interest features of the support set based on the support set features and the support set image using a region of interest alignment network; A second average value calculation module is used to average the features of the region of interest of the support set to obtain a second category prototype; A category score-offset determination module is used to construct a multi-granularity matching network for bidirectional feature alignment, and input the region of interest features of the query set and the second category prototype into the multi-granularity matching network for bidirectional feature alignment to obtain a category score and a prediction box offset; The target detection module is used to obtain the target detection result based on the category score and the prediction box offset.

9. A computer device comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the small sample target detection method according to any one of claims 1 to 7.

Images