A few-sample infrared target detection method, device and system

Abstract

The application discloses a few-sample infrared target detection method, device and system, belongs to the few-sample image recognition technical field, and few-sample fine-tuning model is obtained by using base class pre-training data set and few-sample fine-tuning data set training, and the few-sample fine-tuning model includes domain adaptation feature extraction module, query feature support feature fusion module and Transform module;Current infrared image is input into the trained few-sample fine-tuning model, domain related features are extracted by using domain adaptation feature extraction module to adapt to infrared scene, feature fusion is carried out based on meta-learning by using query feature support feature fusion module to adapt to few-sample scene, global feature map of current infrared image is obtained by using the Transform module based on global attention mechanism, so that target subject recognition is realized, and infrared target detection can be accurately realized in few-sample scene, so that the technical problem that the target detection calculation complexity is low and the accuracy is low in the existing few-sample scene is solved.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of few-shot image recognition, and more particularly to a few-shot infrared target detection method, device and system. BACKGROUND

[0002] In recent years, with the application and development of convolutional neural networks (CNN) in the image field and the strong promotion of high-performance computing devices, image classification recognition technology has achieved great success. From AlexNet in 2012 to VGG, GoogleNet, and ResNet later. Target detection technology based on deep learning fully absorbs the research results in the field of image classification, and target detection technology can also be applied to more fields including autonomous driving, security protection, and intelligent monitoring, making target detection technology a new research hotspot in the field of computer vision. Many mature algorithm models have been successfully deployed in practical application scenarios. At present, the target detection model based on deep learning needs a large number of labeled samples for training, but in actual scenarios, it is often difficult to obtain corresponding high-quality labeled samples of a large scale. Few-shot learning can realize the rapid classification and recognition of new types of targets under the condition of a small number of labeled samples, reducing the dependence on large-scale labeled data. In recent years, few-shot image target detection technology research has gradually attracted widespread attention.

[0003] In the military field, for example, through a small amount of intelligence information, enemy targets are identified to obtain infrared images, and when new targets are intercepted, rapid learning is performed, which has good application prospects. Because specific military projects involve two problems at the data level, one is the difficulty of data acquisition, and the other is data security. Therefore, in this field, specific traditional target detection algorithms are often developed based on target characteristics including target size, outline, and relative relationship with the background. However, when facing various interferences including clouds, fog, traditional image algorithms may not be able to adapt, the target is partially occluded, and the target has a small relative difference with the background due to interference. In such scenarios, traditional target detection algorithms often have high computational complexity and low accuracy. SUMMARY

[0004] In view of the above defects or improvement needs of the prior art, the present application provides a few-shot infrared target detection method, device and system, which aims to provide a few-shot infrared target detection method; input the current infrared image into the trained few-shot fine-tuning model, use the domain adaptation feature extraction module to extract domain-related features to adapt to the infrared scene, use the query feature support feature fusion module to perform feature fusion based on meta-learning to adapt to the few-shot scene, and use the global attention mechanism-based Transformer module to obtain the global feature map of the current infrared image, thereby realizing target subject recognition and accurately realizing infrared target detection in a few-shot scene, thereby solving the technical problems of low computational complexity and low accuracy of target detection in the prior art few-shot scene.

[0005] To achieve the above-mentioned purpose, according to one aspect of the present application, a few-shot infrared target detection method is provided, comprising:

[0006] S1: obtaining a base pre-training data set and a few-shot fine-tuning data set, wherein the base pre-training data set comprises a plurality of visible light images, and the few-shot fine-tuning data set comprises a plurality of infrared images and part of the visible light images in the base pre-training data set;

[0007] S2: training an initial target detection network using the base pre-training data set to obtain a base pre-training model; training the base pre-training model using the few-shot fine-tuning data set to obtain a few-shot fine-tuning model;

[0008] Wherein, the few-shot fine-tuning model comprises a domain adaptation feature extraction module, a query feature support feature fusion module and a Transformer module; the domain adaptation feature extraction module is used to extract the features of the input query image and support image to obtain query features and support features; the query feature support feature fusion module is used to adapt to the few-shot scene by meta-learning method, aggregate the query features and support features to obtain fusion features; the Transformer module is used to encode and decode the fusion features to obtain the target features corresponding to the query image, and the target feature map carries the position information and category information of the target subject;

[0009] S3: inputting the all-class support set and the query image corresponding to the current infrared image into the few-shot fine-tuning model to obtain the target feature map corresponding to the current infrared image, and obtaining the position information and category information of the target subject in the current infrared image from the target feature map.

[0010] In one embodiment, the query feature support feature fusion module comprises:

[0011] An encoding unit is configured to perform feature cropping on the support feature to obtain a bounding box region, add a class prototype represented by the bounding box region and a corresponding class code of the class prototype to obtain a comprehensive support feature;

[0012] An operation unit is configured to perform single-head cross-attention operation on the comprehensive support feature and the query feature to obtain a part of interest in the support feature;

[0013] A fusion unit is configured to fuse the query feature and the part of interest in the support feature to obtain the fusion feature.

[0014] In one of the embodiments, the fusion unit is configured to perform dot product operation on the query feature and the part of interest in the support feature, so that the query feature contains the part of interest in the support feature; and finally, the fusion feature is obtained through a fully connected layer.

[0015] In one of the embodiments, the class code corresponding to the class prototype is represented as:

[0016] CE (c,2i) =sin(c / 10000 2i / d )

[0017] CE (c,2i+1) =cos(c / 10000 2i / d )

[0018] wherein d represents a feature dimension, c represents a current class prototype, CE (c,2i) represents an encoding of the 2i-th feature in the c-th class, and CE (c,2i) represents an encoding of the 2i+1-th feature in the c-th class.

[0019] In one of the embodiments, the domain adaptation feature extraction module comprises:

[0020] A plurality of compression excitation units are configured to serve as a domain adaptation library for obtaining features of an input image X in different domains to obtain features X USE corresponding to each domain.

[0021] A domain attention unit is configured to perform global pooling on the features of each domain, and sequentially pass through a fully connected layer and a softmax layer to obtain weights S DA corresponding to the features of each domain.

[0022] A calculation unit is configured to load the weights S DA on the corresponding features, i.e., X DA =X USE S DA ; and perform dot product operation on X DA after sigmoid operation and the input image X to obtain an output feature.

[0023] wherein, when the input image of the compression excitation unit is the query image, the output of the calculation unit is the query feature; when the input image of the compression excitation unit is an image in the support set, the output of the calculation unit is the support feature.

[0024] In one of the embodiments, the Transformer module comprises:

[0025] The encoder unit comprises a plurality of cascaded first sub-layers, each of which comprises a multi-head self-attention and a feedforward network, and adjacent first sub-layers are connected by a residual connection, the encoder unit being configured to input the fusion feature and its corresponding position encoding, and output an encoded feature;

[0026] The decoder unit comprises a plurality of cascaded second sub-layers, each of which comprises two multi-head self-attentions and a feedforward network, and adjacent first sub-layers are connected by a residual connection, the decoder unit being configured to input the encoded feature, and output a target feature corresponding to the query image.

[0027] In one of the embodiments, the total loss function of the initial target detection network comprises: a bipartite matching loss L match , an additional loss L aux , a support set category loss L p and a boundary loss L m ; the total loss function is represented as: L = L match + L max + λ p L p + λ m L m ; λ p and λ m are preset weight hyperparameters.

[0028] In one of the embodiments, the boundary loss L m is represented as:

[0029]

[0030] wherein, N is the total number of categories; IntraDistance i represents an internal distance corresponding to the i-th category, InterDistance i represents an inter-category distance corresponding to the i-th category, p ik represents the k-th prototype vector of category i, and the average prototype is represented as m j represents the average prototype of category j.

[0031] According to another aspect of the present application, there is provided a few-shot infrared target detection device for performing the few-shot infrared target detection method, comprising:

[0032] a obtaining module, configured to obtain a base pre-training dataset and a few-shot fine-tuning dataset, wherein the base pre-training dataset comprises a plurality of visible light images, and the few-shot fine-tuning dataset comprises a plurality of infrared images and part of the visible light images in the base pre-training dataset;

[0033] a training module, configured to train an initial target detection network by using the base pre-training dataset to obtain a base pre-training model, and train the base pre-training model by using the few-shot fine-tuning dataset to obtain a few-shot fine-tuning model;

[0034] wherein the few-shot fine-tuning model comprises a domain adaptation feature extraction module, a query feature support feature fusion module and a Transformer module; the domain adaptation feature extraction module is configured to extract features of input query images and support images to obtain query features and support features; the query feature support feature fusion module is configured to adapt to a few-shot scene by using a meta-learning method, aggregate the query features and the support features to obtain fused features; and the Transformer module is configured to perform encoding and decoding processing on the fused features to obtain target features corresponding to the query images, wherein the target feature maps carry position information and category information of target subjects;

[0035] a detection module, configured to input the all-class support set and query images corresponding to a current infrared image into the few-shot fine-tuning model to obtain target feature maps corresponding to the current infrared image, and obtain position information and category information of target subjects in the current infrared image from the target feature maps.

[0036] According to another aspect of the present application, there is provided a few-shot infrared target detection system, comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the above method when executing the computer program.

[0037] According to another aspect of the present application, there is provided a computer readable storage medium, which stores a computer program, and the computer program implements the steps of the above method when executed by a processor.

[0038] Overall, the above technical solutions conceived by the present application can achieve the following beneficial effects compared with the prior art:

[0039] (1) The present application provides a few-shot infrared target detection method; the current infrared image is input into the trained few-shot fine-tuning model; the domain adaptation feature extraction module can enable the network to learn general knowledge from the visible light data set, and then fine-tune the few-shot, so that the network is suitable for the infrared scene; the query feature support feature fusion module is used to support feature fusion based on meta-learning to adapt to the few-shot scene, and the global feature map of the current infrared image is obtained by using the global attention mechanism based on the Transformer module, so as to realize target subject recognition, and the infrared target detection can be accurately realized in the few-shot scene.

[0040] (2) The query feature support feature fusion module of the present application cuts the support feature to obtain the bounding box region, adds the class prototype represented by the bounding box region and the corresponding class code to obtain the comprehensive support feature; the comprehensive support feature and the query feature are subjected to single-head cross-attention operation to obtain the interested part in the support feature; the query feature and the interested part in the support feature are fused to obtain the fusion feature; the query feature support feature fusion module is used to fuse the support feature and the query feature, which is applied to the field of infrared few-shot target detection, without traditional region proposal and non-maximum suppression (NMS) post-processing, and belongs to an end-to-end model, which reduces the computational complexity of the infrared few-shot target detection method.

[0041] (3) The present application trains the initial target detection network by using a total loss function including boundary loss, and the feature space in the training process is constrained, so that samples of the same category are close to each other, and samples of different categories are as far away from each other as possible, so that the detection accuracy of the few-shot fine-tuning model is higher. BRIEF DESCRIPTION OF DRAWINGS

[0042] Figure 1 The flowchart of the few-shot infrared target detection method provided for the embodiment 1 of the present application.

[0043] Figure 2 The structure of the few-shot target detection network provided for the embodiment 1 of the present application is a schematic diagram.

[0044] Figure 3 The structure of the query feature support feature fusion module provided for the embodiment 2 of the present application is a schematic diagram.

[0045] Figure 4 The structure of the domain adaptation feature extraction module provided for the embodiment 5 of the present application is a schematic diagram.

[0046] Figure 5 The structure of the Transformer module provided for the embodiment 6 of the present application is a structural diagram.

[0047] Figure 6 A result simulation diagram of infrared detection image detection provided for Embodiment 8 of the present application. DETAILED DESCRIPTION

[0048] In order to make the objectives, technical solutions and advantages of the present application clearer, the present application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and do not limit the present application. In addition, the technical features involved in each embodiment of the present application described below can be combined with each other as long as they do not conflict with each other.

[0049] Embodiment 1

[0050] As shown in Figure 1 , the present embodiment provides a few-shot infrared target detection method, comprising:

[0051] S1: obtaining a base pre-training data set and a few-shot fine-tuning data set, the base pre-training data set comprising a plurality of visible light images, and the few-shot fine-tuning data set comprising a plurality of infrared images and part of the visible light images in the base pre-training data set.

[0052] S2: training an initial target detection network using the base pre-training data set to obtain a base pre-training model; training the base pre-training model using the few-shot fine-tuning data set to obtain a few-shot fine-tuning model.

[0053] As shown in Figure 2 , the few-shot fine-tuning model comprises a domain adaptation feature extraction module, a query feature support feature fusion module and a Transformer module; the domain adaptation feature extraction module is used to extract the features of the input query image and support image to obtain query features and support features; the query feature support feature fusion module is used to adapt to the few-shot scene through a meta-learning method, aggregate the query features and support features to obtain fusion features; the Transformer module is used to encode and decode the fusion features to obtain target features corresponding to the query image, and the target feature map carries the position information and class information of the target subject.

[0054] S3: inputting all class support sets and query images corresponding to the current infrared image into the few-shot fine-tuning model to obtain target feature maps corresponding to the current infrared image, and obtaining the position information and class information of the target subject in the current infrared image from the target feature maps.

[0055] It should be noted that, taking infrared image data collected from naval vessels as an example, the base class pre-training dataset uses 15 categories from the visible light Pascal VOC dataset as base classes, each with rich labeled data. The few-shot fine-tuning dataset includes infrared few-shot data and some Pascal VOC visible light data, introducing infrared as a new class. The infrared data uses infrared images of sea surface scenes simulated by Vega. The sample images are 640×512 pixel single-channel images with random black cloud interference superimposed on the background. The infrared training data contains k (k values ​​are generally 1, 5, or 10) labels. To balance the data distribution between the base class and the new class, the few-shot fine-tuning dataset also includes some visible light data samples from the 15 base class categories, with each class containing only 3k labels. The training data for base class pre-training and few-shot fine-tuning are organized according to the Episode method. The data input into the network in each forward operation is called a task, and each task consists of a query set and a support set. The query set contains only one query image, and the support set contains N support images. N is set to 5, and each support image corresponds to a category. The annotation information of the support set is visible during training, while the annotation information of the query set is only used to calculate the loss and measure the detection accuracy of the network.

[0056] Example 2

[0057] like Figure 3 As shown, the query feature support feature fusion module includes:

[0058] The encoding unit is used to crop the supporting features to obtain the bounding box region, and add the category prototype represented by the bounding box region and its corresponding category code to obtain the comprehensive supporting features.

[0059] The operation unit is used to perform a single-head cross-attention operation on the comprehensive support features and the query features to obtain the part of interest in the support features.

[0060] The fusion unit is used to merge the parts of interest from the query features and supporting features to obtain fused features.

[0061] Example 3

[0062] The fusion unit is used to perform a dot product between the parts of interest in the query features and the supporting features, so that the query features include the parts of interest in the supporting features; finally, the fused features are obtained through a fully connected layer.

[0063] Specifically, the query feature and support feature fusion module aggregates query features and multiple support set category features. First, RoIAlign is used on the support features to obtain bounding box information from the image as prototypes representing categories. Then, the categories are encoded, and the category codes and category prototypes are added together to obtain the final comprehensive support features. Next, a single-head cross-attention operation is performed on the comprehensive support features and query features, and the output of the single-head attention is used as an adjustment coefficient to adjust the query features. Finally, a dot product operation is used to ensure that the query features include the parts of interest from the support features, and finally, a fully connected layer is used to obtain the final fused features.

[0064] Example 4

[0065] The category code corresponding to the category prototype is represented as:

[0066] CE (c,2i) =sin(c / 10000) 2i / d );

[0067] CE (c,2i+1) =cos(c / 10000) 2i / d );

[0068] Where d represents the feature dimension, c represents the current category prototype, and CE (c,2i) This represents the encoding of the 2i-th feature in category c, CE (c,2i) This represents the encoding of the (2i+1)th feature in category c.

[0069] Example 5

[0070] like Figure 4 As shown, the domain adaptation feature extraction module includes:

[0071] Multiple compression excitation units are used as a domain adaptation library to extract features from the input image X in different domains, obtaining the features X corresponding to each domain. USE ;

[0072] The domain attention unit is used to perform global pooling on the features of each domain, and then pass them through a fully connected layer and a softmax layer to obtain the weights S corresponding to the features of each domain. DA ;

[0073] The calculation unit is used to calculate the weight S. DA Loaded onto the corresponding feature, i.e., X DA =X USE S DA ; X DA After passing through the sigmoid function, the output feature is obtained by performing a dot product with the input image X.

[0074] Specifically, a domain adaptation feature extraction module is used to learn image features and learn different image features for different domains. A plurality of SE modules are introduced as domain adaptation libraries for different domain feature acquisition on the basis of ResNet, and the specific formula is as follows:

[0075]

[0076] wherein N represents the number of SE modules, represent the output of each SE module; a domain attention mechanism is introduced, and the domain attention part is used to generate the weight corresponding to each domain for combining each SE module. The domain attention module first applies global pooling to the feature map, and then passes through a fully connected layer followed by a softmax layer to provide weights for the SE module, and the specific formula is as follows:

[0077] S DA =F DA (X)=softmax(W DA F avg (X));

[0078] The obtained S DA is used as the weight of the general domain adaptation library output, and the output X DA of the domain adaptation module is obtained, and the specific formula is as follows:

[0079]

[0080] X DA After sigmoid, the final output is obtained by dot product of the weight and the original input, and the specific formula is as follows:

[0081]

[0082] When the input image of the compression excitation unit is an image in the query set, the output of the calculation unit is the query feature; when the input image of the compression excitation unit is an image in the support set, the output of the calculation unit is the support feature.

[0083] Embodiment 6

[0084] As shown in Figure 5 , the Transformer module includes:

[0085] The encoder unit includes a plurality of cascaded first sub-layers, each first sub-layer including a multi-head self-attention and a feedforward network, adjacent first sub-layers being connected by a residual connection, the encoder unit being used for input fusion features and corresponding position encodings, and outputting to obtain encoded features;

[0086] The decoder unit comprises a plurality of cascaded second sub-layers, each second sub-layer comprising two multi-head self-attention and one feed-forward network, adjacent first sub-layers being connected by a residual connection, the decoder unit being configured to input an encoded feature and output a target feature corresponding to a query image.

[0087] Specifically, the Transformer encoder and decoder comprise two parts of an encoder and a decoder, and the structure is shown with reference to Figure 5 The encoder comprises six layers with the same structure, and the input of the encoder comprises a feature map after feature extraction and position encoding.

[0088] PE (posx,2i) = sin(posx / 10000 4i / d )

[0089] PE (posx,2i+1) = cos(posx / 10000 4i / d )

[0090] PE (posy,2i) = sin(posy / 10000 4i / d )

[0091] PE (posy,2i+1) = cos(posy / 10000 4i / d )

[0092] wherein posx and posy represent the x and y coordinates of the feature map respectively, and d represents the feature dimension. Each layer of the encoder comprises two sub-layers: multi-head self-attention and feed-forward network, each group of sub-layers is connected by a residual connection, and normalization processing is performed at the end; the input of the decoder comprises the output of the encoder and object queries, the object queries are N learnable embedding variables, which can be randomly initialized at the beginning of training, and the structure of the decoder is similar to that of the encoder, which also comprises six sub-layers with the same structure, but each sub-layer has an additional multi-head attention layer for modeling the relationship between the output of the encoder and the object queries.

[0093] Embodiment 7

[0094] The total loss function of the initial target detection network comprises: a bipartite matching loss L match , an additional loss L aux , a support set category loss L p and a boundary loss L m ; the total loss function is represented as: L = L match + L aux + λ p L p + λ m Lm ; λ p with λ m is a preset weight hyper-parameter.

[0095] Specifically, the network training total loss function can be represented as follows, including four parts, respectively, binary matching loss L match , additional loss L aux , support set class loss L p and boundary loss L m :

[0096] L = L match + L aux + λ p L p + λ m L m .

[0097] Wherein, the binary matching loss function L match is represented as:

[0098]

[0099] When ct i ≠ φ, the value is 1, otherwise 0, wherein the network output result is represented as The ground-truth label box is represented as The binary matching loss function L match is mainly composed of two parts, class loss L cls and box loss L box , and the specific formula is represented as follows:

[0100]

[0101]

[0102]

[0103] In the class loss, p i represents the sigmoid output probability, α i represents the current belonging to which class, and the γ parameter is used to balance the difficult and easy division samples; the box loss uses L1loss and GIou loss to calculate, wherein λ cls , λ iou and λ L1 are weight hyper-parameters, and the present application uses λ cls = 2, λ iou = 2, and λ L1 = 5 during training.

[0104] The additional loss L auxis represented as: For each decoder layer in the Transformer, the output of the previous layer is added to a fully connected layer to get the output result, all the fully connected layers share the weights, and the output result and the labeled box and the labeled class are used to calculate the loss, where the loss L cls and L box The overall additional loss function is calculated as follows, where m represents the number of decoder layers:

[0105]

[0106] Support set class loss L p A fully connected layer with dimension (d, C) is used to map the support set prototype, C is the total number of classes, so that support sets of different classes are distinguished from each other, and here cross-entropy loss is used to measure.

[0107] Boundary loss L m Each class prototype vector needs to be close enough to its average prototype vector, and needs to be far enough away from different classes, and the kth prototype vector p ik of class i is represented as The distance within the class is The distance between classes is

[0108] For the support set feature vector extracted by the feature extractor, the boundary distance M ij between two classes i and j is defined as the sum of the distances of the feature vectors in the two classes closest to the decision boundary, where j represents the class closest to class i, and M ij It is difficult to directly calculate and measure, here an estimate of M ij is considered, for M ij its upper bound is actually the distance between classes, that is, InterDistance i , and its lower bound is the distance between classes minus the intra-class distance of class i minus the intra-class distance of class j, that is, IntraDistance i -IntraDistance i -IntraDistance j .

[0109] Example 8

[0110] Boundary loss L m is represented as:

[0111]

[0112] wherein N is the total number of categories; IntraDistance i represents the intra-distance corresponding to the i-th category, IntraDistance i represents the inter-distance corresponding to the i-th category, p ik represents the k-th prototype vector of the i-th category, and the average prototype is represented as m j represents the average prototype of the j-th category.

[0113] Specifically, in order to increase the boundary distance M ij , it can be increased by increasing its upper bound and lower bound, that is, increasing InterDistance i while reducing IntraDistance i and IntraDistance j , therefore, the boundary loss L m is designed to make the intra-distance of each category as small as possible and the inter-distance as large as possible, and the boundary loss L m is represented as:

[0114]

[0115] In order to verify the effectiveness of the embodiment for infrared target in the few-shot scene, it is compared with the existing mainstream few-shot target detection model Meta-RCNN, FSCE. In the experiment, all models use the same training set and test set, and use 1-shot, 5-shot and 10-shot and average precision 50 (AP50) as evaluation indexes, wherein k-shot represents that the infrared category has only k labeled samples in the few-shot fine-tuning stage, and the experimental results are shown in Table 1. According to the results, the method of the present application achieves the highest detection accuracy under various few-shot settings, and in addition, the detection results of part of the test data set under the 10-shot setting of the present application are shown in Table 2. Figure 6

[0116] Table 1

[0117]

[0118] Embodiment 9

[0119] The embodiment provides a few-shot infrared target detection device for executing a few-shot infrared target detection method, comprising:

[0120] The acquisition module is configured to acquire a base class pre-training data set and a few-shot fine-tuning data set, the base class pre-training data set comprising a plurality of visible light images, and the few-shot fine-tuning data set comprising a plurality of infrared images and part of the visible light images in the base class pre-training data set.​

[0121] a training module configured to train an initial target detection network by using a base class pre-training dataset to obtain a base class pre-training model, and train the base class pre-training model by using a few-shot fine-tuning dataset to obtain a few-shot fine-tuning model;

[0122] The few-shot fine-tuning model comprises a domain adaptation feature extraction module, a query feature support feature fusion module, and a Transformer module. The domain adaptation feature extraction module is configured to extract features of an input query image and a support image to obtain query features and support features. The query feature support feature fusion module is configured to aggregate the query features and the support features by using a meta-learning method to adapt to a few-shot scenario to obtain fused features. The Transformer module is configured to encode and decode the fused features to obtain target features corresponding to the query image, the target feature map carrying position information and category information of a target subject.

[0123] The detection module is configured to input all class support sets and a query image corresponding to a current infrared image into the few-shot fine-tuning model to obtain a target feature map corresponding to the current infrared image, and obtain position information and category information of a target subject in the current infrared image from the target feature map.

[0124] Embodiment 10

[0125] The embodiment provides a few-shot infrared target detection system, comprising a memory and a processor, the memory stores a computer program, and the processor implements the steps of the above method when executing the computer program.

[0126] Embodiment 11

[0127] The embodiment provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the steps of the above method.

[0128] Those skilled in the art can easily understand that the above is only a preferred embodiment of the present application, and is not intended to limit the present application, and any modification, equivalent replacement and improvement made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A method for detecting infrared targets with a small number of samples, characterized in that, include: S1: Obtain the base class pre-training dataset and the few-shot fine-tuning dataset. The base class pre-training dataset includes multiple visible light images, and the few-shot fine-tuning dataset includes: several infrared images and some visible light images from the base class pre-training dataset. S2: Train the initial object detection network using the base class pre-training dataset to obtain the base class pre-training model; train the base class pre-training model using the few-shot fine-tuning dataset to obtain the few-shot fine-tuning model; The few-shot fine-tuning model includes: a domain-adaptive feature extraction module, a query feature-support feature fusion module, and a Transformer module. The domain-adaptive feature extraction module extracts features from the input query set image and support set image to obtain query features and support features. The query feature-support feature fusion module adapts to the few-shot scenario using a meta-learning method, aggregating the query features and support features to obtain fused features. The Transformer module performs encoding and decoding processing on the fused features to obtain target features corresponding to the query set image. The target feature map carries the location and category information of the target subject. S3: Input all class support sets and the query set image corresponding to the current infrared image into the few-sample fine-tuning model to obtain the target feature map corresponding to the current infrared image, and obtain the location information and category information of the target subject in the current infrared image from it; The Transformer module includes: The encoder unit includes multiple cascaded first sub-layers, each of which includes a multi-head self-attention network and a feedforward network. Adjacent first sub-layers are connected by residuals. The encoder unit is used to input the fused features and their corresponding position codes, and output coded features. The decoder unit includes multiple cascaded second sub-layers, each of which includes two multi-head self-attention networks and a feedforward network. Adjacent first sub-layers are connected by residuals. The decoder unit is used to input the encoded features and output the target features corresponding to the query set image. The total loss function of the initial target detection network includes: binary matching loss. Additional losses Support set category loss and boundary loss The total loss function is expressed as: ; and These are preset weight hyperparameters; The boundary loss Represented as: ； Where N is the total number of categories; This represents the internal distance corresponding to the i-th category. ; This represents the inter-category distance corresponding to the i-th category. ; Let the k-th prototype vector of category i be represented as . , This represents the average prototype of category j.

2. The method for detecting infrared targets with few samples as described in claim 1, characterized in that, The query feature support feature fusion module includes: The encoding unit is used to perform feature clipping on the supporting features to obtain the annotation box region, and to add the category prototype represented by the annotation box region and its corresponding category code to obtain the comprehensive supporting features; The operation unit is used to perform a single-head cross-attention operation on the comprehensive support features and the query features to obtain the part of interest in the support features; The fusion unit is used to fuse the query features and the parts of interest in the supporting features to obtain the fused features.

3. The method for detecting infrared targets with few samples as described in claim 2, characterized in that, The fusion unit is used to perform a dot product between the query feature and the part of interest in the supporting feature, so that the query feature includes the part of interest in the supporting feature; finally, the fused feature is obtained through a fully connected layer.

4. The method for detecting infrared targets with few samples as described in claim 2, characterized in that, The category code corresponding to the category prototype is represented as follows: Where d represents the feature dimension and c represents the current category prototype. This represents the encoding of the 2i-th feature in category c. This represents the encoding of the (2i+1)th feature in category c.

5. The method for detecting infrared targets with few samples as described in claim 1, characterized in that, The domain adaptation feature extraction module includes: Multiple compression excitation units are used as a domain adaptation library to extract features from the input image X in different domains, obtaining the features corresponding to each domain. ; Domain attention units are used to perform global pooling on features from each domain, which are then passed sequentially through fully connected layers and softmax layers to obtain the weights corresponding to the features of each domain. ; The calculation unit is used to calculate the weights. Loaded onto the corresponding feature, i.e. ;Will After passing through the sigmoid function, a dot product is performed with the input image X to obtain the output features; Specifically, when the input image of the compression excitation unit is the query set image, the output of the calculation unit is the query feature; when the input image of the compression excitation unit is an image in the support set, the output of the calculation unit is the support feature.

6. A few-sample infrared target detection device, characterized in that, A method for performing the few-sample infrared target detection method according to any one of claims 1-5, comprising: The acquisition module is used to acquire a base class pre-training dataset and a few-shot fine-tuning dataset. The base class pre-training dataset includes multiple visible light images, and the few-shot fine-tuning dataset includes several infrared images and a portion of the visible light images in the base class pre-training dataset. The training module is used to train the initial object detection network using the base class pre-training dataset to obtain the base class pre-trained model; and to train the base class pre-trained model using the few-shot fine-tuning dataset to obtain the few-shot fine-tuning model. The few-shot fine-tuning model includes: a domain-adaptive feature extraction module, a query feature-support feature fusion module, and a Transformer module. The domain-adaptive feature extraction module extracts features from the input query set image and support set image to obtain query features and support features. The query feature-support feature fusion module adapts to the few-shot scenario using a meta-learning method, aggregating the query features and support features to obtain fused features. The Transformer module performs encoding and decoding processing on the fused features to obtain target features corresponding to the query set image. The target feature map carries the location and category information of the target subject. The detection module is used to input all class support sets and the query set image corresponding to the current infrared image into the few-sample fine-tuning model to obtain the target feature map corresponding to the current infrared image, and obtain the location information and category information of the target subject in the current infrared image.

7. A few-sample infrared target detection system, comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 5.

Images