Zero-shot infrared target recognition method and system based on cosine similarity matching

Abstract

This invention relates to the field of infrared weak target recognition technology, and discloses a zero-shot infrared target recognition method and system based on cosine similarity matching. The method includes: preprocessing an infrared image of an unseen category to generate an enhanced image to be recognized; inputting the enhanced image to be recognized into an infrared weak target feature extraction network with an encoder-decoder structure to extract multi-scale target features from the image; generating text features of multiple unseen categories and image features of multi-scale target features through a contrastive language-image pre-trained model; and matching the text features and image features using cosine similarity to obtain text recognition labels for the unseen category infrared image. This invention improves the recognition capability of infrared weak targets through a target detection network with an encoder-decoder structure.

Description

Technical Field

[0001] This invention relates to the field of infrared weak target recognition technology, and more specifically, to a zero-sample infrared target recognition method and system based on cosine similarity matching. Background Technology

[0002] With the development of convolutional neural networks, the application of intelligent target detection and recognition technology in the field of recognition has become increasingly mature. Infrared weak target recognition plays an important role in early warning, reconnaissance, interception and other fields, and has become a research hotspot with extremely important engineering application prospects and practical value.

[0003] However, infrared targets are small in size, have few imaging pixels, low resolution, and little structural and texture information. In addition, current intelligent detection models are driven by data sources, resulting in a small number of samples and poor diversity. These two difficulties restrict the accuracy of intelligent target detection and greatly limit the transferability and generalization ability of the model in a wide variety of real-world environments.

[0004] Conventional image classification models are often trained in a fully supervised manner on image datasets with class labels, which makes them highly dependent on data and unsuitable for small sample recognition applications. They also limit the applicability and generalization ability of the models and are not suitable for task transfer. Summary of the Invention

[0005] The purpose of this invention is to provide a zero-sample infrared target recognition method and system based on cosine similarity matching, which can solve at least one of the aforementioned technical problems. The specific solution is as follows:

[0006] According to specific embodiments disclosed in this invention, the first aspect of this invention discloses a zero-sample infrared target recognition method based on cosine similarity matching, which preprocesses infrared images of unseen categories to generate enhanced images to be recognized;

[0007] The image to be identified and enhanced is input into an infrared weak target feature extraction network with an encoder-decoder structure, and the infrared weak target feature extraction network is used to extract multi-scale target feature information of the image to be identified and enhanced.

[0008] The unseen multiple categories and the multi-scale target feature information are input into the contrastive language-image pre-trained model to generate text features for the unseen multiple categories. And the image features I of the multi-scale target feature information, using cosine similarity to combine the text features The text recognition label, confidence level, and location information of the infrared image of the unseen category are obtained by matching it with the image feature I.

[0009] Preferably, the infrared images of unidentified categories are preprocessed to generate enhanced images to be identified, including:

[0010] A one-stage blind denoising network is used to preprocess infrared images of unidentified categories to generate enhanced images for identification; wherein, the one-stage blind denoising network includes:

[0011] The feature extraction module includes multiple stacked convolutional layers, with a ReLU activation function input after each convolutional operation;

[0012] The feature learning module based on residual structure consists of two stacked ordinary convolutional layers, with a ReLU activation function input after each convolutional layer;

[0013] The image reconstruction module, taking the output of the residual structure-based feature learning module as input, includes: an average pooling layer and multiple stacked convolutional layers.

[0014] Preferably, the feature extraction module consists of a convolutional layer. A convolutional layer with an expansion coefficient of 2 and convolutional layers with an expansion coefficient of 4 The features are stacked sequentially, and the output of the feature extraction module is:

[0015] ;

[0016] in, X For the input of the unseen category of infrared image, To pass through the convolutional layer The output after that;

[0017] For the first p The convolutional kernels of each convolutional layer p It is a natural number from 1 to 8.

[0018] Preferably, in the image reconstruction module, the output of the feature learning module based on the residual structure is input to the average pooling layer to obtain the pooled result. The expression is:

[0019] ;

[0020] in, It is a sliding window. The number of elements in the sliding window;

[0021] This represents the center element of the sliding window. The element representing a sliding window, i , j It is a natural number;

[0022] This represents the sum of the values ​​of all elements in the sliding window;

[0023] The pooling result is then passed through a convolutional layer. and convolutional layers Feature extraction is performed to obtain feature output. The expression is:

[0024] ;

[0025] Output the features The output of the feature extraction module Perform a dot product to obtain the result. The expression is:

[0026] ;

[0027] The result of the dot product Input convolutional layer To obtain the convolutional layer Feature output The expression is:

[0028] ;

[0029] Convolutional layer Output The feature output OUT of the enhanced image to be identified is generated by adding the input of the unseen category infrared image X, and the expression is:

[0030] ;

[0031] ReLU is the activation function.

[0032] Preferably, the infrared weak target feature extraction network includes:

[0033] Encoder layers, each with a different scale, are used to downsample the image to be recognized and enhanced to generate multi-scale feature maps;

[0034] The decoder layer, whose scale corresponds to that of the encoder layer, upsamples the multi-scale feature map to extract texture structure and / or contour features.

[0035] The context information extraction module is set between two adjacent encoder layers and two adjacent decoder layers, respectively, and is used to extract features of the image to be recognized and enhanced at different scales and to fuse the texture structure information of the image to be recognized and enhanced obtained by each decoder layer.

[0036] Preferably, the infrared weak target feature extraction network is used to extract multi-scale target feature information of the image to be identified and enhanced, including:

[0037] Multi-scale features of the image to be enhanced are extracted by using full-scale skip connections between the encoder and decoder based on the UNet3+ network structure.

[0038] The multi-scale features of the image to be identified and enhanced are fused to generate a multi-scale fused feature map;

[0039] The multi-scale fused feature map is fed into a region proposal network to generate proposed regions for candidate boxes.

[0040] The proposed region is pooled with the multi-scale fused feature map;

[0041] The pooled features are then used for classification and bounding box regression through a fully connected network to output the target's category, confidence level, and location information.

[0042] Preferably, the context information extraction module includes:

[0043] Two consecutive cross-center convolutional modules are used to obtain gradient features and expand the receptive field, with dilatation rates of 1 and 3, respectively.

[0044] The cross-attention mechanism module takes the feature map output by the cross-center convolution module as input and passes it through two... Convolution and dimensionality reduction yield two feature maps. Affinity and aggregation operations are then performed on the two feature maps obtained by dimensionality reduction to obtain features of different fine-grained sizes.

[0045] According to specific embodiments disclosed in this invention, a second aspect of this invention discloses a zero-sample infrared target recognition device based on cosine similarity matching, comprising:

[0046] Image processing unit: preprocesses infrared images of unidentified categories to generate enhanced images to be identified;

[0047] Without a target detection unit, the image to be identified and enhanced is input into an infrared weak target feature extraction network with an encoder-decoder structure, and the infrared weak target feature extraction network is used to extract multi-scale target feature information of the image to be identified and enhanced.

[0048] The unseen target recognition unit inputs multiple unseen categories and the multi-scale target feature information into a contrastive language-image pre-training model to generate text features of the multiple unseen categories and image features of the multi-scale target feature information. The text features and image features are matched using cosine similarity to obtain the text recognition label, confidence level, and location information of the infrared image of the unseen category.

[0049] According to specific embodiments disclosed in this invention, a third aspect of this invention discloses a computer-readable storage medium having a computer program stored thereon, wherein the program, when executed by a processor, implements the method for editing content in a document as described in any of the preceding claims.

[0050] According to a specific embodiment of the present invention, in a fourth aspect, the present invention discloses an electronic device, comprising: one or more processors; and a storage device for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the method for editing content in a document as described in any of the preceding claims.

[0051] Compared with the prior art, the above-described solution disclosed in this invention has at least the following beneficial effects:

[0052] This invention improves the recognition capability of infrared weak targets by using a target detection network with an encoder-decoder structure. By comparing a language-image pre-trained model with cosine similarity matching, it mitigates the suppression of the detection model's classification domain generalization ability by limited training data samples in diverse real domains and improves the robustness of zero-sample infrared weak target recognition, thereby achieving a synergistic improvement in the accuracy and robustness of infrared weak target recognition. Attached Figure Description

[0053] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the disclosure of this invention and, together with the description, serve to explain the principles of the invention. It is obvious that the drawings described below are merely some embodiments of the invention, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort. In the drawings:

[0054] Figure 1 This is a flowchart of a zero-sample infrared target recognition method based on cosine similarity matching provided by the present invention;

[0055] Figure 2 This is a schematic diagram of the network structure of an image denoising module according to an embodiment of the present invention;

[0056] Figure 3This is an architecture diagram of an infrared weak target feature extraction and network with an encoder-decoder structure according to an embodiment of the present invention;

[0057] Figure 4 This is a schematic diagram of the recognition algorithm based on cosine similarity matching according to an embodiment of the present invention;

[0058] Figure 5 A schematic diagram of the structure of a zero-sample infrared target recognition device based on cosine similarity matching provided by the present invention;

[0059] Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0060] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are merely a part of the embodiments disclosed in this invention, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments disclosed in this invention without creative effort are within the scope of protection of this invention.

[0061] The terminology used in the disclosed embodiments of this invention is for the purpose of describing particular embodiments only and is not intended to limit the scope of the invention. The singular forms “a,” “the,” and “the” as used in the disclosed embodiments and appended claims are also intended to include the plural forms, and “multiple” generally includes at least two unless the context clearly indicates otherwise.

[0062] It should be understood that the term "and / or" used in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / " in this article generally indicates that the preceding and following related objects have an "or" relationship.

[0063] It should be understood that although the terms first, second, third, etc., may be used in the descriptions of the disclosed embodiments of the present invention, these descriptions should not be limited to these terms. These terms are only used to distinguish the descriptions. For example, first may also be referred to as second without departing from the scope of the disclosed embodiments of the present invention, and similarly, second may also be referred to as first.

[0064] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that an article or device that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such an article or device. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the article or device that includes said element.

[0065] The following is in conjunction with the appendix Figure 1-5 The optional embodiments disclosed in this invention are described in detail. Example 1

[0066] Contrastive Language–Image Pretraining (CLIP) is a multimodal model developed by OpenAI that understands the relationships between images and text. It uses an image encoder (such as VisionTransformer or ResNet) to convert images into feature vectors, and a text encoder (such as Transformer) to convert text into feature vectors. By calculating the similarity between image and text feature vectors and comparing positive and negative samples, it learns to distinguish between matching and mismatched image-text pairs, performing inference and classification on unseen categories.

[0067] When using fully supervised learning to identify weak infrared targets in relevant fields, the limited sample size in some areas leads to poor model diversity, making it unsuitable for task transfer. In contrast, the CILP model, trained on 400 million text-image pairs, can be applied to complex environments. It addresses the issues of low accuracy and poor robustness in identifying weak infrared targets under domain shift conditions in wide-area backgrounds, effectively improving the model's domain translation invariance and zero-shot detection capability for weak targets.

[0068] Therefore, the first embodiment of the present invention provides a zero-sample infrared target recognition method based on cosine similarity matching, comprising the following steps:

[0069] Step S102: Preprocess the infrared images of unidentified categories to generate enhanced images to be identified.

[0070] This embodiment employs a one-stage blind denoising network based on a feature attention mechanism to preprocess the obtained infrared images of unseen categories, such as... Figure 2 As shown, it contains 8 convolutional layers and 1 average pooling layer. p The convolution kernel for each convolution operation is , p The number is a natural number from 1 to 8, with a step size of 1. s All are 1.

[0071] Specifically, the one-stage blind denoising network includes:

[0072] The feature extraction module consists of a regular convolutional layer. A hollow convolutional layer with an expansion coefficient of 2 and a hollow convolutional layer with an expansion coefficient of 4 The convolutional units are stacked sequentially, and each convolution operation is followed by a Reduced Linear Unit (ReLU) to output the feature extraction module. , The expression is:

[0073] ;

[0074] ;

[0075] in, X For the input of the unseen category of infrared image, To pass through convolutional layers The output after that.

[0076] The feature learning module based on residual structure consists of two ordinary convolutional layers. , The features are stacked, and each convolution operation is followed by a ReLU activation function. The expression for the output of the feature learning module based on the residual structure is as follows:

[0077] .

[0078] The image reconstruction module includes: one average pooling layer and three stacked convolutional layers. , and .

[0079] Specifically, in the image reconstruction module, the output of the feature learning module is first... Pooling in the average pooling layer is expressed as follows:

[0080] ;

[0081] in, It is a sliding window. The number of elements in the sliding window;

[0082] This represents the center element of the sliding window. The element representing a sliding window, i , j It is a natural number;

[0083] This represents the sum of the values ​​of all elements in the sliding window.

[0084] The pooling result is then passed through a convolutional layer. and convolutional layers Feature extraction is performed to obtain feature output. The expression is:

[0085] ;

[0086] The output here With image reconstruction module Perform a dot product operation on the input, that is:

[0087] ;

[0088] Input the result into the convolutional layer The expression is:

[0089] ;

[0090] Finally, the convolutional layer The output is added to the original input to generate the image to be recognized and enhanced, expressed as:

[0091] .

[0092] Step S104: Input the image to be identified and enhanced into an infrared weak target feature extraction network with an encoder-decoder structure, and use the infrared weak target feature extraction network to extract multi-scale target feature information of the image to be identified and enhanced.

[0093] In this embodiment, by adopting, as follows Figure 3 The infrared weak target feature extraction network RAMF-Net shown performs feature extraction on the image to be identified and generates the target category, confidence level, and location information in the image to be identified and enhanced.

[0094] like Figure 3 As shown, it includes: a feature extraction network, a feature fusion network, a region proposal network, a ROI pooling network, and a fully connected layer network.

[0095] The feature extraction network employs a UNet3+ residual attention multi-scale skip connection network based on an encoder-decoder structure. Furthermore, a context information extraction module (DCCA) based on differential convolution and cross-attention mechanisms is introduced into the encoding layer to extract multi-scale features from the image to be enhanced. The combination of the UNet3+ network and the DCCA module adaptively refines feature extraction, deeply mines infrared weak target features, and improves the ability to perceive and recognize targets in detail.

[0096] The feature fusion network fuses the extracted multi-scale features of the image to be enhanced to generate a multi-scale fused feature map.

[0097] The Region Proposal Network (RPN) feeds multi-scale fused feature maps into the RPN to generate proposed regions for candidate boxes.

[0098] The ROI pooling network refines the proposed regions and multi-scale fused feature maps by inputting them into the ROI pooling network; the regression network classifies the pooled features and regresses bounding boxes, outputting the target's category and precise location.

[0099] Specifically, in the feature extraction network, each encoder layer has a different scale, used to downsample the image to be recognized and enhanced, generating multi-scale feature maps. The scale of each decoder layer corresponds to the scale of the encoder layer, upsampling the multi-scale feature maps to extract the texture structure and contour features of the image to be recognized and enhanced.

[0100] The multi-scale contextual feature extraction module DCCA, based on differential convolution and residual attention mechanism, is set between two adjacent encoder layers and two adjacent decoder layers, respectively. It is used to extract features of the image to be recognized and enhanced at different scales and to fuse the texture structure information of the image to be recognized and enhanced obtained from each decoder layer.

[0101] The context information extraction module DCCA includes:

[0102] Two consecutive cross-center convolutional modules are used to obtain more gradient features and a larger receptive field. The dilatation rates of the cross-center convolutions are 1 and 3, respectively.

[0103] The cross-attention mechanism module takes the feature map output by the cross-center convolution module as input and passes it through two... Convolution and dimensionality reduction yield two feature maps. Affinity and aggregation operations are then performed on these two feature maps to obtain features of different fine-grained sizes.

[0104] In this embodiment, the DCCA module effectively applies cross-center differential convolution to the residual attention mechanism module. Cross-center differential convolution is a plug-and-play convolution module that can improve the network's ability to represent detailed features while reducing network redundancy through the decoupling of two symmetrical cross sub-operators: horizontal, vertical, and diagonal.

[0105] Since cross-center differential convolution can effectively extract detailed features, two consecutive cross-center convolutions with dilation rates of 1 and 3 are used here to obtain more gradient features and a larger receptive field, thereby providing more powerful spatial feature information for infrared weak target recognition.

[0106] Then, the feature map output by the cross-center convolution module is fed into the repeated cross-attention mechanism module, denoted as the input feature map. The input features are respectively passed through two Convolution and dimensionality reduction yield two feature maps, Q and K. ， 。

[0107] After obtaining the Q and K matrices, an attention map is generated by performing an affinity operation and applying a softmax operation. .

[0108] At each position in Q space u Corresponding to a vector It can also be obtained by extracting the position from K. u The set is obtained from the feature vectors in the same row or column. .

[0109] represent The i-th element.

[0110] The formula for calculating affinity operations is as follows:

[0111] ;

[0112] in, yes and The degree of relevance.

[0113] The feature map is obtained after performing Softmax normalization on D. A , .

[0114] Add to the input feature map The filter adapts to the feature dimension, and for the spatial dimension V Each position u has: vector and set The aggregation operation can then be defined as:

[0115] ;

[0116] In the formula, .

[0117] Furthermore, with This example illustrates how the UNet3+ network constructs multi-scale feature maps.

[0118] First, feature maps from encoder layers of the same scale are received. The two skip connections above use different max-pooling operations to connect the smaller-scale encoder layer. and After pooling downsampling to unify the resolution of the feature map, low-level semantic information is passed on. The resolution needs to be reduced by a factor of 4. The resolution needs to be reduced by a factor of 2. The other two lower skip connections are then processed using bilinear interpolation in the decoder. and Upsampling is performed to magnify the feature map resolution by 4x and 2x respectively. After unifying the resolution, the number of feature maps needs to be unified through convolution operations with kernels of the same size. Here, 64 3x3 filters are used for convolution operations to obtain feature maps with 64 channels. .

[0119] Feature map The calculation formula is shown below.

[0120] ;

[0121] In the formula, i Indicates the first [number] along the encoding direction i There are 1 downsampling layer, where N represents the number of encoders.

[0122] Figure 3 The UNet3+ network uses 5 encoders. C Represents the convolution operation, function D sum function U These represent downsampling and upsampling, respectively. Concat indicates channel-level splicing and fusion. H The function represents the feature aggregation mechanism (convolutional layer + batch normalization + ReLU activation function).

[0123] feature map After being fed into the DCCA module, the feature aggregation mechanism performs convolutional layer + batch normalization + ReLU activation function operations to obtain the desired result. .

[0124] Therefore, by The constructed multi-scale feature map expression is:

[0125] .

[0126] In the formula, H represents the feature aggregation mechanism, and DCCA represents the multi-scale context feature extraction module based on differential convolution and residual attention mechanism.

[0127] Step S106: Input the unseen multiple categories and the multi-scale target features into the contrastive language-image pre-trained model CLIP to generate text features for the unseen multiple categories. And image features I with multi-scale target features, using cosine similarity to combine text features Matching with image feature I yields text recognition labels, confidence scores, and location information for infrared images of unseen categories, such as... Figure 4 As shown.

[0128] The model's performance degrades in the target domain due to the distribution differences between the training data (source domain) and the test data (target domain). Therefore, to maintain robust object detection invariance to real-world "domain shifts," this embodiment uses the CLIP model, combined with contrastive language-image pre-training.

[0129] First, the N undefined categories, such as Kongming lanterns, kites, and helium balloons, are converted into text data. Then, the N converted text data are input into the text encoder of the CLIP model to obtain N text features. .

[0130] Then, the multi-scale target feature information extracted by the infrared weak target feature extraction network RAMF-Net in step S104 is input into the image encoder of the CLIP model to obtain image features. I ,

[0131] Calculate separately N Text features The cosine similarity between the image and image feature I is calculated using the following formula:

[0132] ;

[0133] The text label corresponding to the maximum cosine similarity is the label corresponding to the best matching category. Finally, the text recognition label, confidence score and location information are output to complete the classification of unseen category images, thereby achieving a synergistic improvement in the robustness and accuracy of infrared weak target recognition.

[0134] Example 2

[0135] The present invention also provides an apparatus embodiment that follows the above embodiments, for implementing the method steps described in the above embodiments. The interpretation of the same names is the same as that of the above embodiments, and the same technical effects are achieved as those of the above embodiments. Therefore, it will not be described again here.

[0136] like Figure 5 As shown, this invention discloses a zero-sample infrared target recognition device based on cosine similarity matching, comprising:

[0137] Image processing unit 302: preprocesses infrared images of unidentified categories to generate enhanced images to be identified;

[0138] The unseen target detection unit 304 inputs the image to be identified and enhanced into a target detection network with an encoder-decoder structure, and uses the target detection network to extract multi-scale target features of the image to be identified and enhanced.

[0139] The unseen target recognition unit 306 generates text features for multiple unseen categories by comparing them with a language-image pre-trained model. And the image features I of the multi-scale target features, using cosine similarity to combine the text features The text recognition label of the infrared image of the unseen category is obtained by matching it with the image feature I.

[0140] Example 3

[0141] like Figure 6 As shown, this embodiment provides an electronic device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, which are executed by the at least one processor to enable the at least one processor to perform the method steps described in the above embodiment.

[0142] Example 4

[0143] The present invention discloses a non-volatile computer storage medium storing computer-executable instructions that can perform the steps described in the above embodiments.

[0144] Example 5

[0145] The following is for reference. Figure 6 The diagram illustrates a structural schematic of an electronic device suitable for implementing embodiments of the present invention. The terminal devices in the embodiments of the present invention may include, but are not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. Figure 6 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of use of the disclosed embodiments of the present invention.

[0146] like Figure 6As shown, the electronic device may include a processing unit (e.g., a central processing unit, a graphics processing unit, etc.) 401, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 402 or a program loaded from a storage device 408 into a random access memory (RAM) 403. The RAM 403 also stores various programs and data required for the operation of the electronic device. The processing unit 401, ROM 402, and RAM 403 are interconnected via a bus 404. An input / output (I / O) interface 405 is also connected to the bus 404.

[0147] Typically, the following devices can be connected to I / O interface 405: input devices 406 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 407 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 408 including, for example, magnetic tapes, hard disks, etc.; and communication devices 409. Communication device 409 allows electronic devices to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 6 Electronic devices with various devices are shown, but it should be understood that it is not required to implement or have all of the devices shown. More or fewer devices may be implemented or have alternatively.

[0148] In particular, according to the embodiments disclosed in this invention, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this invention include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device 409, or installed from a storage device 408, or installed from a ROM 402. When the computer program is executed by the processing device 401, it performs the functions defined in the methods of the embodiments disclosed in this invention.

[0149] It should be noted that the computer-readable medium disclosed in this invention may be a computer-readable signal medium, a computer-readable storage medium, or any combination thereof. A computer-readable storage medium may be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this invention, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this invention, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.

[0150] The aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device.

[0151] Computer program code for performing the operations disclosed herein can be written in one or more programming languages ​​or a combination thereof. These programming languages ​​include object-oriented programming languages—such as Java, Smalltalk, and C++—and conventional procedural programming languages—such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0152] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

Claims

1. A zero-shot infrared target recognition method based on cosine similarity matching, characterized in that, include: Preprocessing is performed on infrared images of unidentified categories to generate enhanced images to be identified; The image to be identified and enhanced is input into an infrared weak target feature extraction network with an encoder-decoder structure, and the infrared weak target feature extraction network is used to extract multi-scale target feature information of the image to be identified and enhanced. The unseen multiple categories and the multi-scale target feature information are input into the contrastive language-image pre-trained model to generate text features for the unseen multiple categories. and the image features of the multi-scale target feature information. The text features are analyzed using cosine similarity. and the image features The matching process yields the text recognition tags, confidence scores, and location information for the infrared images of the unseen categories. The preprocessing of infrared images of unidentified categories to generate enhanced images to be identified includes: A one-stage blind denoising network is used to preprocess infrared images of unidentified categories to generate enhanced images for identification; wherein, the one-stage blind denoising network includes: The feature extraction module includes multiple stacked convolutional layers, with a ReLU activation function input after each convolutional operation; The feature learning module based on residual structure consists of two stacked ordinary convolutional layers, with a ReLU activation function input after each convolutional layer; The image reconstruction module, taking the output of the residual structure-based feature learning module as input, includes: an average pooling layer and multiple stacked convolutional layers; The feature extraction module consists of a convolutional layer. A convolutional layer with an expansion coefficient of 2 and convolutional layers with an expansion coefficient of 4 The features are stacked sequentially, and the output of the feature extraction module is: ； in, X For the input of the unseen category of infrared image, To pass through the convolutional layer The output after that; For the first p The convolutional kernels of each convolutional layer p It is a natural number from 1 to 8.

2. The method according to claim 1, characterized in that, In the image reconstruction module, The output of the feature learning module based on the residual structure is input into the average pooling layer to obtain the pooled result. The expression is: in, It is a sliding window. The number of elements in the sliding window; This represents the center element of the sliding window. The element representing a sliding window, i , j It is a natural number; This represents the sum of the values ​​of all elements in the sliding window; The pooling result is then passed through a convolutional layer. and convolutional layers Feature extraction is performed to obtain feature output. The expression is: ； Output the features The output of the feature extraction module Perform a dot product to obtain the result. The expression is: ； The result of the dot product Input convolutional layer To obtain the convolutional layer Feature output The expression is: ； Convolutional layer Output The feature output of the enhanced image to be identified is generated by adding it to the input of the unseen category infrared image X. The expression is: ReLU is the activation function.

3. The method according to claim 1, characterized in that, The infrared weak target feature extraction network includes: Encoder layers, each with a different scale, are used to downsample the image to be recognized and enhanced to generate multi-scale feature maps; The decoder layer, whose scale corresponds to that of the encoder layer, upsamples the multi-scale feature map to extract texture structure and / or contour features. The context information extraction module is set between two adjacent encoder layers and two adjacent decoder layers, respectively, and is used to extract features of the image to be recognized and enhanced at different scales and to fuse the texture structure information of the image to be recognized and enhanced obtained by each decoder layer.

4. The method according to claim 1, characterized in that, The step of extracting multi-scale target feature information from the image to be identified using the infrared weak target feature extraction network includes: Multi-scale features of the image to be enhanced are extracted by using full-scale skip connections between the encoder and decoder based on the UNet3+ network structure. The multi-scale features of the image to be identified and enhanced are fused to generate a multi-scale fused feature map; The multi-scale fused feature map is fed into a region proposal network to generate proposed regions for candidate boxes. The proposed region is pooled with the multi-scale fused feature map; The pooled features are then used for classification and bounding box regression through a fully connected network to output the target's category, confidence level, and location information.

5. The method according to claim 3, characterized in that, The context information extraction module includes: Two consecutive cross-center convolutional modules are used to obtain gradient features and expand the receptive field, with dilatation rates of 1 and 3, respectively. The cross-attention mechanism module takes the feature map output by the cross-center convolution module as input and passes it through two... Convolution and dimensionality reduction yield two feature maps. Affinity and aggregation operations are then performed on the two feature maps obtained by dimensionality reduction to obtain features of different fine-grained sizes.

6. A zero-sample infrared target recognition device based on cosine similarity matching, characterized in that, For implementing the method as described in any one of claims 1-5, comprising: Image processing unit: preprocesses infrared images of unidentified categories to generate enhanced images to be identified; The unseen target detection unit inputs the image to be identified and enhanced into a target detection network with an encoder-decoder structure, and uses the target detection network to extract multi-scale target features from the image to be identified and enhanced. The unseen object recognition unit generates text features for multiple unseen categories by comparing them with a language-image pre-trained model. and the image features of the multi-scale target features The text features are analyzed using cosine similarity. and the image features The matching process yields text recognition tags for the infrared images of the unseen categories.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the method as described in any one of claims 1 to 5.

8. An electronic device, characterized in that, include: One or more processors; A storage device for storing one or more programs, which, when executed by one or more processors, cause the one or more processors to implement the method as described in any one of claims 1 to 5.

Images