An infrared target detection method decoupled from language guidance features

Abstract

The application discloses an infrared target detection method with language guidance feature decoupling, comprising the following steps: S1, generating a text description corresponding to an infrared image in combination with target label information to obtain infrared image-text data; S2, extracting visual features of the infrared image by using a target detection backbone network, and decomposing the visual features into target visual features and non-target visual features; using the paired infrared image-text data obtained in step S1, inputting the text into a text encoder to extract target text features; S3, projecting the extracted target visual features and target text features into a feature space to realize decoupling of the visual features; S4, inputting the decoupled target visual features into a constructed infrared target detection model to perform infrared target detection; the application can improve the feature extraction capability of the model on the infrared target and improve the infrared image target detection precision.

Description

Technical Field

[0001] This invention relates to the field of target detection technology, and in particular to an infrared target detection method that decouples language-guided features. Background Technology

[0002] In recent years, due to the rapid development of deep learning technology, deep learning-based object detection methods have surpassed traditional methods and become the main research direction in object detection technology. Researchers both domestically and internationally have made significant progress in object detection, particularly with YOLOv5, YOLOv6, YOLOv7, YOLOX, and PP-YOLOE achieving real-time, high-performance object detection, leading to the widespread application of the YOLO series of object detection methods in the industry. Furthermore, DAMO-YOLO introduces a series of new technologies to improve detection performance based on the YOLO framework, including a new detection backbone structure based on NAS search, a deeper neck structure, a simplified head structure, and the introduction of distillation techniques to further enhance performance.

[0003] Deep learning-based object detection technology has developed rapidly, but mainstream research remains limited to the visible light domain. The imaging quality of visible images is easily affected by environmental changes, exhibiting extremely poor imaging performance in low-light environments such as nighttime, fog, and rain. Infrared images, which reflect the temperature difference of objects, remain reliable even in these conditions, offering all-day operational capability. Due to advancements in infrared imaging technology, its applications are becoming increasingly widespread. Object detection technology in infrared scenes has become one of the key research directions for scholars both domestically and internationally in recent years.

[0004] However, infrared images present new challenges to target detection technology in infrared scenes due to their low resolution, low contrast, high noise, and limited features. Traditional infrared target detection technologies require sophisticated and complex data acquisition equipment, and high-quality infrared target detection data is scarce. Furthermore, existing infrared target detection technologies often only design model structures based on infrared or visible light modes, failing to adequately characterize the target and thus limiting the development of infrared target detection technology.

[0005] Infrared target detection in complex scenes faces challenges such as the small size of infrared targets and the indistinct edges and textures. When the infrared detector is far away, the target may only occupy one or a few pixels in the infrared image, making it difficult to extract its shape and structural features. Furthermore, the infrared imaging system itself introduces various noises, resulting in low signal-to-noise ratios and low contrast in infrared target images, further increasing the difficulty of extracting infrared target identification features and leading to inaccurate modeling. Existing infrared target detection technologies often only design model structures for infrared or visible light modes, failing to adequately represent the target. Therefore, achieving robust infrared image target detection remains a pressing problem to be solved. Summary of the Invention

[0006] Purpose of the invention: The purpose of this invention is to provide an infrared target detection method that decouples language-guided features. By introducing textual modalities for feature enhancement when extracting visual features, it can achieve accurate, efficient and robust detection results.

[0007] Technical solution: An infrared target detection method based on language-guided feature decoupling, comprising the following steps:

[0008] S1, combine the target label information to generate the corresponding text description of the infrared image, and obtain infrared image-text data;

[0009] S2, the target detection backbone network is used to extract the visual features of the infrared image and decompose the visual features into target visual features and non-target visual features;

[0010] Using the paired infrared image-text data obtained in step S1, the text is input into a text encoder to extract target text features;

[0011] S3, Project the target visual features and target text features extracted in step S2 into the feature space to achieve decoupling of visual features;

[0012] S4. Input the decoupled target visual features into the constructed infrared target detection model to perform infrared target detection.

[0013] Furthermore, the specific steps to obtain infrared image-text data are as follows:

[0014] S11, given Target label information of infrared image samples traversal One infrared image sample was obtained ,in, Indicates the first The label information for each sample, including target category, quantity, and location; Indicates the first The category of the t-th target in a sample. Indicates the first The location of the t-th target in a sample Indicates the first The number of the t-th target in a sample; Indicates the target quantity;

[0015] S12, label information for each infrared image sample The corresponding text descriptions are generated using a rule-based method, resulting in an infrared image-text dataset. , Indicates the first An infrared image-text data pair.

[0016] Furthermore, the visual features are further decomposed into target visual features and non-target visual features as follows:

[0017] For paired infrared image-text data Image data The backbone network for object detection extracts visual features from infrared images. The output visual features are divided into two parts using tensor clipping:

[0018] ,

[0019] in, These are the raw infrared image features extracted by the detector. W and L represent the height, width, and number of channels of the feature map, respectively; , These are the target visual features and non-target visual features after segmentation, respectively; R represents the real number field; Split() represents the cropping operation.

[0020] Furthermore, the steps to decouple visual features are as follows:

[0021] S31, Align the dimensions of the target visual features with the target text features;

[0022] S32, Calculate the cosine similarity between the visual features of the target and the text features of the target;

[0023] S33, for the obtained similarity matrix between the target text features and the target visual features Contrast loss is used to encourage bringing in matching image-text sample pairs and pulling away mismatched image-text sample pairs.

[0024] Furthermore, the steps to align the dimensions of the target visual features with the target text features are as follows:

[0025] S311, the target visual features obtained through channel segmentation Input to Module;

[0026] S312 uses adaptive pooling to dynamically adjust feature size;

[0027] S313 employs a multi-head self-attention layer to capture complex dependencies;

[0028] S314, iterate through all samples in each batch to obtain visual features that match the text features.

[0029] Furthermore, while using text modalities to supervise the extraction of target features, a loss function is constructed. To more thoroughly decouple non-target visual features from target visual features, the specific process is as follows:

[0030] SD1, average pooling the non-target visual features and target visual features obtained in step 2, as shown in the following expression:

[0031] ,

[0032] ,

[0033] in, This indicates the average pooling operation. , These are the pooled non-target visual features and target visual features, respectively.

[0034] SD2, for and We perform cosine similarity calculation and use the cosine similarity as the value of the loss function for optimization, as shown in the following expression:

[0035] ,

[0036] in, This represents the cosine similarity.

[0037] Furthermore, in step S4, the specific process of infrared target detection is as follows:

[0038] S41 takes the decoupled target visual features as input and uses the pre-trained CSPDarknet backbone as a feature extractor.

[0039] S42, which fuses and enhances the target visual features extracted by the neck network;

[0040] S43, the detection head uses multiple convolutional layers and anchor box mechanism to classify and regress bounding boxes on the fused feature map, predicting the category and location of the target;

[0041] S44 removes redundant detection boxes by nonmaximum suppression, retaining the detection result with the highest confidence.

[0042] Compared with the prior art, the significant advantages of this invention are as follows:

[0043] 1. This invention employs a language-guided feature decoupling method to decouple the visual features of an infrared image into target visual features and non-target visual features. The degree of decoupling of the visual features is measured by using target text features aligned with the target visual features of the image. In other words, the distance between target visual features and non-target visual features is increased in the feature space, thereby more thoroughly decoupling the irrelevant features of the infrared target.

[0044] 2. This invention introduces a text modality for feature enhancement when extracting visual features. Compared with single-modality infrared target detection methods, the visual feature extraction of this invention has greater generalizability and can achieve accurate, efficient and robust detection results. Attached Figure Description

[0045] Figure 1 This is a flowchart of the present invention;

[0046] Figure 2 This is a schematic diagram of a language-guided feature decoupling infrared target detection method.

[0047] Figure 3 This is a schematic diagram comparing the detection results of the method of this invention and the baseline method. The first row is the true label, which is the correct response result of the target. The second row is the result of the baseline method. The third row is the correct response result of the method of this invention. Among them, columns (a)-(c) in the second row represent weak responses, and columns (d)-(f) represent incorrect responses. Detailed Implementation

[0048] The present invention will now be described in further detail with reference to the accompanying drawings and specific embodiments.

[0049] This invention proposes a language-guided feature decoupling method for infrared target detection. It combines existing label information from an infrared dataset to generate corresponding text data for infrared images, constructing a high-quality image-text matching infrared dataset. Because existing lightweight image encoders suffer from insufficient feature extraction, the extracted infrared target features contain irrelevant information, which directly impacts model performance. Therefore, this invention designs a feature decoupling module to decouple the visual features of the infrared image into target visual features and non-target visual features. The degree of decoupling is measured using target text features aligned with the image's target visual features, effectively increasing the distance between target and non-target visual features in the feature space. This more thoroughly decouples irrelevant features of the infrared target, indirectly enhancing the target visual features and improving the model's discrimination and generalization capabilities.

[0050] like Figure 1 , Figure 2 As shown, this invention provides an infrared target detection method with language-guided feature decoupling, comprising the following steps:

[0051] Step 1: Combine the target label information to generate corresponding text descriptions for the infrared images, thus obtaining the infrared image-text dataset for training;

[0052] The specific process is as follows:

[0053] Step 11, given Target label information of infrared image samples ,in Indicates the first Label information for each sample. This can be represented by the target category, location, and quantity. By traversing N infrared image samples, we can obtain... ,in Indicates the target quantity. Indicates the first The category of the t-th target in a sample. Indicates the first The location of the t-th target in a sample Indicates the first The number of the t-th target in a sample.

[0054] Step 12: Label information for each infrared image sample A rule-based method is used to generate corresponding text descriptions, including information such as target category, quantity, and location, to obtain an infrared image-text dataset for training. ,in, Indicates the first An infrared image-text data pair.

[0055] Step 2: Extract visual features from infrared images using a target detection backbone network, and decompose the visual features into target visual features and non-target visual features;

[0056] For paired infrared image-text data Image data The backbone network for object detection extracts visual features from infrared images. The output visual features are divided into two parts using tensor clipping:

[0057] (1)

[0058] in, These are the raw infrared image features extracted by the detector. W and L represent the height, width, and number of channels of the feature map, respectively, and R represents the real number field; Split() represents the cropping operation. , These are the segmented target visual feature map and the non-target visual feature map, respectively. At this point, neither feature has a corresponding meaning to its symbol.

[0059] Using the paired infrared image-text data obtained in step 1, the text data... The input is fed into a text encoder to obtain the target text features;

[0060] The specific process is as follows:

[0061] (2)

[0062] Here, TextEncoder() represents a text encoder; the text encoder uses the BERT model to extract text features from paired image-text infrared data. The value ranges from 1 to Batch, indicating that all samples in each batch are traversed.

[0063] Step 3: Project the target visual features and target text features extracted in Step 2 into the feature space to reduce the distance between the target visual features and target text features, thereby achieving decoupling of the visual features;

[0064] The specific process is as follows:

[0065] Step 31: Align the dimensions of the visual features with the text features. First, align the target visual features obtained from channel segmentation in Step 2. Input to The module then uses adaptive pooling to dynamically adjust the feature size. Next, a multi-head self-attention layer is employed to capture complex dependencies. Finally, visual features matching the target text features are obtained. The specific process is as follows:

[0066] (3)

[0067] in, This indicates a 1×1 convolutional layer ( Batch processing is grouped into one layer ( ) and rectifier linear unit ( ) is a block composed of . Indicates the pooling layer. This indicates a multi-head self-attention layer. The value ranges from 1 to Batch, indicating that all samples in each batch are traversed.

[0068] Step 32: Calculate the cosine similarity between visual features and text features. Embed the target visual features and target text features obtained in Step 2 into groups respectively. and Calculate the similarity matrix:

[0069] (4)

[0070] in, Indicates the first The visual features of the target and the first Similarity scores between target text features This represents the L2 norm of the eigenvector.

[0071] Step 33: For the obtained similarity matrix between the target text features and the target visual features... Contrast loss is used to encourage closer matching image-text sample pairs and to distance mismatched image-text sample pairs. Contrast loss The expression is as follows:

[0072] (5)

[0073] in, Here, b represents the temperature parameter, and b represents the batch size during training.

[0074] Preferably, in this embodiment, while using the text modality to supervise the extraction of target visual features, a loss function is constructed to more thoroughly decouple non-target visual features from target visual features; that is, a loss function is designed. Minimize the similarity between the visual features of the target and the visual features of the noise;

[0075] The specific process is as follows:

[0076] Step 321: First, average pool the non-target visual features and target visual features obtained in Step 2, respectively. The specific process is as follows:

[0077] (6)

[0078] (7)

[0079] in, This indicates the average pooling operation. , These are the pooled non-target visual features and target visual features, respectively.

[0080] Step 322, for the two features obtained in step 41 , Cosine similarity is calculated and used as the value of the loss function for optimization, achieving better decoupling of non-target visual features and target visual features in the image. The calculation method is as follows:

[0081] (8)

[0082] in, This represents the cosine similarity.

[0083] Therefore, target features can be effectively learned by establishing a strong correlation between visual features and semantic information.

[0084] Step 4: Using YOLOv7 as the basic framework, construct an infrared target detection model to achieve infrared target detection;

[0085] First, the decoupled target visual features obtained through steps 1 to 3 above are used as input, and a pre-trained CSPDarknet backbone is used as a feature extractor. Second, the extracted features are fused and enhanced through a neck network (such as PAFPN). Next, the detection head uses multiple convolutional layers and anchor box mechanisms to classify and regress bounding boxes on the fused feature map to predict the target's category and location. Finally, redundant detection boxes are removed through non-maximum suppression, retaining the detection results with the highest confidence, thus achieving more accurate, efficient, and robust infrared target detection.

[0086] Infrared targets were detected using both the method of this invention and the baseline method. The detection results of the two methods are as follows: Figure 3 As shown, the response results indicate that the method of the present invention is superior to the baseline method.

[0087] The above embodiments are merely illustrative of the technical concept of the present invention and should not be construed as limiting the scope of protection of the present invention. Any modifications made to the technical solutions based on the technical concept proposed in this invention shall fall within the scope of protection of this invention.

Claims

1. An infrared target detection method based on language-guided feature decoupling, characterized in that, Includes the following steps: S1, combine the target label information to generate the corresponding text description of the infrared image, and obtain infrared image-text data; S2, utilizes a target detection backbone network to extract visual features from infrared images, and decomposes these visual features into target visual features and non-target visual features, specifically: For paired infrared image-text data Image data The backbone network for object detection extracts visual features from infrared images. The output visual features are divided into two parts using tensor clipping: ， in, These are the raw infrared image features extracted by the detector. W and L represent the height, width, and number of channels of the feature map, respectively; , These are the target visual features and non-target visual features after segmentation, respectively; R represents the real number field; Split() represents the cropping operation; Using the paired infrared image-text data obtained in step S1, the text is input into a text encoder to extract target text features; S3, Project the target visual features and target text features extracted in step S2 into the feature space to achieve decoupling of visual features; S4. Input the decoupled target visual features into the constructed infrared target detection model to perform infrared target detection; In this process, while using text modality to supervise the extraction of target features, a loss function is constructed. To more thoroughly decouple non-target visual features from target visual features, the specific process is as follows: SD1, average pooling the non-target visual features and target visual features obtained in step 2, as shown in the following expression: ， ， in, This indicates the average pooling operation. , These are the pooled non-target visual features and target visual features, respectively. SD2, for and We perform cosine similarity calculation and use the cosine similarity as the value of the loss function for optimization, as shown in the following expression: ， in, This represents the cosine similarity.

2. The infrared target detection method with language-guided feature decoupling according to claim 1, characterized in that, The specific steps to obtain infrared image-text data are as follows: S11, given Target label information of infrared image samples traversal One infrared image sample was obtained ,in, Indicates the first The label information for each sample, including target category, quantity, and location; Indicates the first The category of the t-th target in a sample. Indicates the first The location of the t-th target in a sample Indicates the first The number of the t-th target in a sample; Indicates the target quantity; S12, label information for each infrared image sample The corresponding text descriptions are generated using a rule-based method, resulting in an infrared image-text dataset. , Indicates the first An infrared image-text data pair.

3. The infrared target detection method with language-guided feature decoupling according to claim 1, characterized in that, The steps to decouple visual features are as follows: S31, Align the dimensions of the target visual features with the target text features; S32, Calculate the cosine similarity between the visual features of the target and the text features of the target; S33, for the obtained similarity matrix between the target text features and the target visual features Contrast loss is used to encourage bringing in matching image-text sample pairs and pulling away mismatched image-text sample pairs.

4. The infrared target detection method with language-guided feature decoupling according to claim 3, characterized in that, The steps to align the dimensions of the target visual features with the target text features are as follows: S311, the target visual features obtained through channel segmentation Input to Module; S312 uses adaptive pooling to dynamically adjust feature size; S313 employs a multi-head self-attention layer to capture complex dependencies; S314, iterate through all samples in each batch to obtain visual features that match the text features.

5. The infrared target detection method with language-guided feature decoupling according to claim 1, characterized in that, In step S4, the specific process of infrared target detection is as follows: S41 takes the decoupled target visual features as input and uses the pre-trained CSPDarknet backbone as a feature extractor. S42, which fuses and enhances the target visual features extracted by the neck network; S43, the detection head uses multiple convolutional layers and anchor box mechanism to classify and regress bounding boxes on the fused feature map, predicting the category and location of the target; S44 removes redundant detection boxes by nonmaximum suppression, retaining the detection result with the highest confidence.

Images