An end-to-end anti-interference recognition method based on infrared dual-band images

Abstract

The application discloses an end-to-end anti-interference recognition method based on infrared dual-band images and belongs to the technical field of target recognition in digital image processing. The method first extracts the features of the infrared dual-band images through a deep convolution network, customizes a double-branch backbone network for the feature extraction of the infrared dual-band images, independently extracts the feature information of each band image, improves the feature expression ability and reduces the feature channel redundancy by constructing the correlation between channels, constructs a cross-attention fusion module to learn the complementary relationship between the infrared mid-wave and long-wave features, and finally obtains an end-to-end anti-interference recognition method based on infrared dual-band images, so that the anti-interference performance of the infrared aerial target recognition algorithm under a complex interference situation is improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of target recognition in digital image processing, and in particular to an end-to-end anti-interference recognition method based on infrared dual-band images. Background Art

[0002] The ability to detect and identify objects using thermal radiation is valuable in many ways. However, the effectiveness of infrared imaging systems in detecting and identifying targets can be compromised by interference from various sources. Among these, targets employ jamming techniques to disrupt infrared sensors and deceive the detection system. Traditional single-band infrared recognition algorithms are susceptible to interference, resulting in reduced accuracy and reliability, further complicating the recognition process.

[0003] Given these limitations, driven by the need to enhance target discrimination and anti-interference capabilities, the demand for multi-band infrared data fusion technology has been steadily growing. Mid-wave infrared (MWIR) and long-wave infrared (LWIR) are two common sensor technologies within the infrared spectrum, differing in their wavelength ranges and application characteristics. MWIR, with a wavelength range of 3 to 5 microns, has a high reflective component, enabling it to capture the reflective characteristics of a target. Furthermore, due to its low atmospheric attenuation, MWIR possesses strong penetration, enabling it to obtain relatively clear images even in adverse weather conditions. LWIR, with a wavelength range of 8 to 14 microns, is primarily characterized by its high thermal radiation component, enabling it to capture the target's self-radiation characteristics. Furthermore, with its low atmospheric attenuation, it possesses strong penetration, enabling it to capture clear images at night or in low-light conditions. Furthermore, LWIR has high sensitivity, enabling it to detect even small temperature differences. Fusion of these two infrared bands can provide more comprehensive target information, enhancing target detection, identification, and tracking capabilities in adverse weather or low-visibility conditions, such as at night. The dual-band imaging system exploits the complementary advantages of two different infrared wavelengths to improve anti-interference recognition performance in challenging environments.

[0004] Despite the potential advantages of dual-band infrared fusion, existing research on aerial target recognition still faces several challenges. One of the main issues is the need to preserve key target features while reducing noise and artifacts when integrating medium-wave infrared and long-wave infrared data to generate a fused image. Achieving seamless fusion requires complex algorithms that can effectively combine and enhance the feature information from the two bands without introducing distortion or inconsistencies. However, traditional methods only support specific scenes or target radiation characteristics. Due to the diverse nature of aerial targets and the complex and changing interference in the scenes, traditional recognition methods are difficult to adapt to situations with high interference and changing scenes. Their lack of adaptive capabilities leads to reduced recognition accuracy. Summary of the Invention

[0005] The purpose of the present invention is to overcome the shortcomings of the above-mentioned prior art and provide an end-to-end anti-interference recognition method based on infrared dual-band images to solve the problem that the recognition method of medium-wave infrared and long-wave infrared data images in the prior art only supports specific scenes or target radiation characteristics, is difficult to interfere with complex and changeable scenes, and has poor adaptability.

[0006] In order to achieve the above object, the present invention adopts the following technical solutions:

[0007] An end-to-end anti-interference recognition method based on infrared dual-band images includes the following steps:

[0008] S1, collects infrared long-wave images and infrared medium-wave images;

[0009] S2, through the two branch backbone networks in the dual-branch backbone network, respectively extract infrared long-wave depth features and infrared medium-wave depth features. Each branch backbone network includes a six-layer feature extraction module;

[0010] Each of the feature extraction module channels is divided into two channels, and the features extracted by the two channels are merged to obtain a merged feature, which is input into the channel cross-attention module. The channel cross-attention module obtains weight coefficients of the two channels based on the merged feature, and the features extracted by the two channels are multiplied by the corresponding weight coefficients. After adding the products and performing convolution processing, infrared long-wave depth features and infrared medium-wave depth features are obtained;

[0011] S3: Input the infrared long-wave depth features and infrared medium-wave depth features extracted by the feature extraction modules of the same layer in the two branch backbone networks into the infrared dual-band feature cross-fusion module. After the infrared long-wave depth features and infrared medium-wave depth features are fused through the infrared dual-band feature cross-fusion module, the fused features are output.

[0012] S4 detects the target by fusing features and outputs the target location and category.

[0013] A further improvement of the present invention is:

[0014] Preferably, in S2, the two channels of each feature extraction module are aC and bC channels respectively, where a+b=1, a and b are weights of the two channels respectively, and C is the total number of channels of the feature extraction module.

[0015] Preferably, in S2, the process of merging the features extracted from the two channels to obtain the merged features is as follows: each channel is first compressed by convolution so that the number of channels of the two channels is equal, and then the spatial weights are adjusted by calculating the features through spatial and channel attention. The two channels are summed pixel by pixel based on their respective spatial weights to obtain the corresponding weights.

[0016] Preferably, in S2, the channel cross attention module inputs the merged features into two convolution layers with different convolution kernels, and respectively calculates the global average pooling operation features across the spatial dimension and the global average pooling operation processing features across the channel dimension; after adding and fusing the two features, the process weight coefficient is obtained. Each channel associated with the process weight coefficient is shuffled by the channels, and all the channels are rearranged in an alternating manner to obtain the weight coefficient Q.

[0017] Preferably, in S2, in the process of calculating the global average pooling operation across the channel dimension to process the features: two 1×1 convolution operations are performed, the first convolution operation is used to reduce the channel dimension, and the second convolution operation is used to expand the channel dimension.

[0018] Preferably, in S3, the depth features of the third to sixth layers in the two branch backbone networks are input into the infrared dual-band feature cross-fusion module.

[0019] Preferably, in S3, the infrared dual-band feature cross fusion module enhances the infrared long-wave depth feature and the infrared medium-wave depth feature respectively to obtain enhanced infrared long-wave depth feature and enhanced infrared medium-wave depth feature, and the two enhanced depth features are fused through the fusion function Ψ Fusion (·) After fusion, the fusion features are obtained.

[0020] Preferably, in S3, for the infrared long-wave depth feature, the enhancement process is: infrared long-wave depth feature F L and infrared mid-wave depth characteristics F M Through two 1×1 convolutional layers, two feature maps K are obtained. L ,Q M , the feature map K L ,Q M After normalizing the relevant scores through the softmax function, the normalized result is obtained; after adding the normalized result and the long-wave local feature, the local enhanced feature is obtained, and the local enhanced feature and the infrared long-wave depth feature F are added together. L Collected by aggregation operation, infrared long-wave enhanced depth features are obtained

[0021] Preferably, in S3, for the infrared medium wave depth feature, the enhancement process is: infrared long wave depth feature F L and infrared mid-wave depth characteristics F M Through two 1×1 convolutional layers, two feature maps K are obtained. L ,Q M , the feature map K L ,Q MAfter normalizing the relevant scores through the softmax function, the normalized result is obtained; after adding the normalized result and the medium-wave local feature, the sum is added to the infrared medium-wave depth feature F M Obtain infrared mid-wave enhanced depth features through aggregation operation

[0022] Preferably, in S4, the target position and category are output through the network neck module and the detection head.

[0023] Compared with the prior art, the present invention has the following beneficial effects:

[0024] The present invention discloses an end-to-end anti-interference recognition method based on infrared dual-band images. The method first uses a deep convolutional network to customize a dual-branch backbone network specifically for feature extraction of infrared dual-band image pairs, and independently extracts feature information of each band image. Constructing inter-channel correlation improves feature expression capabilities and reduces feature channel redundancy. Constructing a cross-attention fusion module to learn the complementary relationship between infrared medium-wave and long-wave features, an end-to-end anti-interference recognition method based on infrared dual-band images is obtained, which improves the anti-interference performance of the infrared aerial target recognition algorithm under complex interference situations. This method constructs the deep features independently extracted from the infrared dual-band images into a black box model, rather than separately constructing an image fusion network model and a target detection network model, effectively reducing the complexity and scale of the network model while significantly improving the algorithm's target feature enhanced representation and anti-interference recognition capabilities in complex interference scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 The feature extraction module of the present invention;

[0026] Figure 2 The channel cross attention module of the present invention;

[0027] Figure 3 It is the infrared dual-band feature cross-fusion module of the present invention;

[0028] Figure 4 It is the infrared dual-band target recognition algorithm framework of the present invention;

[0029] Figure 5 This is a partial anti-interference recognition result of the present invention. DETAILED DESCRIPTION

[0030] Hereinafter, the terms "first," "second," "third," and "fourth" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the quantity of the technical features indicated. Thus, a feature identified as "first," "second," "third," or "fourth" may explicitly or implicitly include one or more of such features.

[0031] The co-shooting method provided in the embodiments of the present application can be applied to terminal devices such as mobile phones, tablet computers, wearable devices, vehicle-mounted devices, augmented reality (AR) / virtual reality (VR) devices, laptop computers, ultra-mobile personal computers (UMPCs), netbooks, and personal digital assistants (PDAs). The embodiments of the present application do not impose any restrictions on the specific types of terminal devices.

[0032] It should be noted that the terms "first," "second," and the like in the description and drawings of the present invention are used to distinguish similar objects and are not necessarily used to describe a specific order or precedence. It should be understood that the terms used in this manner are interchangeable where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having," as well as any variations thereof, are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or apparatus comprising a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units that are not explicitly listed or that are inherent to these processes, methods, products, or apparatus.

[0033] To achieve the above objectives, the present invention discloses an end-to-end anti-interference recognition method based on infrared dual-band images, which includes the following steps:

[0034] S1, collect and acquire infrared long-wave images and infrared medium-wave images.

[0035] The specific process is to obtain infrared dual-band image data respectively and construct infrared dual-band image dataset Data={I1,I2,I3,…,I n}, where I represents the image, n represents the total number of images, and the infrared long-wave band images and infrared mid-band images each have corresponding data sets.

[0036] S2: Input the dual-band image dataset described in step 1 into the dual-branch backbone network of the network model, and independently extract deep features from each band image.

[0037] S201, input the dual-band image dataset described in step 1 into the dual-branch backbone network of the network model, and extract infrared long-wave depth features and infrared medium-wave depth features respectively through the two branch backbone networks in the dual-branch backbone network. Each branch backbone network includes six layers of feature extraction modules, and the input of each layer of feature extraction module is the output of the feature extraction module of the previous layer; it is composed of the superposition of cross-channel cross-attention modules.

[0038] The formulas of the two network models in the dual-branch network are:

[0039] F i L =Φ b_L (I L ,φ L )

[0040] F i M =Φ b_M (I M ,φ M )

[0041] in, Represent the feature maps of the i-th layer (i=3, 4, 5) of the infrared long-wave and infrared medium-wave branches, respectively. Only layers 3, 4, and 5 are used. W, H, and C represent the height, width, and number of channels of the feature map. Represent the input infrared long-wave image and infrared medium-wave image respectively, and the number of channels is 1. Φ b_L (·) and Φ b_M (·) represent the feature extraction backbone network of infrared long-wave and medium-wave branches, respectively, and the parameter is φ L and φ M .

[0042] In step 202, each of the feature extraction module channels is divided into two channels, and the two channels merge the features extracted by each channel to obtain a merged feature, and input the merged feature into the channel cross attention module.

[0043] Specifically, for a given infrared dual-band image, a convolution layer operation is performed to obtain spatial refinement features. First, the number of channels C of F1 is divided into two parts, with aC and bC channels respectively, as follows Figure 1 As shown, where a+b=1. Subsequently, 1×1 convolution is further used to compress the channels of the upper and lower branch feature maps to keep them consistent to improve computational efficiency. The compression ratio r is introduced here to control the feature channel to balance the computational cost of the channel cross attention module (the present invention sets r1=2). After splitting and compression operations, the spatial refinement feature F1 is divided into the upper F up and F down Next, we choose to use spatial and channel attention to calculate the spatial weights of feature modulation, and use F up and F down After summing each pixel, the weight is calculated and then merged through the weighted summation method to obtain the merged feature. In this process, since infrared images are grayscale images and express less information, the convolution operation removes the redundant information of each channel and retains the useful information.

[0044] S202, such as Figure 2 As shown in the figure, a channel cross attention module (CCAM) is constructed. Based on spatial attention and channel attention, the merged features obtained above are processed by global average pooling operations across spatial dimensions and global average pooling operations across channel dimensions.

[0045] The goal of the channel-wise attention module is to generate channel-specific They have the same dimensions as F1, and first calculate the corresponding F according to the formula c and F s .

[0046]

[0047]

[0048] Among them C m×m (·) indicates the convolution kernel size is m×m The convolutional layer. represents the features after global average pooling operation across spatial dimensions, Represents the features processed by the global average pooling operation across the channel dimension. In order to reduce the number of parameters and limit the model complexity, the first 1×1 convolution reduces the channel dimension from C to (In the present invention, r2 is set to 16), and the second 1×1 convolution expands it back to C. Then, F is converted to s and F c Fused together to obtain a rough process weight coefficient

[0049]

[0050] To obtain the final Q, Each channel of is adjusted according to the corresponding branch features. The content of the upper and lower branch features is used as a guide to generate the final channel weight coefficient Q. Specifically, Each channel of is rearranged in an alternating manner by a channel shuffle operation.

[0051]

[0052] Where s represents the sigmoid operation and CS(·) represents the channel shuffle operation.

[0053] S203, under the guidance of the weight coefficient matrix Q, the output features F of the upper and lower segmentation stages are combined by weighting each channel up and F down , the channel refinement feature Y can be obtained.

[0054] Y=C 1×1 (F up Q+F down (1-Q))

[0055] Each feature extraction module obtains the channel refinement feature Y through the above S201-S203, which is the infrared long-wave depth feature F obtained by the feature extraction module of this layer. i L or infrared medium wave depth feature F i M .

[0056] S3, the infrared long-wave depth features and infrared medium-wave depth features extracted by the feature extraction modules of the same layer in the two branch backbone networks are input into the infrared dual-band feature cross-fusion module, and the dual FCA module is used to collect complementary information. After the infrared long-wave depth features and infrared medium-wave depth features are fused by the infrared dual-band feature cross-fusion module, the fused features are output. For example, the infrared long-wave depth features F obtained by the third-layer feature extraction modules of the two branch backbone networks are connected. i L and infrared mid-wave depth characteristics F i M The two branches are input into a channel cross attention module together. Similarly, the fourth, fifth and sixth layers of the two branch backbone networks are connected and the same operation is performed.

[0057] By utilizing the cross-attention mechanism, the perceptual advantages of infrared dual-band images are effectively integrated, so that the model can learn the complementary relationship between infrared dual-band features and model the long-range dependencies between cross-band and cross-channel features.

[0058] Construct a cross attention fusion module (CCAM), sum the features of the two bands, perform channel mixing (splicing), shuffle, and convolve.

[0059] For infrared long-wave images, such as Figure 3 As shown, the module first detects the infrared long-wave depth feature F L and medium wave depth feature F M Two convolutional layers with 1×1 filters are applied to generate two feature maps The correlation matrix A is then constructed by dot product operations, and the correlation scores are normalized using the softmax function, which represents the similarity between different bands of medium and long waves. L Apply another convolutional layer with 1×1 filters to generate Used for feature adaptation, C' is the number of channels, which is less than C and used for dimensionality reduction to obtain long-wave local features V L , the long wave local result V LAfter adding the normalized result, the local enhanced features are obtained

[0060]

[0061] in, A is the feature vector at position A in the output feature map. i,j is a scalar value at channel i and position j in matrix A.

[0062] Then the local enhancement feature and infrared long-wave depth feature F L Collected by aggregation operation, infrared long-wave enhanced depth features are obtained In the above process, context information is added to the local feature V L To enhance local features and improve pixel-wise representation.

[0063] Similar to the infrared long-wave branch, another FCA module is also used to enhance the function of the infrared medium-wave branch: This module first L and medium wave depth feature F M Two convolutional layers with 1×1 filters are applied to generate two feature maps The correlation matrix A is then constructed by dot product operations, and the correlation scores are normalized using the softmax function, which represents the similarity between different bands of medium and long waves. M Apply another convolutional layer with 1×1 filters to generate Used for feature adaptation, C' is the number of channels, which is less than C and used for dimensionality reduction to obtain the medium wave local feature V M , the medium wave local result V M After adding the normalized result, the local enhanced features are obtained

[0064]

[0065] in, A is the feature vector at position A in the output feature map. i,j is a scalar value at channel i and position j in matrix A.

[0066] Then the local enhancement feature and infrared medium wave depth feature F M Collected by aggregation operation, infrared long-wave enhanced depth features are obtained In the above process, context information is added to the local feature V M To enhance local features and improve pixel-wise representation.

[0067] Get the enhanced dual-band branch features and Then, input the fusion function ΨFusion (·) Get the fusion features.

[0068]

[0069] S4, uses the fusion features obtained in step 5 to predict the target, inputs the network neck module and detection head, and outputs the target location and category.

[0070] The following is further described with reference to specific embodiments.

[0071] Example

[0072] The performance of the algorithm of the present invention is tested in the simulation interference scene. The experimental results are as follows: Figure 5 As shown, from Figure 5 As can be seen in the fourth scene in the first row and the first scene in the third row, for objects with overlapping and occluded areas, the method proposed in the present invention can fully identify the target and interference areas. In the third scene in the second row, the method proposed in the present invention can accurately identify targets of different scales. In this case, the target scale varies, and the method proposed in the present invention can accurately detect them. In the scenes in the first column, the method proposed in the present invention also performs well in identifying small-sized targets. The surrounding interference areas in the infrared image are brighter than the target, and the surrounding scene can easily drown out the flying target features. Therefore, it is challenging to detect such objects.

Claims

1. An end-to-end anti-interference recognition method based on infrared dual-band images, characterized in that: The following steps are involved: S1, collects infrared long-wave images and infrared medium-wave images; S2, through the two branch backbone networks in the dual-branch backbone network, respectively extract infrared long-wave depth features and infrared medium-wave depth features. Each branch backbone network includes a six-layer feature extraction module; Each of the feature extraction module channels is divided into two channels, and the features extracted by the two channels are merged to obtain a merged feature, which is input into the channel cross-attention module. The channel cross-attention module obtains weight coefficients of the two channels based on the merged feature, and the features extracted by the two channels are multiplied by the corresponding weight coefficients. After adding the products and performing convolution processing, infrared long-wave depth features and infrared medium-wave depth features are obtained; S3: Input the infrared long-wave depth features and infrared medium-wave depth features extracted by the feature extraction modules of the same layer in the two branch backbone networks into the infrared dual-band feature cross-fusion module. After the infrared long-wave depth features and infrared medium-wave depth features are fused through the infrared dual-band feature cross-fusion module, the fused features are output. S4 detects the target by fusing features and outputs the target location and category.

2. The end-to-end anti-interference recognition method based on infrared dual-band images according to claim 1 is characterized in that: In S2, the two channels of each feature extraction module are aC and bC channels respectively, where a+b=1, a and b are weights of the two channels respectively, and C is the total number of channels of the feature extraction module.

3. The end-to-end anti-interference recognition method based on infrared dual-band images according to claim 2 is characterized in that: In S2, the process of merging the features extracted from the two channels to obtain the merged features is as follows: each channel is first compressed by convolution so that the number of channels of the two channels is equal, and then the spatial weights are adjusted by calculating the features through spatial and channel attention. The two channels are summed pixel by pixel based on their respective spatial weights to obtain the corresponding weights.

4. The end-to-end anti-interference recognition method based on infrared dual-band images according to claim 1 is characterized in that: In S2, the channel cross attention module inputs the merged features into two convolutional layers with different convolution kernels, and respectively calculates the global average pooling operation features across the spatial dimension and the global average pooling operation processing features across the channel dimension; After adding and fusing the two features, we can get the process weight coefficient Each channel associated with the process weight coefficient is shuffled by the channels, and all the channels are rearranged in an alternating manner to obtain the weight coefficient Q.

5. The end-to-end anti-interference recognition method based on infrared dual-band images according to claim 4 is characterized in that: In S2, the global average pooling operation across the channel dimension is calculated to process the features: it is processed by two 1×1 convolution operations, the first convolution operation is used to reduce the channel dimension, and the second convolution operation is used to expand the channel dimension.

6. The end-to-end anti-interference recognition method based on infrared dual-band images according to claim 1 is characterized in that: In S3, the deep features from the third to the sixth layers in the two branch backbone networks are input into the infrared dual-band feature cross-fusion module.

7. The end-to-end anti-interference recognition method based on infrared dual-band images according to claim 1 is characterized in that: In S3, the infrared dual-band feature cross fusion module enhances the infrared long-wave depth feature and the infrared medium-wave depth feature respectively to obtain enhanced infrared long-wave depth feature and enhanced infrared medium-wave depth feature, and combines the two enhanced depth features through the fusion function Ψ Fusion (·) After fusion, the fusion features are obtained.

8. The end-to-end anti-interference recognition method based on infrared dual-band images according to claim 7 is characterized in that: In S3, for the infrared long-wave depth feature, the enhancement process is: infrared long-wave depth feature F L and infrared mid-wave depth characteristics F M Through two 1×1 convolutional layers, two feature maps K are obtained. L ,Q M , the feature map K L ,Q M After normalizing the relevant scores through the softmax function, the normalized result is obtained; after adding the normalized result and the long-wave local feature, the local enhanced feature is obtained, and the local enhanced feature and the infrared long-wave depth feature F are added together. L Collected by aggregation operation, infrared long-wave enhanced depth features are obtained 9. The end-to-end anti-interference recognition method based on infrared dual-band images according to claim 7 is characterized in that: In S3, for the infrared medium wave depth feature, the enhancement process is: infrared long wave depth feature F L and infrared mid-wave depth characteristics F M Through two 1×1 convolutional layers, two feature maps K are obtained. L ,Q M , the feature map K L ,Q M After normalizing the relevant scores through the softmax function, the normalized result is obtained; after adding the normalized result and the medium-wave local feature, the sum is added to the infrared medium-wave depth feature F M Obtain infrared mid-wave enhanced depth features through aggregation operation 10. The end-to-end anti-interference recognition method based on infrared dual-band images according to claim 1 is characterized in that: In S4, the target location and category are output through the network neck module and detection head.

Images