End-to-end anti-interference identification method based on infrared dual-band image

Abstract

The invention discloses an end-to-end anti-interference identification method based on an infrared dual-band image, and belongs to the technical field of target identification in digital image processing. The method comprises the following steps: firstly, customizing a double-branch backbone network for feature extraction of an infrared dual-band image through a deep convolutional network, independently extracting feature information of each band image, constructing inter-channel correlation to improve feature expression ability and reduce feature channel redundancy; and constructing a cross attention fusion module to learn a complementary relationship between infrared medium wave and long wave features, and finally obtaining an end-to-end anti-interference identification method based on an infrared dual-band image, thereby improving the anti-interference performance of an infrared aerial target identification algorithm under a complex interference situation.

Description

Technical Field

[0001] The invention belongs to the technical field of target recognition in digital image processing, and in particular is an end-to-end anti-interference recognition method based on infrared dual-band images. Background Art

[0002] The ability to detect and identify objects using thermal radiation is valuable in many ways. However, the effectiveness of infrared imaging systems in detecting and identifying targets can be compromised by interference from a variety of sources. Among them, the targets use jamming techniques to damage infrared sensors and deceive the detection system. Traditional single-band infrared recognition algorithms are susceptible to interference, resulting in reduced accuracy and reliability, further complicating the recognition process.

[0003] In view of the above limitations, driven by the need to enhance target discrimination and anti-interference capabilities, the demand for infrared data fusion technology in multiple bands has been steadily growing. Among them, mid-wave infrared (MWIR) and long-wave infrared (LWIR) are two common sensor technologies in the infrared spectrum range, which differ in wavelength range and application characteristics. The wavelength range of MWIR is between 3 and 5 microns, and it has the characteristics of high reflective components, which enables it to obtain the reflective characteristic information of the target. At the same time, due to the small atmospheric attenuation, MWIR has strong penetration ability and can still obtain relatively clear images under bad weather conditions. The wavelength range of long-wave infrared is between 8 and 14 microns, and its main feature is that the thermal radiation component is high, which enables long-wave infrared to obtain the self-radiation characteristic information of the target. At the same time, it has small atmospheric attenuation and strong penetration ability, and can still obtain clear images at night or under low illumination conditions. In addition, long-wave infrared has high sensitivity and can detect small temperature differences. By fusing two infrared bands, more comprehensive target information can be obtained, and the target detection and identification and tracking capabilities can be enhanced under low visibility conditions such as bad weather or at night. The dual-band imaging system exploits the complementary strengths of two different infrared wavelengths to improve interference-resistant recognition performance in challenging environments.

[0004] Despite the potential advantages of dual-band infrared fusion, there are still some challenges in the existing research work on aerial target recognition. One of the main problems is that when integrating medium-wave infrared and long-wave infrared data to generate a fused image, key target features need to be retained while reducing noise and artifacts. Achieving seamless fusion requires complex algorithms that can effectively combine and enhance feature information from the two bands without introducing distortion or inconsistency. However, traditional methods only support specific scenes or target radiation characteristics. Due to the changeable scenes where aerial targets are located and the complex and changeable interference in the scenes, traditional recognition methods are difficult to adapt to situations with a lot of interference and changeable scenes. They have no adaptive ability, which reduces the recognition accuracy. Summary of the invention

[0005] The purpose of the present invention is to overcome the shortcomings of the above-mentioned prior art and to provide an end-to-end anti-interference recognition method based on infrared dual-band images, so as to solve the problem that the recognition method of medium-wave infrared and long-wave infrared data images in the prior art only supports specific scenes or target radiation characteristics, is difficult to interfere with complex and changeable scenes, and has poor adaptability.

[0006] In order to achieve the above object, the present invention adopts the following technical solutions:

[0007] An end-to-end anti-interference recognition method based on infrared dual-band images includes the following steps:

[0008] S1, collecting and acquiring infrared long-wave images and infrared medium-wave images;

[0009] S2, extracts infrared long-wave depth features and infrared medium-wave depth features respectively through the two branch backbone networks in the dual-branch backbone network, and each branch backbone network includes a six-layer feature extraction module;

[0010] Each of the feature extraction module channels is divided into two channels, and the features extracted from the two channels are merged to obtain a merged feature, and the merged feature is input into a channel cross-attention module, and the channel cross-attention module obtains weight coefficients of the two channels based on the merged feature, and the features extracted from the two channels are multiplied by the corresponding weight coefficients, and the products are added and convolved to obtain infrared long-wave depth features and infrared medium-wave depth features;

[0011] S3, inputting the infrared long-wave depth features and infrared medium-wave depth features extracted by the feature extraction modules of the same layer in the two branch backbone networks into the infrared dual-band feature cross-fusion module, fusing the infrared long-wave depth features and the infrared medium-wave depth features through the infrared dual-band feature cross-fusion module, and outputting the fused features;

[0012] S4 detects the target by fusing features and outputs the target location and category.

[0013] A further improvement of the present invention is:

[0014] Preferably, in S2, the two channels of each of the feature extraction modules are aC and bC channels respectively, wherein a+b=1, a and b are weights of the two channels respectively, and C is the total number of channels of the feature extraction module.

[0015] Preferably, in S2, the process of merging the features extracted from each of the two channels to obtain the merged features is as follows: each channel is first compressed by convolution so that the number of channels of the two channels is equal, and then the spatial weight is adjusted by calculating the features through space and channel attention. The two channels are summed pixel by pixel based on their respective spatial weights to obtain the corresponding weights.

[0016] Preferably, in S2, the channel cross attention module inputs the merged features into two convolution layers with different convolution kernels, and respectively calculates the global average pooling operation features across the spatial dimension and the global average pooling operation processing features across the channel dimension; after adding and fusing the two features, the process weight coefficient is obtained. Each channel associated with the process weight coefficient is shuffled by the channels, and all the channels are rearranged in an alternating manner to obtain the weight coefficient Q.

[0017] Preferably, in S2, in the process of calculating the global average pooling operation across the channel dimension to process the features: through two 1×1 convolution operations, the first convolution operation is used to reduce the channel dimension, and the second convolution operation is used to expand the channel dimension.

[0018] Preferably, in S3, the depth features of the third to sixth layers in the two branch backbone networks are input into the infrared dual-band feature cross-fusion module.

[0019] Preferably, in S3, the infrared dual-band feature cross-fusion module enhances the infrared long-wave depth feature and the infrared medium-wave depth feature respectively to obtain an enhanced infrared long-wave depth feature and an enhanced infrared medium-wave depth feature, and combines the two enhanced depth features through a fusion function Ψ Fusion (·) After fusion, the fusion features are obtained.

[0020] Preferably, in S3, for the infrared long-wave depth feature, the enhancement process is: infrared long-wave depth feature F L and infrared mid-wave depth feature F M Through two 1×1 convolutional layers respectively, two feature maps K are obtained L ,Q M , the feature map K L ,Q M After normalizing the relevant scores through the softmax function, the normalized result is obtained; after adding the normalized result and the long-wave local feature, the local enhanced feature is obtained, and the local enhanced feature and the infrared long-wave depth feature F are added together. L Collected by aggregation operation, infrared long-wave enhanced depth features are obtained

[0021] Preferably, in S3, for the infrared medium wave depth feature, the enhancement process is: infrared long wave depth feature F L and infrared mid-wave depth feature F M Through two 1×1 convolutional layers respectively, two feature maps K are obtained L ,Q M , the feature map K L ,Q MAfter normalizing the relevant scores through the softmax function, the normalized result is obtained; after adding the normalized result and the medium-wave local feature, the summed result and the infrared medium-wave depth feature F are added. M Through the aggregation operation, infrared mid-wave enhanced depth features are obtained

[0022] Preferably, in S4, the target position and category are output through the network neck module and the detection head.

[0023] Compared with the prior art, the present invention has the following beneficial effects:

[0024] The present invention discloses an end-to-end anti-interference recognition method based on infrared dual-band images. The method first uses a deep convolutional network to customize a dual-branch backbone network specifically for feature extraction of infrared dual-band image pairs, and independently extracts feature information of each band image. Constructing inter-channel correlation improves feature expression capabilities and reduces feature channel redundancy. Constructing a cross-attention fusion module to learn the complementary relationship between infrared medium-wave and long-wave features, an end-to-end anti-interference recognition method based on infrared dual-band images is obtained, which improves the anti-interference performance of infrared aerial target recognition algorithms under complex interference situations. The method constructs the deep features independently extracted from infrared dual-band images into a black box model, rather than separately constructing an image fusion network model and a target detection network model, which effectively reduces the complexity and scale of the network model while significantly improving the algorithm's ability to enhance target feature representation and anti-interference recognition in complex interference scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 The feature extraction module of the present invention;

[0026] Figure 2 The channel cross attention module of the present invention;

[0027] Figure 3 It is the infrared dual-band feature cross-fusion module of the present invention;

[0028] Figure 4 It is the infrared dual-band target recognition algorithm framework of the present invention;

[0029] Figure 5 This is a partial anti-interference recognition result of the present invention. DETAILED DESCRIPTION

[0030] In the following, the terms "first", "second", "third", and "fourth" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, a feature defined as "first", "second", "third", and "fourth" may explicitly or implicitly include one or more of the features.

[0031] The co-shooting method provided in the embodiment of the present application can be applied to terminal devices such as mobile phones, tablet computers, wearable devices, vehicle-mounted devices, augmented reality (AR) / virtual reality (VR) devices, laptop computers, ultra-mobile personal computers (UMPC), netbooks, personal digital assistants (PDA), etc. The embodiment of the present application does not impose any restrictions on the specific type of the terminal device.

[0032] It should be noted that the terms "first", "second", etc. in the specification and drawings of the present invention are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0033] To achieve the above object, the present invention discloses an end-to-end anti-interference recognition method based on infrared dual-band images, which comprises the following steps:

[0034] S1, collect and acquire infrared long-wave images and infrared medium-wave images.

[0035] The specific process is to obtain infrared dual-band image data respectively and construct an infrared dual-band image dataset Data = {I1, I2, I3, …, I n}, where I represents the image, n represents the total number of images, and the infrared long-wave band images and infrared mid-wave band images each have a corresponding data set.

[0036] S2, input the dual-band image dataset described in step 1 into the dual-branch backbone network of the network model, and independently extract deep features from each band image.

[0037] S201, input the dual-band image data set described in step 1 into the dual-branch backbone network of the network model, and extract infrared long-wave depth features and infrared medium-wave depth features respectively through the two branch backbone networks in the dual-branch backbone network. Each branch backbone network includes six layers of feature extraction modules, and the input of each layer of feature extraction module is the output of the feature extraction module of the previous layer; it is composed of the superposition of cross-channel cross-attention modules.

[0038] The formulas of the two network models in the dual-branch network are:

[0039] F i L =Φ b_L (I L ,φ L )

[0040] F i M =Φ b_M (I M ,φ M )

[0041] in, They represent the feature maps of the i-th layer (i=3, 4, 5) of the infrared long-wave and infrared medium-wave branches respectively, and only use layers 3, 4, and 5. W, H, and C represent the height, width, and number of channels of the feature map. Respectively represent the input infrared long-wave image and infrared medium-wave image, and the number of channels is 1. Φ b_L (·) and Φ b_M (·) represent the feature extraction backbone network of infrared long-wave and medium-wave branches, respectively, and the parameter is φ L and φ M .

[0042] In step 202, each of the feature extraction module channels is divided into two channels, and the two channels merge the features extracted by each channel to obtain merged features, and input the merged features into the channel cross-attention module.

[0043] Specifically, for a given infrared dual-band image, a convolution layer operation is performed to obtain the spatial refinement feature First, the number of channels C of F1 is divided into two parts, with aC and bC channels respectively, as Figure 1 As shown, a+b=1. Subsequently, 1×1 convolution is further used to compress the channels of the upper and lower branch feature maps to keep them consistent to improve computational efficiency. The compression ratio r is introduced here to control the feature channel to balance the computational cost of the channel cross attention module (the present invention sets r1=2). After splitting and compression operations, the spatial refinement feature F1 is divided into the upper F up and F down Next, we choose to use spatial and channel attention to calculate the spatial weights of feature modulation and convert F up and F down After summing each pixel, the weight is calculated, and then the weighted summation method is used to merge and obtain the merged features. In this process, since the infrared image is a grayscale image and expresses little information, the convolution operation is performed to remove the redundant information of each channel and retain the useful information.

[0044] S202, such as Figure 2 As shown in the figure, a channel cross attention module (CCAM) is constructed. Based on spatial attention and channel attention, the merged features obtained above are processed by global average pooling operations across spatial dimensions and global average pooling operations across channel dimensions.

[0045] The goal of the channel cross attention module is to generate channel-specific They have the same dimensions as F1, and first calculate the corresponding F according to the formula c and F s .

[0046]

[0047]

[0048] Among them C m×m (·) indicates the convolution kernel size is m×m The convolutional layer. represents the features after global average pooling across spatial dimensions, represents the features processed by the global average pooling operation across the channel dimension. In order to reduce the number of parameters and limit the model complexity, the first 1×1 convolution reduces the channel dimension from C to (In the experiment of the present invention, r2=16 is set), and the second 1×1 convolution expands it back to C. Then, F is added by a simple addition operation 1. s and F c fused together to obtain a rough process weight coefficient

[0049]

[0050] To obtain the final Q, Each channel of is adjusted according to the corresponding branch features. The content of the upper and lower branch features is used as a guide to generate the final channel weight coefficient Q. Specifically, Each channel of is rearranged in an alternating manner by a channel shuffle operation.

[0051]

[0052] Where s represents the sigmoid operation and CS(·) represents the channel shuffle operation.

[0053] S203, under the guidance of the weight coefficient matrix Q, the output features F of the upper and lower segmentation stages are weighted and merged channel by channel up and F down , the channel refinement feature Y can be obtained.

[0054] Y=C 1×1 (F up ·Q+F down (1-Q)

[0055] Each feature extraction module obtains the channel refinement feature Y through the above S201-S203, which is the infrared long-wave depth feature F obtained by the feature extraction module of this layer. i L Or infrared medium wave depth feature F i M .

[0056] S3, the infrared long-wave depth features and infrared medium-wave depth features extracted by the feature extraction modules of the same layer in the two branch backbone networks are input into the infrared dual-band feature cross-fusion module, and the dual FCA module is used to collect complementary information. After the infrared long-wave depth features and infrared medium-wave depth features are fused by the infrared dual-band feature cross-fusion module, the fused features are output. For example, the infrared long-wave depth features F obtained by the feature extraction modules of the third layer in the two branch backbone networks are connected. i L and infrared mid-wave depth feature F i M They are input together into a channel cross attention module. Similarly, the fourth, fifth and sixth layers of the two branch backbone networks are connected and the same operation is performed.

[0057] By utilizing the cross-attention mechanism, the perceptual advantages of infrared dual-band images are effectively integrated, so that the model can learn the complementary relationship between infrared dual-band features and model the long-range dependencies of cross-band and cross-channel features.

[0058] To build a cross-attention fusion module (CCAM), the features of the two bands are summed, the channels are mixed (concatenated), shuffled, and convolved.

[0059] For infrared long-wave images, such as Figure 3 As shown in the figure, the module first detects the infrared long-wave depth feature F L and the medium wave depth feature F M Two convolutional layers with 1×1 filters are applied to generate two feature maps respectively. The correlation matrix A is then constructed by dot product operations, and the correlation scores are then normalized using the softmax function, which represents the similarity between different bands of medium and long waves. L Another convolutional layer with a 1×1 filter is applied to generate Used for feature adaptation, C' is the number of channels, which is less than C. Used for dimensionality reduction to obtain long-wave local features V L , the long wave local result V LAfter adding the normalized result, the local enhanced features are obtained

[0060]

[0061] in, is the feature vector at position A in the output feature map. i,j is a scalar value at channel i and position j in matrix A.

[0062] Then the local enhancement feature and infrared long-wave depth feature F L Collected by aggregation operation, infrared long-wave enhanced depth features are obtained In the above process, context information is added to the local feature V L To enhance local features and improve pixel-by-pixel representation.

[0063] Similar to the infrared long-wave branch, another FCA module is also used to enhance the function of the infrared medium-wave branch: this module firstly L and the medium wave depth feature F M Two convolutional layers with 1×1 filters are applied to generate two feature maps respectively. The correlation matrix A is then constructed by dot product operations, and the correlation scores are then normalized using the softmax function, which represents the similarity between different bands of medium and long waves. M Another convolutional layer with a 1×1 filter is applied to generate Used for feature adaptation, C' is the number of channels, which is less than C. Used for dimensionality reduction to obtain the medium wave local feature V M , the local result V M After adding the normalized result, the local enhanced features are obtained

[0064]

[0065] in, is the feature vector at position A in the output feature map. i,j is a scalar value at channel i and position j in matrix A.

[0066] Then the local enhancement feature and infrared mid-wave depth feature F M Collected by aggregation operation, infrared long-wave enhanced depth features are obtained In the above process, context information is added to the local feature V M To enhance local features and improve pixel-by-pixel representation.

[0067] Get the enhanced dual-band branch features and After that, input the fusion function ΨFusion (·) Get the fusion features.

[0068]

[0069] S4, use the fusion features obtained in step 5 to predict the target, input the network neck module and detection head, and output the target location and category.

[0070] The following is further described in conjunction with specific embodiments.

[0071] Example

[0072] The performance of the algorithm tested in the simulated interference scenario is shown in the experimental results. Figure 5 As shown, from Figure 5 It can be seen in the fourth scene in the first row and the first scene in the third row that for objects with overlapping and occluded areas, the method proposed in the present invention can fully identify the target and interference areas. In the third scene in the second row, the method proposed in the present invention can accurately identify targets of different scales. In this case, the target scale varies, and the method proposed in the present invention can accurately detect them. In the scenes in the first column, the method proposed in the present invention also performs well in identifying small-sized targets. The surrounding interference areas in the infrared image are brighter than the target, and the surrounding scenes can easily drown out the flying target features. Therefore, it is challenging to detect such objects.

Claims

1. An end-to-end anti-interference recognition method based on infrared dual-band images, characterized in that: The following steps are involved: S1, collecting and acquiring infrared long-wave images and infrared medium-wave images; S2, extracts infrared long-wave depth features and infrared medium-wave depth features respectively through the two branch backbone networks in the dual-branch backbone network, and each branch backbone network includes a six-layer feature extraction module; Each of the feature extraction module channels is divided into two channels, and the features extracted from the two channels are merged to obtain a merged feature, and the merged feature is input into a channel cross-attention module, and the channel cross-attention module obtains weight coefficients of the two channels based on the merged feature, and the features extracted from the two channels are multiplied by the corresponding weight coefficients, and the products are added and convolved to obtain infrared long-wave depth features and infrared medium-wave depth features; S3, inputting the infrared long-wave depth features and infrared medium-wave depth features extracted by the feature extraction modules of the same layer in the two branch backbone networks into the infrared dual-band feature cross-fusion module, fusing the infrared long-wave depth features and the infrared medium-wave depth features through the infrared dual-band feature cross-fusion module, and outputting the fused features; S4 detects the target by fusing features and outputs the target location and category.

2. The end-to-end anti-interference recognition method based on infrared dual-band images according to claim 1 is characterized in that: In S2, the two channels of each feature extraction module are aC and bC channels respectively, wherein a+b=1, a and b are weights of the two channels respectively, and C is the total number of channels of the feature extraction module.

3. The end-to-end anti-interference recognition method based on infrared dual-band images according to claim 2 is characterized in that: In S2, the process of merging the features extracted from the two channels to obtain the merged features is as follows: each channel is first compressed by convolution to make the number of channels of the two channels equal, and then the spatial weights are adjusted by calculating the features through spatial and channel attention. The two channels are summed pixel by pixel based on their respective spatial weights to obtain the corresponding weights.

4. The end-to-end anti-interference recognition method based on infrared dual-band images according to claim 1 is characterized in that: In S2, the channel cross attention module inputs the merged features into two convolutional layers with different convolution kernels, and respectively calculates the global average pooling operation features across the spatial dimension and the global average pooling operation processing features across the channel dimension; After adding and fusing the two features, we get the process weight coefficient Each channel associated with the process weight coefficient is shuffled by the channels, and all the channels are rearranged in an alternating manner to obtain the weight coefficient Q.

5. The end-to-end anti-interference recognition method based on infrared dual-band images according to claim 4 is characterized in that: In S2, the global average pooling operation across the channel dimension is calculated to process the features: it is processed by two 1×1 convolution operations, the first convolution operation is used to reduce the channel dimension, and the second convolution operation is used to expand the channel dimension.

6. The end-to-end anti-interference recognition method based on infrared dual-band images according to claim 1 is characterized in that: In S3, the deep features from the third to the sixth layers in the two branch backbone networks are input into the infrared dual-band feature cross-fusion module.

7. The end-to-end anti-interference recognition method based on infrared dual-band images according to claim 1 is characterized in that: In S3, the infrared dual-band feature cross-fusion module enhances the infrared long-wave depth feature and the infrared medium-wave depth feature respectively to obtain enhanced infrared long-wave depth feature and enhanced infrared medium-wave depth feature, and combines the two enhanced depth features through the fusion function Ψ Fusion (·) After fusion, the fusion features are obtained.

8. The end-to-end anti-interference recognition method based on infrared dual-band images according to claim 7 is characterized in that: In S3, for the infrared long-wave depth feature, the enhancement process is: infrared long-wave depth feature F L and infrared mid-wave depth feature F M Through two 1×1 convolutional layers respectively, two feature maps K are obtained L ,Q M , the feature map K L ,Q M After normalizing the relevant scores through the softmax function, the normalized result is obtained; after adding the normalized result and the long-wave local feature, the local enhanced feature is obtained, and the local enhanced feature and the infrared long-wave depth feature F are added together. L Collected by aggregation operation, infrared long-wave enhanced depth features are obtained 9. The end-to-end anti-interference recognition method based on infrared dual-band images according to claim 7 is characterized in that: In S3, for the infrared medium wave depth feature, the enhancement process is: infrared long wave depth feature F L and infrared mid-wave depth feature F M Through two 1×1 convolutional layers respectively, two feature maps K are obtained L ,Q M , the feature map K L ,Q M After normalizing the relevant scores through the softmax function, the normalized result is obtained; after adding the normalized result and the medium-wave local feature, the summed result and the infrared medium-wave depth feature F are added. M Through the aggregation operation, infrared mid-wave enhanced depth features are obtained 10. The end-to-end anti-interference recognition method based on infrared dual-band images according to claim 1 is characterized in that: In S4, the target location and category are output through the network neck module and the detection head.

Images