Feature integration method of interactive convolution and dynamic focusing for infrared images

Abstract

This invention discloses a feature integration method for interactive convolution and dynamic focusing of infrared images, comprising: obtaining infrared image pairs of varying resolutions in various real-world scenarios using an infrared camera; performing degradation preprocessing on a portion of the original high-resolution images to obtain low-quality high-resolution images to form a mixed low-resolution dataset; and dividing the processed dataset into a training set and a test set; constructing a two-layer feature extraction module for feature modeling; training a network using the processed training set and optimizing a loss function; and inputting the low-resolution infrared image into the trained network to output a high-resolution reconstruction result. By fusing local and global features, this method significantly improves the super-resolution reconstruction quality of infrared images in complex scenarios such as low contrast and blurred edges, while maintaining high computational efficiency.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a feature integration method for interactive convolution and dynamic focusing of infrared images. Background Art

[0002] Infrared imaging technology has broad applications in military reconnaissance, night vision security, medical diagnosis, and industrial inspection. However, due to the inherent resolution limitations of infrared sensors, signal-to-noise interference, and the uneven thermal radiation in complex environments, acquired infrared images often suffer from low resolution, blurred details, blurred edges, and insufficient contrast, severely impacting subsequent target recognition and information extraction. Traditional infrared image enhancement methods primarily rely on interpolation algorithms (such as bilinear interpolation and bicubic interpolation) or filtering (such as non-uniformity correction). However, such methods can only smooth image textures and struggle to recover missing high-frequency detail, resulting in limited super-resolution reconstruction.

[0003] In recent years, deep learning-based super-resolution technology has made significant progress. Super-resolution convolutional neural networks (SRCNNs) extract features and reconstruct high-resolution images through multi-layer convolution. However, the receptive field of fixed convolution kernels is difficult to adapt to the diverse feature distribution of infrared images. Furthermore, while the introduction of generative adversarial networks (GANs) has improved image perception quality, their training instability and pattern collapse limit their robustness. While the introduction of dynamic filtering modules and attention mechanisms can enhance key feature extraction, most methods use a single manifold optimization strategy and lack the interactive fusion of multi-scale information, resulting in insufficient fine-grained texture reconstruction capabilities.

[0004] Existing dynamic focusing strategies dynamically adapt feature weights through adjustable attention parameters. However, most designs focus solely on local feature interactions, neglecting the collaborative modeling of cross-scale features. Furthermore, while residual connection-based architectures can alleviate the vanishing gradient problem, their feature transfer efficiency in the complex backgrounds of infrared images still needs to be improved. Therefore, a method that combines dynamic feature interactions guided by multi-directional information with a focused attention mechanism is urgently needed to improve detail reconstruction while suppressing noise, providing a more efficient solution for infrared imaging. Summary of the Invention

[0005] The purpose of the present invention is to provide a feature integration method for interactive convolution and dynamic focusing of infrared images, which can effectively solve the problems of insufficient multi-scale feature fusion, poor dynamic adaptability and weak detail recovery ability existing in traditional infrared image super-resolution technology.

[0006] To achieve the above object, the present invention provides a feature integration method for interactive convolution and dynamic focusing of infrared images, comprising the following steps:

[0007] S1. Obtain infrared image pairs of different resolutions in various real scenes through infrared cameras;

[0008] S2. Perform degradation preprocessing on a portion of the original high-resolution images to obtain low-quality high-resolution images to form a mixed low-resolution dataset, and divide the processed dataset into a training set and a test set;

[0009] S3, build a two-layer feature extraction module for feature modeling;

[0010] S4. Use the processed training set to train the network and optimize the loss function;

[0011] S5. Input the low-resolution infrared image into the trained network and output the high-resolution reconstruction result.

[0012] Furthermore, step S1 uses an infrared camera to obtain infrared image data of different scenes by adjusting optical zoom and focal length, specifically:

[0013] S1.1. Optical zoom uses long-wave (8-14μm) and medium-wave (3-5μm) infrared cameras that support motorized zoom. By adjusting the lens focal length, switching from wide-angle to telephoto is achieved, directly generating raw image pairs with different spatial resolutions.

[0014] S1.2. To obtain infrared image pairs with different resolutions, the long-wave camera outputs a resolution of 640×480 at a focal length of 50 mm and 1280×1024 at a focal length of 200 mm. The medium-wave camera generates images of different resolutions by changing the lens focal length to 30 mm or 90 mm. This directly utilizes the hardware characteristics to generate the real physical resolution difference, which is consistent with the actual working mode of the infrared imaging system.

[0015] Furthermore, in S2, a portion of the original high-resolution images are subjected to degradation preprocessing to obtain low-quality high-resolution images to form a mixed low-resolution dataset, and the processed dataset is divided into a training set and a test set. The steps are:

[0016] S2.1-1. The blur effect of the optical system is simulated by Gaussian function, and resolution degradation is achieved by combining downsampling operation. A part of the original high-resolution image is artificially degraded. The Gaussian blur is achieved by convolving the image with a two-dimensional Gaussian kernel. Defined as:

[0017] ;

[0018] Where, is the standard deviation of the Gaussian kernel, which controls the degree of blur;

[0019] S2.1-2. Use Gaussian blur to simulate optical system aberrations, and then generate a low-resolution image by downsampling:

[0020] ;

[0021] Where, is the Gaussian kernel G for high-resolution images Convolution and downsampling are downsampling operations that average adjacent pixels. Synthetic LR-HR pairs generated through controllable degradation preprocessing provide accurate pixel-level supervision signals, helping the model learn basic degradation patterns. Integrating the synthesized pairs with the collected real LR-HR pairs ensures that the training data covers a variety of degradation scenarios and improves model robustness.

[0022] S2.2. Furthermore, the mixed low-resolution dataset needs to be divided into training and test sets for subsequent network training and validation. The data is divided proportionally into a training set (70%), a validation set (15%), and a test set (15%), ensuring that each subset covers different scenes, resolutions, and noise levels to verify generalization ability.

[0023] Furthermore, a two-layer feature extraction module is constructed in S3 for feature modeling, including two stages: shallow feature extraction and deep feature extraction. First, shallow features are extracted by a smaller convolution kernel to capture local details and high-frequency information (such as edges and textures) in the image. Then, the deep extraction module focuses on global context and semantic information to understand the overall structure and semantic relationship of the image. The specific steps are as follows:

[0024] S3.1-1. The shallow feature extraction module uses a 3×3 convolutional layer to perform preliminary feature extraction on the input low-resolution infrared image and output a shallow feature map. The input low-resolution image is set to:

[0025] ;

[0026] Where, is the spatial resolution of the image (height × width), Is the number of input channels. Infrared images usually have a single channel of 1. When the convolution kernel is set to In the case of , each pixel value of the output feature map F is calculated by the following formula:

[0027] ;

[0028] Where, is the dimension space of the convolution kernel, is the pixel coordinate of the output feature map, c is the output channel index, where ,and is the number of output channels; is the bias term, which is initialized to 0 here; It is the boundary processing after filling;

[0029] S3.1-2, input size is , the convolution kernel size is kxk, the step size is s=1, and there is no padding, that is, p=0, then the output size is as follows:

[0030] ;

[0031] ;

[0032] Where, and Output length and width without padding.

[0033] Furthermore, the deep feature extraction module (DFB) in S3 is composed of multiple feature fusion blocks (FBs) stacked together. The module uses a two-way information interaction mechanism to enhance feature expression capabilities, including a source information branch based on deep convolution and an information branch based on an attention mechanism. The specific steps are as follows:

[0034] S3.2-1, Deep Feature Fusion Block (DFB) generates a channel attention map by passing the features of the convolution branch through global average pooling, 1×1 convolution layer and Sigmoid activation function, which is used to enhance the feature representation of the self-attention branch. Among them, global average pooling can capture global context information and avoid local bias. The global mean is calculated for each channel to obtain the channel descriptor. :

[0035] ;

[0036] in, is the c-th channel value of the input feature X at position (i, j), It is all channels;

[0037] S3.2-2. Enhance feature representation through channel attention mechanism. The core idea is to use convolution branches to extract local features, self-attention branches to model global dependencies, and channel attention maps to dynamically adjust feature weights, combining the advantages of both. 1×1 convolution is used to compress the number of channels (usually reducing the dimension to C / r, where r is the compression ratio):

[0038] ;

[0039] Where, , s is the compressed eigenvector, and ; In addition, the ReLU activation function is used in the channel attention map, and the formula is as follows:

[0040] ;

[0041] The ReLU in the formula is a nonlinear transformation, which can alleviate the gradient disappearance problem;

[0042] S3.2-3. Another information branch of the bidirectional information interaction mechanism comes from the spatial interaction module. The features of the self-attention branch are passed through a 1×1 convolution layer and a Sigmoid activation function to generate a spatial attention map, which is used to enhance the feature representation of the convolution branch. When generating spatial attention weights, the fully connected layer (FC) and the Sigmoid activation function are mainly used to generate spatial attention weights:

[0043] ;

[0044] ;

[0045] Where, is the fully connected layer weight, Belongs to the real number range and is a bias term; It is a sigmoid function, ensuring the weight value .

[0046] Furthermore, the deep feature fusion block DFB adopts a dual-branch parallel structure, including a convolution branch and a self-attention branch, and realizes feature enhancement in channel and spatial dimensions through a two-way information interaction mechanism. The specific steps are as follows:

[0047] S3.3-1. The convolution branch adopts a dual-path deep convolution structure. The first path uses 3×3 deep convolution to extract local detail features, and the second path uses 5×5 deep convolution to expand the receptive field and enhance texture information. The outputs of the two paths are spliced after dimensionality reduction through 1×1 convolution to form the final convolution branch features. The feature outputs of the two paths are calculated as follows:

[0048] ;

[0049] ;

[0050] In the formula, X belongs to is the input feature, H×W is the spatial resolution, C is the number of channels, K represents the convolution kernel, and b is the optional bias term under the convolution kernel size. Then, the output feature maps of the two branches are directly spliced in the channel dimension to form the final fusion feature. The splicing operation is element merging, without parameter calculation, only increasing the number of channels, and the computational overhead is small. The formula for fusing dual-path features is as follows:

[0051] ;

[0052] Where, and are the features output by the two paths, It is a 1×1 dimensionality reduction convolution operation with a convolution kernel size of 1×1 (i.e., covering only a single pixel of the input feature map). It is used to adjust the number of channels, fuse features, or implement cross-channel information interaction. Concat fusion merges two-branch features along the channel dimension, combining local and global information in a simple and efficient way. It is one of the core operations for improving model performance.

[0053] S3.3-2. The self-attention branch uses window partitioning and multi-head self-attention mechanisms to model global dependencies. Specifically, the input features are divided into multiple non-overlapping windows, and self-attention is calculated within each window. The multi-head self-attention mechanism is used to divide the features into multiple subspaces for parallel calculation. The window offset strategy is used to enhance the information interaction between different windows. In this step, the main task is to generate attention weights by calculating the correlation between features. The specific formula is as follows:

[0054] ;

[0055] Where Z is the feature representation after self-attention weighting, which contains global context information, D is the dimension of the feature, Softmax is the normalized exponential function, Q, K, and V are the mappings of the input feature X to the query, key, and value, respectively.

[0056] Furthermore, in S4, the processed training set is used to train the network and optimize the loss function to improve the quality and details of the reconstructed image. The pixel-level loss, cross entropy loss, and reconstruction loss are combined. The specific steps are as follows:

[0057] S4.1. To enhance the distinguishing ability of feature representation, the model can be implemented by minimizing the loss function. The expression of the pixel-level loss function is:

[0058] ;

[0059] Where L is the high-resolution image and reconstructed image The pixel difference, It refers to the pixel value of the high-resolution image with coordinates (i, h, w);

[0060] S4.2. Based on the attention mechanism, fine-tuning the model through supervised segmentation tasks, calculating the segmentation loss function, and optimizing the network weights through backpropagation can help the model learn richer spatial structure information from the data. This task uses the cross-entropy loss function to calculate the segmentation error:

[0061] ;

[0062] In the formula, given input x and corresponding label ,here is the true label of the i-th pixel, is the probability that the i-th pixel belongs to a certain category predicted by the model;

[0063] S4.3. After data processing and feature extraction, the model needs to generate new samples by learning the reconstruction process. The training goal is to minimize the reconstruction error, using the following loss function:

[0064] ;

[0065] in, is the result of the network reconstructing the image, and the loss function is the difference between the predicted result and the real original high-resolution data The error between .

[0066] Furthermore, the S5 inputs the low-resolution infrared image into the trained network, fuses the shallow features with the deep features through long skip connections, performs upsampling using the PixelShuffle operation, and finally uses a 3×3 convolutional layer to restore the target resolution and output a high-resolution reconstruction result. The specific steps are as follows:

[0067] S5.1. Long skip connections can directly transmit shallow features and enhance gradient backpropagation. The main task in this step is channel dimension splicing. The formula is as follows:

[0068] ;

[0069] Where, is the l-th layer feature map, is the k-th layer feature map, and concat is the fusion method;

[0070] S5.2. Next, we use the PixelShuffle operation for upsampling. The core idea is to expand the spatial dimensions (H, W) of the low-resolution feature map while reducing the number of channels (C) to achieve efficient upsampling. The calculation formula is as follows:

[0071] ;

[0072] Where r is the upper adoption factor, 、 、 where is the number of channels, height, and width after reconstructing; Y and X are the output and input features, respectively. Compared to traditional interpolation (such as bilinear interpolation), this method better preserves high-frequency details. Therefore, the image reconstruction module uses pixel shuffle and a 3×3 convolutional layer to restore the feature map to a high-resolution image.

[0073] Beneficial Effects: The present invention first collects 3D point cloud data of underground spaces from multiple perspectives and directions by deploying multiple sensors, including lidar and vision sensors, at different locations. Data from different perspectives or time periods are then registered and aligned. Feature extraction and map construction are completed by extracting meaningful local features and removing the influence of dynamic objects (such as vehicles or pedestrians) on map construction. A sparse 3D U-Net module is designed to extract 3D features. Based on feature extraction, a self-supervised learning module is introduced to enhance feature representation using unlabeled point cloud data. Missing point cloud data is filled in through an iterative generation process, supplementing missing spatial details. A feature projection model is used to map features from different modalities into a shared latent space. Ultimately, breakthroughs are made in key technologies such as 3D semantic segmentation and 3D reconstruction under information-missing conditions, and multi-agent data splicing. A method for detecting and characterizing underground confined spaces is established, providing technical support for the development of new autonomous underground space detection and environmental characterization equipment, and enhancing the ability to acquire intelligence on targets in underground confined spaces. BRIEF DESCRIPTION OF THE DRAWINGS

[0074] Figure 1 It is a schematic flow diagram of the present invention;

[0075] Figure 2 is a schematic diagram of the two-way information interaction adopted by the present invention;

[0076] Figure 3 It is a schematic diagram of the feature fusion block FB proposed in the present invention;

[0077] Figure 4 It is a fusion structure diagram of the two layers of features of the present invention. DETAILED DESCRIPTION

[0078] The present invention will be further described below with reference to the accompanying drawings.

[0079] like Figure 1 As shown in FIG, the feature integration method for interactive convolution and dynamic focusing of infrared images includes the following steps:

[0080] S1. Obtain infrared image pairs of different resolutions in various real scenes through infrared cameras;

[0081] S2. Perform degradation preprocessing on a portion of the original high-resolution images to obtain low-quality high-resolution images to form a mixed low-resolution dataset, and divide the processed dataset into a training set and a test set;

[0082] S3, build a two-layer feature extraction module for feature modeling;

[0083] S4. Use the processed training set to train the network and optimize the loss function;

[0084] S5. Input the low-resolution infrared image into the trained network and output the high-resolution reconstruction result.

[0085] As a preferred embodiment, step S1 uses an infrared camera to obtain infrared image data of different scenes by adjusting optical zoom and focal length, and the steps are as follows:

[0086] S1.1. Optical zoom uses long-wave (8-14μm) and medium-wave (3-5μm) infrared cameras that support motorized zoom. By adjusting the lens focal length, switching from wide-angle to telephoto is achieved, directly generating raw image pairs with different spatial resolutions.

[0087] S1.2. To obtain infrared image pairs with different resolutions, the long-wave camera outputs a resolution of 640×480 at a focal length of 50 mm and 1280×1024 at a focal length of 200 mm. The medium-wave camera generates images of different resolutions by replacing the lens with a focal length of 30 mm or 90 mm. This directly utilizes hardware characteristics to generate real physical resolution differences, which is consistent with the actual working mode of the infrared imaging system.

[0088] As a preferred embodiment, in S2, the steps of performing degradation preprocessing on a portion of the original high-resolution images to obtain low-quality high-resolution images to form a mixed low-resolution dataset, and dividing the processed dataset into a training set and a test set are as follows:

[0089] S2.1-1. The blur effect of the optical system is simulated by Gaussian function, and resolution degradation is achieved by combining downsampling operation. A part of the original high-resolution image is artificially degraded. The Gaussian blur is achieved by convolving the image with a two-dimensional Gaussian kernel. Defined as:

[0090] ;

[0091] Where, is the standard deviation of the Gaussian kernel, which controls the degree of blur;

[0092] S2.1-2. Use Gaussian blur to simulate optical system aberrations, and then generate a low-resolution image by downsampling:

[0093] ;

[0094] Where, is the Gaussian kernel G for high-resolution images Convolution and downsampling are downsampling operations that average adjacent pixels. Synthetic LR-HR pairs generated through controllable degradation preprocessing provide accurate pixel-level supervision signals, helping the model learn basic degradation patterns. Integrating the synthesized pairs with the collected real LR-HR pairs ensures that the training data covers a variety of degradation scenarios and improves model robustness.

[0095] S2.2. The mixed low-resolution dataset needs to be divided into training and test sets for subsequent network training and validation. The data should be divided into training set (70%), validation set (15%) and test set (15%) in proportion to ensure that each subset covers different scenes, resolutions and noise levels to verify the generalization ability.

[0096] As a preferred embodiment, a two-layer feature extraction module is constructed in S3 for feature modeling, including two stages: shallow feature extraction and deep feature extraction. First, shallow features are extracted by a smaller convolution kernel to capture local details and high-frequency information (such as edges and textures) in the image. Then, a deep extraction module is used to focus on global context and semantic information to understand the overall structure and semantic relationship of the image. The specific steps are as follows:

[0097] S3.1-1. The shallow feature extraction module uses a 3×3 convolutional layer to perform preliminary feature extraction on the input low-resolution infrared image and output a shallow feature map. The input low-resolution image is set to:

[0098] ;

[0099] Where, is the spatial resolution of the image (height × width), Is the number of input channels. Infrared images usually have a single channel of 1. When the convolution kernel is set to In the case of , each pixel value of the output feature map F is calculated by the following formula:

[0100] ;

[0101] Where, is the dimension space of the convolution kernel, is the pixel coordinate of the output feature map, c is the output channel index, where ,and is the number of output channels; is the bias term, which is initialized to 0 here; It is the boundary processing after filling;

[0102] S3.1-2, input size is , the convolution kernel size is kxk, the step size is s=1, and there is no padding, that is, p=0, then the output size is as follows:

[0103] ;

[0104] ;

[0105] Where, and Output length and width without padding.

[0106] As a preferred embodiment, the deep feature extraction module (DFB) in S3 is composed of multiple feature fusion blocks (FB) stacked together, wherein the module uses a two-way information interaction mechanism to enhance feature expression capabilities, including a source information branch based on deep convolution and an information branch based on an attention mechanism, such as Figure 3 The Feature Fusion Block (FFB) achieves efficient fusion of local and global features through a dual-branch parallel structure. Its convolution branch uses 3×3 and 5×5 dual-path deep convolution to extract multi-scale local features, which are then spliced after 1×1 convolution compression. The attention branch models long-range dependencies through window partitioning and a multi-head self-attention mechanism, and introduces window offsets to enhance cross-window interactions. The two branches dynamically complement each other through a two-way interaction mechanism: the features of the convolution branch are subjected to global average pooling and a fully connected layer to generate channel attention weights, which are used to enhance the value matrix of the attention branch; the features of the attention branch are used to generate a spatial attention mask through 1×1 convolution to refine the output features of the convolution branch. Finally, feature fusion is achieved through element-by-element addition, and residual connections are retained to ensure gradient flow. This design simultaneously optimizes channel and spatial dimension representations on the H×W×C feature map, where C is the number of channels, M is the attention window size, and h is the number of attention heads. The specific steps are as follows:

[0107] S3.2-1, Deep Feature Fusion Block (DFB) generates a channel attention map by passing the features of the convolution branch through global average pooling, 1×1 convolution layer and Sigmoid activation function, which is used to enhance the feature representation of the self-attention branch. Among them, global average pooling can capture global context information and avoid local bias. The global mean is calculated for each channel to obtain the channel descriptor. :

[0108] ;

[0109] in, is the c-th channel value of the input feature X at position (i, j), It is all channels;

[0110] S3.2-2. Enhance feature representation through channel attention mechanism. The core idea is to use convolution branches to extract local features, self-attention branches to model global dependencies, and channel attention maps to dynamically adjust feature weights, combining the advantages of both. 1×1 convolution is used to compress the number of channels (usually reducing the dimension to C / r, where r is the compression ratio):

[0111] ;

[0112] Where, , s is the compressed eigenvector, and ; In addition, the ReLU activation function is used in the channel attention map, and the formula is as follows:

[0113] ;

[0114] The ReLU in the formula is a nonlinear transformation, which can alleviate the gradient disappearance problem;

[0115] S3.2-3. Another information branch of the bidirectional information interaction mechanism comes from the spatial interaction module. The features of the self-attention branch are passed through a 1×1 convolution layer and a Sigmoid activation function to generate a spatial attention map, which is used to enhance the feature representation of the convolution branch. When generating spatial attention weights, the fully connected layer (FC) and the Sigmoid activation function are mainly used to generate spatial attention weights:

[0116] ;

[0117] ;

[0118] Where, is the fully connected layer weight, Belongs to the real number range and is a bias term; It is a sigmoid function, ensuring the weight value .

[0119] As a preferred embodiment, the deep feature fusion block DFB adopts a dual-branch parallel structure, including a convolution branch and a self-attention branch, and realizes feature enhancement of channel and spatial dimensions through a two-way information interaction mechanism, such as Figure 2As shown, the convolution branch generates channel attention weights (σ activation) through global average pooling (GAP) and a fully connected layer (FC), which are used to modulate the value matrix of the attention branch. The attention branch generates a spatial attention mask through 1×1 convolution and σ activation, which is used to enhance the key spatial regions of the convolution branch. Finally, the dual-branch features are fused through element-by-element addition, and residual connections are retained to ensure training stability. This design simultaneously optimizes channel and spatial dimension representations on the H×W×C feature map, where M is the attention window size and C is the number of feature channels. The specific steps are:

[0120] S3.3-1. The convolution branch adopts a dual-path depth convolution structure. The first path adopts 3×3 depth convolution to extract local detail features, and the second path adopts 5×5 depth convolution to expand the receptive field and enhance texture information. The outputs of the two paths are spliced after dimensionality reduction through 1×1 convolution to form the final convolution branch features. The feature outputs of the two paths are calculated as follows:

[0121] ;

[0122] ;

[0123] In the formula, X belongs to is the input feature, H×W is the spatial resolution, C is the number of channels, K represents the convolution kernel, and b is the optional bias term under the convolution kernel size. Then, the output feature maps of the two branches are directly spliced in the channel dimension to form the final fusion feature. The splicing operation is element merging, without parameter calculation, only increasing the number of channels, and the computational overhead is small. The formula for fusing dual-path features is as follows:

[0124] ;

[0125] Where, and are the features output by the two paths, It is a 1×1 dimensionality reduction convolution operation with a convolution kernel size of 1×1 (i.e., covering only a single pixel of the input feature map). It is used to adjust the number of channels, fuse features, or implement cross-channel information interaction. Concat fusion merges two-branch features along the channel dimension, combining local and global information in a simple and efficient way. It is one of the core operations for improving model performance.

[0126] S3.3-2. The self-attention branch uses window partitioning and a multi-head self-attention mechanism to model global dependencies. Specifically, the input features are divided into multiple non-overlapping windows, self-attention is calculated within each window, the multi-head self-attention mechanism is used to divide the features into multiple subspaces for parallel calculation, and a window offset strategy is used to enhance information interaction between different windows. In this step, the main task is to generate attention weights by calculating the correlation between features. The specific formula is as follows:

[0127] ;

[0128] Where Z is the feature representation weighted by self-attention, which contains global context information, D is the dimension of the feature, Softmax is the normalized exponential function, and Q, K, and V are the mappings of the input feature X to the query, key, and value, respectively.

[0129] As a preferred embodiment, the S4 uses the processed training set to train the network and optimize the loss function to improve the quality and details of the reconstructed image. The loss function combines pixel-level loss, cross entropy loss, and minimization of reconstruction loss. The specific steps are as follows:

[0130] S4.1. To enhance the distinguishing ability of feature representation, the model can be implemented by minimizing the loss function. The expression of the pixel-level loss function is:

[0131] ;

[0132] Where L is the high-resolution image and reconstructed image The pixel difference, It refers to the pixel value of the high-resolution image with coordinates (i, h, w);

[0133] S4.2. Based on the attention mechanism, fine-tuning the model through supervised segmentation tasks, calculating the segmentation loss function, and optimizing the network weights through backpropagation can help the model learn richer spatial structure information from the data. This task uses the cross-entropy loss function to calculate the segmentation error:

[0134] ;

[0135] In the formula, given input x and corresponding label ,here is the true label of the i-th pixel, is the probability that the i-th pixel belongs to a certain category predicted by the model;

[0136] S4.3. After data processing and feature extraction, the model needs to generate new samples by learning the reconstruction process. The training goal is to minimize the reconstruction error, using the following loss function:

[0137] ;

[0138] in, is the result of the network reconstructing the image, and the loss function is the difference between the predicted result and the real original high-resolution data The error between

[0139] As a preferred embodiment, Figure 4 , the S5 inputs the low-resolution infrared image into the trained network, fuses the shallow features with the deep features through long skip connections, uses the PixelShuffle operation for upsampling, and finally uses a 3×3 convolutional layer to restore the target resolution and output a high-resolution reconstruction result. The specific steps are:

[0140] S5.1. Long skip connections can directly transmit shallow features and enhance gradient backpropagation. The main task in this step is channel dimension splicing. The formula is as follows:

[0141] ;

[0142] Where, is the l-th layer feature map, is the k-th layer feature map, and concat is the fusion method;

[0143] S5.2. Next, we use the PixelShuffle operation for upsampling. The core idea is to expand the spatial dimensions (H, W) of the low-resolution feature map while reducing the number of channels (C) to achieve efficient upsampling. The calculation formula is as follows:

[0144] ;

[0145] Where r is the upper adoption factor, 、 、 where is the number of channels, height, and width after reorganization; Y and X are the output and input features, respectively. Compared to traditional interpolation (such as bilinear interpolation), it better preserves high-frequency details. Therefore, the image reconstruction module uses pixel shuffle and a 3×3 convolutional layer to restore the feature map to a high-resolution image.

[0146] This infrared image super-resolution interactive convolution and dynamic focusing feature integration method is based on multi-directional information feature interactive fusion technology, adaptive attention dynamic focusing technology, and multi-task collaborative loss optimization strategy. It can effectively extract and enhance key information in low-resolution infrared images, thereby achieving a significant improvement in infrared image resolution and detail quality. It can provide core algorithm support and technical guarantee for target recognition, precise monitoring and high-definition imaging in complex infrared environments.

Claims

1. A feature integration method for interactive convolution and dynamic focusing of infrared images, characterized in that: The steps include: S1. Obtain infrared image pairs of different resolutions in various real scenes through infrared cameras; S2. Perform degradation preprocessing on a portion of the original high-resolution images to obtain low-quality high-resolution images to form a mixed low-resolution dataset, and divide the processed dataset into a training set and a test set; S3. Build a two-layer feature extraction module for feature modeling, including shallow feature extraction and deep feature extraction. First, a 3×3 convolution kernel is used to extract shallow features to capture local details and high-frequency information in the image. Then, a deep extraction module is used to focus on global context and semantic information to understand the overall structure and semantic relationship of the image. Specifically: S3.1-1. The shallow feature extraction module uses a 3×3 convolutional layer to perform preliminary feature extraction on the input low-resolution infrared image and output a shallow feature map. The input low-resolution image is set to: ； Where, is the spatial resolution of the image, i.e. height × width, is the number of input channels. The infrared image has a single channel of 1. When the convolution kernel is set to In the case of , each pixel value of the output feature map F is calculated by the following formula: ； Where, is the dimension space of the convolution kernel, is the pixel coordinate of the output feature map, c is the output channel index, , is the number of output channels; is the bias term, initialized to 0; It is the boundary processing after filling; S3.1-2, input size is , the convolution kernel size is k k, step size s=1, no padding (p=0), then the output size is as follows: ； ； Where, and Output length and width without padding; The deep feature extraction module is composed of multiple feature fusion blocks stacked together. The module uses a two-way information interaction mechanism to enhance feature expression capabilities, including a source information branch based on deep convolution and an information branch based on the attention mechanism. Specifically: S3.2-1. The deep feature fusion block generates a channel attention map by passing the features of the convolution branch through global average pooling, 1×1 convolution layer and Sigmoid activation function to enhance the feature representation of the self-attention branch. Among them, global average pooling captures global context information and avoids local bias. The global mean is calculated for each channel to obtain the channel descriptor. : ； in, is the c-th channel value of the input feature X at position (i, j), It is all channels; S3.2-2. Enhance feature representation through channel attention mechanism, that is, use convolution branches to extract local features, self-attention branches to model global dependencies, and channel attention maps to dynamically adjust feature weights, combining the advantages of both. 1×1 convolution is used to compress the number of channels and reduce the dimensionality to C / r, where r is the compression ratio: ； Where, , s is the compressed eigenvector, and ; In addition, the activation function is used in the channel attention map, and the formula is as follows: ； The ReLU in the formula is a nonlinear transformation; S3.2-3. Another information branch of the two-way information interaction mechanism comes from the spatial interaction module. The features of the self-attention branch are passed through a 1×1 convolution layer and a Sigmoid activation function to generate a spatial attention map, which is used to enhance the feature representation of the convolution branch. When generating the spatial attention weight, a fully connected layer and a Sigmoid activation function are used to generate the spatial attention weight: ； ； Where, is the fully connected layer weight, Belongs to the real number range and is a bias term; It is a sigmoid function, ensuring the weight value ; The deep feature fusion block adopts a dual-branch parallel structure, including a convolution branch and a self-attention branch, and realizes feature enhancement in channel and spatial dimensions through a two-way information interaction mechanism. The specific steps are as follows: S3.3-1. The convolution branch adopts a dual-path depth convolution structure. The first path adopts 3×3 depth convolution, and the second path adopts 5×5 depth convolution. The outputs of the two paths are concatenated after dimensionality reduction through 1×1 convolution to form the final convolution branch features. The feature outputs of the two paths are calculated as follows: ； ； In the formula, X belongs to is the input feature, H×W is the spatial resolution, C is the number of channels, K represents the convolution kernel, and b is the optional bias term under the convolution kernel size; the output feature maps of the two branches are directly spliced in the channel dimension to form the final fusion feature. The formula for fusing the dual-path features is as follows: ； Where, and are the features output by the two paths, It is a 1×1 dimensionality reduction convolution operation, and its convolution kernel size is 1×1; S3.3-2. The self-attention branch uses window partitioning and multi-head self-attention mechanism to model global dependencies. That is, the input features are divided into multiple non-overlapping windows, self-attention is calculated in each window, the multi-head self-attention mechanism is used to divide the features into multiple subspaces for parallel calculation, and the window offset strategy is used to enhance the information interaction between different windows. The attention weight is generated by calculating the correlation between features. The specific formula is as follows: ； Where Z is the feature representation after self-attention weighting, which includes global context information, D is the dimension of the feature, Softmax is the normalized exponential function, Q, K and V are the mappings of the input feature X to the query, key and value respectively; S4. Use the processed training set to train the network and optimize the loss function; S5. Input the low-resolution infrared image into the trained network and output the high-resolution reconstruction result.

2. The feature integration method for interactive convolution and dynamic focusing of infrared images according to claim 1, characterized in that: Step S1 uses an infrared camera to obtain infrared image data of different scenes by adjusting optical zoom and focal length, specifically: S1.

1. Optical zoom uses long-wave and medium-wave infrared cameras that support motorized zoom. By adjusting the lens focal length, the camera switches from wide angle to telephoto, directly generating raw image pairs with different spatial resolutions. S1.

2. To obtain infrared image pairs with different resolutions, the long-wave camera outputs a resolution of 640 × 480 at a focal length of 50 mm and a resolution of 1280 × 1024 at a focal length of 200 mm. The medium-wave camera generates images with different resolutions by replacing the lens with a focal length of 30 mm or 90 mm.

3. The feature integration method for interactive convolution and dynamic focusing of infrared images according to claim 1, characterized in that: Step S2 is specifically as follows: S2.1-1. The blur effect of the optical system is simulated by Gaussian function, and resolution degradation is achieved by combining downsampling operation. A part of the original high-resolution image is artificially degraded. The Gaussian blur is achieved by convolving the image with a two-dimensional Gaussian kernel. Defined as: ； Where, is the standard deviation of the Gaussian kernel, which controls the degree of blur; S2.1-2. Use Gaussian blur to simulate optical system aberrations, and then generate a low-resolution image by downsampling: ； Where, is the Gaussian kernel G for high-resolution images Convolution, Downsample is a downsampling operation, where adjacent pixels are averaged; S2.

2. Divide the mixed low-resolution dataset into training and test sets for subsequent network training and validation. Divide the data into training, validation, and test sets proportionally to ensure that each subset covers different scenes, resolutions, and noise levels to verify generalization ability.

4. The feature integration method for interactive convolution and dynamic focusing of infrared images according to claim 1, characterized in that: In step S4, the processed training set is used to train the network. The specific steps for optimizing the loss function are as follows: S4.

1. To enhance the distinguishing ability of feature representation, the model is implemented by minimizing the loss function. The expression of the pixel-level loss function is: ； Where L is the high-resolution image and reconstructed image The pixel difference, It refers to the pixel value of the high-resolution image with coordinates i, h, w; S4.

2. Based on the attention mechanism, fine-tune the model through supervised segmentation tasks, calculate the segmentation loss function, optimize the network weights through backpropagation, and use the cross-entropy loss function to calculate the segmentation error: ； In the formula, given input x and corresponding label ,here is the true label of the i-th pixel, is the probability that the i-th pixel belongs to a certain category predicted by the model; S4.

3. After data processing and feature extraction, the model needs to generate new samples by learning the reconstruction process. The training goal is to minimize the reconstruction error, using the following loss function: ； in, is the result of the network reconstructing the image, and the loss function is the difference between the predicted result and the real original high-resolution data The error between .

5. The feature integration method for interactive convolution and dynamic focusing of infrared images according to claim 1, characterized in that: Step S5 is specifically as follows: S5.

1. Long skip connections directly transfer shallow features and enhance gradient backpropagation. The task in this step is channel dimension splicing. The formula is as follows: ； Where, is the l-th layer feature map, is the k-th layer feature map, Concat is the fusion method; S5.

2. Use pixel reorganization to perform upsampling, expand the spatial dimensions H, W of the low-resolution feature map, and reduce the number of channels C, so as to achieve efficient upsampling. The calculation formula is as follows: ； Where r is the upper adoption factor, 、 、 is the number of channels, height, and width after reorganization; Y and X are the output and input features, respectively; finally, the image reconstruction module restores the feature map to a high-resolution image after pixel reorganization and a 3×3 convolutional layer.

Images