An infrared image super-resolution reconstruction method, system, medium and application program

Abstract

The application discloses an infrared image super-resolution reconstruction method, system, medium and application program, and the method comprises the following steps: inputting a low-resolution infrared image into a shallow feature fusion network to extract shallow embedding features; extracting prior shared features through a shared feature extraction main line from the shallow embedding features; predicting three degradation prior maps, namely a degradation intensity map, a thermal boundary map and an uncertainty map; performing degradation prior guided shallow modulation on the shallow embedding features according to the degradation prior maps to obtain shallow features; sequentially passing the shallow features through deep feature modeling and thermal ordering reconstruction to generate backbone reconstruction features; generating a difficult area mask of a difficult recovery area according to the degradation prior maps, and performing local refinement enhancement on the difficult recovery area in the backbone reconstruction features to obtain a final high-resolution infrared image. The application aims to fully utilize the thermal structure characteristics of the infrared image, and realize high-fidelity detail recovery and stable super-resolution reconstruction of the infrared image under complex degradation conditions.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, specifically to an infrared image super-resolution reconstruction method, system, medium, and application. Background Technology

[0002] Infrared imaging achieves target imaging in a scene by detecting the thermal radiation of objects in the infrared band. It does not rely on external visible light illumination and has advantages such as all-weather operation (day and night) and strong adaptability to low-light and complex lighting environments. Compared with visible light images, infrared images can better reflect the differences in thermal distribution between targets and backgrounds, making them valuable for applications such as night surveillance, target search, security early warning, unmanned platform sensing, industrial inspection, and emergency rescue. However, limited by factors such as infrared detector pixel size, cooled or uncooled processes, imaging optical system aperture, and imaging distance, existing infrared imaging equipment generally suffers from low spatial resolution, blurred edge details, insufficient local contrast, and significant noise interference. Especially under conditions of long distance, small targets, and complex backgrounds, the acquired infrared images often fail to directly meet the application requirements of fine identification, measurement analysis, and high-reliability sensing. To address the problem of improving the clarity of low-resolution infrared images, existing technologies typically employ image super-resolution methods to enhance the spatial resolution of the image. Most existing super-resolution methods originate from the field of visible light image processing, mainly using convolutional neural networks, attention mechanism networks, Transformer networks, or generative models to reconstruct high-resolution images from low-resolution images. These methods have achieved good results on visible light natural images, but they typically assume that the input image has rich texture information and a stable degradation model. Infrared images, however, differ significantly from visible light images in their imaging mechanisms, radiation characteristics, and degradation forms. Infrared images generally have weaker textures, with target boundaries appearing more as thermal difference boundaries than distinct texture edges. They are also susceptible to various factors such as thermal diffusion, non-uniform response, quantization errors, sensor noise, and blurring degradation. This leads to problems such as insufficient detail recovery, pseudo-high-frequency texture generation, and distortion of thermal radiation relationships when directly applying visible light super-resolution methods. Furthermore, existing infrared image super-resolution methods mostly focus on improving pixel-domain reconstruction accuracy, typically using pixel loss, perceptual loss, or adversarial loss to constrain the reconstruction results, with less emphasis on specifically modeling the physically significant thermal structures and relative thermal intensity relationships in infrared images. For infrared images, the relative hot and cold order of targets and background within local regions, the location of thermal boundaries, and the temperature difference transition relationships are crucial. This information not only affects the visual quality of the image but also directly relates to the reliability of subsequent target detection, recognition, and analysis tasks. Existing methods that focus solely on texture enhancement for image restoration are prone to introducing false details in local areas that are inconsistent with the true thermal distribution, or to producing over-smoothing and contour drift in boundary regions, making it difficult to balance image sharpness and thermal radiation consistency. Furthermore, infrared image degradation typically exhibits significant spatial non-uniformity, with different regions potentially experiencing varying degrees of blurring, noise, contrast attenuation, and thermal diffusion effects simultaneously. Traditional super-resolution frameworks based on a uniform degradation assumption often struggle to accurately characterize degradation distributions under complex infrared imaging conditions and lack mechanisms for differentiated processing of difficult-to-restore and easily-restored regions.In this situation, applying a uniform enhancement strategy to the entire image can easily lead to over-correction of easily recoverable areas while failing to effectively compensate for difficult-to-recover areas, thus affecting the overall reconstruction quality and method stability. In summary, existing infrared images suffer from low spatial resolution, blurred edge details, and insufficient thermal structure representation. Furthermore, most current super-resolution methods focus only on direct mapping from low to high resolution, directly employing a uniform reconstruction strategy, making it difficult to differentiate restoration based on the degree of degradation and thermal boundary characteristics of different regions in the infrared image. Summary of the Invention

[0003] The technical problem to be solved by this invention is to provide an infrared image super-resolution reconstruction method, system, medium, and application program that addresses the above-mentioned problems in the prior art. This invention aims to fully utilize the thermal structural characteristics of infrared images to achieve high-fidelity detail restoration and stable super-resolution reconstruction of infrared images under complex degradation conditions.

[0004] To solve the above-mentioned technical problems, the technical solution adopted by the present invention is as follows: A method for super-resolution reconstruction of infrared images, comprising converting an input low-resolution infrared image into a super-resolution infrared image. Generate high-resolution infrared images using a pre-trained infrared image super-resolution reconstruction model Includes: S101, which displays low-resolution infrared images Input shallow feature fusion network to extract shallow embedding features S102, embedding shallow features Prior shared features are extracted from the trunk path through shared feature extraction. Based on prior shared features The degradation intensity map was predicted. Thermal boundary map and uncertainty diagram Three degenerate prior maps, and shallow embedding features based on the degenerate prior maps. Performing degenerate prior-guided shallow modulation to obtain shallow features S103, shallow features The backbone reconstruction features are generated sequentially through deep feature modeling and hot sorting reconstruction. According to the degradation intensity map Thermal boundary map and uncertainty diagram Difficult-to-recover region mask S104, based on the difficult-to-recover region mask. Reconstruction features of the main trunk The difficult-to-recover regions in the image are locally refined, enhanced, and reconstructed to obtain the final high-resolution infrared image. .

[0005] Optionally, step S101 includes: S201, low-resolution infrared image Normalization yields normalized features ; S202, normalized features Feature extraction is performed using a 3×3 convolutional layer to obtain small-scale features. Normalized features Large-scale features are extracted using a 7×7 depthwise convolutional layer and channel mapping using a 1×1 convolutional layer. ; S203, small-scale features Perform two-dimensional discrete wavelet decomposition to obtain even-numbered row features and odd row features Even row features and odd row features The column directions were divided into even-numbered and odd-numbered columns, resulting in four sub-sampling features. ~ Four sub-sampling features ~ After dimensional alignment, construct low-frequency approximate components. Horizontal detail components Vertical detail components and diagonal detail components : ; ; ; ; Using bicubic interpolation to approximate low-frequency components Horizontal detail components Vertical detail components and diagonal detail components Restored to small-scale features For the same spatial dimensions, low-frequency approximate wavelet components were obtained. Horizontal detail wavelet components Vertical detail wavelet components and diagonal detail wavelet components ; Approximate low-frequency wavelet components Horizontal detail wavelet components Vertical detail wavelet components and diagonal detail wavelet components Wavelet detail features are obtained by splicing. ; S204, based on wavelet detail features Generate modulation parameters Using modulation parameters For large-scale features Modulation is performed to generate large-scale modulation features. ; S205 will modulate large-scale features Small-scale features The features are stitched together along the channel dimension and then fused using a 1×1 convolution to obtain shallow fused features. ; S206, shallow fusion features This is achieved by mapping it to a unified embedding feature space through an embedding feature mapping layer to obtain shallow embedding features. : ; in, and For the convolution kernel parameters and bias parameters of the embedded feature mapping layer, This is a convolution operation.

[0006] Optionally, step S102 includes: S301, Shallow Embedded Features Prior shared features are extracted from the trunk path through shared feature extraction. The shared feature extraction trunk includes two convolutional layers, and the shared feature extraction trunk extracts prior shared features. The function expression is: ; ; in, and These are the kernel parameters and bias parameters of the first convolutional layer, respectively. and These are the kernel parameters and bias parameters of the second convolutional layer, respectively. This is a convolution operation; This is a convolution operation grouped by channel; It is a non-linear activation function. These are the intermediate prior features output by the first convolutional layer; S302 will share prior features Inputting three parallel 1×1 convolutional branches respectively yields the degradation intensity map. Thermal boundary map and uncertainty diagram Three degenerate a priori diagrams: ; ; ; in, , and These represent the 1×1 convolution kernel parameters of the three parallel 1×1 convolution branches; , and These are the bias parameters for three parallel 1×1 convolutional branches; Use the Sigmoid activation function; S303, degradation intensity map Thermal boundary map and uncertainty diagram Three degenerate prior images are stitched together along the channel dimension to obtain the prior stack tensor. ; S304, prior stacking tensors The modulation parameter tensor is obtained by generating prior modulation parameters through a prior modulation parameter generation unit consisting of a 1×1 convolutional layer and a nonlinear activation layer. S305, the modulation parameter tensor is divided into three tensors along the channel dimension to obtain the scale modulation term. Bias modulation term and gating items ; S306, Scale modulation term and gating items By applying nonlinear transformations, the final scale modulation term is obtained. and final gating item According to the bias modulation term The final scale modulation term and final gating item Performing degenerate prior-guided shallow modulation to obtain shallow features : ; in, For element-wise multiplication, To be compatible with the final scale modulation term A single sheet of the same size.

[0007] Optionally, in step S103, shallow features The backbone reconstruction features are generated sequentially through deep feature modeling and hot sorting reconstruction. include: S401, shallow features Expanded into one-dimensional sequence features For one-dimensional sequence features The input sequence features are obtained by performing random deactivation processing. ; S402, input sequence features Input by The deep reconstruction backbone, composed of a series of residual state space groups, generates deep sequence features. In the deep reconstruction backbone, any first... The functional expression for the group of residual state-space units is: ; in, For the first Deep sequence features generated from a set of residual state spaces For the first A set of residual state spaces, For the first The input sequence features of a group of residual state spaces The image height is the two-dimensional feature map. The image width of the two-dimensional feature map. For two-dimensional feature serialization operators, For group-level convolutional mapping units, An operator for recovering sequence features into a two-dimensional feature map. Indicates will Restore to size Two-dimensional feature map; S403, deep sequence features Restore to two-dimensional feature map : ; ; in, These are the normalized deep sequence features. For normalization operators; S404, two-dimensional feature map and shallow features Residual fusion is performed to obtain backbone reconstruction features. : ; in, A convolutional mapping unit, used for processing two-dimensional feature maps. Apply a convolutional mapping to the two-dimensional feature map. Transform to suit shallow features Construct the feature space for residual fusion.

[0008] Optionally, in step S103, based on the degradation intensity map Thermal boundary map and uncertainty diagram Difficult-to-recover region mask This refers to generating data using a mask unit comprising two convolutional layers, based on the degradation intensity map. Thermal boundary map and uncertainty diagram The prior stack tensor obtained by stitching together three degenerate prior maps along the channel dimension Generating units generate masks for difficult regions : ; ; in, The output features of the first convolutional layer of the mask generation unit. and These are the convolution kernel parameters for the first and second convolutional layers of the mask generation unit, respectively. and These are the bias parameters for the first and second convolutional layers of the mask generation unit, respectively. It is a non-linear activation function. It is the Sigmoid activation function. This is a convolution operation.

[0009] Optionally, step S104 includes: S501, reconstructing the main trunk features Candidate refined residuals are generated using a residual refinement unit consisting of two convolutional layers. : ; ; in, The output features of the first convolutional layer of the residual refinement unit. and These are the convolution kernel parameters for the first and second convolutional layers of the residual refinement unit, respectively. and These are the bias parameters for the first and second convolutional layers of the residual refinement unit, respectively. It is a non-linear activation function; S502, utilizing a difficult-to-recover region mask. Refine the candidate residuals Position-by-position weighting is performed, and the residuals are superimposed onto the backbone reconstruction features. The selectively refined features are obtained above : ; in, The preset refinement factor, This is element-wise multiplication; S503, selectively refined features The input reconstruction head reconstructs the final high-resolution infrared image. .

[0010] Optionally, the loss function used during training of the infrared image super-resolution reconstruction model includes pixel reconstruction loss. Local thermal sorting loss Consistency loss due to re-degradation Weighted loss in difficult areas and mask coverage loss The expression for the calculation function of the pixel reconstruction loss, in part or all of the above, is as follows: ; in, For the final high-resolution infrared image, For reference, high-resolution infrared images, It is an L1 norm; The expression for the calculation function of the local hot sorting loss is: ; in, For local hot sorting loss weights, For an effective mask generated from a reference thermal difference threshold, For element-wise multiplication, It is a soft positive function. for The single-channel heatmap obtained by mapping The thermal difference between the center of the local window and its neighboring locations This is the temperature scaling factor. For symbolic functions, for The single-channel heatmap obtained by mapping The thermal difference between the center of the local window and its neighboring locations To prevent positive numbers with a denominator of zero; The expression for the calculation function of the re-degradation consistency loss is: ; in, To further degrade the consistency loss weight, for Low-resolution infrared images obtained by degenerate mapping transformation The input is a low-resolution infrared image; the degradation mapping transformation includes converting the input low-resolution infrared image into a final high-resolution infrared image. Low-resolution infrared images are obtained through convolutional feature extraction, bicubic interpolation downsampling, and output convolution. The calculation function expression for the weighted loss of the difficult region is as follows: ; in, Weighted loss for the difficult area pixel position The mean absolute error at that point For high-resolution masks In coordinates The mask value at the location, the high-resolution mask For difficult area masking Interpolated to the final high-resolution infrared image The same spatial dimensions are obtained; The expression for calculating the mask coverage loss is as follows: ; in, Weighting is applied to mask coverage loss. To obtain the maximum value, Masking for difficult areas The average value of all elements. This represents the upper bound of the average mask coverage.

[0011] The present invention also provides an infrared image super-resolution reconstruction system, including a microprocessor and a memory interconnected thereto, the microprocessor being programmed or configured to execute the infrared image super-resolution reconstruction method.

[0012] The present invention also provides a computer-readable storage medium storing a computer program or instructions that are programmed or configured to execute the infrared image super-resolution reconstruction method by a processor.

[0013] The present invention also provides a computer application, including a computer program or instructions, which are programmed or configured to execute the infrared image super-resolution reconstruction method via a processor.

[0014] Compared with the prior art, the present invention can mainly achieve the following beneficial effects: (1) The present invention introduces feature fusion and degradation prior prediction mechanism in the shallow stage, extracts multi-scale features and wavelet detail information from the input infrared image at the same time, and further generates degradation intensity map, thermal boundary map and uncertainty map. Then, the prior is injected into the shallow features through gated modulation, so that the subsequent backbone network can obtain the input representation with degradation perception capability, and improve the reconstruction stability under complex degradation conditions. (2) The present invention adopts a deep reconstruction backbone based on residual state space group, performs long-range dependency modeling on feature sequence, and completes infrared structure restoration by combining local enhancement; at the same time, local thermal sorting constraint and re-degradation consistency constraint are introduced in the training stage, so that the output result can better maintain the local relative thermal intensity relationship and input consistency in the infrared image while improving spatial resolution. (3) This invention uses degradation intensity map, thermal boundary map and uncertainty map to generate difficult area mask, selectively refines only the difficult recovery area in the features of the trunk end, and strengthens the recovery ability of local key areas and suppresses global over-correction by weighted reconstruction loss of difficult area and mask coverage constraint, thereby taking into account the overall reconstruction stability and local detail enhancement ability. Attached Figure Description

[0015] Figure 1 This is a schematic diagram of the basic process of the method in an embodiment of the present invention.

[0016] Figure 2 This is a schematic diagram of the network structure of the shallow feature fusion network in an embodiment of the present invention.

[0017] Figure 3 This is a schematic diagram comparing the infrared super-resolution results of the method of this embodiment and the existing method, wherein (a) is a high-resolution red true value image, (b) is the result obtained by the FSRCNN method, (c) is the result obtained by the SRCNN method, (d) is the result obtained by the IRSRMamba method, (e) is the result obtained by the RGT method, (f) is the result obtained by the PSRGAN method, and (g) is the result obtained by the method of this embodiment. Detailed Implementation

[0018] To enable those skilled in the art to better understand the technical solutions of the present invention, the technical solutions of the present invention will be further described in detail below with reference to the accompanying drawings in the embodiments of the present invention.

[0019] like Figure 1 As shown, the infrared image super-resolution reconstruction method in this embodiment includes converting the input low-resolution infrared image into a super-resolution image. Generate high-resolution infrared images using a pre-trained infrared image super-resolution reconstruction model Includes: S101, which displays low-resolution infrared images Input shallow feature fusion network to extract shallow embedding features S102, embedding shallow features Prior shared features are extracted from the trunk path through shared feature extraction. Based on prior shared features The degradation intensity map was predicted. Thermal boundary map and uncertainty diagram Three degenerate prior maps, and shallow embedding features based on the degenerate prior maps. Performing degenerate prior-guided shallow modulation to obtain shallow features S103, shallow features The backbone reconstruction features are generated sequentially through deep feature modeling and hot sorting reconstruction. According to the degradation intensity map Thermal boundary map and uncertainty diagram Difficult-to-recover region mask S104, based on the difficult-to-recover region mask. Reconstruction features of the main trunk The difficult-to-recover regions in the image are locally refined, enhanced, and reconstructed to obtain the final high-resolution infrared image. .

[0020] Step S101 takes a low-resolution infrared image as input, performs shallow feature fusion, predicts a degradation intensity map, a thermal boundary map, and an uncertainty map, and uses the prior map to perform gated modulation on the shallow features, thereby obtaining an input representation with degradation perception capability. Figure 2 As shown, step S101 in this embodiment includes: S201, low-resolution infrared image Normalization yields normalized features ; In this embodiment, low-resolution infrared images Normalization yields normalized features The function expression is: ; in, For normalized features, For low-resolution infrared images, When the input is a single-channel infrared image, the mean tensor is taken as the input mean tensor. , The image dynamic range scaling factor is used; the above normalization operation obtains the normalized input features by performing mean normalization and dynamic range scaling on the low-resolution infrared image. ; S202, normalized features Feature extraction is performed using a 3×3 convolutional layer to obtain small-scale features. Normalized features Large-scale features are extracted using a 7×7 depthwise convolutional layer and channel mapping using a 1×1 convolutional layer. ; can be represented as: ; ; ; in, and These are the kernel parameters and bias parameters of a 3×3 convolutional layer, respectively. and These are the kernel parameters and bias parameters for a 7×7 convolutional layer. The output features of a 7×7 convolutional layer and These are the kernel parameters and bias parameters for a 1×1 convolutional layer, respectively. This is a convolution operation; This is a convolution operation grouped by channel; S203, small-scale features Perform two-dimensional discrete wavelet decomposition to obtain even-numbered row features and odd row features , can be represented as: ; ; in, Indicates small-scale features Tensor slicing starts from index 0 and uses even-numbered indices with a step size of 2. Indicates small-scale features Starting from index 1, the tensor slices are taken at odd indices with a step size of 2, and the size is adjusted by dividing by 2. Even row features and odd row features The column directions were divided into even-numbered and odd-numbered columns, resulting in four sub-sampling features. ~ , can be represented as: ; ; ; ; Four sub-sampling features ~ After dimensional alignment, construct low-frequency approximate components. Horizontal detail components Vertical detail components and diagonal detail components : ; ; ; ; Using bicubic interpolation to approximate low-frequency components Horizontal detail components Vertical detail components and diagonal detail components Restored to small-scale features For the same spatial dimensions, low-frequency approximate wavelet components were obtained. Horizontal detail wavelet components Vertical detail wavelet components and diagonal detail wavelet components ; Approximate low-frequency wavelet components Horizontal detail wavelet components Vertical detail wavelet components and diagonal detail wavelet components Wavelet detail features are obtained by splicing. , can be represented as: ; in, This is a splicing operation along the channel dimension; S204, based on wavelet detail features Generate modulation parameters Using modulation parameters For large-scale features Modulation is performed to generate large-scale modulation features. ; In this embodiment, based on wavelet detail features Generate modulation parameters This refers to the use of wavelet detail features Modulation parameters are generated using two 1×1 convolutional layers and a SiLU activation function. : ; ; in, The output features are obtained after the first 1×1 convolutional layer passes through the SiLU activation function. The SiLU activation function is used. and The kernel parameters and bias parameters for the first 1×1 convolutional layer. and The kernel parameters and bias parameters for the second 1×1 convolutional layer. This is a convolution operation; in this embodiment, modulation parameters are used. For large-scale features Modulation is performed to generate large-scale modulation features. The function expression is: ; in, This is element-wise multiplication; S205 will modulate large-scale features Small-scale features The features are stitched together along the channel dimension and then fused using a 1×1 convolution to obtain shallow fused features. ; S206, shallow fusion features This is achieved by mapping it to a unified embedding feature space through an embedding feature mapping layer to obtain shallow embedding features. : ; in, and For the convolution kernel parameters and bias parameters of the embedded feature mapping layer, This is a convolution operation.

[0021] In this embodiment, step S102 includes: S301, Shallow Embedded Features Prior shared features are extracted from the trunk path through shared feature extraction. The shared feature extraction trunk includes two convolutional layers, and the shared feature extraction trunk extracts prior shared features. The function expression is: ; ; in, and These are the kernel parameters and bias parameters of the first convolutional layer, respectively. and These are the kernel parameters and bias parameters of the second convolutional layer, respectively. This is a convolution operation; This is a convolution operation grouped by channel; It is a nonlinear activation function (Gaussian error linear unit activation function can be used). These are the intermediate prior features output by the first convolutional layer; S302 will share prior features Inputting three parallel 1×1 convolutional branches respectively yields the degradation intensity map. Thermal boundary map and uncertainty diagram Three degenerate a priori diagrams: ; ; ; in, , and These represent the 1×1 convolution kernel parameters of the three parallel 1×1 convolution branches; , and These are the bias parameters for three parallel 1×1 convolutional branches; The activation function is Sigmoid; the three degenerate prior maps constitute the prior mapping set P: ; in, Represents the degradation intensity map. Larger indicates position The higher the degree of ambiguity, thermal diffusion, or complex degradation at a given location; Represents a thermal boundary map. Larger indicates position It is more likely to belong to the target outline, background thermal boundary, or local temperature difference transition area; Representing uncertainty graphs, The larger the value, the greater the difficulty and instability of restoration during subsequent reconstruction. S303, degradation intensity map Thermal boundary map and uncertainty diagram Three degenerate prior images are stitched together along the channel dimension to obtain the prior stack tensor. , can be represented as: ; S304, prior stacking tensors The modulation parameter tensor is obtained by generating prior modulation parameters through a prior modulation parameter generation unit consisting of a 1×1 convolutional layer and a nonlinear activation layer. S305, the modulation parameter tensor is divided into three tensors along the channel dimension to obtain the scale modulation term. Bias modulation term and gating items ; S306, Scale modulation term and gating items By applying nonlinear transformations, the final scale modulation term is obtained. and final gating item According to the bias modulation term The final scale modulation term and final gating item Performing degenerate prior-guided shallow modulation to obtain shallow features : ; in, For element-wise multiplication, To be compatible with the final scale modulation term A single sheet of the same size.

[0022] To address the issues of weak detail, blurred boundaries, and complex degradation coexisting in infrared images, step S103 employs prior-guided shallow features (i.e., prior shared features). Deep feature modeling and hot-ranking reconstruction are performed, and local hot-ranking constraints and re-degradation consistency constraints are combined during training to improve the main structure recovery ability and suppress the generation of false details, ultimately outputting a high-resolution image. In this embodiment, step S103 involves shallow features... The backbone reconstruction features are generated sequentially through deep feature modeling and hot sorting reconstruction. include: S401, shallow features Expanded into one-dimensional sequence features For one-dimensional sequence features The input sequence features are obtained by performing random deactivation processing. ; S402, input sequence features Input by The deep reconstruction backbone, composed of a series of residual state space groups, generates deep sequence features. In the deep reconstruction backbone, any first... The functional expression for the group of residual state-space units is: ; in, For the first Deep sequence features generated from a set of residual state spaces For the first A set of residual state spaces, For the first The input sequence features of a group of residual state spaces The image height is the two-dimensional feature map. The image width of the two-dimensional feature map. For two-dimensional feature serialization operators, For group-level convolutional mapping units, An operator for recovering sequence features into a two-dimensional feature map. Indicates will Restore to size Two-dimensional feature map; S403, deep sequence features Restore to two-dimensional feature map : ; ; in, These are the normalized deep sequence features. For normalization operators; S404, two-dimensional feature map and shallow features Residual fusion is performed to obtain backbone reconstruction features. : ; in, A convolutional mapping unit, used for processing two-dimensional feature maps. Apply a convolutional mapping to the two-dimensional feature map. Transform to suit shallow features Construct the feature space for residual fusion.

[0023] In this embodiment, step S103 involves using the degradation intensity map. Thermal boundary map and uncertainty diagram Difficult-to-recover region mask This refers to generating data using a mask unit comprising two convolutional layers, based on the degradation intensity map. Thermal boundary map and uncertainty diagram The prior stack tensor obtained by stitching together three degenerate prior maps along the channel dimension Generating units generate masks for difficult regions : ; ; in, The output features of the first convolutional layer of the mask generation unit. and These are the convolution kernel parameters for the first and second convolutional layers of the mask generation unit, respectively. and These are the bias parameters for the first and second convolutional layers of the mask generation unit, respectively. It is a non-linear activation function. It is the Sigmoid activation function. This is a convolution operation.

[0024] In step S104, the difficult-to-recover region mask is used. Reconstruction features of the main trunk The difficult-to-recover regions in the image are locally refined, enhanced, and reconstructed to obtain the final high-resolution infrared image. Local enhancement is performed only on areas with severe degradation, complex thermal boundaries, or high uncertainty in restoration, thereby improving the detail recovery capability of key local areas and avoiding global over-correction of the entire image. In this embodiment, step S104 includes: S501, reconstructing the main trunk features Candidate refined residuals are generated using a residual refinement unit consisting of two convolutional layers. : ; ; in, The output features of the first convolutional layer of the residual refinement unit. and These are the convolution kernel parameters for the first and second convolutional layers of the residual refinement unit, respectively. and These are the bias parameters for the first and second convolutional layers of the residual refinement unit, respectively. It is a non-linear activation function; S502, utilizing a difficult-to-recover region mask. Refine the candidate residuals Position-by-position weighting is performed, and the residuals are superimposed onto the backbone reconstruction features. The selectively refined features are obtained above : ; in, The preset refinement factor, This is element-wise multiplication; S503, selectively refined features The input reconstruction head reconstructs the final high-resolution infrared image. .

[0025] In this embodiment, the loss function used during training of the infrared image super-resolution reconstruction model includes pixel reconstruction loss. Local thermal sorting loss Consistency loss due to re-degradation Weighted loss in difficult areas and mask coverage loss The expression for the calculation function of the pixel reconstruction loss, in part or all of the above, is as follows: ; in, For the final high-resolution infrared image, For reference, high-resolution infrared images, It is an L1 norm.

[0026] In this embodiment, the calculation function expression for the local hot sorting loss is: ; in, For local hot sorting loss weights, For an effective mask generated from a reference thermal difference threshold, For element-wise multiplication, It is a soft positive function. for The single-channel heatmap obtained by mapping The thermal difference between the center of the local window and its neighboring locations This is the temperature scaling factor. For symbolic functions, for The single-channel heatmap obtained by mapping The thermal difference between the center of the local window and its neighboring locations To prevent positive numbers with zero denominators, when generating an effective mask from the reference thermal difference threshold, each pixel in the generated effective mask has a value of 1 or 0. This is used to determine whether the thermal difference between the center pixel and its neighboring pixels within a local window of the reference thermal map is sufficiently significant. Only when the absolute value of the reference thermal difference is greater than the threshold will the pixel in the generated effective mask have a value of 1, and the corresponding position will participate in the thermal sorting loss calculation; otherwise, it is considered that the thermal difference at that position is too small, and the pixel in the generated effective mask has a value of 0, and it will not participate in the sorting constraint, in order to avoid introducing unstable supervision due to extremely small thermal differences.

[0027] In this embodiment, the calculation function expression for the re-degradation consistency loss is: ; in, To further degrade the consistency loss weight, for Low-resolution infrared images obtained by degenerate mapping transformation The input is a low-resolution infrared image; the degradation mapping transformation includes converting the input low-resolution infrared image into a final high-resolution infrared image. Low-resolution infrared images are obtained through convolutional feature extraction, bicubic interpolation downsampling, and output convolution.

[0028] In this embodiment, the calculation function expression for the hard-region weighted loss is: ; in, Weighted loss for the difficult area pixel position The mean absolute error at that point For high-resolution masks In coordinates The mask value at the specified location is either 1 or 0; the high-resolution mask... For difficult area masking Interpolated to the final high-resolution infrared image The same spatial dimensions can be represented as: ; in, For size interpolation operators, and For the final high-resolution infrared image Height and width; weighted loss in difficult areas Used to enhance the reconstruction optimization of difficult areas.

[0029] In this embodiment, the calculation function expression for the mask coverage loss is: ; in, Weighting is applied to mask coverage loss. To obtain the maximum value, Masking for difficult areas The average value of all elements. The upper limit of the average mask coverage is set to 0.08 in this embodiment. Only when the average mask response exceeds the threshold It takes effect at the same time, thereby inhibiting the mask from continuing to expand in the later stages of training.

[0030] To accurately evaluate the performance of the infrared image super-resolution reconstruction method in this embodiment and to more intuitively compare the advantages and disadvantages of different methods, this embodiment selects Peak Signal-to-Noise Ratio (PSNR) (the higher the better) and Structural Similarity Index (SSIM) (the higher the better) as objective evaluation indicators to measure the fidelity and structural consistency between the reconstructed result and the reference high-resolution image. Simultaneously, a subjective comparison of the visual effect of the reconstructed result is performed to comprehensively evaluate the reconstruction capability of the method of this invention. Specifically, this embodiment optimizes the network parameters based on the Adam optimizer, with an initial learning rate set to 0.0001, and uses a cosine annealing strategy to decay the learning rate. During training, the degenerate prior-guided shallow modulation module (step S101) and the deep hot-sorting reconstruction module (steps S102 and S103) are trained first. After obtaining a stable backbone model, the prior-guided selective refinement module (step S104) is then activated for a second stage of short-range fine-tuning. The training loss consists of several components, including pixel reconstruction loss, local hot sorting constraint, re-degradation consistency constraint, hard-region weighted loss, and mask coverage constraint, which work together to ensure overall reconstruction accuracy and local hard-region recovery capability. To recover clearer infrared details, the output image resolution is set to 2 or 4 times the input image resolution, with a focus on analyzing the experimental results under the ×4 setting. In the experiments, the infrared image super-resolution reconstruction method of this embodiment is compared with five existing super-resolution and infrared super-resolution methods, including FSRCNN, SRCNN, PSRGAN, RGT, and IRSRMamba. The results are shown in Table 1.

[0031] Table 1: Performance Comparison of the Method in this Embodiment with Five Super-resolution and Infrared Super-resolution Methods

[0032] In addition to objective evaluation indicators, this embodiment also includes visual evaluation to provide a more intuitive understanding of the effects of different super-resolution methods. Figure 3 This diagram illustrates a comparison of infrared super-resolution results between the method of this invention and existing methods, where (a) is a high-resolution red true image, (b) is the result obtained by the FSRCNN method, (c) is the result obtained by the SRCNN method, (d) is the result obtained by the IRSRMamba method, (e) is the result obtained by the RGT method, (f) is the result obtained by the PSRGAN method, and (g) is the result obtained by the method of this embodiment. Figure 3 The comparison of (a) to (g) shows that the method of this embodiment achieves the best super-resolution visual effect, which indicates that the method of this embodiment can perform infrared super-resolution tasks very well.

[0033] In summary, the method in this embodiment includes using degradation prior-guided shallow modulation to perform shallow feature fusion on low-resolution infrared images, further predicting degradation intensity maps, thermal boundary maps, and uncertainty maps, and then performing gated modulation on the shallow features to provide a degradation-aware input representation for subsequent reconstruction. Deep thermal ranking reconstruction is used to perform deep structure modeling and upsampling reconstruction on the prior-modulated features, and local thermal ranking constraints and re-degradation consistency constraints are combined during training to improve the recovery capability of the main structure of the infrared image and suppress the generation of false details. Prior-guided selective refinement generates a mask for difficult areas based on the degradation intensity map, thermal boundary map, and uncertainty map, selectively enhancing only severely degraded, complex-boundary, or highly uncertain-to-recovery regions, thereby further improving the detail recovery capability of key local areas. This invention organically combines shallow feature fusion, degradation prior prediction, prior-guided feature modulation, deep state-space modeling, and selective refinement for difficult areas, achieving super-resolution reconstruction results with strong detail recovery capability, good thermal structure preservation, and good adaptability to complex degradation regions, relying only on low-resolution infrared input.

[0034] Those skilled in the art will understand that the technical solutions provided by this invention can take the form of a method, system, or computer program product. Therefore, this invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this invention can take the form of a computer program product implemented on one or more computer-readable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code. For example, this invention can provide an infrared image super-resolution reconstruction system including an interconnected microprocessor and a memory, the microprocessor being programmed or configured to execute the infrared image super-resolution reconstruction method. This invention can provide a computer-readable storage medium storing a computer program or instructions programmed or configured to execute the infrared image super-resolution reconstruction method via a processor. This invention can provide a computer application program including a computer program or instructions programmed or configured to execute the infrared image super-resolution reconstruction method via a processor. This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It should be understood that each block of a flowchart and / or block diagram, and combinations of blocks in a flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing device, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 The computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to operate in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The functions specified in one or more boxes. These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable apparatus for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0035] The above description is merely a preferred embodiment of the present invention. The scope of protection of the present invention is not limited to the above embodiments. All technical solutions falling within the scope of the present invention's concept are within the scope of protection of the present invention. It should be noted that for those skilled in the art, any improvements and modifications made without departing from the principles of the present invention should also be considered within the scope of protection of the present invention.

Claims

1. A method for super-resolution reconstruction of infrared images, characterized in that, This includes inputting low-resolution infrared images. Generate high-resolution infrared images using a pre-trained infrared image super-resolution reconstruction model Includes: S101, which displays low-resolution infrared images Input shallow feature fusion network to extract shallow embedding features S102, embedding shallow features Prior shared features are extracted from the trunk path through shared feature extraction. Based on prior shared features The degradation intensity map was predicted. Thermal boundary map and uncertainty diagram Three degenerate prior maps, and shallow embedding features based on the degenerate prior maps. Performing degenerate prior-guided shallow modulation to obtain shallow features S103, shallow features The backbone reconstruction features are generated sequentially through deep feature modeling and hot sorting reconstruction. According to the degradation intensity map Thermal boundary map and uncertainty diagram Difficult-to-recover region mask S104, based on the difficult-to-recover region mask. Reconstruction features of the main trunk The difficult-to-recover regions in the image are locally refined, enhanced, and reconstructed to obtain the final high-resolution infrared image. ; Step S102 includes: S301, embedding shallow features Prior shared features are extracted from the trunk path through shared feature extraction. S302 will share prior features. Inputting three parallel 1×1 convolutional branches respectively yields the degradation intensity map. Thermal boundary map and uncertainty diagram Three degradation prior maps; S303, degradation intensity map Thermal boundary map and uncertainty diagram Three degenerate prior images are stitched together along the channel dimension to obtain the prior stack tensor. S304, stacking prior tensors A priori modulation parameter tensor is generated by a priori modulation parameter generation unit consisting of a 1×1 convolutional layer and a nonlinear activation layer; S305, the modulation parameter tensor is divided into three tensors along the channel dimension to obtain the scale modulation term. Bias modulation term and gating items S306, for the scale modulation term and gating items By applying nonlinear transformations, the final scale modulation term is obtained. and final gating item According to the bias modulation term The final scale modulation term and final gating item Performing degenerate prior-guided shallow modulation to obtain shallow features : ； in, For element-wise multiplication, To be compatible with the final scale modulation term A single sheet of the same size; In step S103, shallow features are... The backbone reconstruction features are generated sequentially through deep feature modeling and hot sorting reconstruction. Includes: S401, which will include shallow features Expanded into one-dimensional sequence features For one-dimensional sequence features The input sequence features are obtained by performing random deactivation processing. S402, input sequence features Input by The deep reconstruction backbone, composed of a series of residual state space groups, generates deep sequence features. In the deep reconstruction backbone, any first... The functional expression for the group of residual state-space units is: ； in, For the first Deep sequence features generated from a set of residual state spaces For the first A set of residual state spaces, For the first The input sequence features of a group of residual state spaces The image height is the two-dimensional feature map. The image width of the two-dimensional feature map. For two-dimensional feature serialization operators, For group-level convolutional mapping units, An operator for recovering sequence features into a two-dimensional feature map. Indicates will Restore to size Two-dimensional feature map; S403, deep sequence features Restore to two-dimensional feature map : ； ； in, These are the normalized deep sequence features. For the normalization operator; S404, the two-dimensional feature map and shallow features Residual fusion is performed to obtain backbone reconstruction features. : ； in, A convolutional mapping unit, used for processing two-dimensional feature maps. Apply a convolutional mapping to the two-dimensional feature map. Transform to suit shallow features Construct the feature space for residual fusion; Step S104 includes: S501, reconstructing the main trunk features Candidate refined residuals are generated using a residual refinement unit consisting of two convolutional layers. ; S502, utilizing a difficult-to-recover region mask. Refine the candidate residuals Position-by-position weighting is performed, and the residuals are superimposed onto the backbone reconstruction features. The selectively refined features are obtained above : ； in, The preset refinement factor, This is element-wise multiplication; S503, selectively refined features The input reconstruction head reconstructs the final high-resolution infrared image. .

2. The infrared image super-resolution reconstruction method according to claim 1, characterized in that, Step S101 includes: S201, low-resolution infrared image Normalization yields normalized features ; S202, normalized features Feature extraction is performed using a 3×3 convolutional layer to obtain small-scale features. Normalized features Large-scale features are extracted using a 7×7 depthwise convolutional layer and channel mapping using a 1×1 convolutional layer. ; S203, small-scale features Perform two-dimensional discrete wavelet decomposition to obtain even-numbered row features and odd row features Even row features and odd row features The column directions were divided into even-numbered and odd-numbered columns, resulting in four sub-sampling features. ~ Four sub-sampling features ~ After dimensional alignment, construct low-frequency approximate components. Horizontal detail components Vertical detail components and diagonal detail components : ； ； ； ； Using bicubic interpolation to approximate low-frequency components Horizontal detail components Vertical detail components and diagonal detail components Restored to small-scale features For the same spatial dimensions, low-frequency approximate wavelet components were obtained. Horizontal detail wavelet components Vertical detail wavelet components and diagonal detail wavelet components ; Approximate low-frequency wavelet components Horizontal detail wavelet components Vertical detail wavelet components and diagonal detail wavelet components Wavelet detail features are obtained by splicing. ; S204, based on wavelet detail features Generate modulation parameters Using modulation parameters For large-scale features Modulation is performed to generate large-scale modulation features. ; S205 will modulate large-scale features Small-scale features The features are concatenated along the channel dimension and then fused using a 1×1 convolution to obtain shallow fused features. ; S206, shallow fusion features This is achieved by mapping it to a unified embedding feature space through an embedding feature mapping layer to obtain shallow embedding features. : ； in, and For the convolution kernel parameters and bias parameters of the embedded feature mapping layer, This is a convolution operation.

3. The infrared image super-resolution reconstruction method according to claim 1, characterized in that, The shared feature extraction trunk includes two convolutional layers, and the shared feature extraction trunk extracts prior shared features. The function expression is: ； ； in, and These are the kernel parameters and bias parameters of the first convolutional layer, respectively. and These are the kernel parameters and bias parameters of the second convolutional layer, respectively. This is a convolution operation; This is a convolution operation grouped by channel; It is a non-linear activation function. The intermediate prior features are the output of the first convolutional layer; the shared prior features are... Inputting three parallel 1×1 convolutional branches respectively yields the degradation intensity map. Thermal boundary map and uncertainty diagram The functional expressions for the three degenerate prior maps are: ； ； ； in, , and These represent the 1×1 convolution kernel parameters of the three parallel 1×1 convolution branches; , and These are the bias parameters for three parallel 1×1 convolutional branches; This is the Sigmoid activation function.

4. The infrared image super-resolution reconstruction method according to claim 1, characterized in that, In step S103, based on the degradation intensity map Thermal boundary map and uncertainty diagram Difficult-to-recover region mask This refers to generating data using a mask unit comprising two convolutional layers, based on the degradation intensity map. Thermal boundary map and uncertainty diagram The prior stack tensor obtained by stitching together three degenerate prior maps along the channel dimension Generating units generate masks for difficult regions : ； ； in, The output features of the first convolutional layer of the mask generation unit. and These are the convolution kernel parameters for the first and second convolutional layers of the mask generation unit, respectively. and These are the bias parameters for the first and second convolutional layers of the mask generation unit, respectively. It is a non-linear activation function. It is the Sigmoid activation function. This is a convolution operation.

5. The infrared image super-resolution reconstruction method according to claim 1, characterized in that, The feature of reconstructing the trunk Candidate refined residuals are generated using a residual refinement unit consisting of two convolutional layers. The function expression is: ； ； in, The output features of the first convolutional layer of the residual refinement unit. and These are the convolution kernel parameters for the first and second convolutional layers of the residual refinement unit, respectively. and These are the bias parameters for the first and second convolutional layers of the residual refinement unit, respectively. It is a non-linear activation function.

6. The infrared image super-resolution reconstruction method according to claim 1, characterized in that, The loss function used during training of the infrared image super-resolution reconstruction model includes pixel reconstruction loss. Local thermal sorting loss Consistency loss due to re-degradation Weighted loss in difficult areas and mask coverage loss The expression for the calculation function of the pixel reconstruction loss, in part or all of the above, is as follows: ； in, For the final high-resolution infrared image, For reference, high-resolution infrared images, It is an L1 norm; The expression for the calculation function of the local hot sorting loss is: ； in, For local hot sorting loss weights, For an effective mask generated from a reference thermal difference threshold, For element-wise multiplication, It is a soft positive function. for The single-channel heatmap obtained by mapping The thermal difference between the center of the local window and its neighboring locations This is the temperature scaling factor. For symbolic functions, for The single-channel heatmap obtained by mapping The thermal difference between the center of the local window and its neighboring locations To prevent positive numbers with a denominator of zero; The expression for the calculation function of the re-degradation consistency loss is: ； in, To further degrade the consistency loss weight, for Low-resolution infrared images obtained by degenerate mapping transformation The input is a low-resolution infrared image; the degradation mapping transformation includes converting the input low-resolution infrared image into a final high-resolution infrared image. Low-resolution infrared images are obtained through convolutional feature extraction, bicubic interpolation downsampling, and output convolution. The calculation function expression for the weighted loss of the difficult region is as follows: ； in, Weighted loss for the difficult area pixel position The mean absolute error at that point For high-resolution masks In coordinates The mask value at the location, the high-resolution mask For difficult area masking Interpolated to the final high-resolution infrared image The same spatial dimensions are obtained; The expression for calculating the mask coverage loss is as follows: ； in, Weighting is applied to mask coverage loss. To obtain the maximum value, Masking for difficult areas The average value of all elements This represents the upper bound of the average mask coverage.

7. An infrared image super-resolution reconstruction system, comprising a microprocessor and a memory interconnected, characterized in that, The microprocessor is programmed or configured to perform the infrared image super-resolution reconstruction method according to any one of claims 1 to 6.

8. A computer-readable storage medium storing a computer program or instructions, characterized in that, The computer program or instructions are programmed or configured to execute the infrared image super-resolution reconstruction method according to any one of claims 1 to 6 via a processor.

9. A computer application program, comprising a computer program or instructions, characterized in that, The computer program or instructions are programmed or configured to execute the infrared image super-resolution reconstruction method according to any one of claims 1 to 6 via a processor.

Images