Infrared video super-resolution reconstruction method and system based on spatio-temporal feature fusion

Abstract

The application discloses an infrared video super-resolution reconstruction method and system based on space-time feature fusion, and relates to the technical field of video super-resolution reconstruction.The application obtains double-camera calibration parameters, a low-resolution infrared video sequence and a high-resolution visible light image; a composite degradation model is constructed to generate various degradation distribution maps, and according to the degradation distribution maps, infrared image degradation compensation is completed; through global transformation and local residual correction, infrared and visible light sub-pixel registration is realized, cross-modal reliability is calculated, and reliable visible light structural features are screened; according to the gate weight, the visible light feature injection position and intensity are controlled, and the fusion reconstruction features are constructed in combination with adjacent frame infrared features; in the reconstruction process, multiple thermal radiation consistency constraints are introduced to guarantee that the infrared thermal information is not distorted; through degradation operator, degradation back projection verification is carried out, the reconstruction result is optimized, and a high-resolution infrared image is output.The application realizes infrared video high-fidelity super-resolution reconstruction which takes into account the fidelity of thermal information and the clearness of details.

Description

Technical Field

[0001] This invention relates to the field of video super-resolution reconstruction technology, specifically to an infrared video super-resolution reconstruction method and system based on spatiotemporal feature fusion. Background Technology

[0002] With the increasing demand for infrared visual perception in fields such as security monitoring, industrial thermal detection, drone reconnaissance, autonomous driving, and fire rescue, infrared imaging equipment has been widely deployed. However, due to limitations in detector size, hardware cost, and optical systems, real-world infrared videos generally suffer from multiple degradation problems, including low resolution, blurred edges, noise, stripes, and thermal drift. To improve image detail recognition capabilities at low cost, infrared super-resolution reconstruction has become the mainstream technology. Current research focuses on three main directions: single-frame infrared reconstruction, visible light-guided single-frame reconstruction, and multi-frame infrared video reconstruction. Many deep learning-based solutions improve visual clarity through feature fusion and attention networks, but issues such as dynamic device degradation, cross-modal misalignment, thermal information distortion, and video ghosting in real-world scenarios remain unresolved. In this context, the industry continues to develop algorithms that balance detail enhancement with the thermal physical characteristics of infrared.

[0003] Chinese patent (publication number CN120495087A) discloses a multi-level attention-guided infrared super-resolution method. This method extracts multi-scale features of infrared and visible light, achieves bidirectional feature fusion through cross-attention, and outputs high-definition infrared images through multi-scale decoding. However, this method directly superimposes visible light textures with fixed fusion weights, without distinguishing the matching degree between visible light textures and infrared thermal structures. It is very easy to generate false textures such as clothing patterns and markings that have no temperature correspondence in the reconstruction results. At the same time, it lacks targeted processing for various composite degradations of infrared light.

[0004] Chinese patent (publication number CN121563780A) discloses an infrared super-resolution reconstruction method that integrates high-frequency information from visible light. This method extracts high-frequency feature channels of two modes and splices them together. It completes detail enhancement through cascaded sensing modules and optimizes image smoothness with multiple loss functions. However, this method only completes modal registration through simple interpolation and does not evaluate local registration errors and occlusion areas. It has problems such as target contour offset and double artifacts. Furthermore, it does not set specific constraints to protect the original infrared cold and hot sorting and hot spot distribution.

[0005] Chinese patent (publication number CN118279148A) discloses a visible light adaptive dual-stream guided thermal infrared super-resolution method. This method constructs a dual-stream network to extract the optimal matching visible light features and completes the reconstruction through multi-dimensional attention. However, this method is only for processing single static infrared images and cannot improve details through sub-pixel information of adjacent video frames. At the same time, it does not distinguish between inter-frame geometric motion and changes in the thermal state of the target. Directly reusing historical frames is prone to producing trailing and temporal flicker.

[0006] In summary, existing infrared super-resolution schemes cannot simultaneously meet the requirements of realistic composite degradation adaptation, cross-modal reliable guidance, video temporal stability, infrared thermophysical information fidelity, and reconstruction result verification. They are also unable to adapt to complex real-world scenarios with stringent requirements for thermal fidelity and result interpretability, such as industrial inspection and military reconnaissance. There is an urgent need for a complete, closed-loop, multi-constraint collaborative infrared video super-resolution reconstruction scheme. Summary of the Invention

[0007] To address the aforementioned technical problems, this application discloses a method and system for infrared video super-resolution reconstruction based on spatiotemporal feature fusion, wherein the method specifically comprises: Obtain the calibration parameters of the infrared camera and the visible light camera, and acquire the low-resolution infrared video frame sequence and high-resolution visible light image to be reconstructed; The blurring, noise, non-uniform degradation, row and column stripes, bad pixels, thermal drift and local saturation degradation information in infrared images are analyzed to construct a degradation distribution map corresponding to the spatial location of the infrared image. Based on the degradation distribution map, the degradation region of the infrared image is identified and the feature enhancement processing is performed to obtain the degradation-compensated infrared features. Based on the structural feature correspondence between the infrared image and the high-resolution visible light image, spatial mapping is performed to generate a cross-modal confidence map, and visible light structural features are extracted based on the cross-modal confidence map. Based on the visible light structural features, the injection position and injection intensity in the infrared reconstruction process are analyzed, and combined with the degradation-compensated infrared features, visible light structural features, and features of adjacent infrared video frames, a fused reconstruction feature is constructed. Reconstruction is performed based on the fused reconstruction features. During the reconstruction process, thermal radiation consistency constraints are used to maintain the consistency of thermal distribution relationships, thermal gradient trends, and hotspot locations in the reconstructed image. Based on the reconstructed high-resolution infrared image, a degenerate back projection verification is performed, and the reconstruction result is corrected according to the verification result to obtain a high-resolution infrared reconstructed image and a corresponding region-level confidence map.

[0008] Preferably, the construction of the degradation distribution map corresponding to the spatial location of the infrared image specifically involves: performing multi-dimensional feature analysis on the input infrared image using a composite degradation algorithm adapted to the real infrared imaging scene, extracting multi-scale infrared basic features, frequency domain degradation features, spatial anomalous degradation features, and temporal degradation features respectively; fusing and modeling the multi-dimensional features using a preset degradation parameter fusion algorithm to output a spatial degradation distribution map covering all types of degradation; stitching the degradation distribution maps of each type by channel to generate a unified spatial degradation tensor; generating position-by-position and channel-by-channel modulation parameters based on the spatial degradation tensor to perform degradation condition modulation on the infrared basic features, thereby obtaining the degradation-compensated infrared features.

[0009] Preferably, the generation of the cross-modal confidence map specifically involves: determining the global transformation matrix based on the dual-sensor calibration parameters and generating an initial mapping relationship; Based on the initial mapping relationship, the local residual displacement field is estimated, the global mapping error is corrected, and the target space mapping field is obtained. Based on the target space mapping field, the high-resolution visible light image and visible light structural feature sampling are mapped to the infrared image coordinate system to complete the dual-modal space alignment; An occlusion mask is generated through consistency verification of forward and reverse displacement fields. The bimodal feature consistency error and edge direction error are calculated to generate a registration confidence map. The formula is as follows: in, for Time coordinates The confidence map of the bimodal registration at the location. To cover the mask, This refers to the consistency error between forward and reverse mapping. For feature consistency error, This refers to the edge direction error. For spatiotemporal synchronization error, These are preset weighting coefficients.

[0010] Preferably, the extraction of visible light structural features specifically includes: visible light structural features including at least visible light image quality, cross-modal edge consistency, regional structure consistency, temporal structure support, and infrared degradation compensation requirements; The formula for calculating the quality of visible light images is as follows: in, for Time pixel position The corresponding overall quality reliability of visible light images, To enhance credibility, For clarity and credibility, For the sake of contrast credibility, For effective visibility and credibility; The formula for calculating cross-modal edge direction consistency is: in, For the first Layer features time Reliability of infrared-visible edge orientation consistency at location. For infrared images Layer features at the pixel The edge gradient vector at that point, For visible light images Layer features at the pixel The edge gradient vector at that point, For vector dot product operation, For L2 norm operations, It is a very small constant; The formula for calculating regional structural consistency is: in, for time Reliability of positional bimodal region structural consistency For infrared image pixels The local structural feature vector of the location, Visible light image pixels The local structural feature vector of the location; The formula for calculating the support of temporal structures is: in, for time Temporal structure support of location, The set of adjacent frame indices that participate in the timing calculation. This represents the total number of adjacent frames. For the present Real-time infrared images The structural characteristic response value of the location, For adjacent After inter-frame alignment, the frames are mapped to the infrared structural response values ​​of the current frame. This refers to the time-series smoothing scaling coefficient; By integrating multi-dimensional evaluation indicators, registration confidence, and degradation confidence, a regional-level cross-modal confidence map is generated. Furthermore, considering infrared degradation compensation requirements, the target visible light feature gating weights are obtained, as shown in the formula: in, For cross-modal trusted gated weight graphs, for Time-region-level cross-modal reliability graph This is a graph showing the infrared degradation compensation requirement. The height and width dimensions of the feature map. For pixel-level point-by-point multiplication, the Hadamard product operator is used. Based on the aforementioned gated weights, reliable visible light structural features are screened by scale and frequency band.

[0011] Preferably, the analysis of injection location and injection intensity during the infrared reconstruction process specifically involves: based on the cross-modal gating weights The system locates the injection position of visible light features and injects visible light structural features in regions where the gating weight is greater than a preset threshold. Different injection intensities are set for visible light features of different frequency bands. Among them, the mid-frequency structural features are controlled by full-domain gating weight, and the high-frequency structural features are constrained by dual thresholds of edge consistency and registration reliability. The controlled fusion of visible light features and infrared features is completed by using a residual fusion method.

[0012] Preferably, the construction of the fused reconstruction features specifically involves: acquiring a sequence of consecutive infrared video frames before and after the current frame; performing degradation modulation preprocessing on the features of adjacent frames to suppress noise, stripes, and dead pixel interference; completing inter-frame alignment through global motion compensation and local dense motion field estimation; generating an occlusion mask based on bidirectional motion consistency verification; generating regional temporal reliability by combining geometric reliability, thermal stability, degradation reliability, and temporal distance attenuation factor; normalizing the weights of adjacent frames based on temporal reliability; adaptively aggregating sub-pixel features from multiple frames; extracting temporal detail residuals; and fusing temporal compensation features, degradation compensation infrared features, and reliable visible light structure features to obtain the fused reconstruction features.

[0013] Preferably, the thermal radiation consistency constraint specifically includes: regional average thermal response constraint, thermal ranking consistency constraint, thermal gradient consistency constraint, and hotspot location consistency constraint, wherein the formula for the regional average thermal response consistency constraint is: in, Let the regional heat mean consistency loss function be used. This represents the total number of local block regions in the image. For the first A local statistical image sub-region, The calculation is performed to determine the average pixel value of the region. For rebuilding output Real-time high-resolution infrared images, The original low-resolution infrared image corresponds to the first The true average thermal response reference value for each sub-region; The hot sorting consistency constraint formula is: Let the hot sorting consistency loss function be... This is a set of pixel region comparison sample pairs. This represents the total number of sample pairs. To preset the hot sorting interval margin, Symbols indicating the hot and cold relationships in the original image. , To reconstruct the first high-resolution image , The thermal response temperature characteristic value of the region; The formula for thermal gradient consistency constraint is: in, Let thermal gradient consistency loss function be used. For the spatial gradient solver, To reconstruct the low-frequency thermal gradient feature map of a high-resolution infrared image, This is a low-frequency thermal gradient reference feature map obtained by upsampling and matching the resolution of the original low-resolution infrared image. For norm-1 constraint operations; The formula for the consistency constraint between hotspot location and range is: in, Let the hotspot consistency loss function be used. This represents the actual hotspot mask region obtained after upsampling the original low-resolution infrared image. To reconstruct the hotspot mask region predicted from the high-resolution infrared image, It is a very small constant.

[0014] Preferably, the degradation backprojection verification specifically involves: constructing a real-time degradation operator based on the degradation parameters corresponding to the degradation distribution map. The initially reconstructed high-resolution infrared image is degraded and projected onto a low-resolution space to obtain a reconstructed low-resolution image. ,in, for Real-time dynamic degradation operator, For degenerate mapping operation functions, This is the complete set of infrared image degradation parameters estimated at the current moment. This is a reconstructed low-resolution infrared image obtained through degenerate back projection. The reconstructed low-resolution infrared image obtained through degenerate back projection; Calculate the back projection error between the original low-resolution infrared image and the reconstructed image, and iteratively correct the high-resolution image based on the back projection error. Repeat the iterative correction until the back projection error is lower than the preset error threshold or the number of iterations reaches the maximum preset number, then terminate the iterative optimization and output the optimal high-resolution infrared image with the highest accuracy and the smallest residual distortion.

[0015] Preferably, the regional-level credibility map specifically includes: extracting multi-dimensional uncertainty indicators such as degradation estimation credibility, cross-modal gating credibility, temporal fusion credibility, back projection error, and bi-branch reconstruction difference; The reliability of regional reconstruction is calculated using the exponential decay function, as follows: in, To enhance the credibility of regional-level reconstruction, These are the preset weighting coefficients. This is a plot of the upsampling back projection error. To reconstruct differences in the two branches, For the confidence plot of degradation estimation, For cross-modal visible light feature gating weights, For pixel-level timing overall reliability, To ensure the fidelity and reliability of infrared intrinsic features; Based on the confidence level of the image region types, including the original infrared observation support region, the adjacent frame compensation region, the confidence visible light guidance region, and the model prior low confidence region, a region-level confidence map is obtained.

[0016] The system includes a data acquisition module, an infrared imaging degradation self-sensing module, an infrared-visible light dynamic registration module, a cross-modal trusted gating module, an infrared video temporal consistency fusion module, and a thermal radiation consistency constraint reconstruction module, as detailed below: The data acquisition module acquires low-resolution infrared video frame sequences, high-resolution visible light images, dual sensor calibration parameters, and real-time device status data. The infrared imaging degradation self-sensing module analyzes multi-dimensional degradation features, generates a comprehensive degradation parameter set and a global spatial degradation tensor, and outputs the degradation-compensated infrared features and degradation estimation confidence map. The infrared-visible light dynamic registration module generates a sub-pixel-level target space mapping field based on the global transformation matrix and the local residual displacement field, and outputs a registration confidence map and an occlusion mask. The cross-modal trust gating module integrates multi-dimensional trust evaluation indicators to generate a cross-modal trust map and cross-modal trust gating weights, and filters and outputs trustworthy visible light structural features. The infrared video temporal consistency fusion module calculates the inter-frame geometric alignment reliability and thermal stability to obtain the pixel-level temporal comprehensive reliability, and adaptively aggregates multi-frame features to generate temporal compensation features. The thermal radiation consistency constraint reconstruction module outputs a high-resolution infrared reconstruction image and a regional reconstruction confidence map.

[0017] Compared with the prior art, the technical solution of this application has the following technical effects: This invention can simultaneously identify various types of spatial non-uniform degradation, such as noise, non-uniformity, row and column stripes, bad pixels, local saturation, motion blur, and thermal drift, and generate a region-by-region degradation distribution map to dynamically modulate infrared features, thereby significantly reducing reconstruction distortion under different cameras and complex environments.

[0018] This invention generates a region-level fusion weight by comprehensively considering registration accuracy, edge and structure consistency, visible light imaging quality, occlusion, and temporal stability. It can controllably inject visible light structures in frequency bands and automatically reduce or even shut down guidance when registration fails or visible light quality is low, thus avoiding false hot edges and target contour offset problems caused by direct stitching and fusion.

[0019] This invention simultaneously calculates the reliability of inter-frame geometric alignment and thermal stability. It reduces the weight of historical frames for drastic thermal changes, new targets, and occlusion areas, reuses reliable sub-pixel details, and improves the temporal continuity of infrared video. It also sets four types of exclusive constraints: regional mean, hot and cold sorting, thermal gradient, and hot spot range. The reconstruction is split into two branches: thermal information preservation and structural details. Details are supplemented only with controlled residuals, making it suitable for high-reliability scenarios such as temperature measurement and fault detection.

[0020] This invention constructs a real-time degradation operator based on estimated degradation, back-projects the initial high-resolution image to the low-resolution spatial comparison residual and iteratively optimizes it to form a complete self-verification, ensuring that the reconstruction result fits the original observation; and automatically switches to pure infrared single-frame / multi-frame reconstruction mode when there is visible light occlusion, overexposure or underexposure, complete registration failure or low confidence of adjacent frames, and has the ability to guide failure adaptive back-off, making the system more robust.

[0021] The above description is only an overview of the technical solution of this application. In order to better understand the technical means of this application and implement it in accordance with the contents of the specification, and to make the above and other objects, features and advantages of this application more obvious and understandable, the preferred embodiments of this application are described in detail below with reference to the accompanying drawings.

[0022] The above and other objects, advantages and features of this application will become more apparent to those skilled in the art from the following detailed description of specific embodiments in conjunction with the accompanying drawings. Attached Figure Description

[0023] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. In all drawings, similar elements or parts are generally identified by similar reference numerals. In the drawings, the elements or parts are not necessarily drawn to scale.

[0024] Based on the description of the figures and their corresponding technical content in the document, the titles of the figures are as follows: Figure 1 The flowchart shows the overall process of infrared video super-resolution reconstruction based on spatiotemporal feature fusion. Figure 2 This is an architecture diagram of an infrared video super-resolution reconstruction system based on spatiotemporal feature fusion. Detailed Implementation

[0025] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. In the following description, specific details such as specific configurations and components are provided merely to help fully understand the embodiments of this application. Therefore, those skilled in the art should understand that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this application. In addition, for clarity and brevity, descriptions of known functions and structures are omitted in the embodiments.

[0026] It should be understood that the phrase "an embodiment" or "this embodiment" throughout the specification means that a specific feature, structure, or characteristic related to the embodiment is included in at least one embodiment of this application. Therefore, "an embodiment" or "this embodiment" appearing throughout the specification does not necessarily refer to the same embodiment. Furthermore, these specific features, structures, or characteristics can be combined in any suitable manner in one or more embodiments.

[0027] Furthermore, reference numerals and / or letters may be repeated in different examples within this application. Such repetition is for the purpose of simplification and clarity and does not in itself indicate a relationship between the various embodiments and / or settings discussed.

[0028] In this article, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can mean: A exists alone, B exists alone, and A and B exist simultaneously. The term " / and" in this article describes another type of relationship between related objects, indicating that two relationships can exist. For example, A / and B can mean: A exists alone, and A and B exist alone. In addition, the character " / " in this article generally indicates that the related objects before and after it are in an "or" relationship.

[0029] In this article, the term "at least one" is merely a description of the relationship between related objects, indicating that there can be three relationships. For example, "at least one of A and B" can mean: A exists alone, A and B exist simultaneously, or B exists alone.

[0030] It should also be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion.

[0031] Example 1 mainly describes an infrared video super-resolution reconstruction method based on spatiotemporal feature fusion, such as... Figure 1 As shown, it specifically includes: Obtain the calibration parameters of the infrared camera and the visible light camera, and acquire the low-resolution infrared video frame sequence and high-resolution visible light image to be reconstructed; The blurring, noise, non-uniform degradation, row and column stripes, bad pixels, thermal drift and local saturation degradation information in infrared images are analyzed to construct a degradation distribution map corresponding to the spatial location of the infrared image. Based on the degradation distribution map, the degradation region of the infrared image is identified and the feature enhancement processing is performed to obtain the degradation-compensated infrared features. Based on the structural feature correspondence between the infrared image and the high-resolution visible light image, spatial mapping is performed to generate a cross-modal confidence map, and visible light structural features are extracted based on the cross-modal confidence map. Based on the visible light structural features, the injection position and injection intensity in the infrared reconstruction process are analyzed, and combined with the degradation-compensated infrared features, visible light structural features, and features of adjacent infrared video frames, a fused reconstruction feature is constructed. Reconstruction is performed based on the fused reconstruction features. During the reconstruction process, thermal radiation consistency constraints are used to maintain the consistency of thermal distribution relationships, thermal gradient trends, and hotspot locations in the reconstructed image. Based on the reconstructed high-resolution infrared image, a degenerate back projection verification is performed, and the reconstruction result is corrected according to the verification result to obtain a high-resolution infrared reconstructed image and a corresponding region-level confidence map.

[0032] Furthermore, the low-resolution infrared video frame sequence and high-resolution visible light image to be reconstructed are specifically as follows: Pre-acquire and store dual sensor calibration parameters from both infrared and visible light cameras. The calibration parameters include dual-camera intrinsic parameters, extrinsic parameters, and distortion parameters; Real-time acquisition of infrared camera equipment parameters Parameters of visible light camera equipment Simultaneously, it acquires device status information corresponding to camera motion status and time synchronization error. The complete set of camera parameters is obtained by integration. ; At present, when reconstruction is needed Centered on a target location, a predetermined number of consecutive images are selected forward and backward to construct a low-resolution infrared video frame sequence. ,in This represents the number of adjacent frames selected before and after. Characterizes low-resolution properties; Acquire high-resolution visible light images that are approximately time-synchronized and field-matched with the current infrared frame to be reconstructed. This enables the synchronous acquisition and pairing input of dual-modal raw data and equipment calibration parameters.

[0033] Furthermore, the specific steps for constructing a degradation distribution map corresponding to the spatial location of the infrared image are as follows: by using a composite degradation algorithm adapted to the real infrared imaging scene, multi-dimensional feature analysis is performed on the input infrared image to extract multi-scale infrared basic features, frequency domain degradation features, spatial anomaly degradation features, and temporal degradation features. By fusing multi-dimensional features using a pre-defined degradation parameter fusion algorithm, a spatial degradation distribution map covering all types of degradation is output, and the corresponding comprehensive degradation description formula is: By stitching together the degradation distribution maps of various types according to channels, a unified spatial degradation tensor is generated, using the following formula: in, This is the set of comprehensive degradation parameters for the infrared image at the current moment. Let be the global spatial degradation tensor of the infrared image at the current moment. To estimate the spatial variation fuzzy kernel, This is a distribution map of random noise intensity. This is a non-uniform degradation distribution map. This is a row and column stripe distribution diagram. This is a distribution map of abnormal pixels with bad pixels. This is a distribution map of locally saturated regions. This is a motion fuzzy distribution map. This is a thermal drift distribution map. For the confidence plot of degradation estimation, For pixel-level point-by-point multiplication, the Hadamard product operator is used. Based on the spatial degradation tensor, position-by-position and channel-by-channel modulation parameters are generated, and degradation condition modulation is applied to the basic infrared features to obtain the degradation-compensated infrared features. The modulation formula is as follows: in, , For degraded modulation parameters, For degraded modulation mapping networks, For degraded bias modulation mapping networks, The basic characteristics of the infrared encoder output. The infrared features are after degradation compensation.

[0034] Furthermore, the composite degradation algorithm simulates the end-to-end degradation generation mechanism of high-resolution real infrared original images through fuzzing operations, downsampling, superposition of multiple types of noise, and detector distortion truncation, thus characterizing the complete imaging degradation generation process of low-resolution infrared images. The corresponding imaging formula for low-resolution infrared images is: in, This is the low-resolution infrared image to be reconstructed acquired at the current moment. The desired solution is the ideal, non-degradable, high-resolution infrared ground truth image at the current moment. The spatial variation blur kernel corresponding to the image at the current moment. For convolution operations or spatial variation blur operations, used to characterize blur degradation caused by optics and motion; Super-resolution magnification The corresponding image downsampling process; Random noise introduced into the imaging process; Non-uniform degradation caused by inconsistent response of infrared detectors; This refers to row and column stripe noise or fixed pattern noise present in the image. To reduce sparse degradation of images, such as bad pixels and abnormal pixels; Characterizes the imaging distortion process of detector dynamic range constraint, pixel quantization, numerical truncation and local saturation.

[0035] Furthermore, the generation of the cross-modal confidence map specifically involves: determining the global transformation matrix based on the dual-sensor calibration parameters, and generating the initial mapping relationship, as shown in the formula: in, These are the initial visible light image coordinates obtained after global transformation matrix mapping. This is the global transformation matrix. For homogeneous coordinate normalization operation, The coordinates are homogeneous coordinates of the infrared image; Based on the initial mapping relationship, the local residual displacement field is estimated, and the global mapping error is corrected to obtain the target space mapping field, as shown in the formula: in, For the local residual displacement field, For local residual correction network function, These are regular structural features used for registration in infrared images. Regular structural features for registration of visible light images. For real-time device status, It is a sub-pixel level target space mapping field; Based on the target space mapping field, the high-resolution visible light image and visible light structural feature sampling are mapped to the infrared image coordinate system to complete the dual-modal space alignment; An occlusion mask is generated through consistency verification of forward and reverse displacement fields. The bimodal feature consistency error and edge direction error are calculated to generate a registration confidence map. The formula is as follows: in, for Time coordinates The confidence map of the bimodal registration at the location. To cover the mask, This refers to the consistency error between forward and reverse mapping. For feature consistency error, This refers to the edge direction error. For spatiotemporal synchronization error, These are preset weighting coefficients.

[0036] Furthermore, the extraction of visible light structural features specifically includes: visible light structural features at least visible light image quality, cross-modal edge consistency, regional structure consistency, temporal structure support, and infrared degradation compensation requirements; The formula for calculating the quality of visible light images is as follows: in, for Time pixel position The corresponding overall quality reliability of visible light images, To enhance credibility, For clarity and credibility, For the sake of contrast credibility, For effective visibility and credibility; The formula for calculating cross-modal edge direction consistency is: in, For the first Layer features time Reliability of infrared-visible edge orientation consistency at location. For infrared images Layer features at the pixel The edge gradient vector at that point, For visible light images Layer features at the pixel The edge gradient vector at that point, For vector dot product operation, For L2 norm operations, It is a very small constant; The formula for calculating regional structural consistency is: in, for time Reliability of positional bimodal region structural consistency For infrared image pixels The local structural feature vector of the location, Visible light image pixels The local structural feature vector of the location; The formula for calculating the support of temporal structures is: in, for time Temporal structure support of location, The set of adjacent frame indices that participate in the timing calculation. This represents the total number of adjacent frames. For the present Real-time infrared images The structural characteristic response value of the location, For adjacent After inter-frame alignment, the frames are mapped to the infrared structural response values ​​of the current frame. This refers to the time-series smoothing scaling coefficient; By integrating multi-dimensional evaluation indicators, registration confidence, and degradation confidence, a regional-level cross-modal confidence map is generated. Furthermore, considering infrared degradation compensation requirements, the target visible light feature gating weights are obtained, as shown in the formula: in, For cross-modal trusted gated weight graphs, for Time-region-level cross-modal reliability graph This is a graph showing the infrared degradation compensation requirement. The height and width dimensions of the feature map. For pixel-level point-by-point multiplication, the Hadamard product operator is used. Based on the aforementioned gated weights, reliable visible light structural features are screened by scale and frequency band.

[0037] Furthermore, the visible light features are decomposed into low-frequency, mid-frequency, and high-frequency features, and a strict screening threshold is set for high-frequency features to suppress the injection of false textures.

[0038] Furthermore, the analysis of the injection location and injection intensity during the infrared reconstruction process specifically involves: based on the aforementioned cross-modal gating weights... Locate the injection position of visible light features and inject visible light structural features in regions where the gating weight is greater than a preset threshold; Differential injection intensities are set for visible light features of different frequency bands. Among them, the mid-frequency structural features are controlled by global gating weights, and the high-frequency structural features are subject to dual threshold constraints of edge consistency and registration reliability. Controlled fusion of visible light and infrared features is achieved using a residual fusion method, with the following formula: in, For adaptive fusion weights, These are the selected reliable visible light features. It is a pixel-level point-by-point Hadamard product operator.

[0039] Furthermore, the construction of fusion reconstruction features specifically involves: acquiring a sequence of consecutive infrared video frames before and after the current frame, performing degradation modulation preprocessing on the features of adjacent frames, and suppressing noise, stripes, and dead pixel interference; Inter-frame alignment is achieved through global motion compensation and local dense motion field estimation. An occlusion mask is generated based on bidirectional motion consistency verification. The inter-frame geometric alignment reliability and thermal stability are calculated respectively, using the following formulas: in, for time Position relative to the first Reliability of geometric alignment between adjacent frames. For temporal occlusion masking, To preset the balance weight coefficient, This refers to the inter-frame forward and reverse motion consistency error. This refers to the inter-frame geometric feature matching error. for time Position relative to the first Thermal stability between adjacent frames This is the error caused by the change in thermal response between frames. This is the scaling factor for smoothing local thermal changes. It is a very small constant; By combining geometric reliability, thermal stability, degradation reliability, and time-distance decay factor, a regional-level time series reliability is generated, using the following formula: in, For pixel-level timing overall reliability, To degrade reliability and credibility, For the reliability of time-series distance attenuation; Based on temporal reliability, the weights of adjacent frames are normalized, multi-frame sub-pixel features are adaptively aggregated, temporal detail residuals are extracted, and temporal compensation features, degradation compensation infrared features, and reliable visible light structure features are fused to obtain fused reconstruction features.

[0040] Furthermore, the thermal radiation consistency constraints specifically include: regional average thermal response constraints, thermal ranking consistency constraints, thermal gradient consistency constraints, and hotspot location consistency constraints. The formula for the regional average thermal response consistency constraint is: in, Let the regional heat mean consistency loss function be used. This represents the total number of local block regions in the image. For the first A local statistical image sub-region, The calculation is performed to determine the average pixel value of the region. For rebuilding output Real-time high-resolution infrared images, The original low-resolution infrared image corresponds to the first The true average thermal response reference value for each sub-region; The hot sorting consistency constraint formula is: Let the hot sorting consistency loss function be... This is a set of pixel region comparison sample pairs. This represents the total number of sample pairs. To preset the hot sorting interval margin, Symbols indicating the hot and cold relationships in the original image. , To reconstruct the first high-resolution image , The thermal response temperature characteristic value of the region; The formula for thermal gradient consistency constraint is: in, Let thermal gradient consistency loss function be used. For the spatial gradient solver, To reconstruct the low-frequency thermal gradient feature map of a high-resolution infrared image, This is a low-frequency thermal gradient reference feature map obtained by upsampling and matching the resolution of the original low-resolution infrared image. For norm-1 constraint operations; The formula for the consistency constraint between hotspot location and range is: in, Let the hotspot consistency loss function be used. This represents the actual hotspot mask region obtained after upsampling the original low-resolution infrared image. To reconstruct the hotspot mask region predicted from the high-resolution infrared image, It is a very small constant; Through the synergistic effect of multiple constraints, the reconstructed image is guaranteed not to change the thermal distribution pattern, thermal gradient trend, and hotspot core features of the original infrared image.

[0041] Furthermore, the degradation backprojection verification specifically involves constructing a real-time degradation operator based on the degradation parameters corresponding to the degradation distribution map. The initially reconstructed high-resolution infrared image is degraded and projected onto a low-resolution space to obtain a reconstructed low-resolution image. ,in, for Real-time dynamic degradation operator, For degenerate mapping operation functions, This is the complete set of infrared image degradation parameters estimated at the current moment. This is a reconstructed low-resolution infrared image obtained through degenerate back projection. The reconstructed low-resolution infrared image obtained through degenerate back projection; The back projection error between the original low-resolution infrared image and the reconstructed image is calculated, and the back projection loss function is constructed as follows: in, This represents the pixel-level back projection residual error. For the back-projection consistency loss function, for The original low-resolution infrared images to be reconstructed are acquired in real time. Solving operators for a norm; The iterative update formula for correcting high-resolution images based on back projection error is as follows: in, For the first The high-resolution infrared image after the next iteration update. For the first The adaptive iterative learning rate for each iteration. For the first The amount of high-resolution image pixel correction corresponding to the next iteration; Repeated iterative corrections are performed until the back projection error is lower than the preset error threshold or the number of iterations reaches the maximum preset number. Then, the iterative optimization is terminated, and the optimal high-resolution infrared image with the highest accuracy and the least residual distortion is output.

[0042] Furthermore, the regional-level credibility map specifically includes: extracting multi-dimensional uncertainty indicators such as degradation estimation credibility, cross-modal gating credibility, temporal fusion credibility, back projection error, and bi-branch reconstruction difference; The reliability of regional reconstruction is calculated using the exponential decay function, as follows: in, To enhance the credibility of regional-level reconstruction, These are the preset weighting coefficients. This is a plot of the upsampling back projection error. To reconstruct differences in the two branches, For the confidence plot of degradation estimation, For cross-modal visible light feature gating weights, For pixel-level timing overall reliability, To ensure the fidelity and reliability of infrared intrinsic features; Based on the confidence level of the image region types, including the original infrared observation support region, the adjacent frame compensation region, the confidence visible light guidance region, and the model prior low confidence region, a region-level confidence map is obtained.

[0043] Furthermore, this method also includes a cross-modal guidance failure adaptive backoff mechanism and a temporal fusion backoff mechanism, wherein the cross-modal guidance failure backoff mechanism specifically involves: calculating the global visible light guidance availability. When any of the following parameters—guidance availability, visible light image quality, or region occlusion ratio—falls below a preset threshold, the visible light feature injection weight is forcibly reset to zero, and the system switches to pure infrared single-frame or multi-frame reconstruction mode. The temporal fusion rollback mechanism is as follows: when the number of effective adjacent frames is zero or the global temporal reliability is lower than the preset threshold, the temporal compensation features are cleared, and only the infrared features and reliable visible light features of the current frame are used to complete the reconstruction, thereby suppressing interference from invalid temporal information.

[0044] This embodiment details an infrared video super-resolution reconstruction method based on spatiotemporal feature fusion. It acquires dual-camera calibration parameters, a low-resolution infrared video sequence, and a high-resolution visible light image. A composite degradation model is constructed to generate various degradation distribution maps, which are then used to perform infrared image degradation compensation. Sub-pixel registration of infrared and visible light is achieved through global transformation and local residual correction. Cross-modal reliability is calculated, and reliable visible light structural features are selected. The injection position and intensity of visible light features are controlled according to gating weights, and fused reconstruction features are constructed by combining infrared features from adjacent frames. Multiple thermal radiation consistency constraints are introduced during the reconstruction process to ensure that infrared thermal information is not distorted. Degradation backprojection verification is performed using degradation operators to optimize the reconstruction results and output a high-resolution infrared image.

[0045] Example 2, based on Example 1, describes in detail an infrared video super-resolution reconstruction system based on spatiotemporal feature fusion, such as... Figure 2 As shown, it includes a data acquisition module, an infrared imaging degradation self-sensing module, an infrared-visible light dynamic registration module, a cross-modal trusted gating module, an infrared video temporal consistency fusion module, and a thermal radiation consistency constraint reconstruction module, as detailed below: The data acquisition module is connected to the infrared imaging degradation self-sensing module, the infrared-visible light dynamic registration module, and the infrared video temporal consistency fusion module, respectively, to acquire low-resolution infrared video frame sequences, high-resolution visible light images, dual sensor calibration parameters, and real-time device status data. The infrared imaging degradation self-sensing module is connected to the cross-modal reliable gating module and the thermal radiation consistency constraint reconstruction module, respectively. It analyzes multi-dimensional degradation features, generates a comprehensive degradation parameter set and a global spatial degradation tensor, and outputs the degradation-compensated infrared features and degradation estimation reliability map. The infrared-visible light dynamic registration module is connected to the cross-modal trusted gating module. It generates a sub-pixel level target space mapping field based on the global transformation matrix and the local residual displacement field, and outputs the registration confidence map and the occlusion mask. The cross-modal trustworthy gating module is connected to the thermal radiation consistency constraint reconstruction module and the infrared video temporal consistency fusion module, respectively. It integrates multi-dimensional trustworthiness evaluation indicators to generate a cross-modal trustworthiness map and cross-modal trustworthy gating weights, and filters and outputs trustworthy visible light structural features. The infrared video temporal consistency fusion module is connected to the thermal radiation consistency constraint reconstruction module. It calculates the inter-frame geometric alignment reliability and thermal state stability to obtain the pixel-level temporal comprehensive reliability, and adaptively aggregates multi-frame features to generate temporal compensation features. The thermal radiation consistency constraint reconstruction module is the system output unit. It receives the degraded infrared features, reliable visible light structural features and temporal compensation features, and performs reconstruction under multiple thermal radiation consistency constraints. It completes the degradation back projection iterative correction through a real-time dynamic degradation operator and outputs a high-resolution infrared reconstructed image and a regional reconstruction reliability map.

[0046] This embodiment describes in detail an infrared video super-resolution reconstruction system based on spatiotemporal feature fusion, including six interconnected functional modules: data acquisition, infrared imaging degradation self-sensing, infrared-visible dynamic registration, cross-modal trusted gating, infrared video temporal consistency fusion, and thermal radiation consistency reconstruction. Through multi-module collaboration, infrared degradation correction, cross-modal trusted feature fusion, and temporal information adaptive aggregation are completed, outputting a high-fidelity infrared image and a corresponding region confidence map.

[0047] The above are merely preferred embodiments of the present invention and are not intended to limit the scope of protection of the present invention. For those skilled in the art, the present invention can have various modifications and variations. Any changes, modifications, substitutions, integrations, and parameter changes made to these embodiments within the spirit and principles of the present invention, without departing from the principles and spirit of the present invention, through conventional substitutions or to achieve the same function, fall within the scope of protection of the present invention.

Claims

1. A method for super-resolution reconstruction of infrared video based on spatiotemporal feature fusion, characterized in that, include: Obtain the calibration parameters of the infrared camera and the visible light camera, and acquire the low-resolution infrared video frame sequence and high-resolution visible light image to be reconstructed; The blurring, noise, non-uniform degradation, row and column stripes, bad pixels, thermal drift and local saturation degradation information in infrared images are analyzed to construct a degradation distribution map corresponding to the spatial location of the infrared image. Based on the degradation distribution map, the degradation region of the infrared image is identified and the feature enhancement processing is performed to obtain the degradation-compensated infrared features. Based on the structural feature correspondence between the infrared image and the high-resolution visible light image, spatial mapping is performed to generate a cross-modal confidence map, and visible light structural features are extracted based on the cross-modal confidence map. Based on the visible light structural features, the injection position and injection intensity in the infrared reconstruction process are analyzed, and combined with the degradation-compensated infrared features, visible light structural features, and features of adjacent infrared video frames, a fused reconstruction feature is constructed. Reconstruction is performed based on the fused reconstruction features. During the reconstruction process, thermal radiation consistency constraints are used to maintain the consistency of thermal distribution relationships, thermal gradient trends, and hotspot locations in the reconstructed image. Based on the reconstructed high-resolution infrared image, a degenerate back projection verification is performed, and the reconstruction result is corrected according to the verification result to obtain a high-resolution infrared reconstructed image and a corresponding regional confidence map.

2. The infrared video super-resolution reconstruction method based on spatiotemporal feature fusion according to claim 1, characterized in that, The construction of the degradation distribution map corresponding to the spatial location of the infrared image is specifically as follows: by using a composite degradation algorithm adapted to the real infrared imaging scene, multi-dimensional feature analysis is performed on the input infrared image to extract multi-scale infrared basic features, frequency domain degradation features, spatial anomaly degradation features and temporal degradation features respectively. Multi-dimensional features are fused and modeled using a preset degradation parameter fusion algorithm to output a spatial degradation distribution map covering all types of degradation. The degradation distribution maps of each type are stitched together by channel to generate a unified spatial degradation tensor. Based on the spatial degradation tensor, position-by-position and channel-by-channel modulation parameters are generated to perform degradation condition modulation on the basic infrared features to obtain the degradation-compensated infrared features.

3. The infrared video super-resolution reconstruction method based on spatiotemporal feature fusion according to claim 1, characterized in that, The generation of the cross-modal reliability map specifically involves: determining the global transformation matrix based on the dual-sensor calibration parameters and generating an initial mapping relationship; Based on the initial mapping relationship, the local residual displacement field is estimated, the global mapping error is corrected, and the target space mapping field is obtained. Based on the target space mapping field, the high-resolution visible light image and visible light structural feature sampling are mapped to the infrared image coordinate system to complete the dual-modal space alignment; An occlusion mask is generated through consistency verification of forward and reverse displacement fields. The bimodal feature consistency error and edge direction error are calculated to generate a registration confidence map. The formula is as follows: in, for Time coordinates The confidence map of the bimodal registration at the location. To cover the mask, This refers to the consistency error between forward and reverse mapping. For feature consistency error, This refers to the edge direction error. For spatiotemporal synchronization error, These are preset weighting coefficients.

4. The infrared video super-resolution reconstruction method based on spatiotemporal feature fusion according to claim 3, characterized in that, The extraction of visible light structural features specifically includes: visible light structural features include at least visible light image quality, cross-modal edge consistency, regional structure consistency, temporal structure support, and infrared degradation compensation requirements. The formula for calculating the quality of visible light images is as follows: in, for Time pixel position The corresponding overall quality reliability of visible light images, To enhance credibility, For clarity and credibility, For the sake of contrast credibility, For effective visibility and credibility; The formula for calculating cross-modal edge direction consistency is: in, For the first Layer features time Reliability of infrared-visible edge orientation consistency at location. For infrared images Layer features at the pixel The edge gradient vector at that point, For visible light images Layer features at the pixel The edge gradient vector at that point, For vector dot product operation, For L2 norm operations, It is a very small constant; The formula for calculating regional structural consistency is: in, for time Reliability of positional bimodal region structural consistency For infrared image pixels The local structural feature vector of the location, Visible light image pixels The local structural feature vector of the location; The formula for calculating the support of temporal structures is: in, for time Temporal structure support of location, The set of adjacent frame indices that participate in the timing calculation. This represents the total number of adjacent frames. For the present Real-time infrared images The structural characteristic response value of the location, For adjacent After inter-frame alignment, the frames are mapped to the infrared structural response values ​​of the current frame. For time series smoothing scaling coefficients; By integrating multi-dimensional evaluation indicators, registration confidence, and degradation confidence, a regional-level cross-modal confidence map is generated. Furthermore, considering infrared degradation compensation requirements, the target visible light feature gating weights are obtained, as shown in the formula: in, For cross-modal trusted gated weight graphs, for Time-region-level cross-modal reliability graph This is a graph showing the infrared degradation compensation requirement. The height and width dimensions of the feature map. For pixel-level point-by-point multiplication, the Hadamard product operator is used. Based on the aforementioned gated weights, reliable visible light structural features are screened by scale and frequency band.

5. The infrared video super-resolution reconstruction method based on spatiotemporal feature fusion according to claim 1, characterized in that, The analysis of the injection location and injection intensity during the infrared reconstruction process specifically involves: based on the cross-modal gating weights The system locates the injection position of visible light features and injects visible light structural features in regions where the gating weight is greater than a preset threshold. Different injection intensities are set for visible light features of different frequency bands. Among them, the mid-frequency structural features are controlled by full-domain gating weight, and the high-frequency structural features are constrained by dual thresholds of edge consistency and registration reliability. The controlled fusion of visible light features and infrared features is completed by using a residual fusion method.

6. The infrared video super-resolution reconstruction method based on spatiotemporal feature fusion according to claim 5, characterized in that, The construction of fused reconstruction features specifically involves: acquiring a sequence of consecutive infrared video frames before and after the current frame; performing degradation modulation preprocessing on the features of adjacent frames to suppress noise, stripes, and dead pixel interference; completing inter-frame alignment through global motion compensation and local dense motion field estimation; generating an occlusion mask based on bidirectional motion consistency verification; and generating regional temporal reliability by combining geometric reliability, thermal stability, degradation reliability, and temporal distance attenuation factor. Based on temporal reliability, the weights of adjacent frames are normalized, multi-frame sub-pixel features are adaptively aggregated, temporal detail residuals are extracted, and temporal compensation features, degradation compensation infrared features, and reliable visible light structure features are fused to obtain fused reconstruction features.

7. The infrared video super-resolution reconstruction method based on spatiotemporal feature fusion according to claim 1, characterized in that, The thermal radiation consistency constraint specifically includes: regional average thermal response constraint, thermal ranking consistency constraint, thermal gradient consistency constraint, and hotspot location consistency constraint. The formula for the regional average thermal response consistency constraint is: in, Let the regional heat mean consistency loss function be used. This represents the total number of local block regions in the image. For the first A local statistical image sub-region, The calculation is performed to determine the average pixel value of the region. For rebuilding output Real-time high-resolution infrared images, The original low-resolution infrared image corresponds to the first The true average thermal response reference value for each sub-region; The hot sorting consistency constraint formula is: Let the hot sorting consistency loss function be... This is a set of pixel region comparison sample pairs. This represents the total number of sample pairs. To preset the hot sorting interval margin, Symbols indicating the hot and cold relationships in the original image. , To reconstruct the first high-resolution image , The thermal response temperature characteristic value of the region; The formula for thermal gradient consistency constraint is: in, Let thermal gradient consistency loss function be used. For the spatial gradient solver, To reconstruct the low-frequency thermal gradient feature map of a high-resolution infrared image, This is a low-frequency thermal gradient reference feature map obtained by upsampling and matching the resolution of the original low-resolution infrared image. For norm-1 constraint operations; The formula for the consistency constraint between hotspot location and range is: in, Let the hotspot consistency loss function be used. This represents the actual hotspot mask region obtained after upsampling the original low-resolution infrared image. To reconstruct the hotspot mask region predicted from the high-resolution infrared image, It is a very small constant.

8. The infrared video super-resolution reconstruction method based on spatiotemporal feature fusion according to claim 1, characterized in that, The degradation backprojection verification specifically involves: constructing a real-time degradation operator based on the degradation parameters corresponding to the degradation distribution map. The initially reconstructed high-resolution infrared image is degraded and projected onto a low-resolution space to obtain a reconstructed low-resolution image. ,in, for Real-time dynamic degradation operator, For degenerate mapping operation functions, This is the complete set of infrared image degradation parameters estimated at the current moment. This is a reconstructed low-resolution infrared image obtained through degenerate back projection. The reconstructed low-resolution infrared image obtained through degenerate back projection; Calculate the back projection error between the original low-resolution infrared image and the reconstructed image, and iteratively correct the high-resolution image based on the back projection error. Repeat the iterative correction until the back projection error is lower than the preset error threshold or the number of iterations reaches the maximum preset number, then terminate the iterative optimization and output the optimal high-resolution infrared image with the highest accuracy and the smallest residual distortion.

9. The infrared video super-resolution reconstruction method based on spatiotemporal feature fusion according to claim 8, characterized in that, The regional-level credibility map specifically includes: extracting multi-dimensional uncertainty indicators such as degradation estimation credibility, cross-modal gating credibility, temporal fusion credibility, back projection error, and bi-branch reconstruction difference; The reliability of regional reconstruction is calculated using the exponential decay function, as follows: in, To enhance the credibility of regional-level reconstruction, These are the preset weighting coefficients. This is a plot of the upsampling back projection error. To reconstruct differences in the two branches, For the confidence plot of degradation estimation, For cross-modal visible light feature gating weights, For pixel-level timing synthesis reliability, To ensure the fidelity and reliability of infrared intrinsic features; Based on the confidence level of the image region types, including the original infrared observation support region, the adjacent frame compensation region, the confidence visible light guidance region, and the model prior low confidence region, a region-level confidence map is obtained.

10. An infrared video super-resolution reconstruction system based on spatiotemporal feature fusion, used to implement the infrared video super-resolution reconstruction method based on spatiotemporal feature fusion as described in any one of claims 1-9, comprising a data acquisition module, an infrared imaging degradation self-sensing module, an infrared-visible light dynamic registration module, a cross-modal trusted gating module, an infrared video temporal consistency fusion module, and a thermal radiation consistency constraint reconstruction module, as detailed below: The data acquisition module acquires low-resolution infrared video frame sequences, high-resolution visible light images, dual sensor calibration parameters, and real-time device status data. The infrared imaging degradation self-sensing module analyzes multi-dimensional degradation features, generates a comprehensive degradation parameter set and a global spatial degradation tensor, and outputs the degradation-compensated infrared features and degradation estimation confidence map. The infrared-visible light dynamic registration module generates a sub-pixel-level target space mapping field based on the global transformation matrix and the local residual displacement field, and outputs a registration confidence map and an occlusion mask. The cross-modal trustworthy gating module integrates multi-dimensional trustworthiness evaluation indicators to generate a cross-modal trustworthiness map and cross-modal trustworthy gating weights, and filters and outputs trustworthy visible light structural features; The infrared video temporal consistency fusion module calculates the inter-frame geometric alignment reliability and thermal stability to obtain pixel-level temporal comprehensive reliability, and adaptively aggregates multi-frame features to generate temporal compensation features. The thermal radiation consistency constraint reconstruction module outputs a high-resolution infrared reconstruction image and a regional reconstruction confidence map.

Images