Smoke image restoration method, system and device based on infrared structure prior guidance

Abstract

The present application relates to the technical field of image enhancement, in particular to a smoke image restoration method, system and device based on infrared structure prior guidance; the method of the present application firstly estimates the transmittance map and the atmospheric light based on the smoke RGB image and the infrared image, generates a coarse restoration image of stable condition prior; then, the multi-scale structure correction features of the infrared image are extracted, and the uncertainty perception gate is adaptively generated according to the direction similarity of the RGB and infrared features under each scale, the infrared structure injection is actively enhanced in the heavy smoke shielding area to make up for the missing information of the RGB, and unnecessary cross-modal interference is suppressed in the light smoke area; finally, the coarse restoration image is encoded into the latent space condition, and the Gaussian noise latent variable is taken as the initial state, the multi-scale structure features adjusted by the gate are combined with the latent space condition to carry out few-step probability flow sampling restoration, so that the clear visible light image reconstruction with high quality and high texture reality is realized while the low reasoning overhead is maintained.

Description

Technical Field

[0001] This invention relates to the field of image enhancement technology, specifically to a method, system, and device for smoke image restoration based on infrared structure prior guidance. Background Technology

[0002] In complex scenarios such as drone traffic monitoring, emergency inspection, and fire rescue, acquiring clear and reliable visual information is a crucial foundation for scene understanding and autonomous decision-making. However, in environments with fire smoke or heavy particulate matter, the air medium significantly scatters and absorbs visible light, leading to decreased contrast, color distortion, and loss of structural information in RGB images. This significantly weakens the robustness of downstream visual tasks such as object detection and semantic segmentation.

[0003] Existing research has promoted the development of visual enhancement for harsh environments from the perspectives of single-modal recovery, physical modeling, multimodal perception, and generative reconstruction, but the following shortcomings still exist: most methods still mainly rely on single-modal information, and their recovery capabilities are limited under dense smoke occlusion and real-world scene domain shifts; existing multimodal methods mostly focus on fusion or high-level visual tasks, lacking selective cross-modal modeling for smoke recovery; although generative methods have strong information completion capabilities, a balance still needs to be struck between inference efficiency and physical consistency; at the same time, real-world smoke scenes lack strictly paired ground truth, causing model evaluations to deviate from real-world application requirements. Summary of the Invention

[0004] The purpose of this invention is to provide a method, system, and device for smoke image recovery based on infrared structure prior guidance.

[0005] The technical solution of this invention is as follows: A smoke image recovery method based on infrared structure prior guidance includes the following operations: S1. Obtain the RGB image and the corresponding infrared image of smoke in the same scene; S2. Based on the smoke RGB image and infrared image, estimate the transmittance map and atmospheric light, and generate a coarse recovery image according to the atmospheric scattering model; obtain the multi-scale structural correction features of the infrared image, and generate corresponding scale cross-modal uncertainty perception gating according to the feature direction similarity between the smoke RGB image and the infrared image at each scale, so as to adaptively adjust the infrared structure injection intensity at the corresponding scale and obtain the multi-scale gating adjustment structure features. S3. The coarsely restored image is encoded into latent space conditions, and based on the probability flow model, with Gaussian noise latent variables as the initial state, combined with latent space conditions and multi-scale gating adjustment structure features, a few-step sampling restoration is performed to obtain a clear visible light image.

[0006] The method for obtaining multi-scale structural correction features in S2 is as follows: the infrared image is processed by a multi-scale VMamba encoder to obtain multi-scale infrared features; the infrared features of each scale are fused with time coding, mapped to the corresponding channel space with the same scale, and then processed by convolution to obtain multi-scale structural correction features.

[0007] The feature direction similarity between the RGB image with smoke and the infrared image in S2 is obtained based on the RGB control features of the RGB image with smoke and the structural correction features of the infrared image; the RGB control features are obtained by processing the RGB image with smoke using the ControlNet module.

[0008] The adaptive adjustment of infrared structural injection intensity in S2 is calculated using the following formula: , For the first l Scale-based gating control structure characteristics, For the RGB image with smoke l Scale-based RGB control features For infrared images l Scale-gated graphs For the first l Scale-based cross-modal uncertainty sensing gating These are the structural correction features after amplitude normalization and variance alignment. For the first l The scale injection coefficient.

[0009] The probability flow model in S3 constructs a linear path from data latent variables to Gaussian noise latent variables in the latent space, with the intermediate states and the target velocity field as follows: , , This is an intermediate state. For the latent variables corresponding to the clear target image, For normalized time variables, For Gaussian noise latent variables, For the target velocity field.

[0010] In S3, the operation of few-step sampling recovery is as follows: starting from the Gaussian noise latent variable, it is updated in reverse along the predicted velocity field through few-step Euler integration, gradually mapping back to the latent variable corresponding to the clear target image, and then decoded by the variational autoencoder to obtain the recovery result as a clear visible light image.

[0011] The coarsely restored image in S2 is calculated using the following formula: , For RGB images with smoke, This is a transmittance diagram. Atmospheric light, This is the lower limit of dynamic transmittance.

[0012] A smoke image restoration system based on infrared structure prior guidance, used to implement the aforementioned smoke image restoration method based on infrared structure prior guidance, includes: The scene image acquisition module acquires RGB images and corresponding infrared images of smoke in the same scene. The multi-scale gated adjustment structure feature generation module is used to estimate the transmittance map and atmospheric light based on the smoke RGB image and infrared image, generate a coarse recovery image according to the atmospheric scattering model, obtain the multi-scale structure correction features of the infrared image, and generate the corresponding scale cross-modal uncertainty perception gate according to the feature direction similarity between the smoke RGB image and the infrared image at each scale, so as to adaptively adjust the infrared structure injection intensity at the corresponding scale, and obtain the multi-scale gated adjustment structure features. The clear visible light image generation module encodes the coarsely restored image into latent space conditions, and based on the probability flow model, uses Gaussian noise latent variables as the initial state, combines latent space conditions and multi-scale gating adjustment structure features to perform a few-step sampling restoration to obtain a clear visible light image.

[0013] A smoke image restoration device based on infrared structure prior guidance includes a processor and a memory, wherein the processor executes a computer program stored in the memory to implement the above-described smoke image restoration method based on infrared structure prior guidance.

[0014] A computer-readable storage medium for storing a computer program, wherein the computer program, when executed by a processor, implements the above-described smoke image recovery method based on infrared structure prior guidance.

[0015] The beneficial effects of this invention are as follows: This invention provides a smoke image restoration method based on infrared structure prior guidance. First, it estimates transmittance maps and atmospheric light based on RGB and infrared images of smoke-covered areas, and generates a physically interpretable coarse restoration image according to an atmospheric scattering model. This explicitly incorporates the physical laws of atmospheric scattering into the restoration process, providing stable prior conditions consistent with the actual degradation mechanism for subsequent probabilistic flow models. This effectively avoids artifacts in dense smoke regions where the predicted collapse is inconsistent with the physical reality, generated by purely data-driven models. Then, by extracting multi-scale structural correction features from the infrared image and adaptively generating uncertainty-aware gating based on the directional similarity between RGB and infrared features at each scale, it actively enhances infrared structure injection in dense smoke-covered areas to compensate for missing RGB information, and suppresses inconsistencies in light smoke regions. Necessary cross-modal interference enables spatial adaptive selective compensation of infrared structural features, significantly improving the structural recovery stability and edge sharpness in heavily smoked areas. Finally, the coarsely recovered image is encoded into latent space conditions, and with Gaussian noise latent variables as the initial state, a few-step probability flow sampling recovery is performed by combining the latent space conditions with gated multi-scale structural features. Generative modeling completes the mapping from degenerate distribution to sharp distribution in the latent space, achieving high-quality, high-texture realism sharp visible light image reconstruction while maintaining low inference overhead. The method of this invention achieves significant improvements in structural integrity, texture realism, and cross-domain generalization ability in dense smoke-covered scenes through the triple synergy of physical prior guidance, structural adaptive injection, and generative latent space sampling. Attached Figure Description

[0016] The solutions and advantages of this application will become clear to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the scope of the invention.

[0017] In the attached diagram: Figure 1 This is a visualization comparison of different methods on the synthetic pairing test set, as shown in the example. Figure 2 This example illustrates a visual comparison of different methods in a real, unpaired smoke scenario. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of the exemplary embodiments of this application clearer, the technical solutions in the exemplary embodiments of this application are described clearly and completely below. Obviously, the described exemplary embodiments are only some embodiments of this application, and not all embodiments.

[0019] This embodiment provides a smoke image recovery method based on infrared structure prior guidance, including the following operations: S1. Obtain the RGB image and the corresponding infrared image of smoke in the same scene; S2. Based on the smoke RGB image and infrared image, estimate the transmittance map and atmospheric light, and generate a coarse recovery image according to the atmospheric scattering model; obtain the multi-scale structural correction features of the infrared image, and generate corresponding scale cross-modal uncertainty perception gating according to the feature direction similarity between the smoke RGB image and the infrared image at each scale, so as to adaptively adjust the infrared structure injection intensity at the corresponding scale and obtain the multi-scale gating adjustment structure features. S3. Encode the coarsely restored image into latent space conditions, and based on the probability flow model, with Gaussian noise latent variables as the initial state, combine the latent space conditions and multi-scale gating adjustment structure features to perform a few-step sampling restoration to obtain a clear visible light image. The specific steps are detailed below.

[0020] S1. Obtain RGB images and corresponding infrared images of the same scene with smoke. Utilize the natural advantages of infrared images in smoke environments, such as strong penetration and structural stability, to compensate for the inherent defects of visible light in the absence of structural information under dense smoke, thus providing crucial cross-modal complementary information for subsequent recovery.

[0021] S2. Based on the smoke RGB image and infrared image, estimate the transmittance map and atmospheric light, and generate a coarse recovery image according to the atmospheric scattering model; obtain the multi-scale structural correction features of the infrared image, and generate corresponding scale cross-modal uncertainty perception gating according to the feature direction similarity between the smoke RGB image and the infrared image at each scale, so as to adaptively adjust the infrared structural injection intensity at the corresponding scale and obtain the multi-scale gating adjustment structural features.

[0022] First, based on the RGB and infrared images with smoke, the transmittance map and atmospheric light are estimated, and a coarse reconstruction image is generated according to the atmospheric scattering model.

[0023] After obtaining the transmittance map and atmospheric light, the coarsely reconstructed image is obtained. It is calculated using the following formula: , For RGB images with smoke, This is a transmittance diagram. Atmospheric light, A dynamic transmittance lower limit is used to avoid numerical instability in extremely dense smoke areas. For nighttime or low-brightness areas, a dark area noise suppression strategy is further introduced to reduce the risk of erroneously amplified noise. In this embodiment, the coarsely restored image is not directly used as the final output, but rather as a physically interpretable conditional prior to constrain subsequent probabilistic flow restoration.

[0024] Then, multi-scale structural correction features of the infrared image are obtained. Specifically, the infrared image is processed by a multi-scale VMamba encoder to obtain multi-scale infrared features. The infrared features of each scale are fused with the temporal code, mapped to the corresponding channel space with the same scale, and then processed by convolution to obtain multi-scale structural correction features.

[0025] The multi-scale structural correction feature is calculated using the following formula: , , For the first l Scale-based structural correction features. For the first l Scale fusion characteristics () is the first l Scale-based channel mapping and convolution processing, For the first l Infrared characteristics at the scale, For time encoding, For channel projection processing, This is for time-encoded mapping processing.

[0026] Next, since RGB control features typically lack reliable structure in heavily smoked areas, while infrared features can provide supplementary clues; and in lightly degraded areas, excessive infrared feature injection may suppress RGB textures. Therefore, this embodiment designs a cross-modal uncertainty-aware gating system based on the feature direction similarity between the RGB and infrared images with smoke at each scale, to adaptively adjust the infrared structure injection intensity.

[0027] The cross-modal uncertainty sensing gating is calculated using the following formula: , , For the first l Scale-based cross-modal uncertainty sensing gating To compare RGB images and infrared images with smoke at different scales The feature orientation similarity is obtained based on the RGB control features of the smoky RGB image and the structural correction features of the infrared image; the RGB control features are obtained by processing the smoky RGB image using the ControlNet module. For the RGB image with smoke l Scale-based RGB control features For infrared images l Scale-based structural correction features. Operation for 00 It is a very small constant term.

[0028] Finally, based on cross-modal uncertainty-aware gating, the infrared structural injection intensity at corresponding scales is adaptively adjusted to obtain multi-scale gated structural features. Specifically, when... When the value is large, it indicates that the RGB control features and the structural correction features of the infrared image are relatively consistent in direction, and the RGB branch can provide a relatively reliable structural response. Smaller, reducing infrared injection intensity; when When the values ​​are low or even negative, it indicates a significant difference in the structural responses of the two modes. The RGB features may be affected by smoke occlusion or degradation. Increase the size, thereby enhancing infrared structural compensation.

[0029] The above-mentioned adaptive adjustment of infrared structural injection intensity is calculated using the following formula: , , For the first l Scale-based gating control structure characteristics, For infrared images l Scale-gated graphs For the first l Spatial gating prediction processing at scale Processed using the sigmoid activation function. For the first l Scale-based cross-modal uncertainty sensing gating These are the structural correction features after amplitude normalization and variance alignment. For the first l The scale injection coefficient is calculated based on the spatial location and degree of degradation. This formula can dynamically utilize infrared information to enhance structure recovery in heavily smoked regions, while avoiding the over-smoothing texture problem caused by indiscriminate cross-modal fusion.

[0030] S3. The coarsely restored image is encoded into latent space conditions, and based on the probability flow model, with Gaussian noise latent variables as the initial state, combined with latent space conditions and multi-scale gating adjustment structure features, a few-step sampling restoration is performed to obtain a clear visible light image.

[0031] The probabilistic flow model constructs a linear path from latent data variables to latent Gaussian noise variables in the latent space, with the intermediate states and the target velocity field as follows: , , It is Gaussian noise. For the latent variables corresponding to the clear target image, For normalized time variables, For Gaussian noise latent variables, Let be the target velocity field. Given... With cross-modal conditions c Predicting the velocity field under these conditions This allows us to learn the transport direction from the data distribution to the noise distribution in the latent space.

[0032] The coarse-reconstructed image is mapped to latent space conditions by a conditional encoder, which serve as conditional cues to guide velocity field prediction. The initial state of the probability flow in the probabilistic flow model is a Gaussian noise latent variable, rather than the latent variable of the coarse-reconstructed image. This allows the model to perform generative recovery guided by physical priors, rather than a simple deterministic mapping. Simultaneously, multi-scale infrared structure correction features, adjusted by uncertainty-aware gating, are injected into the corresponding levels of the recovery backbone, enhancing structure recovery capabilities in heavily smoked regions.

[0033] In the inference phase, the probabilistic flow model uses Gaussian noise latent variables as the initial state. Combining latent space conditions and multi-scale gating adjustment structure features, it performs few-step sampling recovery. The latent variables are updated backward along the predicted velocity field through few-step Euler integration, gradually mapping back to the latent variables corresponding to the clear target image. The recovered result is then decoded by a variational autoencoder, serving as the clear visible light image. The formula for calculating the updated latent variables is: , For the first i +1 latent variable corresponding to sampling time step For the first i The latent variables corresponding to each sampling time step The integration step size is the time interval between two adjacent sampling time steps. For the probability flow model at time step Based on the current latent variables and condition information The predicted velocity field, For the first i Each sampling time step It gradually decreases from 1 to 0.

[0034] The objective of the probabilistic flow model is to constrain the model to learn the linear velocity field from the noisy latent variables to the data latent variables. The probabilistic flow loss is: , For the velocity field predicted by the model, For the target velocity field.

[0035] Furthermore, to refine the training of the probabilistic flow model, in addition to the probabilistic flow loss, image domain structural constraints for transmittance and atmospheric light are introduced for supervised prediction, physical consistency constraints for supervised rendering of smoky images and constraining them to be consistent with the input degraded image, edge constraints for enhancing edge structure, perceptual constraints for improving perceptual distance, color constraints for mitigating color deviation, and frequency domain constraints for constraining frequency domain amplitude distribution. These constraints are then weighted and summed to construct the total training loss. Simultaneously, a course-based learning strategy is adopted during the training process. The first stage (early stage) primarily uses the probabilistic flow loss to stabilize the latent space velocity field; the second stage (mid-stage) introduces physical consistency and structural constraints; and the third stage (late stage) gradually incorporates perceptual loss, edge loss, color loss, and frequency domain loss.

[0036] To verify the effectiveness of the method in this embodiment, the following experiment was conducted.

[0037] Dataset and Evaluation Metrics. The source domain training data was constructed based on the DroneVehicle dataset released by Tianjin University. This dataset contains visible light RGB images and corresponding infrared images from the perspective of UAVs, providing basic image pairs for RGB-IR cross-modal smoke recovery. Since some images in the original data contain invalid white border regions for annotation or registration, the experiment first simultaneously cropped the RGB images and their corresponding IR images during the data preprocessing stage to remove invalid boundary regions and maintain the spatial correspondence between the two modalities. Subsequently, the cropped clear RGB images were used as the supervision target, and smoke-bearing RGB images were synthesized online based on an atmospheric scattering model and a random smoke degradation process, forming ternary training samples. During the training stage, clear RGB images were used as the supervision target, and the synthesized smoke-bearing RGB images and their corresponding infrared images were used as inputs to learn the cross-modal recovery mapping. All input images were uniformly adjusted to 256×256 resolution before training and testing. The training set contains 12,000 RGB-IR-GT image pairs, and the training and test set samples do not overlap.

[0038] To evaluate the model's recovery capability under different smoke concentrations, a fixed synthetic paired test set was constructed in the experiments. This test set contained 1000 images, with 500, 300, and 200 images for light, dense, and very dense smoke concentrations, respectively. The synthetic test set had corresponding sharp ground truth (GT), therefore PSNR, SSIM, and LPIPS were used as full-reference evaluation metrics. PSNR and SSIM were used to measure pixel-level error and structural similarity, while LPIPS evaluated perceptual similarity based on depth feature distance. For real unpaired data, BRISQUE, NIQE, and MUSIQ were used as no-reference evaluation metrics. BRISQUE and NIQE modeled image quality based on statistical modeling of natural scenes, while MUSIQ utilized a multi-scale Transformer to model overall perceptual quality.

[0039] Since it is difficult to obtain strictly paired smoke-free reference images for real-world smoke scenes, the experiment further constructed a real unpaired RGB-IR test set to analyze the model's cross-domain generalization ability. The real test set contains 290 RGB-IR image pairs, of which 30 pairs with good spatial correspondence were selected from the RGBT-3M dataset, and 260 RGB-IR image pairs were supplemented from the FLAME3 dataset. The real test set is only used for no-reference metric evaluation and visualization analysis and is not used for model training.

[0040] The recovery performance of this embodiment on the synthetic paired test set is shown in Table 1. Table 1 presents the full reference index results under different smoke concentrations. Higher PSNR and SSIM are better, and lower LPIPS are better. Figure 1 This paper presents a visual comparison of the proposed method with several existing state-of-the-art methods (including FlowIE, FFA-Net, and DehazeFormer) on a synthetic pairing test set. The synthetic pairing test results are mainly used to analyze the gain of the proposed cross-modal prior on FlowIE. At the same time, the performance of the model in cross-domain smoke recovery is further discussed in conjunction with real unpaired scenarios.

[0041] Table 1. Comparison of full reference indices for different methods at different smoke concentrations on the synthetic paired test set.

[0042] As shown in Table 1, compared to FlowIE, the method in this embodiment exhibits better PSNR, SSIM, and LPIPS in overall, light smoke, dense smoke, and extremely dense smoke scenarios, indicating that infrared-assisted physical coarse recovery and cross-modal structural injection can effectively improve the adaptability in smoke recovery tasks. Figure 1As can be seen, in dense smoke scenes, the method in this embodiment achieves more stable road boundaries and local structures compared to FlowIE. In very dense smoke scenes, the road, vehicle, and green belt structures in the RGB input are severely occluded, and FlowIE still exhibits significant fogging, while FFA-Net and DehazeFormer show structural artifacts or texture discontinuities in local areas. In contrast, the method in this embodiment can maintain a more continuous scene layout by leveraging infrared structural priors, indicating that cross-modal structure injection helps improve the structural recovery stability in heavily smoked regions.

[0043] In addition, for evaluation of real-world smoke scenes, the experiment selected samples with good RGB-IR spatial correspondence from the RGBT-3M dataset. Because some infrared images in this dataset have obvious black borders, missing fields of view, or partial occlusion, making reliable spatial alignment with their corresponding RGB images difficult, the experiment did not use all samples from this dataset. Considering the limited number of RGBT-3M samples after selection, the experiment further supplemented the dataset with RGB-IR image pairs from the FLAME3 dataset to enhance the scene diversity of the real test set. The final real test set contains 290 RGB-IR image pairs, with 30 pairs selected from RGBT-3M and 260 pairs added from FLAME3. This test set is only used for evaluation without reference metrics and visualization analysis and is not used for model training.

[0044] Since there are no strictly paired smoke-free ground planes for real-world mountain smoke scenes, IR does not use full-reference metrics such as PSNR and SSIM for evaluation. Instead, it employs BRISQUE, NIQE, and MUSIQ to assess the no-reference quality of the LQ input image and the restored result. Lower BRISQUE and NIQE values ​​indicate better natural statistical quality, while higher MUSIQ values ​​indicate better overall perceptual quality. Considering the varying sensitivities of different no-reference metrics to smoothness, natural statistics, color, and texture, the experiment combines metric results with visualization to comprehensively analyze the real-world scene restoration performance.

[0045] In real-world unpaired smoke scenes, due to the lack of strictly corresponding smoke-free reference images, BRISQUE, NIQE, and MUSIQ were used in the experiment to evaluate the no-reference visual quality of the method in this embodiment and existing methods (FFA-Net, FlowIE). The results are shown in Table 2. The mean values ​​of BRISQUE, NIQE, and MUSIQ for the original smoke input were 12.193, 4.331, and 51.499, respectively. Compared with the original input, FFA-Net can improve the natural statistics to some extent. Specifically, FFA-Net reduced BRISQUE from 12.193 to 11.220 and NIQE from 4.331 to 4.190, indicating that it can mitigate some smoke degradation by enhancing local contrast and edge response. However, from the visualization results, see... Figure 2 The output of FFA-Net exhibits a significant color temperature shift, with the overall restored result leaning towards a warm tone. Some smoke and background areas display unrealistic yellow or brown enhancements. While this color compensation may improve local natural statistical features, it disrupts the color consistency and visual naturalness of the real scene, resulting in a significant drop in its MUSIQ from 51.499 to 46.067. This phenomenon indicates that unimodal dehazing networks in realistic smoke scenes tend to degenerate the smoke removal process into global color enhancement or contrast enhancement, making it difficult to stably distinguish between smoke, background texture, and true colors.

[0046] Table 2. No-reference image quality assessment results of different methods in real-world unpaired smoke scenes.

[0047] In contrast, FlowIE exhibits strong cross-domain instability in real-world scenarios, with its BRISQUE, NIQE, and MUSIQ scores deteriorating to 41.194, 16.846, and 16.718, respectively. This indicates that relying solely on the original latent space generative recovery model is insufficient for direct generalization to complex real-world smoke scenes. The method in this embodiment achieves the best result on the BRISQUE metric, reducing it from 12.193 to 9.999, a reduction of 2.194. Compared to FFA-Net's 11.220, this embodiment's method reduces the score by 1.2217, demonstrating that the infrared-assisted physical coarse recovery and cross-modal structure injection mechanisms proposed in this embodiment can more effectively mitigate the macroscopic degradation caused by real-world smoke, improving overall image visibility and structural clarity.

[0048] It should be noted that the lack of a reference metric in real-world unpaired scenes is not entirely equivalent to smoke removal effectiveness. BRISQUE and NIQE focus more on the natural statistical distribution of images, while MUSIQ leans more towards overall visual aesthetics and photographic perceived quality. Therefore, a model that preserves more original colors and textures may achieve a higher MUSIQ, but this does not necessarily mean more effective smoke suppression. The method in this embodiment achieves the best results on BRISQUE, indicating that it has a stronger ability to suppress the macroscopic degradation caused by real smoke.

[0049] Depend on Figure 2 As can be seen, FlowIE exhibits significant cross-domain instability in realistic smoke scenes, with the recovery results prone to over-smoothing and structural damage, making it difficult to preserve the contours of mountains, vegetation, and roads in the real scene. FFA-Net can enhance local contrast, but the output is prone to warm color shifts, with some background areas being unnaturally enhanced to yellow or brown. In contrast, the method in this embodiment, by leveraging infrared structural priors and uncertainty-aware injection, better preserves the main scene structure while mitigating the impact of smoke, demonstrating that cross-modal physics and structural priors contribute to improving visibility and recovery stability in realistic unpaired smoke scenes.

[0050] This embodiment also provides a smoke image restoration system based on infrared structure prior guidance, used to implement the above-mentioned smoke image restoration method based on infrared structure prior guidance, including: The scene image acquisition module acquires RGB images and corresponding infrared images of smoke in the same scene. The multi-scale gated adjustment structure feature generation module is used to estimate the transmittance map and atmospheric light based on the smoke RGB image and infrared image, generate a coarse recovery image according to the atmospheric scattering model, obtain the multi-scale structure correction features of the infrared image, and generate the corresponding scale cross-modal uncertainty perception gate according to the feature direction similarity between the smoke RGB image and the infrared image at each scale, so as to adaptively adjust the infrared structure injection intensity at the corresponding scale, and obtain the multi-scale gated adjustment structure features. The clear visible light image generation module encodes the coarsely restored image into latent space conditions, and based on the probability flow model, uses Gaussian noise latent variables as the initial state, combines latent space conditions and multi-scale gating adjustment structure features to perform a few-step sampling restoration to obtain a clear visible light image.

[0051] This embodiment also provides a smoke image restoration device based on infrared structure prior guidance, including a processor and a memory, wherein the processor executes the computer program stored in the memory to implement the above-described smoke image restoration method based on infrared structure prior guidance.

[0052] This embodiment also provides a computer-readable storage medium for storing a computer program, wherein the computer program, when executed by a processor, implements the above-described smoke image recovery method based on infrared structure prior guidance.

[0053] This embodiment provides a smoke image restoration method based on infrared structure prior guidance. First, it estimates transmittance maps and atmospheric light based on RGB and infrared images of smoke-covered areas. Then, it generates a physically interpretable coarse restoration image according to an atmospheric scattering model, explicitly incorporating the physical laws of atmospheric scattering into the restoration process. This provides stable conditional priors consistent with the actual degradation mechanism for subsequent probabilistic flow models, effectively avoiding artifacts in dense smoke regions where predicted collapse is inconsistent with physical reality. Next, it extracts multi-scale structural correction features from the infrared image and adaptively generates uncertainty-aware gating based on the directional similarity between RGB and infrared features at each scale. In dense smoke-covered areas, it actively enhances infrared structure injection to compensate for missing RGB information, while in light smoke areas, it suppresses... Necessary cross-modal interference enables spatial adaptive selective compensation of infrared structural features, significantly improving the structural recovery stability and edge sharpness in heavily smoked areas. Finally, the coarsely recovered image is encoded into latent space conditions, and with Gaussian noise latent variables as the initial state, a few-step probability flow sampling recovery is performed by combining the latent space conditions with gated multi-scale structural features. Generative modeling completes the mapping from degenerate distribution to sharp distribution in the latent space, achieving high-quality, high-texture realism sharp visible light image reconstruction while maintaining low inference overhead. The method of this invention, through the triple synergy of physical prior guidance, structural adaptive injection, and generative latent space sampling, achieves significant improvements in structural integrity, texture realism, and cross-domain generalization ability in dense smoke-covered scenes.

[0054] While exemplary embodiments of the invention have been described herein, many other variations or modifications conforming to the principles of the invention can be directly determined or derived from the disclosure of this invention without departing from its spirit and scope. Therefore, the scope of the invention should be understood and recognized to cover all such other variations or modifications.

Claims

1. A smoke image restoration method based on infrared structural prior guidance, characterized in that, This includes the following operations: S1. Obtain the RGB image and the corresponding infrared image of smoke in the same scene; S2. Based on the RGB and infrared images with smoke, estimate the transmittance map and atmospheric light, and generate a coarse reconstruction image according to the atmospheric scattering model; Infrared images are processed by a multi-scale VMamba encoder to obtain multi-scale infrared features; The infrared features at each scale are fused with the temporal code, mapped to the corresponding channel space with the same scale, and then processed by convolution to obtain multi-scale structure-corrected features. Based on the feature direction similarity between the smoke RGB image and the infrared image corresponding to each scale, a cross-modal uncertainty perception gate is generated for the corresponding scale, which is used to adaptively adjust the infrared structure injection intensity. The cross-modal uncertainty sensing gating is calculated using the following formula: , , For the first l Scale-based cross-modal uncertainty sensing gating To compare RGB images and infrared images with smoke at different scales The feature orientation similarity is obtained based on the RGB control features of the smoky RGB image and the structural correction features of the infrared image; the RGB control features are obtained by processing the smoky RGB image using the ControlNet module. For the RGB image with smoke l Scale-based RGB control features For infrared images l Scale-based structural correction features. for and The inner product, For L2 norm processing, Operation for 00 For constant terms; based on cross-modal uncertainty sensing gating, the infrared structure injection intensity at corresponding scales is adaptively adjusted to obtain multi-scale gating adjustment structure features; The adaptive adjustment of infrared structural injection intensity is calculated using the following formula: , , For the first l Scale-based gating control structure characteristics, For infrared images l Scale-gated graphs For the first l Spatial gating prediction processing at scale Processed using the sigmoid activation function. For the first l Scale-based cross-modal uncertainty sensing gating These are the structural correction features after amplitude normalization and variance alignment. For the first l The scale injection coefficient; S3. The coarsely restored image is encoded as a latent space condition, and based on the probability flow model, with Gaussian noise latent variables as the initial state, combined with the latent space condition and multi-scale gating adjustment structure features, a few-step sampling restoration is performed to obtain a clear visible light image.

2. The smoke image restoration method based on infrared structure prior guidance according to claim 1, characterized in that, The probability flow model in S3 constructs a linear path from data latent variables to Gaussian noise latent variables in the latent space, with the intermediate states and the target velocity field as follows: ， ， This is an intermediate state. For the latent variables corresponding to the clear target image, For normalized time variables, For Gaussian noise latent variables, For the target velocity field.

3. The smoke image restoration method based on infrared structure prior guidance according to claim 1, characterized in that, In S3, the operation of few-step sampling recovery is as follows: starting from the Gaussian noise latent variable, it is updated in reverse along the predicted velocity field through few-step Euler integration, gradually mapping back to the latent variable corresponding to the clear target image, and then decoded by the variational autoencoder to obtain the recovery result as a clear visible light image.

4. The smoke image restoration method based on infrared structure prior guidance according to claim 1, characterized in that, In S2, the coarsely restored image is calculated using the following formula: ， For RGB images with smoke, This is a transmittance diagram. Atmospheric light, This is the lower limit of dynamic transmittance.

5. A smoke image restoration system based on infrared structure prior guidance, used to implement the smoke image restoration method based on infrared structure prior guidance as described in claim 1, characterized in that, include: The scene image acquisition module acquires RGB images and corresponding infrared images of smoke in the same scene. The multi-scale gated adjustment structure feature generation module is used to estimate the transmittance map and atmospheric light based on the smoke RGB image and infrared image, generate a coarse recovery image according to the atmospheric scattering model, obtain the multi-scale structure correction features of the infrared image, and generate the corresponding scale cross-modal uncertainty perception gate according to the feature direction similarity between the smoke RGB image and the infrared image at each scale, so as to adaptively adjust the infrared structure injection intensity at the corresponding scale, and obtain the multi-scale gated adjustment structure features. The clear visible light image generation module encodes the coarsely restored image into latent space conditions, and based on the probability flow model, uses Gaussian noise latent variables as the initial state, combines latent space conditions and multi-scale gating adjustment structure features to perform a few-step sampling restoration to obtain a clear visible light image.

6. A smoke image restoration device based on infrared structural prior guidance, characterized in that, It includes a processor and a memory, wherein the processor executes a computer program stored in the memory to implement the smoke image restoration method based on infrared structure prior guidance as described in any one of claims 1-4.

7. A computer-readable storage medium, characterized in that, Used to store a computer program, wherein the computer program, when executed by a processor, implements the smoke image recovery method based on infrared structure prior guidance as described in any one of claims 1-4.

Images