An image restoration method under infrared guided non-uniform scattering medium interference

Abstract

The application discloses an infrared-guided image restoration method under non-uniform scattering medium interference, comprising the following steps: (1) constructing a progressive guided aggregation module; (2) designing a differential amplification attention module; (3) constructing a structure-guided enhancement module; and (4) designing a physical coupling loss. In view of the uneven distribution of interference areas and the image feature redundancy problem, the application effectively combines the infrared image structure information and the visible light texture expression ability, combines the multi-modal complementary information and the physical priori knowledge, and realizes the adaptive enhancement and structure fine restoration of the visible light image under the non-uniform scattering medium interference. The method has excellent performance on the non-uniform interference dataset, effectively alleviates the problems of structure loss, color deviation and modal redundancy, provides a high-robustness solution for image enhancement in complex environments such as dust, sand and water mist, and provides a new direction for the research of the infrared-guided image restoration method.

Description

Technical Field

[0001] This invention belongs to the field of computer vision and image processing technology, and particularly relates to an image restoration method under infrared-guided non-uniform scattering medium interference. Background Technology

[0002] In urban and industrial fields such as complex weather perception, road monitoring, and metallurgical production, visual data acquired by image sensing devices is often interfered with by non-uniform scattering media such as fog, haze, sandstorms, and dust. The perception quality directly affects subsequent image understanding, target detection, and production safety. In practical applications, due to the combined influence of multiple factors such as the dynamic distribution of non-uniform scattering media, changes in environmental conditions, and differences in device imaging, image degradation exhibits stable characteristics such as strong spatial variations and complex blurring properties. This leads to a significant decrease in image clarity and contrast, thereby affecting the reliability of automated identification and monitoring systems.

[0003] Considering the widespread and typical nature of dust in non-uniform scattering media, this paper takes image dust removal as a representative task to study the image degradation problem caused by such media. Although non-uniform scattering media differ in composition and distribution, they all exhibit significant scattering effects during optical imaging, causing similar impacts on image quality. Therefore, it is feasible to construct a universal image restoration method around the dust removal problem.

[0004] Currently, image deduplication methods mainly fall into two categories: deduplication methods based on single-modal image information and deduplication methods assisted by multimodal data.

[0005] Dust removal methods based on single-modal image information utilize the statistical properties of the image itself or learn priors to model the degradation process, and improve image quality through deep learning networks or physical models. They offer advantages such as end-to-end processing and high efficiency, but struggle to adapt to severely degraded image scenarios, and color restoration is prone to distortion. The generalization ability of these methods remains a challenge. For example, patent CN114881875B, entitled "Single Image Dehazing Network and Dehazing Method Based on U-Net Structure and Residual Network," constructs an end-to-end dehazing network based on U-Net and residual modules. It enhances feature extraction capabilities through smoothed dilated convolutional residual blocks and a feature weighted summation module, achieving good dehazing results on synthetic foggy datasets. However, this method relies solely on visible light image information, without incorporating auxiliary sensor data such as infrared as guiding information, lacking multimodal perception capabilities. In complex foggy scenarios, it struggles to effectively restore texture details and spatial structure information, limiting the model's application in real industrial environments.

[0006] Multimodal data-assisted dust removal methods introduce additional modalities, such as infrared images, as aids. These methods enhance feature reconstruction through information guidance or fusion, achieving some success in complex scenarios. However, they still have shortcomings in multimodal information fusion strategies and guidance accuracy, making it difficult to accurately locate and effectively enhance key structures. This leads to problems such as limited ability to model non-uniform interference and image artifacts. For example, patent CN118781018B, entitled "A Deep Learning-Based Infrared Image-Assisted Image Dehazing Method," introduces infrared images as a priori information on haze density and combines Transformer and an improved CNN network to design a channel-spatial dual attention mechanism, demonstrating good dehazing performance under uniform haze conditions. However, this model is mainly designed for idealized uniform dust scenarios and does not fully consider complex factors such as uneven dust concentration distribution and dynamic particulate matter interference in actual industrial environments. It lacks robust modeling for non-uniform haze, resulting in limited fusion performance in complex industrial scenarios such as real blast furnaces.

[0007] From the perspective of modal characteristics, visible light images and infrared images are somewhat complementary in terms of sensing interference areas: the former retains rich texture and color information, which is beneficial for detail restoration and visual consistency enhancement, but is sensitive to low visibility areas; the latter has a stable advantage in penetrating interference and maintaining structure, and can stably perceive key targets under strong interference conditions.

[0008] Therefore, constructing a well-structured, clearly guided, and information-complementary image restoration strategy, giving full play to the regional guidance capability of infrared images for visible light images, and achieving dynamic perception and adaptive enhancement of non-uniform interference are of great significance for improving the quality of restored images. Summary of the Invention

[0009] To address the aforementioned problems in existing technologies, this invention aims to propose an image restoration method under infrared-guided non-uniform scattering medium interference. To effectively model scattering degradation characteristics and improve the structural restoration capability of key image regions, a progressively guided aggregation module is constructed. A physical-mask coupler, consisting of a multi-layered dual-branch physical perception module and a soft-mask modulation unit, is embedded level by level. The guiding network perceives scattering distribution characteristics at different semantic levels. The dual-branch physical perception module generates atmospheric light maps, transmittance maps, and intermediate restored images. The soft-mask modulation unit replaces traditional skip connections, enabling display adjustment of non-uniform scattering regions, alleviating over-repair in weak interference regions, and effectively preserving image details. Addressing the differences in perceiving structural and texture representations between infrared and visible light images in interference regions of varying intensities, a differential amplification attention module and a structural guidance enhancement module are designed. The former constructs a dual-path modulation mechanism (channel and spatial) to dynamically mine inter-modal differences and enhance cross-modal guidance. The latter adaptively amplifies or suppresses differential features based on the interference intensity estimated from the transmittance map, ensuring that high-concentration interference regions preferentially fuse infrared structural information while low-concentration regions do not overly rely on guidance, thus improving the accuracy and targeting of the overall restoration effect. To ensure consistency between the network output and the physical model, a physical coupling loss is constructed, and the atmospheric scattering model is embedded into the end-to-end training process. This collaboratively optimizes the transmittance map, atmospheric light map, and reconstructed image, maintaining physical consistency among the three under weak supervision, thereby enhancing the network's generalization ability and interpretability. The image restoration method proposed in this invention combines infrared guidance with physical consistency constraints, balancing structural restoration and information integrity. It provides a new path for physical modeling and cross-modal collaboration in image restoration tasks under non-uniform interference environments such as dust, sand, and water mist.

[0010] To achieve the above objectives, the solution of the present invention is to propose an image restoration method under infrared-guided non-uniform scattering medium interference, comprising the following steps:

[0011] (1) Construct a progressively guided aggregation module and embed a physical-mask coupler consisting of a dual-branch physical sensing module and a soft mask modulation unit step by step. The dual-branch physical sensing module guides the network to perceive scattering characteristics at different semantic levels, and the soft mask modulation unit realizes adaptive adjustment of the interference region.

[0012] (2) Design a differential amplification attention module to extract the difference information between infrared and visible light modes from the two dimensions of channel and space, obtain the infrared features after differential enhancement, realize fine modeling of mode differences, and improve the quality of infrared guidance.

[0013] (3) Construct a structure-guided enhancement module, use the multi-scale features of the transmittance map to control the amplification or suppression of modal difference information, finely adjust the dependence of different interference regions on infrared information, and complete the structure enhancement and selective fusion of information under modal guidance.

[0014] (4) Design physical coupling loss to achieve coordinated optimization of the transmittance map, atmospheric light intensity map and restored image, and ensure the overall consistency between the final output image and the scattering physical process.

[0015] As one embodiment of the present invention, the specific implementation scheme of the present invention is as follows:

[0016] The (1) construction of the progressive guided aggregation module:

[0017] In the presence of non-uniform scattering media, traditional dust removal methods often result in the loss of details in thin interference areas. This invention proposes a progressive guided aggregation module, which achieves progressive sensing and suppression through step-by-step stacking. Multiple physical-mask couplers are designed for each stage to provide local adaptive enhancement in non-uniform regions. The specific steps are as follows:

[0018] Step 1: The original interference map is progressively guided through the first level of the aggregation module. Input convolution to obtain initial visible light image features:

[0019]

[0020] in, Represents the original interference image; This indicates the first-order visible light characteristic.

[0021] Step 2: Input the visible light features into the next level module step by step. Each level module contains multiple physical-mask couplers:

[0022]

[0023]

[0024] in, This indicates the progressive bootstrap aggregation module. class, =2,3,4; Indicates the first Level visible light characteristics; Indicates module cascading; Indicates the first Level 1 A physical-mask coupler.

[0025] Step 3: Within each physical-mask coupler, a dual-branch physical sensing module is designed based on an atmospheric scattering model to estimate atmospheric light. Transmittance diagram and intermediate reconstruction image , and by A soft mask is generated by a soft mask modulation unit to mitigate over-fixing caused by global consistency processing.

[0026]

[0027]

[0028]

[0029] in, This indicates a dual-branch physical sensing module; Indicates the first Level 1 Visible light characteristics output by a physical coupler; Indicates a soft mask modulation unit; This indicates a soft mask.

[0030] The differential amplification module (2) is designed as follows:

[0031] Step 1: Extract infrared features from the infrared image using a single-layer convolution:

[0032]

[0033] in, Represents the original infrared image; Indicates infrared characteristics; This represents a single-layer convolution.

[0034] Step 2: Design the differential amplifier module based on the differential amplifier principle:

[0035]

[0036] in, Indicates a differential signal; Indicates common-mode signal; This represents the differential gain, which controls the degree of amplification of differences. This represents the common-mode gain, which controls the proportion of redundant information retained. This invention focuses on amplifying differential-mode signals and retains only the first term.

[0037] Step 3: Construct a differential mode signal using infrared and first-order visible light features:

[0038]

[0039] right Focus on channel direction to guide selective channel amplification:

[0040]

[0041]

[0042] in, Indicates the channel amplification factor; This indicates channel difference enhancement features; This indicates channel-wise global average pooling; Indicates a fully connected layer; This represents the activation function.

[0043] right Focusing on spatial orientation guides the response of texture details:

[0044]

[0045]

[0046] in: Indicates the spatial magnification factor; Indicates spatial difference enhancement features; This indicates pixel-wise average pooling; Represents convolution; This represents the activation function.

[0047] Step 4: Enhance Channel Difference Features With spatial augmentation results Coupling, simulating the nonlinear fused output after dual-path modulation in a differential amplifier circuit:

[0048]

[0049] in, This indicates the output infrared characteristics after differential enhancement.

[0050] The (3) section describes the construction of a structure-guided enhancement module:

[0051] Step 1: Stitch together the multi-level visible light features output by the progressively guided aggregation module, and then perform multi-level attention:

[0052]

[0053]

[0054] in, Indicates channel attention; Represents pixel attention; This indicates an enhanced spatial attention.

[0055] Step 2: The dual-branch physical sensing module outputs the final atmospheric light map. Transmittance diagram and restored image ,based on The design structure guides the enhancement module to obtain infrared information guidance weights. :

[0056]

[0057]

[0058] in, This indicates a structure-guided enhancement module; This indicates a dual-branch physical sensing module.

[0059] Step 3: Utilize the output infrared characteristics after differential enhancement guide Further restore and enhance structural features, and output the final restored image. :

[0060]

[0061] The loss function is designed as described in (4):

[0062] Step 1: Construct a physical coupling loss based on an atmospheric scattering model, and utilize atmospheric light. and transmittance diagram The restored image output by the network Reconstruction was performed to obtain a reconstruction dust map:

[0063]

[0064]

[0065] in: This indicates an image containing dust. This represents a reconstruction of the dust map.

[0066] Physical consistency is achieved by minimizing the L1 gap between the reconstructed dust map and the original disturbance map to achieve truth-free results. Indirect supervision under certain circumstances :

[0067]

[0068] in: Indicates L1 gap, This represents the physical coupling loss.

[0069] Step 2: Construct an enhanced color loss based on color consistency loss, divide the restored image into patches, calculate the color consistency loss for each patch, and enhance local color consistency.

[0070]

[0071] in, Represents a true and clear image; Indicates the first The first patch The average value of the channel; Indicates shared ownership One patch; Indicates shared ownership One channel; This indicates an increase in color loss.

[0072] Step 3: Use L1 loss to calculate the pixel-level differences between the restored image and the original sharp image to ensure the overall restoration effect of the final output:

[0073]

[0074] in, This indicates pixel intensity loss.

[0075] Step 4: Construct TV regularization loss to measure the difference between adjacent pixels in the image, i.e., the total image variation, to promote smoothing of the transmittance map and reduce noise and artifacts.

[0076]

[0077] in, This represents the TV regularization loss of the transmittance map.

[0078] Step 5: Total Loss for:

[0079]

[0080] in, and These represent the weights of the physical coupling loss, enhanced color consistency loss, pixel intensity loss, and TV regularization loss, respectively.

[0081] Compared with the prior art, the present invention has the following beneficial effects:

[0082] (1) In order to improve the structural restoration capability and detail preservation of visible light images in complex interference areas, this invention constructs a progressive guided aggregation module, which embeds a physical-mask coupler consisting of a dual-branch physical sensing module and a soft mask modulation unit step by step. This fully captures the non-uniform scattering interference characteristics under different semantic levels, realizes the effective perception and modeling of scattering characteristics under multiple semantic levels, and realizes physical prior guidance and regional adaptive adjustment.

[0083] (2) To address the issues of structural blurring and insufficient infrared guidance response in visible light images, a differential amplification attention module was designed to jointly model multimodal differences from the channel and spatial dimensions, effectively amplifying the key structural information of infrared images. A structure guidance enhancement module was constructed, which dynamically senses non-uniform scattering intensity by combining the predicted transmittance map, realizing adaptive enhancement or suppression of infrared features in different regions, and strengthening the structural restoration capability of images in complex environments. Through differential attention and structural adjustment, structural completion in high-interference regions and detail preservation in low-interference regions were achieved.

[0084] (3) In order to improve the physical consistency and interpretability of the network under weak supervision, a physical coupling loss based on the atmospheric scattering model was designed, which realized the collaborative optimization of the transmittance map, atmospheric light intensity map and the restored image, and ensured the overall consistency and interpretability of the output restored image and the scattering physical process.

[0085] (5) This invention comprehensively considers the characteristics of image degradation under non-uniform scattering medium interference and the structural prior advantages of infrared guidance, and proposes an infrared-guided image restoration method under non-uniform scattering medium interference. This method effectively combines the structural information of infrared images with the texture expression ability of visible light, and combines multimodal complementary information and physical prior knowledge. It addresses the problems of uneven distribution of interference areas and image feature redundancy, and realizes adaptive enhancement and fine structural restoration of visible light images under non-uniform scattering medium interference. It breaks through the bottleneck of weak adaptability of traditional single-mode dust removal methods to complex environments, and overcomes the problem of rough control of existing guidance methods, effectively improving the image restoration quality under non-uniform scattering medium interference. It performs well on non-uniform interference datasets, effectively alleviating problems such as structural loss, color shift and modal redundancy, providing a highly robust solution for image enhancement in complex environments such as dust, sand, and water mist, and providing a new direction for the research of infrared-guided image restoration methods. Attached Figure Description

[0086] Figure 1This diagram illustrates an image restoration method under infrared-guided non-uniform scattering medium interference.

[0087] Figure 2 This is the original interference image.

[0088] Figure 3 For true and clear images.

[0089] Figure 4 To restore the image.

[0090] Figure 5 This is a transmittance diagram.

[0091] Figure 6 This is an atmospheric light map. Detailed Implementation

[0092] The technical solutions in the embodiments of the present invention will now be clearly and completely described in conjunction with the accompanying drawings.

[0093] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments. These embodiments are based on the technical solution of the present invention and provide detailed implementation methods and specific operating procedures. However, the scope of protection of the present invention is not limited to the following embodiments.

[0094] Example 1:

[0095] Figure 1 This is a flowchart illustrating the implementation of the method proposed in this invention (image restoration method under infrared-guided non-uniform scattering medium interference). This embodiment is based on a publicly available dataset and uses artificial synthesis to generate representative non-uniform scattering medium interference images, with interference patterns closely resembling real industrial dust and natural sandstorm environments. The dataset contains 4200 pairs of visible light and infrared image samples each, with a uniform image spatial size of (1024, 768), fully covering non-uniform scattering scenarios with low, medium, and high concentration distributions, providing a good foundation for generalization evaluation. The aforementioned multimodal images are input into the infrared-guided image restoration method under non-uniform scattering medium interference proposed in this invention to conduct structural restoration and information enhancement experiments on visible light images. Figures 2-6 The method is shown to restore the data on typical samples, where Figure 2 This is the original interference image. Figure 3 For true and clear images, Figure 4 To restore the image, Figure 5 and Figure 6The images show transmittance and atmospheric light intensity maps, respectively. The image comparison results demonstrate that the model can accurately reconstruct high-structure areas such as road and building edges, and maintains texture continuity even in areas with high concentrations of interference. The predicted atmospheric light intensity and transmittance map show good physical consistency. In summary, the experimental results fully verify the non-uniform interference removal capability of this invention under typical scattering environments, exhibiting good physical consistency and perception quality, and possessing the potential for widespread application in practical environmental perception and industrial intelligent perception scenarios.

[0096] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely illustrative of the principles of the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the present invention as claimed. The scope of protection of this invention is defined by the appended claims and their equivalents.

Claims

1. A method for image restoration under infrared-guided non-uniform scattering medium interference, characterized in that, Includes the following steps: (1) Construct a progressively guided aggregation module and embed a physical-mask coupler consisting of a dual-branch physical sensing module and a soft mask modulation unit step by step. The dual-branch physical sensing module guides the network to perceive scattering characteristics at different semantic levels, and the soft mask modulation unit realizes adaptive adjustment of the interference region. (2) Design a differential amplification attention module to extract the difference information between infrared and visible light modes from the two dimensions of channel and space, obtain the infrared features after differential enhancement, realize fine modeling of mode differences, and improve the quality of infrared guidance; (3) Construct a structure-guided enhancement module, use the multi-scale features of the transmittance map to control the amplification or suppression of modal difference information, finely adjust the dependence of different interference regions on infrared information, and complete the structure enhancement and selective fusion of information under modal guidance; (4) Design physical coupling loss to achieve coordinated optimization of the transmittance map, atmospheric light intensity map and restored image, and ensure the overall consistency between the final output image and the scattering physical process.

2. The image restoration method under infrared-guided non-uniform scattering medium interference according to claim 1, characterized in that, The specific steps for constructing the progressive guided aggregation module in (1) are as follows: Step 1: The original interference map is progressively guided through the first level of the aggregation module. Input convolution to obtain initial visible light image features: ； in, Represents the original interference image; Indicates the first-order visible light characteristic; Step 2: Input the visible light features into the next level module step by step. Each level module contains multiple physical-mask couplers: ； ； in, This indicates the progressive bootstrap aggregation module. class, =2,3,4; Indicates the first Level visible light characteristics; Indicates module cascading; Indicates the first Level 1 One physical-mask coupler; Step 3: Within each physical-mask coupler, a dual-branch physical sensing module is designed based on an atmospheric scattering model to estimate atmospheric light. Transmittance diagram and intermediate reconstruction image , and by A soft mask is generated by a soft mask modulation unit to mitigate over-fixing caused by global consistency processing. ； ； ； in, This indicates a dual-branch physical sensing module; Indicates the first Level 1 Visible light characteristics output by a physical coupler; Indicates a soft mask modulation unit; This indicates a soft mask.

3. The image restoration method under infrared-guided non-uniform scattering medium interference according to claim 1, characterized in that, The specific steps for designing the differential amplification module in (2) are as follows: Step 1: Extract infrared features from the infrared image using a single-layer convolution: ； in, Represents the original infrared image; Indicates infrared characteristics; This represents a single-layer convolution; Step 2: Design the differential amplifier module based on the differential amplifier principle: ； in, Indicates a differential signal; Indicates common-mode signal; This represents the differential gain, which controls the degree of amplification of differences. This represents the common-mode gain, which controls the proportion of redundant information remaining. Step 3: Construct a differential mode signal using infrared and first-order visible light features: ； right Focus on channel direction to guide selective channel amplification: ； ； in, Indicates the channel amplification factor; This indicates channel difference enhancement features; This indicates channel-wise global average pooling; Indicates a fully connected layer; Indicates the activation function; right Focusing on spatial orientation guides the response of texture details: ； ； in: Indicates the spatial magnification factor; Indicates spatial difference enhancement features; This indicates pixel-wise average pooling; Represents convolution; Indicates the activation function; Step 4: Enhance Channel Difference Features With spatial augmentation results Coupling, simulating the nonlinear fused output after dual-path modulation in a differential amplifier circuit: ； in, This indicates the output infrared characteristics after differential enhancement.

4. The image restoration method under infrared-guided non-uniform scattering medium interference according to claim 1, characterized in that, The specific steps for constructing the structure guidance enhancement module in (3) are as follows: Step 1: Stitch together the multi-level visible light features output by the progressively guided aggregation module, and then perform multi-level attention: ； ； in, Indicates channel attention; Represents pixel attention; This indicates enhanced spatial attention; Step 2: The dual-branch physical sensing module outputs the final atmospheric light map, indicating channel-by-channel stitching. Transmittance diagram and restored image ,based on The design structure guides the enhancement module to obtain infrared information guidance weights. : ； ； in, This indicates a structure-guided enhancement module; This indicates a dual-branch physical sensing module; Step 3: Utilize the output infrared characteristics after differential enhancement guide Further restore and enhance structural features, and output the final restored image. : 。 5. The image restoration method under infrared-guided non-uniform scattering medium interference according to claim 1, characterized in that, The specific steps for designing the loss function in (4) are as follows: Step 1: Construct a physical coupling loss based on an atmospheric scattering model, and utilize atmospheric light. and transmittance diagram The restored image output by the network Reconstruction was performed to obtain a reconstruction dust map: ； ； in: This indicates an image containing dust. Represents a reconstruction dust diagram; Physical consistency is achieved by minimizing the L1 gap between the reconstructed dust map and the original disturbance map to achieve truth-free results. Indirect supervision under certain circumstances : ； in: Indicates L1 gap, Indicates physical coupling loss; Step 2: Construct an enhanced color loss based on color consistency loss, divide the restored image into patches, calculate the color consistency loss for each patch, and enhance local color consistency. ； in, Represents a true and clear image; Indicates the first The first patch The average value of the channel; Indicates shared ownership One patch; Indicates shared ownership One channel; This indicates an increase in color loss; Step 3: Use L1 loss to calculate the pixel-level differences between the restored image and the original sharp image to ensure the overall restoration effect of the final output: ； in, Indicates pixel intensity loss; Step 4: Construct TV regularization loss to measure the difference between adjacent pixels in the image, i.e., the total image variation, to promote smoothing of the transmittance map and reduce noise and artifacts. ； in, This represents the TV regularization loss in the transmittance map. Step 5: Total Loss for: ； in, and These represent the weights of the physical coupling loss, enhanced color consistency loss, pixel intensity loss, and TV regularization loss, respectively.

Images