Mine dust fog image defogging method and system based on real fog diffusion

Abstract

The application provides a mine dust fog image defogging method and system based on real fog diffusion. In view of the problems of single training data of the existing defogging model, slow diffusion model inference speed, missing of fast sampling texture and poor processing effect of non-uniform fog, the application adopts a four-stage processing flow: first, real pair training data is generated based on a lightweight degradation encoder and a feature modulation mechanism; then, the inference efficiency is improved through a single-step prediction strategy; then, the texture details are migrated by using a slice statistical alignment operation to generate an initial defogging estimation map; finally, a density perception fidelity guide mechanism based on a transmittance map is introduced to adaptively adjust the guide intensity of different fog concentration areas. The application reduces the model parameter quantity, improves the inference efficiency and image restoration quality, realizes the balance between defogging completeness and structure fidelity, is suitable for deployment of mine edge computing devices, and effectively supports underground target detection and intelligent perception tasks.

Description

Technical Field

[0001] This invention relates to the field of computer vision and image processing technology, specifically to a method and system for defogging mine dust and fog images based on real-world fog diffusion. Background Technology

[0002] In underground coal mine operations, long-term mining, transportation, and dust suppression measures such as water spraying result in high concentrations of coal dust and water mist particles suspended in the air, creating a complex dust and fog mixture in the underground environment. Simultaneously, underground lighting relies primarily on artificial light sources such as miners' lamps and fixed LED lights, and the strong reflections from the tunnel walls create a non-uniform lighting environment. Under these conditions, images captured by monitoring equipment generally suffer from low contrast, loss of detail, local overexposure, and halo effects. In particular, the large and unevenly distributed droplets in sprayed areas cause the coexistence of dense and thin fog areas in the images, severely impacting subsequent target detection and intelligent perception tasks. Currently, image restoration for such images mainly employs traditional dehazing algorithms based on atmospheric scattering models or end-to-end dehazing networks based on deep learning. In recent years, diffusion models, due to their excellent generative capabilities, have also been increasingly applied to image dehazing, attempting to fill in lost image details through generative methods.

[0003] However, existing deep learning methods largely rely on synthetic datasets, which are typically generated by physical models and have monotonous textures, failing to simulate the complex distribution of dust and fog in real mines. While some methods introduce ControlNet-guided generation, directly reusing the massive U-Net encoder structure results in a large number of parameters, low computational efficiency, and difficulty in accurately extracting instance-level degradation features. Diffusion models are slow inference; the standard diffusion model dehazing process requires starting with Gaussian noise and performing dozens or even hundreds of iterative denoising samplings. This cumbersome iterative process consumes significant computational resources and has high inference latency, severely limiting its deployment and application on edge computing devices in mines. Furthermore, existing methods typically employ a globally uniform guidance strategy during dehazing, failing to distinguish between dense fog and light fog areas. However, in mine images, light fog areas need to retain their original structure, while dense fog areas need to recover lost content. Uniform processing can lead to structural distortion in light fog areas or incomplete dehazing in dense fog areas, failing to achieve a balance between image fidelity and restoration quality. Summary of the Invention

[0004] To address the aforementioned technical shortcomings, the present invention aims to provide a method and system for dehazing mine dust and fog images based on real-world fog diffusion. This method efficiently extracts fog and fog features from reference images to generate high-quality, paired, realistic mine fog and fog training data; avoids the complete iterative sampling process starting from pure noise, thus improving inference efficiency; rapidly transfers texture details from the real fog image to the prediction results, constructing a high-quality initial dehazing estimation map; and adaptively adjusts the guidance intensity according to the fog concentration, effectively restoring details in dense fog areas while preserving the structure of thin fog areas.

[0005] To achieve the above objectives, the present invention adopts the following technical solution:

[0006] A method for dehazing mine dust and fog images based on real haze diffusion includes the following steps:

[0007] S1. Generate realistic paired training data based on lightweight feature modulation. Input a clear mine background image, a real reference haze image and a text control vector. Extract degradation features through a lightweight degradation encoder and inject them into the generation network through feature modulation to generate a realistic haze image in one step.

[0008] S2. A single-step prediction strategy is adopted, which uses a trained noise prediction network to directly predict noise at an earlier time step and estimates the mean of the early prediction in a single step.

[0009] S3. Generate an initial dehazing estimation map based on slice statistical alignment, extract corresponding overlapping slices of the real haze image and the early prediction mean, and then stitch and fuse them after channel-level statistical alignment.

[0010] S4. Based on density-aware fidelity, the image is refined, the transmittance map is calculated, and the image is updated by weighted gradient of MS-SSIM loss to generate a clear image.

[0011] Preferably, in step S1, the text control vector is obtained by inputting text prompts describing the mine environment into the CLIP text encoder and mapping them into a fixed-length semantic embedding vector.

[0012] Preferably, in step S1, the lightweight degradation encoder consists of one initial convolutional layer followed by four cascaded lightweight residual blocks. Each residual block contains a convolution-ReLU-convolution structure and embeds a channel attention mechanism. Finally, the spatial features of the reference haze image are compressed into a one-dimensional degradation feature vector through a global average pooling layer.

[0013] Preferably, in step S1, feature modulation injection involves converting the extracted degenerate features into style scaling coefficients through a mapping layer, and using a weight modulation-demodulation mechanism to dynamically scale and normalize the convolutional kernel weights of the generator network U-Net.

[0014] Preferably, in step S1, the generating network adopts the SD-Turbo architecture, encodes clear images into latent features, combines text control vectors and degradation features to generate latent features of haze images in a single step inference, and then restores them into realistic haze images by the decoder.

[0015] Preferably, in step S3, the slice statistical alignment involves calculating the channel-level mean and standard deviation of the real haze image slice and the early prediction mean slice, subtracting the mean of the haze slice from its mean, multiplying by the ratio of the two standard deviations, and adding the mean of the prediction slice to complete the alignment operation.

[0016] Preferably, in step S4, the transmittance map is calculated using a dark channel prior algorithm, and the transmittance map has a large value in the thin fog region and a small value in the dense fog region.

[0017] Preferably, in step S4, the weighted update is to multiply the MS-SSIM loss gradient with the transmittance map element-wise, with a high gradient weight in the hazy region to preserve the original structure and a low gradient weight in the dense fog region to restore the occluded content.

[0018] A dehazing system for mine dust and fog images based on real-world haze diffusion includes:

[0019] The realistic data generation module is used to input clear mine background images, realistic reference haze images and text control vectors, and generate realistic pairwise training data through lightweight degradation encoding and feature modulation.

[0020] The single-step prediction module is used to directly predict noise at an earlier time step using a prediction noise network and to estimate the mean of the early predictions in a single step.

[0021] The slice statistical alignment module is used to extract corresponding overlapping slices and perform statistical alignment to generate an initial dehazing estimation map;

[0022] The density-aware guided refinement module is used to calculate the transmittance map, and combined with MS-SSIM loss gradient weighted update, it outputs a clear image.

[0023] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0024] 1. The lightweight degradation encoder and modulation-demodulation mechanism designed in this invention significantly reduce the number of computational parameters compared to traditional, large-scale generation guidance methods. It can accurately extract instance-level degradation features of mine dust and fog, making the generated training data closer to the real environment in terms of texture and distribution, effectively solving the problem of monotonous training data and improving the model's generalization ability in real-world scenarios.

[0025] 2. The single-step prediction strategy proposed in this invention greatly improves inference efficiency. This mechanism directly skips a large number of redundant iterative steps in the diffusion model, quickly locates the intermediate state of the denoising process, and enables the model to have the potential for rapid processing on mine-side equipment.

[0026] 3. The slice statistical alignment operation (AlignOp) constructed in this invention effectively solves the texture loss problem caused by rapid sampling. By utilizing local statistical properties, high-frequency details of the real fog image are quickly transferred to the initial estimation image, significantly improving the clarity and structural integrity of the initial dehazed image with extremely low computational cost.

[0027] 4. The density-aware fidelity guidance mechanism proposed in this invention achieves adaptive processing of non-uniform fog. By dynamically adjusting the guidance intensity through the transmittance map, the original structure of the tunnel wall, equipment edges, etc. is effectively preserved in the thin fog area, while the obscured visual information is successfully recovered by using the model prior in the dense fog area, achieving the optimal balance between defogging and image fidelity. Attached Figure Description

[0028] Figure 1 This is a schematic diagram of the overall implementation process of the present invention;

[0029] Figure 2 This is an overall structural block diagram provided by the embodiment of the present invention;

[0030] Figure 3 This is a detailed network architecture diagram of the realistic image generation module provided in this invention.

[0031] Figure 4 This is a schematic diagram of the principle framework of single-step prediction and guided refinement in this invention;

[0032] Figure 5 This is a comparative schematic diagram showing the effect of applying the method of the present invention to the defogging of real mine dust and fog images. In the diagram, (a1) is the real fog image underground; (a2) is the defogging image using the method of the present invention; (b1) is the real fog image underground; (b2) is the defogging image using the method of the present invention; (c1) is the real fog image underground; and (c2) is the defogging image using the method of the present invention. Detailed Implementation

[0033] The following is in conjunction with the appendix Figures 1 to 5 The present invention will be further described below.

[0034] A method for dehazing mine dust and fog images based on real haze diffusion is proposed. Its specific implementation process mainly includes four stages: realistic data generation, single-step prediction, slice statistical alignment, and guided refinement.

[0035] Phase 1: Generating Realistic Data Based on Lightweight Feature Modulation. To construct a high-quality pairwise dataset for training, this system uses a pre-trained diffusion model (SD-Turbo) as a base and performs a lightweight reconstruction of its conditional control module.

[0036] 1. Image Input: The input consists of a clear background image of the mine, a realistic reference image of haze, and a text control vector describing the dust and haze environment.

[0037] 2. Text Control Vector Generation: To introduce semantic control, this invention uses a pre-trained CLIP (Contrastive Language-Image Pre-Training) text encoder. The input text prompts describing the mine environment are mapped into fixed-length semantic embedding vectors by the CLIP encoder, guiding the semantic consistency of the generated images.

[0038] 3. Lightweight Degradable Encoder To address the issue of excessive parameter count in existing ControlNet technologies, this invention designs a lightweight degradation encoder. Instead of reusing the bulky U-Net structure, this encoder consists of one initial convolutional layer followed by four cascaded lightweight residual blocks. Each residual block contains a convolution-ReLU-convolution structure and embeds a channel attention mechanism to enhance sensitivity to haze concentration features. Finally, a global average pooling layer compresses the spatial features of the reference haze image into a one-dimensional degradation feature vector. .

[0039] 4. Feature Modulation Injection: Extracted Degenerate Features Instead of directly concatenating with the backbone network features, the features are first transformed into style scaling coefficients through a mapping layer. Subsequently, a weight modulation-demodulation mechanism is used to dynamically scale and normalize the convolutional kernel weights of the generator network U-Net. This process "embodies" the hazy style of the reference image into the network parameters, rather than simply injecting it into the intermediate layer features.

[0040] 5. Single-step generation: Combining text control vectors and degradation features The model will pass a clear image through the encoder. Encoded as latent features, via Figure 3 The U-Net network shown can generate latent features with realistic haze textures through single-step inference, and finally pass them through the decoder. The latent spatial features are restored into high-quality synthetic haze images.

[0041] Second stage: Single-step prediction strategy. To address the slow speed of the diffusion model, this method designs a non-iterative fast sampling path.

[0042] 1. Single-step prediction: Utilizing a trained prediction noise network At an earlier time Directly predict noise Single-step estimation of early forecast mean This replaces the traditional step-by-step noise reduction process.

[0043] The third stage: Initial dehazing estimation based on slice statistical alignment. Utilizing the characteristic that the diffusion model's early predictions already include the overall color distribution, the structure is quickly recovered through statistical alignment.

[0044] 1. Slice Extraction: Using the sliding window technique, slices are extracted from the input real haze image and the mean of the early predictions obtained in the second stage. Extract the overlapping slices corresponding to the spatial location.

[0045] 2. Statistical Alignment (AlignOp): Calculates the channel-level mean and standard deviation of the two slices. The variance of the earlier prediction results is used to correct the distribution of the real haze image slices. The mean of the haze slice is subtracted, multiplied by the ratio of the standard deviations of the two slices, and then added to the mean of the predicted slice. The aligned slices are then stitched together, and overlapping areas are merged to generate an initial dehazing estimation map. This image combines the high-frequency texture details of the input image with the statistical features of a clear image, providing a high-quality starting point for subsequent processing.

[0046] Phase 4: Density-Perception Fidelity-Guided Refinement. In the final stage of sampling, adaptive optimization is performed to address the uneven distribution of mine fog.

[0047] 1. Transmittance Map Calculation: A rough transmittance map of the input haze image is calculated using the dark channel prior algorithm. . The value is larger in light fog areas and smaller in dense fog areas, which reflects the different fidelity requirements in different areas.

[0048] 2. Adaptive Guidance: MS-SSIM (Multi-Scale Structural Similarity) loss is introduced as a fidelity metric. In the refinement sampling step, the MS-SSIM loss gradient between the current predicted map and the initial dehazed estimated map is calculated.

[0049] 3. Weighted Update: The calculated gradient is compared with the transmittance map. Perform element-wise multiplication. In the hazy region ( Large values ​​and high gradient weights force the model to retain its original structure; in dense fog regions ( With small values ​​and low gradient weights, the model can utilize its generative capabilities to recover occluded content. Through this differential guidance, high-quality, clear images are ultimately generated.

[0050] Example:

[0051] This embodiment proposes a method and system for dehazing mine dust and fog images based on real-world fog diffusion. The method uses a pre-trained SD-Turbo model as a foundation, constructs a lightweight degradation encoder containing residual blocks to extract fog features, and replaces ControlNet with a feature modulation mechanism to generate realistic fog images, thereby constructing a pairwise training set.

[0052] In the inference phase, this embodiment designs an accelerated fidelity sampling process, utilizes a single-step prediction strategy to improve efficiency, introduces slice statistical alignment operations to quickly transfer texture details, and combines a density-aware fidelity guidance mechanism based on transmittance maps to adaptively balance the guidance intensity for different fog concentrations. This method solves the image restoration problem in non-uniform fog environments while reducing computational costs.

[0053] The specific implementation process includes the following four modules:

[0054] 1. Realistic data generation module based on lightweight feature modulation; 2. Single-step prediction module; 3. Initial dehazing estimation module based on slice statistical alignment; 4. Density-aware fidelity guided refinement module.

[0055] 1. Realistic data generation module based on lightweight feature modulation:

[0056] like Figure 3 As shown, the input to this generative model includes the following three parts:

[0057] Reference Haze Image: Used to provide the style and features of the target haze. A lightweight degenerate encoder was designed, consisting of one initial convolutional layer followed by four cascaded lightweight residual blocks. Each residual block contains a convolution-ReLU-convolution structure. via encoder Encode it and extract degradation features This allows for the controllable adjustment of the properties of smog.

[0058] CleanImage: The source image to be fogged. This is achieved through an image encoder. They are mapped to latent features and used as the initial input for the generation process.

[0059] Text control vectors (TextEmbedding) The text prompts describing the mine environment are input into CLIPTextEncoder and mapped to a fixed-length semantic embedding vector. This vector contains a general prior about the hazy environment to guide the generative network to maintain semantic consistency.

[0060] The core generative network employs the SD-Turbo architecture (U-Net). During inference, all input parameters remain frozen except for the variable reference haze image. The network utilizes features extracted from the reference haze image. This is utilized through a weighted modulation-demodulation mechanism. Specifically, this mechanism first... The weights of the U-Net convolutional layers are dynamically scaled as style vectors, and then normalized and unmodulated to eliminate amplitude bias. This process directly affects the convolutional kernels, enabling the network to adaptively inject the haze physical properties of the reference image while avoiding interference from the background content of the reference image, ultimately generating high-quality, realistic haze images. The specific process is as follows: Figure 3 As shown.

[0061] To optimize the generation results, a single-step loss function is used. The difference between the decoded generated image and the real target haze image is calculated, and this difference is used to supervise the training of controllable parameters. The loss function is defined as follows: (1);

[0062] in, Represents the mathematical expectation. Describing the L1 norm, It's a decoder. To generate the network output in a parameterized manner, These are potential features of a clear image. It refers to the degradation characteristics of fog maps. It is a text control vector. It is a time step. It is a fog map of the actual target.

[0063] 2. Single-step prediction module:

[0064] The forward diffusion process in a general diffusion model can be described by the following formula: (2);

[0065] in It's a noisy image. , These are noise scheduling parameters.

[0066] Formula (2) above describes the forward diffusion process, which is scheduled according to a predefined variance. Gaussian noise is gradually added to the encoded target image. In the single-step prediction strategy proposed in this embodiment, The predicted mean is denoted as The calculation is performed by reversing the forward diffusion process: (3);

[0067] in, Indicates time step The predicted image, Indicates the current input. and These are the scaling factors for the noise term and the signal term, respectively. Used to predict noise, C represents the encoded blurred image.

[0068] 3. Initial Dehazing Estimation Module Based on Slice Statistical Alignment

[0069] This module uses sliding window technology to analyze blurred images. and early prediction images Extract overlapping blocks, and denote these blocks as follows: Calculate the aligned blocks The calculation is as follows: (4);

[0070] in and This represents the channel mean and standard deviation.

[0071] By stitching together these aligned image patches and performing a weighted average of the pixel values ​​in the overlapping areas, a complete dehazing estimation image can be reconstructed. The dehazing estimation preserves the details of the blurred image, and the difference from the actual potentially sharp image is relatively small. Based on this, this embodiment assumes that at time step... After adding Gaussian noise, the distribution of the dehazed image is highly similar to the distribution of the real, sharp image, thus... It can be done in Add noise and use We can directly approximate the result as follows: (5);

[0072] in, Indicates in Step output, and These are the scaling factors for the noise term and the signal term, respectively. This represents the initial dehazing estimation map. It is an encoder that encodes pixel images into latent spatial features.

[0073] The sampling process starts from the time step This recovery step can further refine the image results, especially in helping to restore details lost in areas of dense fog.

[0074] 4. Density-perceived fidelity guidance refinement module

[0075] In the guided refinement stage, to further improve the fidelity and restoration quality of image sampling, this embodiment constructs a fog density-aware fidelity guidance mechanism. This mechanism aims to constrain the denoising path during the refinement process, guiding it towards the initial dehazed estimation image. Approximation. For the non-uniformly distributed fog concentration in the image, a spatially adaptive weighted guidance strategy is implemented: given the positive correlation between the numerical distribution characteristics of the transmittance map (higher values ​​in thin fog areas and lower values ​​in dense fog areas) and the required guidance intensity distribution, this strategy can accurately reflect the differences in fidelity requirements across different regions. First, the input image is calculated... Rough transmittance diagram As a spatial weight matrix, a higher guiding weight is applied in the hazy region to strengthen the structural fidelity constraint on the dehazed estimated map and ensure that the original texture is not lost; while in the dense fog region, given the severe degradation of the original details, the constraint strength is reduced accordingly, and more of the generative prior of the diffusion model is used to infer and recover the potential image content, thereby achieving an adaptive balance between the thoroughness of dehazing and structural fidelity.

[0076] At each time step A clear image is obtained based on the prediction formula. Subsequently, the MS-SSIM loss, which is sensitive to structural similarity, was used as the fidelity measure of this design. The loss function is defined as follows: (6);

[0077] in To predict the image, This is the initial dehazing estimation image.

[0078] This embodiment applies the gradient descent algorithm to... To optimize fidelity, specifically, to reduce fidelity loss related to... gradient and weighted graph Element-wise multiplication is used to selectively reduce the guidance intensity in dense fog areas. The mean prediction at each time step t is calculated in each refinement step. Updated according to the following formula: (7);

[0079] in It is the predicted mean at time step t. Control the guiding intensity. It is a decoder Jacobian matrix, decoder Transform the latent features back into the image space. It is a weight matrix. This indicates element-wise multiplication. It is the fidelity loss function The gradient of the predicted image.

[0080] Finally, the updated predicted image is used. and noisy images Calculate the next noisy image This is used to gradually update the sampling process at each time step.

[0081] This invention achieves the following effects:

[0082] First, it addresses the problem that existing dehazing models suffer from poor generalization ability in the complex dust and fog environment of real mines due to a lack of high-quality paired training data. By constructing a lightweight degenerate encoder with residual blocks and a feature modulation mechanism, it efficiently extracts fog and haze features from reference images, generating high-quality, paired, realistic mine fog and haze training data.

[0083] Second, it solves the problem of slow sampling iterations in traditional diffusion models. By using a single-step prediction strategy, it avoids the complete iterative sampling process starting from pure noise, significantly improving inference efficiency.

[0084] Third, it solves the problem of blurred texture and missing structure in the initial image during the fast sampling process. By using slice statistical alignment, the texture details of the real fog image are quickly transferred to the prediction result, constructing a high-quality initial dehazing estimation map.

[0085] Fourth, it solves the problem of poor image restoration quality in non-uniform fog environments. A density-aware fidelity guidance mechanism based on transmittance maps is introduced, which adaptively adjusts the guidance intensity according to the fog concentration, effectively restoring details in dense fog areas while preserving the structure of thin fog areas.

Claims

1. A method for dehazing mine dust and fog images based on real haze diffusion, characterized in that, Includes the following steps: S1. Realistic paired training data is generated based on lightweight feature modulation. A clear mine background image, a realistic reference haze image, and a text control vector are input. Degradation features are extracted by a lightweight degradation encoder and injected into the generator network via feature modulation to generate a realistic haze image in a single step. The text control vector is obtained by inputting text prompts describing the mine environment into the CLIP text encoder and mapping them to a fixed-length semantic embedding vector. The lightweight degradation encoder consists of one initial convolutional layer followed by four cascaded lightweight residual blocks. Each residual block contains a convolution-ReLU-convolution structure and embeds a channel attention mechanism. Finally, a global average pooling layer compresses the spatial features of the reference haze image into a one-dimensional degradation feature vector. Modulation injection transforms the extracted degradation features into style scaling coefficients through a mapping layer. The weight modulation-demodulation mechanism is used to dynamically scale and normalize the convolutional kernel weights of the generator network U-Net. The generator network adopts the SD-Turbo architecture, which encodes clear images into latent features. It combines text control vectors and degradation features to generate latent features of haze images in a single step inference, and then restores them into realistic haze images through the decoder. S2. A single-step prediction strategy is adopted, which uses a trained noise prediction network to directly predict noise at an earlier time step and estimates the mean of the early prediction in a single step. S3. Generate an initial dehazing estimation map based on slice statistical alignment, extract corresponding overlapping slices of the real haze image and the early prediction mean, and then stitch and fuse them after channel-level statistical alignment. S4. Based on density-aware fidelity, the image is refined, the transmittance map is calculated, and the image is updated by weighted gradient of MS-SSIM loss to generate a clear image.

2. The method for dehazing mine dust and fog images based on real haze diffusion as described in claim 1, characterized in that, In step S3, the slice statistical alignment involves calculating the channel-level mean and standard deviation of the real haze image slice and the early prediction mean slice, subtracting the mean of the haze slice from its mean, multiplying by the ratio of the two standard deviations, and adding the mean of the prediction slice to complete the alignment operation.

3. The method for dehazing mine dust and fog images based on real haze diffusion as described in claim 1, characterized in that, In step S4, the transmittance map is calculated using the dark channel prior algorithm. The transmittance map has a large value in the thin fog region and a small value in the dense fog region.

4. The method for dehazing mine dust and fog images based on real haze diffusion as described in claim 1, characterized in that, In step S4, the weighted update involves element-wise multiplying the MS-SSIM loss gradient with the transmittance map, with higher gradient weights in hazy areas to preserve the original structure and lower gradient weights in dense fog areas to restore occluded content.

5. A defogging system for mine dust and fog images based on real-world haze diffusion, used to implement the method according to any one of claims 1 to 4, characterized in that, include: The realistic data generation module is used to input clear mine background images, realistic reference haze images and text control vectors, and generate realistic pairwise training data through lightweight degradation encoding and feature modulation. The single-step prediction module is used to directly predict noise at an earlier time step using a prediction noise network and to estimate the mean of the early predictions in a single step. The slice statistical alignment module is used to extract corresponding overlapping slices and perform statistical alignment to generate an initial dehazing estimation map; The density-aware guided refinement module is used to calculate the transmittance map, and combined with MS-SSIM loss gradient weighted update, it outputs a clear image.

Images