A low-illumination image zero-reference enhancement method and system for a fully-mechanized coal mining face

Abstract

A kind of low-illumination image zero reference enhancement method and system of fully mechanized coal face in mine, method is to use pre-training diffusion model to start from part from noise graph, realize the guidance on the content of denoising process by prediction estimate value;Only use normal light image data set when training, by extracting low light and normal light image illumination edge invariant and content invariant and the t step estimate value x0 as the input of network, so that network can learn normal light feature sufficiently during training and restore source image according to input information, network constructs illumination feature according to invariant information during testing and constantly refines estimate value x0, realize low light image enhancement.System includes multispectral image acquisition module, illumination invariant extraction module, diffusion model initialization and noise generation module, Prediction-Net network processing module, reverse denoising guide module and post-processing optimization module.The present application can effectively enhance low light image.

Description

Technical Field

[0001] This invention relates to a zero-reference enhancement method and system for low-light images of fully mechanized mining faces, belonging to the field of low-light image enhancement technology. Background Technology

[0002] As coal mining depths continue to increase, the underground environment becomes increasingly complex, especially in fully mechanized mining faces. Under poor lighting conditions, image recognition technology, upon which mining machinery and personnel rely, faces severe challenges. In low-light environments, conventional vision systems suffer from significant loss of image detail due to insufficient light, affecting the identification of mining equipment, personnel, and the mining environment, thus hindering the precise control and scheduling of automated mining machinery. Researching low-light image enhancement technology for fully mechanized mining faces in underground coal mines is of great significance for improving coal mine safety, increasing mining efficiency, and ensuring the safety of miners.

[0003] Currently, traditional low-light image enhancement methods have many limitations. For example, histogram equalization improves contrast by redistributing image gray values, but it easily over-enhances certain areas, leading to loss of image details, blockiness, and is unable to specifically handle complex, low-light, and noisy images in fully mechanized mining faces, resulting in unsatisfactory enhancement effects. While gamma transform can improve image brightness and contrast, it has poor adaptability to uneven lighting and is difficult to meet the complex and variable lighting conditions in mines. In the field of deep learning, supervised methods require a large number of paired low-light and normal-light images for training; however, such data is difficult to obtain, and the enhancement results depend on the training data. Once the training data differs from the actual distribution of mine images, the model's generalization ability will decrease. Unsupervised methods generally use GAN models, which do not require labeled data, but training is difficult, adversarial balance is hard to achieve, and gradient vanishing or exploding problems are prone to occur. Furthermore, both traditional methods and deep learning-based methods struggle to cope with complex image noise, reflected light, and object occlusion in low-light mine environments. Summary of the Invention

[0004] The purpose of this invention is to provide a zero-reference enhancement method and system for low-light images of fully mechanized mining faces. This method and system can effectively enhance low-light images, adapt to complex and variable lighting conditions, and have strong generalization ability.

[0005] To achieve the above objectives, the present invention provides a zero-reference enhancement method for low-light images of fully mechanized mining faces, comprising the following steps:

[0006] S1. Diffusion Model Reverse Denoising: Starting from a noise map conforming to a standard normal distribution, predict the noise level x at the t-th step of denoising. t For the enhanced result of the condition The denoising result of step t-1 is obtained, and this information guides the image generation in step t-1, thus obtaining the final generated result;

[0007] S2. Model Training: Input a normal lighting image and extract the lighting edge invariants and content invariants;

[0008] S3, combine the illumination edge invariant with the x-axis. t The estimated x 0|t Input the Prediction-Net network and predict minimize And the MSE loss of the normal light image, and so on, to obtain x t-1 ,…,x1,Prediction

[0009] S4. During testing, input a low-light image, repeat S2 and S3, and obtain the final enhancement result.

[0010] Furthermore, in the denoising process of S1, x 0|t The result is estimated from the denoising result of step t, and the formula is as follows:

[0011]

[0012] in, α t =1-x t x t The value is a predefined parameter that controls the intensity of noise addition during each diffusion step. Its magnitude is in the range (0, 1), and it gradually decreases as t increases. θ (x t x(t) represents the noise predicted at step t. t This represents the result obtained in step t+1;

[0013] Depend on The denoising result of step t-1 is obtained, and this information guides the image generation in step t-1, as shown in the following formula:

[0014]

[0015] Where N represents a Gaussian distribution, and I is a noise map randomly sampled from a standard normal distribution at step t. and These are the mean and variance of the intermediate noise plot at step t, respectively, and the formulas are as follows:

[0016]

[0017] x obtained through the above steps t-1 It is consistent with the input image in terms of content, and is also transmitted via x. t-1Get x t-2 This process is repeated until an enhanced result is achieved.

[0018] Furthermore, the illumination edge invariant mentioned in S2 is derived based on the Kubelka-Munk theory. Assuming equal energy but non-uniform illumination and a dull, matte surface, invariant N is derived. Simultaneously, assuming equal energy and uniform illumination, and a dull, matte surface, invariant W is derived. The formulas are as follows:

[0019]

[0020] Among them, E x and E y These are the horizontal and vertical gradients of E, respectively; E λx and E λy E respectively λ Horizontal and vertical gradients; E λλx and E λλy E respectively λλ Horizontal and vertical gradients; E λ and E λλ These are the first and second derivatives of E with respect to wavelength λ, respectively.

[0021] E, E λ E λλ The calculations are performed using the following formulas:

[0022]

[0023] Where R(x,y), G(x,y), and B(x,y) are the gray values ​​of the three channels of the original image at the corresponding positions.

[0024] The illumination content invariant is the frequency spectrum obtained by Fourier transforming the original image, and the formula is:

[0025] A I =|FFT(input)|;

[0026] Here, FFT stands for Fourier Transform.

[0027] By combining illumination edge invariants and content invariants as inputs to the network, the structural and content relationships between low-light and normal-light images can be maintained relatively well.

[0028] Furthermore, the Prediction-Net network in S3 adopts a U-Net architecture, which includes three downsampling operations and three upsampling operations, and uses skip connections to fuse features from different levels, as expressed by the following formula:

[0029] ds = DownSample(x1);

[0030] us = UpSample(x2);

[0031] f_result=Conv(Concat(ds,us));

[0032] Where DownSample represents the downsampling operation, UpSample represents the upsampling operation, Concat represents the concatenation operation, and Conv represents the convolution operation;

[0033] The U-Net architecture also includes an APFM module, which employs a multi-stage fusion approach to refine modifications to A. I and x 0|t The phase map is then subjected to inverse Fourier transform with its corresponding phase map and amplitude map. Finally, the feature maps obtained from each stage are fused and represented as follows:

[0034] f c =Relu(Conv(x))*3;

[0035] fussion1 = IFFT(f c ((|FFT(x 0|t )|),A I );

[0036] fussion2 = IFFT(|FFT(x) 0|t )|,f c (A I ));

[0037] fussion3 = IFFT(|FFT(x) 0|t )|,A I );

[0038] fussion r =Relu(Conv(Concat(fussion1,fussion w ,fussion3)));

[0039] Where ReLU is the activation function, *3 means that the previous operation is repeated 3 times, and IFFT is the inverse Fourier transform.

[0040] Minimize during training The mean square error between the image and the normal light image is expressed by the formula:

[0041]

[0042] in, Let I(i,j) represent the final result of the prediction at step t, where I(i,j) represents the input normal light image, and M and N represent the width and height of the image, respectively.

[0043] Exposure control loss is used to control the deviation of grayscale intensity and midtone values. The formula is:

[0044]

[0045] Among them, Y k This represents the average gray value of a local area, where P represents the number of area blocks and E is a constant.

[0046] The total loss is:

[0047] L=MSE+α*L e .

[0048] The overall function of the network is mainly based on the given input illumination edge invariants and content invariants, as well as the x estimated at step t. 0|t This method predicts the original normal light image. It is a zero-reference method, trained only using a normal light image dataset, and exhibits good generalization ability.

[0049] Furthermore, in step S4, by extracting the illumination edge invariants and content invariants of the low-light image, and simultaneously utilizing the illumination prior information learned by the Prediction-Net network, the low-light image is input into the network and undergoes continuous denoising and prediction. This leads to the final enhanced result.

[0050] A zero-reference enhancement system for low-illuminance images of fully mechanized mining faces includes a multispectral image acquisition module, an illumination invariant extraction module, a diffusion model initialization and noise generation module, a Prediction-Net network processing module, a reverse denoising guidance module, and a post-processing optimization module.

[0051] The multispectral image acquisition module is deployed at key locations in the coal mining machine and hydraulic support. It synchronously acquires raw RAW image streams through a visible light low-illuminance camera and a near-infrared auxiliary light source array, and transmits them to the illumination invariant extraction module via a multi-sensor synchronous controller.

[0052] The illumination invariant extraction module uses Kubelka-Munk theory to calculate the illumination edge invariants N and W, and combines Fourier transform to extract the content invariant A. I The system generates a fused feature map and inputs it into the Prediction-Net module; upon system startup, the diffusion model initialization module generates a standard normally distributed noise map x. T The reverse denoising process is adjusted iteratively through time steps.

[0053] The Prediction-Net network processing module adopts the U-Net architecture and fuses the illumination invariant and diffusion intermediate result x through the APFM module. 0|t Predicted enhanced image after frequency domain phase-corrected FFT / IFFT This feedback is then sent to the reverse denoising guidance module; this module iteratively updates the Gaussian distribution parameter μ. t , Generate x t-1 The sequence is processed until the initial enhanced result x0 is output;

[0054] The post-processing optimization module compresses the dynamic range through differentiable tone mapping, performs color correction in conjunction with SSIM and color constancy loss, and adjusts the enhancement intensity based on real-time feedback of mine dust concentration, finally outputting the results to the monitoring terminal.

[0055] This invention utilizes a pre-trained diffusion model's reverse denoising component, starting from the noisy image and guiding the denoising process through predicted estimates. To better adapt to the complex lighting environment of fully mechanized mining faces, training uses only normal light image datasets. Illumination edge invariants and content invariants from both low-light and normal-light images, along with the t-th step estimate x0, are used as network input. This allows the network to fully learn normal lighting features during training and reconstruct the source image based on the input information. During testing, the network constructs lighting features based on the invariant information and continuously refines the estimate x0, ultimately enhancing the low-light image. This invention effectively enhances low-light images, adapts to complex and variable lighting conditions, exhibits strong generalization ability, and can be widely applied to image enhancement in low-light environments at fully mechanized mining faces. Attached Figure Description

[0056] Figure 1 This is a schematic diagram of the workflow of the method of the present invention;

[0057] Figure 2 This is a framework diagram of the method of the present invention;

[0058] Figure 3 This is a diagram of the Prediction-Net network framework in an embodiment of the present invention;

[0059] Figure 4 This is a diagram of the APFM module in an embodiment of the present invention;

[0060] Figure 5 These are comparison images of low-light image enhancement effects in embodiments of the present invention. Detailed Implementation

[0061] The invention will now be further described with reference to the accompanying drawings.

[0062] like Figure 1 and Figure 2 As shown, a zero-reference enhancement method for low-light images of a fully mechanized mining face includes the following steps:

[0063] S1. Diffusion Model Reverse Denoising: Starting from a noise map conforming to a standard normal distribution, predict the noise level x at the t-th step of denoising. t For the enhanced result of the condition The denoising result of step t-1 is obtained, and this information guides the image generation in step t-1, thus obtaining the final generated result;

[0064] S2. Model Training: Input a normal lighting image and extract the lighting edge invariants and content invariants;

[0065] S3, combine the illumination edge invariant with the x-axis. t The estimated x 0|t Input the Prediction-Net network and predict minimize And the MSE loss of the normal light image, and so on, to obtain x t-1 ,…,x1,Prediction

[0066] S4. During testing, input a low-light image, repeat S2 and S3, and obtain the final enhancement result.

[0067] Furthermore, in the denoising process of S1, x 0|t The result is estimated from the denoising result of step t, and the formula is as follows:

[0068]

[0069] in, α t =1-β t ,β t For a predefined value, ∈ θ (x t x(t) represents the noise predicted at step t. t This represents the result obtained in step t+1;

[0070] Depend on The denoising result of step t-1 is obtained, and this information guides the image generation in step t-1, as shown in the following formula:

[0071]

[0072] Where N represents a Gaussian distribution, and I is a noise map randomly sampled from a standard normal distribution at step t. and These are the mean and variance of the intermediate noise plot at step t, respectively, and the formulas are as follows:

[0073]

[0074] x obtained through the above steps t-1 It is consistent with the input image in terms of content, and is also transmitted via x. y-1 Get x t-2 This process is repeated until an enhanced result is achieved.

[0075] Furthermore, such as Figure 3 As shown, the illumination edge invariant in S2 is derived based on the Kubelka-Munk theory. Assuming equal energy but non-uniform illumination and a dull, matte surface, invariant N is derived. Simultaneously, assuming equal energy and uniform illumination, and a dull, matte surface, invariant W is derived. The formulas are as follows:

[0076]

[0077] Among them, E x and E y These are the horizontal and vertical gradients of E, respectively; E λx and E λy E respectively λ Horizontal and vertical gradients; E λλx and E λλy E respectively λλ Horizontal and vertical gradients; E λ and E λλ These are the first and second derivatives of E with respect to wavelength λ, respectively.

[0078] E, E λ E λλ The calculations are performed using the following formulas:

[0079]

[0080] Where R(x,y), G(x,y), and B(x,y) are the gray values ​​of the three channels of the original image at the corresponding positions.

[0081] The illumination content invariant is the frequency spectrum obtained by Fourier transforming the original image, and the formula is:

[0082] A I =|FFT(iuput)|;

[0083] Where FFT stands for Fourier Transform;

[0084] By combining illumination edge invariants and content invariants as inputs to the network, the structural and content relationships between low-light and normal-light images can be maintained relatively well.

[0085] Furthermore, such as Figure 4 As shown, the Prediction-Net network in S3 adopts a U-Net architecture, which includes three downsampling operations and three upsampling operations. It uses skip connections to fuse features from different levels, as expressed by the following formula:

[0086] ds = DownSample(x1);

[0087] us = UpSample(x2);

[0088] f_result=Conv(Concat(ds,us));

[0089] Where DownSample represents the downsampling operation, UpSample represents the upsampling operation, Concat represents the concatenation operation, and Conv represents the convolution operation;

[0090] The U-Net architecture also includes an APFM module, which employs a multi-stage fusion approach to refine modifications to A. I and x 0|t The phase map is then subjected to inverse Fourier transform with its corresponding phase map and amplitude map. Finally, the feature maps obtained from each stage are fused and represented as follows:

[0091] f c =Relu(Conv(x))*3;

[0092] fussion1 = IFFT(f c ((|FFT(x 0|t )|),A I );

[0093] fussion2 = IFFT(|FFT(x) 0|t )|,f c (A I ));

[0094] fussion3 = IFFT(|FFT(x) 0|t )|,A I );

[0095] fussion r =Relu(Conv(Concat(fussion1,fussion2,fussion3)));

[0096] Where ReLU is the activation function, *3 means that the previous operation is repeated 3 times, and IFFT is the inverse Fourier transform.

[0097] Minimize during training The mean square error between the image and the normal light image is expressed by the formula:

[0098]

[0099] in, Let I(i,j) represent the final result of the prediction at step t, where I(i,j) represents the input normal light image, and M and N represent the width and height of the image, respectively.

[0100] Exposure control loss is used to control the deviation of grayscale intensity and midtone values. The formula is:

[0101]

[0102] Among them, Y k In this embodiment, P represents the average grayscale value of a local area of ​​size 16×16, P represents the number of area blocks, and E is a constant, which is 0.6 in this invention.

[0103] The overall function of the network is mainly based on the given input illumination edge invariants and content invariants, as well as the x estimated at step t. 0|t This method predicts the original normal light image. It is a zero-reference method, trained only using a normal light image dataset, and exhibits good generalization ability.

[0104] Furthermore, in step S4, by extracting the illumination edge invariants and content invariants of the low-light image, and simultaneously utilizing the illumination prior information learned by the Prediction-Net network, the low-light image is input into the network and undergoes continuous denoising and prediction. This leads to the final enhanced result.

[0105] The present invention also provides a zero-reference enhancement system for low-illuminance images of fully mechanized mining faces, including a multispectral image acquisition module, an illumination invariant extraction module, a diffusion model initialization and noise generation module, a Prediction-Net network processing module, a reverse denoising guidance module, and a post-processing optimization module;

[0106] The multispectral image acquisition module is deployed at key locations in the coal mining machine and hydraulic support. It synchronously acquires raw RAW image streams through a visible light low-illuminance camera and a near-infrared auxiliary light source array, and transmits them to the illumination invariant extraction module via a multi-sensor synchronous controller.

[0107] The illumination invariant extraction module uses Kubelka-Munk theory to calculate the illumination edge invariants N and W, and combines Fourier transform to extract the content invariant A. I The system generates a fused feature map and inputs it into the Prediction-Net module; upon system startup, the diffusion model initialization module generates a standard normally distributed noise map x. TThe reverse denoising process is adjusted iteratively through time steps.

[0108] The Prediction-Net network processing module adopts the U-Net architecture and fuses the illumination invariant and diffusion intermediate result x through the APFM module. 0|t Predicted enhanced image after frequency domain phase-corrected FFT / IFFT This feedback is then sent to the reverse denoising guidance module; this module iteratively updates the Gaussian distribution parameter μ. t , Generate x t-1 The sequence is processed until the initial enhanced result x0 is output;

[0109] The post-processing optimization module compresses the dynamic range through differentiable tone mapping, performs color correction in conjunction with SSIM and color constancy loss, and adjusts the enhancement intensity based on real-time feedback of mine dust concentration, finally outputting the results to the monitoring terminal.

[0110] This embodiment uses the COCO-2017 dataset for training and low-light images from the LOL dataset as test data. The results are as follows: Figure 5 As shown: Three types of low-light images were selected: indoor images, outdoor images, and low-light images from a mine. Figure 5 (a) and (b) represent the indoor low-light image and the enhancement result, respectively; Figure 5 (c) and (d) show outdoor low-light images and enhancement results, respectively; Figure 5 (e) and (f) are low-light images and enhancement results from real mines, respectively. It can be seen that the present invention has achieved significant enhancement of low-light images in multiple scenarios.

Claims

1. A low-illumination image zero-reference enhancement method for a fully-mechanized coal mining face, characterized in that, Comprising the following steps: S1, diffusion model reverse denoising: starting from the noise map conforming to the standard normal distribution, predicting the enhanced result conditioned on at the t-th step denoising obtaining the denoising result of the t-1th step, guiding the image generation of the t-1th step with the information, so as to obtain the final generation result; S2, model training: input normal light image, extract illumination edge invariant and content invariant; S3, the lightness edge invariant and by estimated input Prediction-Net network, predict minimizing and the MSE loss of normal light image, and so on …, , predict ; S4, during testing, input low light image, repeat S2 and S3, and obtain final enhancement result; The Prediction-Net network in S3 adopts U-Net architecture, which includes three down-sampling and three up-sampling, uses skip connection to fuse features of different levels, and is expressed by the following formula: ； ； ； wherein, denotes a down-sampling operation, denotes an up-sampling operation, is a concatenation operation, and Conv is a convolution operation; The U-Net architecture further comprises an APFM module, which adopts a multi-stage fusion manner for fine modification and the phase map, while performing inverse Fourier transform on the respective corresponding phase map and amplitude map, and finally fusing the feature maps obtained in each stage, represented as: ； ； ； ； ； Wherein, Relu is an activation function, *3 represents that the previous operation is repeated 3 times, and IFFT is inverse Fourier transform; Minimize during training The mean square error between the normal and the light image, given by: ； wherein, represents the final result of the prediction of the t-th step, represents the input normal light image, and respectively represent the width and height of the image; The exposure control loss is used to control the deviation of gray intensity and intermediate tone value, and the formula is: ； wherein, represents the local area average gray value, P represents the number of area blocks, and E is a constant. The total loss is: 。 2. The low-illumination image zero-reference enhancement method for fully mechanized coal mining face of a mine according to claim 1, characterized in that, the denoising process in S1, The estimation from the t-th step denoising result is obtained, and the formula is as follows: ； wherein, , , is a predefined value, is the predicted noise at step t, denotes the result obtained at step t+1; From The de-noising result of the t-1th step is obtained, and the image of the t-1th step is generated based on the information, and the formula is as follows: ； where, denotes a Gaussian distribution, is a noise map randomly sampled from a standard normal distribution at step t, and are the mean and variance of the intermediate noise map at step t, respectively, given by: ； ； The above steps are obtained In the content consistent with the input image, while via Get , this way, eventually get the enhanced results.

3. The low-illumination image zero-reference enhancement method for fully mechanized coal mining face of a mine according to claim 1, characterized in that, The illumination edge invariant in S2 is derived according to KubelkaMunk theory, assuming equal energy but uneven illumination and dull and lusterless object surface, deriving invariant N, and assuming equal energy and uniform illumination and dull and lusterless object surface, deriving invariant W, the formulas are respectively: ； ； wherein and are the horizontal and vertical gradients of E, respectively; and are the horizontal and vertical gradients of , respectively; and are the horizontal and vertical gradients of , respectively; and are the first and second derivatives of E with respect to wavelength , , , ; , , are calculated by the following equations, respectively: ； wherein, , , are the gray values of the corresponding positions of the three channels of the original image, respectively. The illumination content invariant is the frequency diagram obtained after Fourier transform of the original image, and the formula is: ； Wherein, FFT represents Fourier transform.

4. The low-illumination image zero-reference enhancement method for fully mechanized coal mining face of a mine according to claim 1, characterized in that, The illumination edge invariant and content invariant of the low-light image are extracted in the S4, and the low-light image is input into the network after the illumination prior information learned by the Prediction-Net network is used, continuous denoising and prediction are performed , so that the final enhancement result is obtained.

5. A system for the low-illumination image zero-reference enhancement method of the fully mechanized coal mining face of claim 1, characterized in that, It comprises a multispectral image acquisition module, an illumination invariant extraction module, a diffusion model initialization and noise generation module, a Prediction-Net network processing module, a reverse denoising guide module and a post-processing optimization module; The multispectral image acquisition module is arranged at a key position of a coal mining machine and a hydraulic support, and synchronously acquires an original RAW image stream through a visible light low-illumination camera and a near-infrared auxiliary light source array, and transmits the RAW image stream to the illumination invariant extraction module through a multisensor synchronous controller; The illumination invariant extraction module calculates illumination edge invariants N and W by using Kubelka-Munk theory, and extracts content invariants by combining with Fourier transform , generates a fusion feature map and inputs the Prediction-Net module; when the system is started, the diffusion model initialization module generates a standard normal distribution noise map , and adjusts the reverse denoising process through time step iteration; The Prediction-Net network processing module adopts a U-Net architecture, and fuses the illumination invariant and diffusion intermediate results through an APFM module , and the prediction enhanced image is subjected to frequency domain phase correction FFT / IFFT , and is fed back to a reverse denoising guiding module; the module iteratively updates Gaussian distribution parameters , , generates a sequence, and outputs a preliminary enhanced result ; The post-processing optimization module compresses the dynamic range through a differentiable Tone Mapping, performs color correction in combination with SSIM and color constancy loss, adjusts the enhancement intensity based on real-time feedback of mine dust concentration, and finally outputs the result to a monitoring terminal.

Images