An adaptive image defogging and enhancement method and system in a mine dust and fog environment

Abstract

The present invention belongs to the field of image defogging technology, and proposes an adaptive image defogging enhancement method and system in a dusty and foggy environment in a mine. First, an image processing model including an image grading preprocessing module, an adaptive processing module and an optimization module is established; the image processed by the image grading preprocessing module is input into the adaptive processing module; the image processed by the adaptive processing module is further optimized by the image optimization module; and the optimized image is output. In practical application, the present invention can accurately, efficiently and stably defog images in a dusty and foggy environment in a mine, significantly improving the image quality and the ability to identify safety hazards in coal mines.

Description

Technical Field

[0001] The invention belongs to the technical field of image defogging, and proposes an adaptive image defogging and enhancement method and system in a mine dust and fog environment. Background Art

[0002] As an important energy supply base in my country, coal mines have always received extensive attention for their safe production and intelligent construction. However, in the underground working environment of coal mines, due to factors such as coal ash, dust, and water mist generated by dust removal and water spraying, the target objects are difficult to see clearly, which brings great challenges to the identification and monitoring of safety hazards in coal mines. This dust and fog seriously affects the quality of the collected images, which is not conducive to subsequent processing such as target identification and tracking. Therefore, it is of great significance to study the defogging of underground coal mine images.

[0003] In fact, image dehazing has become a research hotspot in the field of computer vision and image processing. Traditional dehazing methods are mostly based on physical models or image processing techniques, such as dark channel priors, atmospheric scattering models, etc. However, these methods often have poor effects when dealing with complex and changeable coal mine dust and fog environments, and are difficult to meet the needs of practical applications.

[0004] In recent years, deep learning technology has made significant progress in the field of image processing and provided a new solution for image defogging. Defogging methods based on deep learning learn the mapping relationship from foggy images to clear images by training a large number of foggy and clear image pairs. However, existing deep learning defogging methods ignore the feature differences of images with different haze levels and use a large and comprehensive unified network to achieve defogging, which wastes performance and has unstable defogging effects.

[0005] Patent application number CN202311273860.8 is a method for constructing a partial twin defogging network based on U-Net. The framework includes two weight-sharing subnetworks. After the foggy image is input into the first subnetwork for defogging, the generated image is passed to the second subnetwork as defogging prior information. The second subnetwork can have stronger defogging and image reconstruction capabilities by inputting the foggy image and prior guidance information. In addition, the present invention designs a dual encoding and decoding module in the subnetwork of the framework. A foggy image is input into the subnetwork, and convolution downsampling is performed to obtain coding features respectively. After that, the coding features are secondary encoded through the secondary encoding module and feature fusion is performed to obtain the fused secondary coding features. The fused secondary coding features obtained in the previous step are passed through the secondary decoding module, and then the obtained secondary decoding features are respectively fused with the features of the corresponding scales in the encoding stage and the decoding stage, and then the final defogging image is obtained by upsampling. Processing images with only two sub-networks causes too much interference, and it is impossible to generate an effective negative sample comparison strategy, and it is impossible to ensure that negative samples can provide sufficient information in contrast learning. In addition, the specific characteristics of images with different haze levels are not fully utilized. For light haze images, an overly complex model is used, resulting in a waste of computing power; and for dense fog images, no specific processing method is made for their low contrast and blurred details. Therefore, it is urgent for those skilled in the art to solve the above technical problems. Summary of the invention

[0006] The technical problem to be solved by the present invention is to address the deficiencies of the above-mentioned prior art and to provide an adaptive image dehazing and enhancement method and system in a dusty and foggy environment in a mine. The adaptive image dehazing and enhancement method and system in a dusty and foggy environment in a mine, by comparing the similarities and differences between different samples, enables the model to better understand and process images with various different hazes. At the same time, in order to address the problem of blurred edge and detail information in the image caused by dust and fog, the gradient information is added to the loss function to help the model better retain the edge information and details of the image, which can effectively remove dust and fog from images in the mine and bring convenience to subsequent image processing tasks.

[0007] In order to solve the above technical problems, the technical solution adopted by the present invention is:

[0008] An adaptive image defogging and enhancement method in a mine dust and fog environment comprises the following steps:

[0009] S101: establishing an image processing model including an image classification preprocessing module, an adaptive processing module and an optimization module;

[0010] S102: inputting the image processed by the image classification preprocessing module into the adaptive processing module;

[0011] S103: further optimizing the image processed by the adaptive processing module through the image optimization module;

[0012] S104: outputting the optimized image;

[0013] In step S102, the adaptive processing module includes a mist processing module and a dense fog processing module;

[0014] The haze processing module includes a Net-1 network, five convolutional layers, and two fusion layers;

[0015] The dense fog processing module includes a Net-2 network, seven convolutional layers, and three fusion layers;

[0016] In the haze processing module: the haze image is input into the Net-1 network and passes through a series of convolutional layers and fusion layers. In the Net-1 network, the haze image is first processed by three consecutive convolutional layers. Then, these feature maps are sent to the first fusion layer for fusion operation, and then processed by two convolutional layers and one fusion layer to obtain the output result.

[0017] In the dense fog processing module: the dense fog image is input into the Net-2 network through a series of convolutional layers and fusion layers. In the Net-2 network, the dense fog image is first processed by three consecutive convolutional layers, and the feature map is sent to the first fusion layer for fusion operation, and then processed by three convolutional layers and two fusion layers to obtain the output result;

[0018] In step S103, the optimization module includes hidden feature training and haze recognition training;

[0019] For hidden feature training, common intermediate features are selected in the pre-trained model, and the model is trained using the quadruple loss function. The following formula is the quadruple loss function:

[0020]

[0021] in And V i , i=1,2...n is the i-th hidden feature extracted from the fixed pre-trained model, D(x,y) is the L1 distance between x and y, ω i is the weight coefficient, Gt is the positive sample image, T1 is the corresponding light fog image, T2 is the corresponding dense fog image, and f(I) is the output image;

[0022] For haze recognition training, the model is trained by introducing the gradient loss function:

[0023] The gradient loss function is introduced to make the image anchor point closer to the positive sample in the feature space and farther away from the simple negative sample. At the same time, the characteristics of multi-haze images are learned, and the similarities and differences between different samples are compared, so that the model can better understand and process images with different haze levels.

[0024] Gradient loss function:

[0025]

[0026] Among them, f represents the output of the network, I represents the input image, Gx represents the clear image, represents the gradient operation;

[0027] The loss value is then calculated through the joint function, and the parameters of the model are updated through back propagation;

[0028] Joint loss function: Loss = min||Gt-f(I)||+α*L1+β*L2

[0029] Among them, α and β are weight parameters, f represents the output of the network, I represents the input image, and Gx represents the clear image.

[0030] Furthermore, in step S102, the collected image to be processed is converted into a grayscale image to focus on the brightness information, the brightness and contrast of the image to be processed are calculated, and the haze level of the image to be processed is evaluated based on the brightness and contrast.

[0031] Furthermore, the image grading preprocessing module includes image receiving and resizing and haze level grading:

[0032] Image reception and resizing: Receive the image to be processed and resize it to 256*256 pixels to meet the input requirements of the subsequent network model;

[0033] The haze grade classification includes the following steps:

[0034] S1011: Normalization processing: normalizing the image data from an integer range of 0-255 to a floating point range of 0-1 to calculate image brightness and contrast;

[0035] The normalization formula used in the normalization process is as follows:

[0036]

[0037] In the formula, NormalizedImg refers to the normalized pixel value, OriginalImg refers to the original pixel value, MinValue refers to the minimum pixel value in the image, and MaxValue refers to the maximum pixel value in the image;

[0038] S1012: converting the input image to be processed into a grayscale image, removing color information, and retaining only brightness information;

[0039] S1013: Calculate brightness and contrast, where the brightness is obtained by calculating the average value of the pixel values, and the contrast is obtained by calculating the standard deviation of the pixel values;

[0040] S1014: evaluating the haze level classification of the image to be processed, and classifying the image to be processed into different haze levels by setting a threshold and combining the haze level classification;

[0041] In step S1013:

[0042] Brightness calculation formula: μ = (1 / N) * ∑I,

[0043] Contrast calculation formula: σ=sqrt(1 / N)*∑(1-μ) 2 , where brightness is denoted as μ, contrast is denoted as σ, N represents the total number of pixels in the image, and I represents the grayscale value of each pixel;

[0044] In step S1014: haze evaluation formula: Haze = m*μ+n*σ, where m and n are haze weight parameters, and Haze represents haze;

[0045] In step S1014: The threshold is denoted by T.

[0046] The efficiency and effect of the defogging algorithm are improved, and the practicality and reliability of the system in complex environments are enhanced. The introduction of the gradient loss function ensures that the features of the defogging image are as close to the clear image as possible in the feature space, while being away from different types of negative sample images. This helps the model to better retain and restore the edge and detail information of the image while removing the fog. The use of gradient information strengthens the model's protection of the image structure during the processing process, avoids edge blurring and information loss, and improves the quality and authenticity of the defogging image. By combining the joint loss function of the quadruple loss and the gradient loss, the model can be optimized in multiple dimensions, not only learning to defog, but also learning how to maintain the structural integrity and clarity of the image under different fog conditions. This multi-angle learning strategy improves the performance of the model under unknown or complex fog conditions and enhances the generalization ability of the model. The use of the gradient loss function enables the model to adaptively adjust the learning focus according to the change of the fog level, and optimize the light fog and heavy fog images respectively. This adaptability ensures good defogging effects under various fog conditions, improves the practicality and flexibility of the entire system, and through the joint loss function and back propagation mechanism, the update of model parameters is more targeted, which can more effectively reduce irrelevant fog components while protecting the useful information of the image. This optimization strategy not only speeds up the training process, but also improves the defogging efficiency and effect of the model.

[0047] In addition, the present invention also proposes an adaptive image defogging and enhancement system in a mine dust and fog environment, including an image processing chip and a data processor, and uses an adaptive image defogging and enhancement method in a mine dust and fog environment as the operating logic, including an image processing chip, a data processor, an image grading preprocessing module, an adaptive processing module and an optimization module; the image grading preprocessing module performs feature extraction on the dust and fog images in the mine, the adaptive processing module completes the haze analysis of the dust and fog images in the mine through the data processor, and the optimization module trains and optimizes the model so that the model can better distinguish the haze levels.

[0048] The present invention has the following beneficial effects:

[0049] 1. The present invention classifies images according to their haze levels and designs different sub-network structures to perform defogging on images with different degrees of dust and fog, thus achieving efficient and accurate defogging of both light and dense fog images. This adaptive method avoids the limitations of using a single model for haze images and improves the stability and pertinence of the defogging effect.

[0050] 2. During the model training process, the four-tuple loss function of contrastive learning is used to introduce clear images, light fog images, and dense fog images as different types of samples to promote the model to learn richer and more complex feature representations to adapt to image processing at different haze levels. At the same time, adding gradient information to the loss function can ensure that the edge and detail information of the image is better preserved during the dehazing process, thereby improving the quality of the dehazed image.

[0051] 3. For light fog images, a shallower network architecture is used to avoid unnecessary waste of computing resources; for dense fog images, a more complex model structure is used to achieve better defogging effect. This resource allocation strategy can ensure both high efficiency in processing light fog images and accuracy in processing dense fog images, thereby achieving effective resource utilization and overall improvement of defogging efficiency at the system level. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] Figure 1 It is a flow chart of the method of the present invention.

[0053] Figure 2 It is a schematic diagram of a thin mist processing module and a dense fog processing module of the present invention.

[0054] Figure 3 It is a framework diagram of the image defogging algorithm provided by an embodiment of the present invention.

[0055] Figure 4 It is a schematic diagram of the optimization module method in an embodiment of the present invention.

[0056] Figure 5 Schematic diagram of defogging results at different haze levels in an embodiment of the present invention. DETAILED DESCRIPTION

[0057] The present invention will be further described in detail below in conjunction with the accompanying drawings and specific preferred embodiments.

[0058] In the description of the present invention, it should be understood that the terms "left side", "right side", "upper part", "lower part" and the like indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings, which is only for the convenience of describing the present invention and simplifying the description, and does not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation. "First", "second" and the like do not indicate the importance of the components, and therefore cannot be understood as limiting the present invention. The specific dimensions used in this embodiment are only for illustrating the technical solution, and do not limit the scope of protection of the present invention.

[0059] Reference Figure 1 It can be seen that an adaptive image defogging and enhancement method in a mine dust and fog environment mainly includes the following steps:

[0060] S101: performing classification preprocessing on the fog of the image in the mine;

[0061] S102: Input the classified mine fog image into the twin network, and adaptively input it into the appropriate twin network sub-network according to the complexity of the image;

[0062] S103: Use the loss function method to extract features from the dust and fog images in the mine, and use the quadruple loss function to improve the model's feature understanding and defogging quality for various types of fog images;

[0063] S104: adding the extracted image gradient difference information to the loss function to protect the image edge and detail information.

[0064] Step S101 specifically includes:

[0065] Image classification is achieved by analyzing the overall brightness and contrast of underground mine images to determine the thickness of dust and fog in the images. Image classification includes image reception, size adjustment and normalization processing;

[0066] Image reception and resizing: Receive the image to be processed and resize it to 256*256 pixels to meet the input requirements of the subsequent network model;

[0067] Normalization: Normalize the image data from the integer range of 0-255 to the floating point range of 0-1 so that the network model can better process the data;

[0068] The normalization formula is as follows:

[0069]

[0070] In the formula, NormalizedImg refers to the normalized pixel value, OriginalImg refers to the original pixel value, MinValue refers to the minimum pixel value in the image, and MaxValue refers to the maximum pixel value in the image;

[0071] When grading the haze level, the following steps are involved:

[0072] S1011: converting the input color image into a grayscale image to remove color information and retain only brightness information;

[0073] S1012: Calculate brightness and contrast. The brightness can usually be obtained by calculating the average value of pixel values, and the contrast can be obtained by calculating the standard deviation of pixel values.

[0074] S1013: Evaluate the haze of the image;

[0075] S1014: Classifying the image into different haze levels by setting a threshold and combining the haze evaluation result;

[0076] In step S1012:

[0077] Brightness calculation formula: μ = (1 / N) * ∑I,

[0078] Contrast calculation formula: σ=sqrt(1 / N)*∑(1-μ) 2 , where brightness is denoted as μ, contrast is denoted as σ, N represents the total number of pixels in the image, and I represents the grayscale value of each pixel;

[0079] In step S1014: haze evaluation formula: Haze = m*μ+n*σ, where m and n are weight parameters, and Haze is haze;

[0080] In step S1014: The threshold is denoted by T.

[0081] Reference Figure 2 ,It can be seen that for hazy images, a model architecture with shallow depth is ,adopted;

[0082] For images with dense fog, a more complex defogging algorithm is needed, increasing the number of model layers;

[0083] First, the haze image enters the Net-1 network as input, and passes through a series of convolutional layers (gConv) and fusion layers (Fusion). Specifically, in Net-1, the haze image is first processed by three consecutive convolutional layers. Then, these feature maps are sent to the first fusion layer for fusion operation, and then processed by two convolutional layers and one fusion layer to finally get the output result.

[0084] When processing dense fog images:

[0085] First, the dense fog image is taken as input into the Net-2 network through a series of convolutional layers (gConv) and fusion layers (Fusion). Specifically, in Net-2, the dense fog image is first processed by three consecutive convolutional layers, and the feature map is sent to the first fusion layer for fusion operation, and then processed by three convolutional layers and two fusion layers to obtain the output result.

[0086] Compared with the processing flow of light fog images, the number of fusion layers in the processing of dense fog images is greater, because dense fog images usually contain more noise and blurry information, and more fusion steps are required to remove interference and restore clear visual effects.

[0087] Reference Figure 3It can be seen that when training the dehazing network, inspired by contrastive learning, this algorithm aims to learn a representation to pull out the "positive" pairs in some metric space and separate them from the representations between the "negative" pairs. The above purpose is achieved through contrastive regularization. First, a positive sample and multiple negative samples are set. By comparing the similarities and differences between different samples, the model is driven to learn richer and more complex feature representations, so that the model can better understand and process images with different haze levels to produce better restored images.

[0088] Reference Figure 4 ,It is known that the contrastive learning algorithm mainly includes ,positive and negative sample data preparation and contrastive loss ,function;

[0089] Positive and negative sample data preparation:

[0090] The clear image is taken as the positive sample, and the light fog and dense fog images corresponding to the original input foggy image are taken as simple negative samples and complex negative samples;

[0091] Contrastive loss function:

[0092] For the latent feature space, common intermediate features are selected from the same fixed pre-trained model;

[0093] The following formula is the quadruple loss function:

[0094]

[0095] in And V i , i=1,2...n is the i-th hidden feature extracted from the fixed VGG-19 pre-trained model, D(x,y) is the L1 distance between x and y, ω i is the weight coefficient, Gt is the positive sample image, T1 is the corresponding light fog image, T2 is the corresponding dense fog image, and f(I) is the output image.

[0096] For this loss function, it emphasizes making the anchor point closer to the positive sample in the feature space, farther away from the simple negative sample, and farther away from the complex negative sample. At the same time, the characteristics of multi-haze images are learned, and the similarities and differences between different samples are compared, so that the model can better understand and process images with different haze levels.

[0097] Gradient loss function:

[0098]

[0099] Among them, f represents the output of the network, I represents the input image, Gx represents the clear image, Represents the gradient operation. The purpose of this loss function is to make the gradient image of the image output by the network as close as possible to the gradient image of the clear image, thereby protecting the edge information.

[0100] Joint loss function:

[0101] Loss=min||Gt-f(I)||+α*L1+β*L2

[0102] The loss value is calculated by the joint function, and the parameters of the network are updated through back propagation.

[0103] Reference Figure 5 It can be seen that when there is mist in the mine, the image is first processed by the mist network. The network is designed to be relatively shallow, with the aim of minimizing the calculation time while ensuring the defogging effect. The average processing time is 0.48 seconds, and the mist image can be effectively processed in a short time, thereby improving the real-time performance of the system. The fast processing speed of the mist network is conducive to rapid response and real-time monitoring, ensuring that clear images can be obtained in time under mist conditions. If the fog in the mine is high, the image will enter the dense fog network for processing. Compared with the mist network, the dense fog network has a deeper structure and contains more convolutional layers and fusion layers to cope with more serious fog problems. Although the processing time is longer (an average of 0.96 seconds), due to the increase in network depth, the dense fog network can remove fog more thoroughly and restore more image details, ensuring that the image can still be defogged with high quality under dense fog conditions. According to the level of fog, the system automatically selects the appropriate network for processing. The thin fog image uses the thin fog network, and the thick fog image uses the thick fog network. The adaptive processing strategy ensures that the most appropriate network can be selected for processing under any fog conditions, thereby maximizing processing efficiency while ensuring image quality. By distinguishing between thin fog and thick fog networks, the system ensures real-time performance while also ensuring image quality. The thin fog network has a fast processing speed and is suitable for real-time monitoring; the thick fog network has a good processing effect and is suitable for complex environments.

[0104] The preferred embodiments of the present invention are described in detail above; however, the present invention is not limited to the specific details in the above embodiments. Within the technical concept of the present invention, various equivalent transformations can be made to the technical solutions of the present invention, and these equivalent transformations all belong to the protection scope of the present invention.

Claims

1. An adaptive image defogging and enhancement method in a mine dust and fog environment, used for defogging and enhancing images in a mine dust and fog environment, characterized in that: The following steps are involved: S101: establishing an image processing model including an image classification preprocessing module, an adaptive processing module and an optimization module; S102: inputting the image processed by the image classification preprocessing module into the adaptive processing module; S103: further optimizing the image processed by the adaptive processing module through the image optimization module; S104: outputting the optimized image; In step S102, the adaptive processing module includes a mist processing module and a dense fog processing module; The haze processing module includes a Net-1 network, five convolutional layers, and two fusion layers; The dense fog processing module includes a Net-2 network, seven convolutional layers, and three fusion layers; In the haze processing module: the haze image is input into the Net-1 network and passes through a series of convolutional layers and fusion layers. In the Net-1 network, the haze image is first processed by three consecutive convolutional layers. Then, these feature maps are sent to the first fusion layer for fusion operation, and then processed by two convolutional layers and one fusion layer to obtain the output result. In the dense fog processing module: the dense fog image is input into the Net-2 network through a series of convolutional layers and fusion layers. In the Net-2 network, the dense fog image is first processed by three consecutive convolutional layers, and the feature map is sent to the first fusion layer for fusion operation, and then processed by three convolutional layers and two fusion layers to obtain the output result; In step S103, the optimization module includes hidden feature training and haze recognition training; For hidden feature training, common intermediate features are selected in the pre-trained model, and the model is trained using the quadruple loss function. The following formula is the quadruple loss function: in And V i , i=1,2...n is the i-th hidden feature extracted from the fixed pre-trained model, D(x,y) is the L1 distance between x and y, ω i is the weight coefficient, Gt is the positive sample image, T1 is the corresponding light fog image, T2 is the corresponding dense fog image, and f(I) is the output image; For haze recognition training, the model is trained by introducing the gradient loss function: The gradient loss function is introduced to make the image anchor point closer to the positive sample in the feature space and farther away from the simple negative sample. At the same time, the characteristics of multi-haze images are learned, and the similarities and differences between different samples are compared, so that the model can better understand and process images with different haze levels. Gradient loss function: Among them, f represents the output of the network, I represents the input image, Gx represents the clear image, represents the gradient operation; The loss value is then calculated through the joint function, and the parameters of the model are updated through back propagation; Joint loss function: Loss = min||Gt-f(I)||+α*L1+β*L2 Among them, α and β are weight parameters, f represents the output of the network, I represents the input image, and Gx represents the clear image.

2. The adaptive image defogging and enhancement method in a mine dust and fog environment according to claim 1 is characterized in that: In step S102, the collected image to be processed is converted into a grayscale image to focus on the brightness information, the brightness and contrast of the image to be processed are calculated, and the haze level of the image to be processed is evaluated based on the brightness and contrast.

3. The adaptive image defogging and enhancement method in a mine dust and fog environment according to claim 2 is characterized in that: The image grading preprocessing module includes image reception and resizing and haze level grading: Image reception and resizing: Receive the image to be processed and resize it to 256*256 pixels to meet the input requirements of the subsequent network model; The haze grade classification includes the following steps: S1011: Normalization processing: normalizing the image data from an integer range of 0-255 to a floating point range of 0-1 to calculate image brightness and contrast; The normalization formula used in the normalization process is as follows: In the formula, NormalizedImg refers to the normalized pixel value, OriginalImg refers to the original pixel value, MinValue refers to the minimum pixel value in the image, and MaxValue refers to the maximum pixel value in the image; S1012: converting the input image to be processed into a grayscale image, removing color information, and retaining only brightness information; S1013: Calculate brightness and contrast, where the brightness is obtained by calculating the average value of the pixel values, and the contrast is obtained by calculating the standard deviation of the pixel values; S1014: evaluating the haze level classification of the image to be processed, and classifying the image to be processed into different haze levels by setting a threshold and combining the haze level classification; In step S1013: Brightness calculation formula: μ = (1 / N) * ∑I, Contrast calculation formula: σ=sqrt(1 / N)*∑(1-μ) 2 , where brightness is denoted as μ, contrast is denoted as σ, N represents the total number of pixels in the image, and I represents the grayscale value of each pixel; In step S1014: haze evaluation formula: Haze = m*μ+p*σ, where m and p are haze weight parameters, and Haze represents haze; In step S1014: The threshold is denoted by T.

4. An adaptive image defogging and enhancement system in a mine dust and fog environment, characterized by: The invention comprises an adaptive image dehazing and enhancement method in a mine dust and fog environment as described in any one of claims 1 to 3, comprising an image processing chip, a data processor, an image grading preprocessing module, an adaptive processing module and an optimization module; the image grading preprocessing module performs feature extraction on the dust and fog images in the mine, the adaptive processing module completes the haze analysis of the dust and fog images in the mine through the data processor, and the optimization module trains and optimizes the model so that the model can better distinguish the haze levels.

Images