Coal mine underground image enhancement method and device, and medium

Abstract

The invention discloses an underground coal mine image enhancement method, equipment and a medium. The method comprises the following steps: acquiring an underground sRGB image of a coal mine and a corresponding infrared image; obtaining a dust density estimation map according to the infrared image; obtaining a dust sensing intensity factor according to the sRGB image and the dust density estimation map; performing brightness enhancement on the sRGB image according to the dust sensing intensity factor to obtain a self-adaptive HVI image; and obtaining an enhanced sRGB image after enhancement according to the self-adaptive HVI image. The method can adapt to complex working conditions of underground dust scattering and uneven illumination, and the coal mine underground image is optimized and enhanced by estimating the dust density of the quantized environment and enhancing illumination.

Description

Technical Field

[0001] This invention relates to the field of underground coal mine production, and more particularly to a method, device, and medium for enhancing underground coal mine images. Background Technology

[0002] The underground environment of coal mines is characterized by poor lighting conditions, high dust concentration, and complex dynamic changes, which leads to problems such as low-light noise, color distortion, and loss of detail in the images collected by monitoring systems.

[0003] Existing methods for enhancing images acquired by monitoring systems are ill-suited to the complex working conditions of underground coal mine production environments, such as dust scattering and uneven lighting, thus hindering the development of intelligent safety production in coal mines. Summary of the Invention

[0004] This invention aims to at least solve one of the technical problems existing in the prior art. To this end, this invention proposes a method for enhancing underground coal mine images, which can adapt to the complex working conditions of underground dust scattering and uneven illumination. The method optimizes and enhances underground coal mine images by estimating and quantifying the dust density of the environment and enhancing illumination.

[0005] The present invention also proposes an apparatus and a medium having the above-described method for enhancing underground coal mine images.

[0006] A method for enhancing underground coal mine images according to a first aspect of the present invention includes: Acquire sRGB images and their corresponding infrared images from underground coal mines; Based on the infrared image, a dust density estimation map is obtained; Based on the sRGB image and the dust density estimation map, the dust perception intensity factor is obtained; The sRGB image is enhanced in brightness according to the dust perception intensity factor to obtain an adaptive HVI image; Based on the adaptive HVI diagram, the enhanced sRGB image is obtained.

[0007] An image enhancement method for underground coal mines according to an embodiment of the present invention has at least the following beneficial effects: The present invention first acquires an sRGB image of the underground coal mine and its corresponding infrared image, and uses the infrared image with low light sensitivity to estimate the dust density of the underground coal mine, thereby obtaining a dust perception intensity factor; the dust perception intensity factor is used to reduce the influence of dust on the underground coal mine image, and brightness is enhanced to clarify the underground coal mine image, finally obtaining an enhanced underground coal mine image; the enhanced image of the present invention can adapt to the complex working conditions of underground dust scattering and uneven illumination, and optimizes the enhancement of underground coal mine images by estimating and quantifying the dust density of the environment and enhancing illumination.

[0008] According to some embodiments of the present invention, obtaining the dust density estimation map based on the infrared image includes: The infrared image is input into a convolutional neural network; In the convolutional neural network, each pixel in the infrared image is normalized, and the image after all pixels are normalized is used as the dust density estimation map.

[0009] According to some embodiments of the present invention, obtaining the dust perception intensity factor based on the sRGB image and the dust density estimation map includes: The maximum value of the R, G, and B channels of each pixel in the sRGB image is used as the value of that pixel to obtain the intensity map corresponding to the sRGB image. Pixel-level parameters are obtained based on the intensity map; The dust sensing intensity factor is obtained based on the intensity map, the pixel-level parameters, the dust scattering attenuation factor, the constant coefficient, and the dust density estimation map.

[0010] According to some embodiments of the present invention, obtaining pixel-level parameters based on the intensity map includes: The intensity map is input into the parameter generation network; In the parameter generation network, the intensity map is sequentially subjected to feature extraction, downsampling, and upsampling; The pixel-level parameters are obtained based on the processed intensity map and the preset mapping relationship.

[0011] According to some embodiments of the present invention, the step of enhancing the brightness of the sRGB image based on the dust perception intensity factor to obtain an adaptive HVI image includes: The sRGB image is mapped to the HVI space to obtain the HVI image; The HVI graph is input into the horizontal-vertical branch of the CIDNet network, and the intensity graph is input into the intensity branch of the CIDNet network. The image output by the CIDNet network is used as the adaptive HVI image.

[0012] According to some embodiments of the present invention, obtaining the enhanced sRGB image based on the adaptive HVI diagram includes: The adaptive HVI image is converted into an RGB image, and the converted RGB image is the enhanced sRGB image; Based on the sRGB image and the enhanced sRGB image, the color space loss terms before and after sRGB enhancement are obtained; Based on the HVI diagram and the adaptive HVI diagram, the color space loss terms before and after HVI enhancement are obtained; The total loss is obtained by concatenating the color space loss terms before and after sRGB enhancement and the color space loss terms before and after HVI enhancement, and then adjusting the dust scattering attenuation factor and the constant coefficient accordingly.

[0013] According to some embodiments of the present invention, the color space loss term includes: the color space loss term before and after sRGB enhancement, and the color space loss term before and after HVI enhancement; The color space loss term is obtained in the following ways: The color space loss term is obtained by calculating the absolute error between each pixel in the image before and after enhancement, the structural similarity loss of the image before and after enhancement, and the Laplacian operator edge loss.

[0014] According to some embodiments of the present invention, acquiring sRGB images and their corresponding infrared images in a coal mine includes: acquiring sRGB image sequences and infrared image sequences; the total loss further includes: a temporal smoothing term; The method further includes: taking a weighted average of the differences between the dust perception intensity factor and the pixel-level parameters corresponding to the current frame and the previous frame to obtain the temporal smoothing term.

[0015] An electronic device according to a second aspect of the present invention includes: Memory, used to store programs; A processor for executing a program stored in the memory, wherein when the processor executes the program stored in the memory, the processor is configured to perform the method as described in any one of the first aspects.

[0016] According to a third aspect of the present invention, a storage medium stores computer-executable instructions for performing the method as described in any one of the first aspects.

[0017] Other features and advantages of the invention will be set forth in the description which follows, and will be apparent in part from the description, or may be learned by practicing the invention. The objects and other advantages of the invention may be realized and obtained by means of the structures particularly pointed out in the description, claims, and drawings. Attached Figure Description

[0018] The accompanying drawings are provided to further understand the technical solutions of the present invention and constitute a part of the specification. They are used together with the embodiments of the present invention to explain the technical solutions of the present invention, and do not constitute a limitation on the technical solutions of the present invention.

[0019] Figure 1 This is a flowchart of a method for enhancing underground coal mine images according to an embodiment of the present invention. Detailed Implementation

[0020] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0021] It should be understood that in the description of the embodiments of the present invention, "multiple" (or "amounts") means two or more, "greater than," "less than," and "exceeding" are understood to exclude the stated number, while "above," "below," and "within" are understood to include the stated number. If "first," "second," etc., are used in the description, they are only for the purpose of distinguishing technical features and should not be construed as indicating or implying relative importance, or implicitly indicating the number of indicated technical features, or implicitly indicating the order of the indicated technical features.

[0022] like Figure 1 As shown, this embodiment of the invention provides a method for enhancing underground coal mine images, including: Step S100: Obtain the sRGB image and its corresponding infrared image from underground in the coal mine; Step S200: Obtain a dust density estimation map based on the infrared image; Step S300: Obtain the dust perception intensity factor based on the sRGB image and the dust density estimation map; Step S400: Enhance the brightness of the sRGB image according to the dust perception intensity factor to obtain an adaptive HVI image; Step S500: Obtain the enhanced sRGB image based on the adaptive HVI diagram.

[0023] This invention first acquires sRGB images and their corresponding infrared images of underground coal mines. The infrared images, which have low light sensitivity, are then used to estimate the dust density in the coal mine, thereby obtaining a dust perception intensity factor. This dust perception intensity factor is used to reduce the impact of dust on the underground coal mine images, and brightness is enhanced to clarify the images, ultimately resulting in an enhanced underground coal mine image. The enhanced image obtained by this invention can adapt to complex working conditions such as dust scattering and uneven lighting underground, optimizing and enhancing the underground coal mine image by estimating and quantifying the dust density of the environment and enhancing lighting.

[0024] In one embodiment, in step S100, the sRGB image and the infrared image are acquired at the same time, and therefore can be considered to correspond to each other.

[0025] It should be noted that sRGB images are obtained through visible light images, which are obtained through general image acquisition equipment and are highly sensitive to light. In contrast, infrared images are obtained through infrared sensors, based on the thermal radiation of objects, and are less sensitive to ambient light levels during image acquisition.

[0026] In one embodiment, in step S200, obtaining the dust density estimation map based on the infrared image includes: Infrared images are input into a convolutional neural network; In a convolutional neural network, each pixel in the infrared image is normalized individually, and the image after all pixels are normalized is used as the dust density estimation map.

[0027] In one embodiment, the convolutional neural network is a pre-trained network with 3 convolutional layers; normalizing each pixel in the infrared image specifically includes: determining whether each pixel in the infrared image has dust; if so, the value of the pixel is 1, otherwise it is 0.

[0028] In one embodiment, in step S300, obtaining the dust perception intensity factor based on the sRGB image and the dust density estimation map includes: The maximum value of the R, G, and B channels of each pixel in the sRGB image is used as the value of that pixel to obtain the intensity map corresponding to the sRGB image; Pixel-level parameters are obtained based on the intensity map; The dust perception intensity factor is obtained based on the intensity map, pixel-level parameters, dust scattering attenuation factor, constant coefficient, and dust density estimation map.

[0029] In one embodiment, obtaining pixel-level parameters based on the intensity map includes: The intensity map is input into the parameter generation network K-Net; In the parameter generation network K-Net, the intensity map is sequentially subjected to feature extraction, downsampling, and upsampling. Pixel-level parameters are obtained based on the processed intensity map and the preset mapping relationship.

[0030] It should be noted that the parameter generation network K-Net is a pre-trained network.

[0031] In one embodiment, the mapping relationship between the intensity map and pixel-level parameters includes: Dark Zone Strongly compressed black noise; bright area Weak compression preserves details; This is equivalent to: when the value of a pixel in the intensity map is less than 0.1, the corresponding pixel-level parameter is 2.5, thereby strongly compressing black noise; When the value of a pixel in the intensity map is greater than 0.7, the corresponding pixel-level parameter is 0.5, thus weakly compressing to preserve details; For the remaining pixels that do not meet the above range, the pixel-level parameter is 1.

[0032] In one embodiment, pixel-level parameters are mapped to [0.5, 2.5].

[0033] In one embodiment, the dust sensing intensity factor is obtained based on the intensity map, dust scattering attenuation factor, constant coefficient, and dust density estimation map, specifically as follows: The expression for the dust sensing intensity factor is as follows:

[0034] in, Dust sensing intensity factor, This is a graph showing the estimated dust density. The dust scattering attenuation factor. Pixel-level parameters For intensity maps, These are constant coefficients. These are the pixel values ​​in the image; The initial value is 2.0.

[0035] In one embodiment, in step S400, the sRGB image (i.e., the image according to the standard red-green-blue protocol) is brightness-enhanced according to the dust perception intensity factor to obtain an adaptive HVI image, including: Mapping the sRGB image to the HVI space yields the HVI image; The HVI image is input into the horizontal-vertical branch of the CIDNet network (color and intensity decoupled network), and the intensity image is input into the intensity branch of the CIDNet network. The image output by the CIDNet network is used as an adaptive HVI image.

[0036] It is easy to understand that mapping an sRGB image to HVI space means converting the traditional HSV space into an orthogonal HVI (Horizontal-Vertical-Intensity) space.

[0037] In one embodiment, the CIDNet network is a pre-trained color and intensity decoupled network that denoises the color of the image through horizontal-vertical branches and enhances the brightness through intensity branches. This dual-branch enhancement reduces the impact of uneven lighting on underground coal mine images.

[0038] In one embodiment, mapping the sRGB image to the HVI space to obtain the HVI image is specifically as follows: Mapping an sRGB image to the HSV space yields an HSV image; Based on the HSV diagram and intensity diagram, the HVI diagram is obtained, specifically including:

[0039] in, , ; Here is the HVI plot at time t. Let H be the H channel of the HVI plot at time t. Let V be the HVI plot at time t. This is the intensity map at time t. The dust sensing intensity factor in step S300. This is the saturation map in the HSV graph. This is the Hue map in the HSV graph.

[0040] It is easy to understand that, For each moment A general term, Similarly, the image obtained in step S100 can be a single image or a sequence of images at multiple time points.

[0041] In one embodiment, in step S500, obtaining the enhanced sRGB image based on the adaptive HVI image includes: The adaptive HVI image is converted into an RGB image, and the converted RGB image is the enhanced sRGB image; Based on the sRGB image and the enhanced sRGB image, the color space loss terms before and after sRGB enhancement are obtained; Based on the HVI diagram and the adaptive HVI diagram, the color space loss terms before and after HVI enhancement are obtained; The total loss is obtained by concatenating the color space loss terms before and after sRGB enhancement and HVI enhancement, and then adjusting the dust scattering attenuation factor and constant coefficient accordingly.

[0042] It is easy to understand that converting an HVI image to an RGB image is the reverse process of converting an RGB image to an HVI image.

[0043] In one embodiment, the total loss obtained by concatenating the color space loss terms before and after sRGB enhancement and HVI enhancement includes: The expression for the total loss is as follows:

[0044] in, For the total loss, The color space loss term before and after HVI enhancement. For the color space loss term before and after sRGB enhancement, For time series smoothing, The preset HVI weights, The preset time series weights.

[0045] In one embodiment, the color space loss term includes: color space loss term before and after sRGB enhancement, and color space loss term before and after HVI enhancement; The color space loss term can be obtained in the following ways: The color space loss term is obtained by calculating the absolute error between each pixel in the image before and after enhancement, the structural similarity loss of the image before and after enhancement, and the Laplacian operator edge loss.

[0046] In one embodiment, the expression for the color space loss term is as follows:

[0047] in, For color space loss terms, For the enhanced image, For the image before enhancement, For absolute error, For structural similarity loss, For the edge loss of the Laplacian operator, , , These are the preset corresponding coefficients.

[0048] In one embodiment, in step S100, acquiring the sRGB image and its corresponding infrared image in the coal mine includes: acquiring the sRGB image sequence and the infrared image sequence; the total loss also includes: a temporal smoothing term; The method also includes: taking a weighted average of the differences between the dust perception intensity factor and the pixel-level parameters corresponding to the current frame and the previous frame to obtain a temporal smoothing term.

[0049] In one embodiment, the timing smoothing term :

[0050] Where t is the video frame number and T is the total number of frames. For time t (i.e., video frame number t), the pixel-level parameter is... Let be the dust perception intensity factor at time t (i.e., video frame number t). To calculate the L2 norm (Euclidean norm) of the pixel-level parameters at pixel x in the preceding and following video frames. To calculate the L2 norm (Euclidean norm) of the dust perception intensity factor at pixel x in the preceding and following video frames.

[0051] Because underground coal mines are relatively enclosed, the monitoring video may experience temporal instability and inter-frame flickering during upload. The temporal smoothing term is used to force the dust perception intensity factor and pixel-level parameters of adjacent frames to change smoothly and suppress flickering. This invention uses the temporal smoothing term to mitigate the impact of inter-frame flickering.

[0052] This invention is essentially an adaptive low-light enhancement method based on an improved HVI-CIDNet color space. This method is designed for complex underground coal mine environments (such as dust interference, uneven lighting, and video timing jitter). Through three core technologies—dust noise modeling, regional adaptive compression, and timing consistency constraints—it significantly improves the detail visibility and color fidelity of underground low-light images. It is applicable to scenarios such as coal mine safety monitoring and equipment inspection, providing reliable visualization technology support for the construction of intelligent mines.

[0053] This invention also provides an electronic device, which includes, but is not limited to: Memory, used to store programs; The processor is used to execute programs stored in memory. When the processor executes the programs stored in memory, it is used to execute the aforementioned method for enhancing images in underground coal mines.

[0054] The processor and memory can be connected via a bus or other means.

[0055] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs, such as the method described in the embodiments of the present invention. The processor implements the above method by running the non-transitory software program and instructions stored in the memory.

[0056] The memory may include a program storage area and a data storage area, wherein the program storage area may store the operating system and application programs required for at least one function; the data storage area may store data for executing the methods described above. Furthermore, the memory may include high-speed random access memory and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, the memory may optionally include memory remotely located relative to the processor, which can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0057] The non-transitory software program and instructions required to implement the above terminal selection method are stored in memory and are executed by one or more processors.

[0058] This invention also provides a storage medium storing computer-executable instructions for performing the above-described methods.

[0059] In one embodiment, the storage medium stores computer-executable instructions that are executed by one or more control processors.

[0060] The embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0061] It will be understood by those skilled in the art that all or some of the steps and systems in the methods disclosed above can be implemented as software, firmware, hardware, and suitable combinations thereof. Some or all of the physical components can be implemented as software executed by a processor, such as a central processing unit, digital signal processor, or microprocessor, or as hardware, or as an integrated circuit, such as an application-specific integrated circuit. Such software can be distributed on a computer-readable medium, which can include computer storage media (or non-transitory media) and communication media (or transient media). As is known to those skilled in the art, the term computer storage media includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules, or other data). Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technologies, CD-ROM, digital versatile disc (DVD) or other optical disc storage, magnetic cartridges, magnetic tape, disk storage or other magnetic storage devices, or any other medium that can be used to store desired information and is accessible to a computer. Furthermore, as is known to those skilled in the art, communication media typically include computer-readable instructions, data structures, program modules, or other data in modulated data signals such as carrier waves or other transmission mechanisms, and may include any information delivery medium.

[0062] This document describes embodiments of the invention, including preferred embodiments known to the inventors for carrying out the invention. Variations of these embodiments will become apparent to those skilled in the art upon reading the foregoing description. The inventors encourage those skilled in the art to adopt such variations as appropriate, and the inventors intend to practice embodiments of the invention in ways other than those specifically described herein. Therefore, the scope of the invention includes all modifications and equivalents of the subject matter set forth in the appended claims, as permitted by applicable law. Furthermore, the scope of the invention covers any combination of the foregoing elements in all possible variations thereof, unless otherwise indicated herein or otherwise clearly contradicted by the context.

Claims

1. A method for enhancing underground coal mine images, characterized in that, include: Acquire sRGB images and their corresponding infrared images from underground coal mines; Based on the infrared image, a dust density estimation map is obtained; Based on the sRGB image and the dust density estimation map, the dust perception intensity factor is obtained; The sRGB image is enhanced in brightness according to the dust perception intensity factor to obtain an adaptive HVI image; Based on the adaptive HVI diagram, the enhanced sRGB image is obtained.

2. The method for enhancing underground coal mine images according to claim 1, characterized in that, The step of obtaining the dust density estimation map based on the infrared image includes: The infrared image is input into a convolutional neural network; In the convolutional neural network, each pixel in the infrared image is normalized, and the image after all pixels are normalized is used as the dust density estimation map.

3. The method for enhancing underground coal mine images according to claim 1, characterized in that, The step of obtaining the dust perception intensity factor based on the sRGB image and the dust density estimation map includes: The maximum value of the R, G, and B channels of each pixel in the sRGB image is used as the value of that pixel to obtain the intensity map corresponding to the sRGB image. Pixel-level parameters are obtained based on the intensity map; The dust sensing intensity factor is obtained based on the intensity map, the pixel-level parameters, the dust scattering attenuation factor, the constant coefficient, and the dust density estimation map.

4. The method for enhancing underground coal mine images according to claim 3, characterized in that, The step of obtaining pixel-level parameters based on the intensity map includes: The intensity map is input into the parameter generation network; In the parameter generation network, the intensity map is sequentially subjected to feature extraction, downsampling, and upsampling; The pixel-level parameters are obtained based on the processed intensity map and the preset mapping relationship.

5. The method for enhancing underground coal mine images according to claim 3, characterized in that, The step of enhancing the brightness of the sRGB image according to the dust perception intensity factor to obtain an adaptive HVI image includes: The sRGB image is mapped to the HVI space to obtain the HVI image; The HVI graph is input into the horizontal-vertical branch of the CIDNet network, and the intensity graph is input into the intensity branch of the CIDNet network. The image output by the CIDNet network is used as the adaptive HVI image.

6. The method for enhancing underground coal mine images according to claim 5, characterized in that, The process of obtaining the enhanced sRGB image based on the adaptive HVI image includes: The adaptive HVI image is converted into an RGB image, and the converted RGB image is the enhanced sRGB image; Based on the sRGB image and the enhanced sRGB image, the color space loss terms before and after sRGB enhancement are obtained; Based on the HVI diagram and the adaptive HVI diagram, the color space loss terms before and after HVI enhancement are obtained; The total loss is obtained by concatenating the color space loss terms before and after sRGB enhancement and the color space loss terms before and after HVI enhancement, and then adjusting the dust scattering attenuation factor and the constant coefficient accordingly.

7. The method for enhancing underground coal mine images according to claim 6, characterized in that, The color space loss term includes: the color space loss term before and after sRGB enhancement, and the color space loss term before and after HVI enhancement; The color space loss term is obtained in the following ways: The color space loss term is obtained by calculating the absolute error between each pixel in the image before and after enhancement, the structural similarity loss of the image before and after enhancement, and the Laplacian operator edge loss.

8. The method for enhancing underground coal mine images according to claim 6, characterized in that, Acquiring sRGB images and their corresponding infrared images from underground coal mines includes: acquiring sRGB image sequences and infrared image sequences; the total loss also includes: a temporal smoothing term; The method further includes: taking a weighted average of the differences between the dust perception intensity factor and the pixel-level parameters corresponding to the current frame and the previous frame to obtain the temporal smoothing term.

9. An electronic device, characterized in that, include: Memory, used to store programs; A processor for executing a program stored in the memory, wherein when the processor executes the program stored in the memory, the processor is configured to perform the method as described in any one of claims 1 to 8.

10. A storage medium, characterized in that, The device stores computer-executable instructions for performing the method as described in any one of claims 1 to 8.

Images