Coal mine underground low-light image enhancement method and device and electronic equipment

Abstract

This application provides a method, apparatus, and electronic device for enhancing low-light images in coal mines, relating to the field of image enhancement technology. The method includes: acquiring a low-light monitoring image from underground coal mines; inputting the low-light monitoring image into an image enhancement model to obtain an image enhancement parameter map corresponding to the low-light monitoring image. The image enhancement model is a model trained on a dual-branch model based on a linear transformation mechanism and a self-attention mechanism, used to fuse local and global image features and generate the image enhancement parameter map. The image enhancement parameter map includes enhancement parameters for each pixel location in the low-light monitoring image; and determining a target enhanced image based on the low-light monitoring image and the image enhancement parameter map. This application aims to solve the problem of uneven local enhancement, improve the enhancement effect of low-light images, and enhance image quality and information clarity.

Description

Technical Field

[0001] This application relates to the field of image enhancement technology, and in particular to a method, apparatus and electronic device for enhancing low-light images in coal mines. Background Technology

[0002] With the continuous advancement of intelligent construction in the coal industry, safety production supervision and equipment operation status monitoring in underground coal mines increasingly rely on intelligent technologies. Among these, technologies such as personnel behavior analysis, safety production supervision, and equipment operation status perception based on video surveillance have become key supports for ensuring safe production in underground coal mines. The monitoring scenario in underground coal mines has its own unique characteristics. Compared to monitoring systems with stable imaging conditions in surface environments, natural light sources are extremely scarce underground, relying mainly on artificial lighting. Furthermore, artificial lighting has significant limitations in spatial distribution and illumination range. At the same time, the large amount of suspended dust generated during operations causes light scattering. These factors result in underground coal mine monitoring images generally exhibiting problems such as low brightness, uneven illumination distribution, and missing details, seriously affecting the accuracy and reliability of subsequent visual analysis and intelligent perception tasks. Therefore, low-light image enhancement technology for underground coal mines, as a key technology for improving monitoring image quality and ensuring the intelligent development of safe coal mine production, has become a focus and hot topic in the current field of intelligent coal mine research.

[0003] In existing technologies, traditional low-light image enhancement methods mostly employ manually designed image processing strategies, such as histogram equalization and gamma correction. Histogram equalization adjusts the gray-level histogram distribution of an image to make the gray-level distribution more uniform, thereby enhancing image contrast. Gamma correction, on the other hand, adjusts the image brightness by performing a non-linear transformation on the gray-level values ​​to improve the visual effect of the image. However, these traditional methods are poorly adaptable to the complex and variable lighting conditions in coal mines, making it difficult to balance overall brightness enhancement with preservation of local details. In recent years, the Zero-Reference Deep Curve Estimation (Zero-DCE) method based on deep learning has received widespread attention. This method learns pixel-level non-linear mapping relationships, achieving image enhancement without paired labeled data and exhibiting low dependence on training data. Its core lies in using a network to learn the curve parameters of the image, and dynamically enhancing brightness and adjusting contrast by adjusting these parameters.

[0004] However, existing methods still suffer from over-enhancement or under-enhancement of local areas when processing low-light monitoring images in coal mines, resulting in poor enhancement effects for low-light images. Summary of the Invention

[0005] The low-light image enhancement method, apparatus, and electronic equipment provided in this application are used to solve the problem of over-enhancement or under-enhancement in local areas when processing low-light monitoring images in coal mines using existing methods, thereby improving the enhancement effect of low-light images.

[0006] In a first aspect, embodiments of this application provide a method for enhancing low-light images in underground coal mines, including:

[0007] Acquire low-light monitoring images from underground coal mines;

[0008] The low-light monitoring image is input into the image enhancement model to obtain the image enhancement parameter map corresponding to the low-light monitoring image. The image enhancement model is a model that is trained by a two-branch model based on a linear transformation mechanism and a self-attention mechanism to fuse local and global features of the image and generate the image enhancement parameter map. The image enhancement parameter map includes the enhancement parameters of the low-light monitoring image at each pixel position.

[0009] Based on low-light surveillance images and image enhancement parameter maps, the target enhanced image is determined.

[0010] In one possible implementation, the image enhancement model includes a global feature extraction branch and a convolutional backbone network. Inputting a low-light monitoring image into the image enhancement model to obtain an image enhancement parameter map corresponding to the low-light monitoring image includes: inputting the low-light monitoring image into the convolutional backbone network to extract local feature maps of the low-light monitoring image; inputting the low-light monitoring image into the global feature extraction branch to extract global feature maps of the low-light monitoring image; fusing the local feature maps and the global feature maps to obtain a fused feature map; and generating an image enhancement parameter map corresponding to the low-light monitoring image based on the fused feature map.

[0011] In one possible implementation, the global feature extraction branch includes, in sequence, image patch embedding, category labeling, location embedding, a Transformer encoder, a linear layer, and a broadcast module; wherein: image patch embedding is used to perform block embedding and dimensional transformation processing on the low-light monitoring image to obtain image patch features in sequence form; category labeling is used to concatenate learnable category labels to the front of the sequence of image patch features to obtain a feature sequence with classification labels; location embedding is used to add location information to the feature sequence with classification labels to obtain a feature sequence with location awareness; the Transformer encoder is used to perform global modeling and feature encoding on the feature sequence with location awareness through a self-attention mechanism, and output the encoded global feature sequence; the linear layer is used to perform dimensional mapping on the category labels extracted from the encoded global feature sequence to obtain a global feature vector of a specified dimension; and the broadcast module is used to perform dimensional expansion and spatial broadcasting on the global feature vector to obtain a global feature map of the low-light monitoring image.

[0012] In one possible implementation, the convolutional backbone network includes multiple sequentially connected Ghost convolutional modules. These Ghost convolutional modules are used to extract features from low-light surveillance images based on a linear transformation mechanism, generating local feature maps of the low-light surveillance images. Specifically, each Ghost convolutional module is used to: perform a convolution operation on the input feature map to generate an intrinsic feature map corresponding to the input feature map; perform a linear transformation operation on the intrinsic feature map to generate the remaining feature maps corresponding to the intrinsic feature map; and concatenate the intrinsic feature map with the remaining feature maps to output a feature map.

[0013] In one possible implementation, the convolutional backbone network further includes at least one global context module, which is connected after a preset Ghost convolutional module. The global context module is used to perform global context modeling on the feature map output by the Ghost convolutional module to obtain an enhanced feature map with global context information, and output the enhanced feature map to the next Ghost convolutional module.

[0014] In one possible implementation, there are 7 Ghost convolutional modules and 2 global context modules. Specifically, the first Ghost convolutional module extracts features from the low-light monitoring image and outputs a first feature map; the second Ghost convolutional module extracts features from the first feature map and outputs a second feature map; the third Ghost convolutional module extracts features from the second feature map and outputs a third feature map; the fourth Ghost convolutional module extracts features from the third feature map and outputs a fourth feature map; the first global context module performs global context modeling on the fourth feature map to obtain a first enhanced feature map; the fifth Ghost convolutional module extracts features from the fusion result of the first enhanced feature map and the third feature map and outputs a fifth feature map; the second global context module performs global context modeling on the fifth feature map to obtain a second enhanced feature map; the sixth Ghost convolutional module extracts features from the fusion result of the second enhanced feature map and the second feature map and outputs a sixth feature map; and the seventh Ghost convolutional module extracts features from the fusion result of the sixth feature map and the first feature map and outputs a seventh feature map, which is then used as a local feature map output by the convolutional backbone network.

[0015] In one possible implementation, determining the target enhanced image based on the low-light monitoring image and the image enhancement parameter map includes: obtaining the target enhanced image based on the low-light monitoring image and the image enhancement parameter map using a preset reciprocal mapping function; wherein the reciprocal mapping function satisfies:

[0016] ;

[0017] in, These are pixel coordinates; For low-light monitoring images Pixel value at; Enhance the image for the target Pixel value at; The corresponding parameters in the image enhancement diagram Enhanced parameters at that location.

[0018] In one possible implementation, before inputting the low-light monitoring image into the image enhancement model to obtain the image enhancement parameter map corresponding to the low-light monitoring image, the method further includes: acquiring monitoring video from underground coal mines; extracting multiple image frames from the monitoring video based on a preset time interval to obtain model training data; and training an initial image enhancement model based on the model training data until the output information of the initial image enhancement model meets the preset training requirements to obtain the image enhancement model.

[0019] Secondly, embodiments of this application provide a low-light image enhancement device for underground coal mines, comprising:

[0020] The acquisition module is used to acquire low-light monitoring images from underground coal mines;

[0021] The processing module is used to input the low-light monitoring image into the image enhancement model to obtain the image enhancement parameter map corresponding to the low-light monitoring image. The image enhancement model is a model that is trained by a dual-branch model based on a linear transformation mechanism and a self-attention mechanism to fuse local and global features of the image and generate the image enhancement parameter map. The image enhancement parameter map includes the enhancement parameters of the low-light monitoring image at each pixel position.

[0022] The determination module is used to determine the target enhanced image based on the low-light surveillance image and the image enhancement parameter map.

[0023] Thirdly, embodiments of this application provide an electronic device, including: a memory and a processor;

[0024] The memory stores instructions that the computer executes;

[0025] The processor executes computer execution instructions stored in memory, causing the processor to perform the first aspect and / or various possible implementations of the first aspect as described above.

[0026] Fourthly, embodiments of this application provide a computer-readable storage medium storing computer-executable instructions, which, when executed, are used to implement the first aspect and / or various possible implementations of the first aspect.

[0027] Fifthly, embodiments of this application provide a computer program product, including a computer program that, when executed, implements the first aspect and / or various possible implementations of the first aspect.

[0028] The low-light image enhancement method, apparatus, and electronic equipment provided in this application acquire low-light monitoring images from underground coal mines. These images are then input into an image enhancement model to obtain an image enhancement parameter map. The image enhancement model is a model trained on a dual-branch model based on a linear transformation mechanism and a self-attention mechanism. This model is used to fuse local and global image features and generate the image enhancement parameter map. The image enhancement parameter map includes enhancement parameters for each pixel location in the low-light monitoring image. Based on the low-light monitoring image and the image enhancement parameter map, a method for determining the target enhanced image is provided. This method can solve the problem of over-enhancement or under-enhancement in local areas, making the image brightness distribution more uniform, the enhancement effect more natural, improving image quality and information clarity, and enhancing adaptability to complex underground coal mine environments such as unstable lighting and dust interference. This provides reliable data support for subsequent visual analysis and intelligent perception tasks. Attached Figure Description

[0029] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0030] Figure 1 A flowchart illustrating the low-light image enhancement method for underground coal mines provided in this application. Figure 1 ;

[0031] Figure 2 Schematic diagram of the image enhancement model provided in this application Figure 1 ;

[0032] Figure 3 Schematic diagram of the global feature extraction branch provided in this application Figure 1 ;

[0033] Figure 4 Schematic diagram of the global feature extraction branch provided in this application Figure 2 ;

[0034] Figure 5 A schematic diagram of the structure of the global context module provided in this application;

[0035] Figure 6 Schematic diagram of the image enhancement model provided in this application Figure 2 ;

[0036] Figure 7 A flowchart illustrating the low-light image enhancement method for underground coal mines provided in this application. Figure 2 ;

[0037] Figure 8 A schematic diagram of the multiple iteration mapping function provided in this application. Figure 1 ;

[0038] Figure 9 A schematic diagram of the multiple iteration mapping function provided in this application. Figure 2 ;

[0039] Figure 10 A schematic diagram of the reciprocal mapping function provided in this application;

[0040] Figure 11 A schematic diagram of the structure of the coal mine low-light image enhancement device provided in this application;

[0041] Figure 12 A schematic diagram of the structure of the electronic device provided in this application.

[0042] The accompanying drawings illustrate specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to particular embodiments. Detailed Implementation

[0043] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0044] First, let me explain the terms used in this application:

[0045] Zero-DCE (Zero-Reference Deep Curve Estimation) is a referenceless deep learning image enhancement method specifically designed for low-light conditions. Its goal is to improve the visibility of images in low-light environments while maintaining a natural visual effect as much as possible. The network structure typically includes seven convolutional layers: the first six layers use Conv+ReLU, and the last layer uses Conv-Than. Symmetrical cascading connections are used, meaning the outputs of layers 1 / 2 / 3 are channel-concatenated with the outputs of layers 6 / 5 / 4, respectively. The network input is a low-light image (H×W×3), and the network output is the enhanced image and enhancement curve parameters A (also of size H×W×3). In other words, each pixel (i,j) has a set of enhancement coefficients A(i,j,c) (c being the channel R / G / B), used to control the magnitude and method of brightness enhancement for that pixel. Subsequently, image enhancement is achieved through a recursive curve function, a process typically repeated multiple times (usually eight iterations) to gradually increase brightness.

[0046] Existing Zero-DCE and its related improvements largely focus on modeling local pixel relationships, resulting in a limited receptive field. This means they primarily focus on feature information within a small area around each pixel, neglecting the overall structure and global features of the image. In low-light monitoring images from underground coal mines, the global brightness distribution and long-range dependencies are crucial for accurate image enhancement. For example, the color and brightness of a region may be affected by distant light sources or other areas. However, due to the insufficient ability of existing methods to model global features, they cannot capture such long-range dependencies, making it difficult to ensure the overall spatial consistency of the enhancement results. This leads to unsatisfactory enhancement effects in local areas, resulting in over-enhancement or under-enhancement.

[0047] Furthermore, in the complex lighting distribution and varied scene structures (including different objects and scene elements such as tunnels, equipment, and personnel) underground, different objects have different reflective properties and brightness characteristics. Existing methods lack sufficient flexibility and adaptability when facing such complex and diverse scenes. They may adopt a uniform enhancement strategy to handle all types of objects and scenes, but cannot make personalized enhancement adjustments according to the characteristics of different objects. This leads to over-enhancement or under-enhancement in local areas when processing certain special objects or scenes, resulting in insufficient consistency in the overall brightness distribution, color reproduction, and detail representation of the enhancement results.

[0048] To address the aforementioned issues, this application provides a method, apparatus, and electronic device for enhancing low-light images in coal mines. It utilizes a linear transformation mechanism to extract and analyze local image features stably and effectively, while a self-attention mechanism captures long-range dependencies and focuses on global features. By combining these two mechanisms, the image enhancement model can simultaneously consider local details and global information such as structure and illumination distribution, avoiding uneven enhancement caused by focusing only on local or global information. Therefore, by fusing local and global image features to generate an image enhancement parameter map containing enhancement parameters for each pixel location, and determining the target enhanced image accordingly, the problem of uneven local enhancement can be solved, resulting in a more uniform brightness distribution, a more natural enhancement effect, and enhanced details in low-light monitoring images of coal mines that were originally blurred due to insufficient lighting, thus improving image quality and clarity.

[0049] The technical solution of this application and how the technical solution of this application solves the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will be described below with reference to the accompanying drawings.

[0050] The execution entity of the low-light image enhancement method for underground coal mines provided in this application embodiment can be a computing device such as a server or server cluster. The server can be a mobile phone, computer, tablet, or other device. This application embodiment does not impose any particular restrictions on the implementation method of the execution entity, as long as the execution entity can acquire low-light monitoring images from underground coal mines; input the low-light monitoring images into an image enhancement model to obtain an image enhancement parameter map corresponding to the low-light monitoring images. The image enhancement model is a model trained on a dual-branch model based on a linear transformation mechanism and a self-attention mechanism, used to fuse local and global image features and generate an image enhancement parameter map. The image enhancement parameter map includes enhancement parameters for each pixel position of the low-light monitoring image; based on the low-light monitoring image and the image enhancement parameter map, the target enhanced image can be determined.

[0051] Figure 1 A flowchart illustrating the low-light image enhancement method for underground coal mines provided in this application. Figure 1 The execution entity of this method can be a system server storing low-light image enhancement data from underground coal mines or other servers. This embodiment does not impose any particular limitations. Figure 1 As shown, the method may include:

[0052] S101. Acquire low-light monitoring images of underground coal mines.

[0053] In this context, "underground coal mine" refers to the environment where coal mining operations take place underground, characterized by insufficient lighting, high dust levels, uneven lighting, and a relatively enclosed space. Low-light monitoring images, on the other hand, are images with low overall or localized brightness, poor contrast, and details obscured in dark areas due to insufficient ambient light, uneven lighting, or limitations in exposure parameters.

[0054] Furthermore, the image can be captured by monitoring equipment installed underground in a coal mine under low-light conditions, typically exhibiting problems such as low brightness, low contrast, and blurred details. It can be a single frame or a frame from a video stream.

[0055] In some examples, explosion-proof surveillance cameras deployed in key locations such as underground mine roadways, working faces, and conveyor belts can be used to capture video streams in real time and extract single-frame images from them. These images naturally exhibit characteristics such as insufficient lighting and dust interference. Alternatively, historically stored surveillance video or image files can be read from the hard disk recorder or database of the coal mine monitoring system, and frames can be extracted to obtain the image to be enhanced. Video streams or image data transmitted in real time from front-end cameras can also be received via an underground industrial ring network.

[0056] S102. Input the low-light monitoring image into the image enhancement model to obtain the image enhancement parameter map corresponding to the low-light monitoring image. The image enhancement model is a model obtained by training a dual-branch model based on a linear transformation mechanism and a self-attention mechanism to fuse local and global features of the image and generate the image enhancement parameter map. The image enhancement parameter map includes the enhancement parameters of the low-light monitoring image at each pixel position.

[0057] In this step, the linear transformation mechanism refers to an efficient way to extract local features, such as extracting details like edges, textures, and corners from an image. It has low computational cost and can preserve image details well. The self-attention mechanism, on the other hand, is a global feature modeling tool. For example, it understands the overall structure of an image, the long-range dependence of illumination distribution, and a wide range of semantic information (e.g., distinguishing between a coal wall, equipment, or people) by calculating the correlation between any two pixels (or image patches) in the image.

[0058] The image enhancement parameter map is a multi-channel image with the same (or proportional) size as the input image. It is not the brightness value of a pixel, but rather the adjustment parameters for pixel brightness. For each pixel in the input image, the parameter map assigns a unique intensity value for subsequent brightness adjustment.

[0059] In some examples, linear transformation mechanisms can be implemented using a Zero-DCE network composed of convolutional modules such as Conv and Ghost, while self-attention mechanisms can be implemented using a network composed of modules such as a Transformer encoder. As a further example, a low-light image is input into a trained dual-branch model. Local feature extraction: The image passes through a backbone network composed of efficient modules such as Ghost convolutions, progressively extracting deep feature maps containing rich local details. Global feature extraction: Using a Vision Transformer (ViT) network, the image is simultaneously segmented into blocks, linearly embedded, and then positionally encoded before global relational modeling is performed using a Transformer encoder. Finally, the extracted global features (such as class labels) are broadcast and aligned with the spatial dimensions of the original image. Then, the aligned local and global features extracted from the two branches are fused to output a multi-channel image enhancement parameter map. The fusion method can be a simple pixel-by-pixel addition or a more complex dynamic weighted fusion based on attention or gating mechanisms, where the weights can be determined by the dynamic characteristics of the input image (such as scene complexity, dust concentration, and illumination distribution patterns).

[0060] In addition to Conv and Ghost convolutions, the linear transformation branch can also use other lightweight convolutions, such as depthwise separable convolutions, grouped convolutions, or efficient convolutional structures automatically found using Neural Architecture Search (NAS). The self-attention branch, besides the standard VisionTransformer, can use more efficient variants, such as the Swin Transformer (using moving window self-attention) and PVT (Pyramid Structure Vision Transformer), to better handle high-resolution surveillance images or reduce computational complexity.

[0061] Furthermore, in terms of model training, the core idea of ​​the Zero-DCE series can be utilized, using a set of loss functions that do not require real image enhancement (such as exposure control loss, color constancy loss, spatial consistency loss, and illumination smoothing loss) for training. Further, during training, a certain difference between the enhanced image and the original image (such as gradient difference) can be calculated as an additional loss term (feedback term) to constrain the model and prevent over-enhancement or artifacts.

[0062] In other examples, deep learning frameworks (such as TensorFlow and PyTorch) can be used to build two-branch models based on linear transformation and self-attention mechanisms. For instance, in TensorFlow, different layer structures are defined to implement the linear transformation branch and the self-attention branch, and then the outputs of the two branches are fused. For model training, a large number of low-light monitoring images from underground coal mines need to be collected as a training dataset, and these images need to be labeled (e.g., the ideally enhanced image or the enhancement parameters for each pixel). Appropriate loss functions (such as mean squared error loss) and optimization algorithms (such as stochastic gradient descent) are used to train the model, enabling it to learn the mapping relationship from low-light monitoring images to image enhancement parameter maps.

[0063] S103. Based on the low-light monitoring image and the image enhancement parameter map, determine the target enhancement image.

[0064] Among them, the target-enhanced image is an image that has been enhanced, with significant improvements in brightness, contrast, and detail, making it more suitable for practical applications.

[0065] In this step, based on the enhancement parameters at each pixel position in the image enhancement parameter map, the corresponding pixels in the low-light monitoring image are adjusted accordingly (such as brightness adjustment, color correction, etc.) to finally obtain the target enhanced image.

[0066] Furthermore, in some examples during the adjustment process, a pixel-by-pixel calculation method can be used to perform linear or non-linear transformations on the brightness and color values ​​of each pixel based on the enhancement parameters. For example, for the brightness enhancement parameter, if the enhancement parameter of a pixel is 2, then the original brightness value of that pixel is multiplied by 2 to achieve brightness enhancement.

[0067] Preset functions can also be used to perform batch processing of pixels, improving processing efficiency. For example, using a mapping based on a reciprocal function, the low-light monitoring image and the image enhancement parameter map can be processed accordingly to quickly obtain the target enhanced image. Another example is the Zero-DCE general processing method, which uses a simple quadratic curve to gradually adjust pixel brightness through multiple iterations. Each iteration has an independent parameter map; therefore, the parameter map in this method may have multiple channels (e.g., 8 channels are needed for 8 iterations).

[0068] The low-light image enhancement method for coal mines provided in this application utilizes an image enhancement model to fuse local and global features and generate a pixel-level enhancement parameter map. This model provides appropriate enhancement parameters for each pixel based on its specific location, local features, and the overall illumination distribution of the image. This avoids the problems of over-enhancement or under-enhancement in local areas caused by traditional methods that only consider local or global information, resulting in a more uniform brightness distribution and a more natural enhancement effect. Simultaneously, these enhancement parameters can more accurately and comprehensively enhance details in low-light monitoring images of coal mines that were originally blurred due to insufficient lighting. For example, the edges and contours of objects such as personnel and equipment in the image become clearer, and the colors become more realistic, thereby improving image quality and information clarity. Furthermore, through precise pixel-level enhancement processing, it can better adapt to the complex and variable environment of coal mines, including unstable lighting conditions and dust interference. Whether in dimly lit areas or areas heavily affected by dust, it can perform reasonable enhancement according to the actual situation, ensuring that the image presents a good visual effect in different areas and improving the stability and reliability of image enhancement technology in coal mine applications.

[0069] While the above methods can utilize a dual-branch model based on linear transformation and self-attention mechanisms to fuse local and global image features and generate enhancement parameter maps, in practical processing of low-light monitoring images from underground coal mines, challenges may arise regarding how to more effectively construct the model structure to accurately extract local and global features, and how to more efficiently fuse the two to generate accurate enhancement parameter maps. Therefore, based on the above embodiments, Figure 2 Schematic diagram of the image enhancement model provided in this application Figure 1 ,like Figure 2 As shown, the image enhancement model described in S102 includes a global feature extraction branch and a convolutional backbone network. The method for inputting a low-light monitoring image into the image enhancement model to obtain the image enhancement parameter map corresponding to the low-light monitoring image may include: inputting the low-light monitoring image into the convolutional backbone network to extract local feature maps of the low-light monitoring image; inputting the low-light monitoring image into the global feature extraction branch to extract global feature maps of the low-light monitoring image; fusing the local feature maps and global feature maps to obtain a fused feature map, and generating the image enhancement parameter map corresponding to the low-light monitoring image based on the fused feature map.

[0070] The global feature extraction branch is a component of the image enhancement model. Its purpose is to capture the long-range dependencies and global semantic information of the image, used to extract global features from low-light surveillance images. Global features reflect the overall attributes of the image, such as the overall brightness distribution, color tone, and the relative positions of major objects.

[0071] Furthermore, after receiving the input low-light monitoring image, the global feature extraction branch extracts a feature map that can represent the global information of the image through a series of internal operations and processing.

[0072] A global feature extraction branch can be constructed using modules based on self-attention mechanisms. For example, a multi-head self-attention layer can be used, allowing the model to simultaneously focus on the relationships between different locations in the image, thereby better capturing global features. In implementation, relevant functions and modules in deep learning frameworks (such as PyTorch) can be used to build the multi-head self-attention layer, setting appropriate parameters such as the number of heads and dimensions.

[0073] Global average pooling and other operations can also be used to help extract global features. First, the image is divided into multiple regions, and average pooling is performed on each region. Then, the features of each region are integrated to obtain the global feature map of the image.

[0074] Optionally, the global feature extraction branch can use the standard Vision Transformer, or a hierarchical Transformer based on a moving window to generate multi-scale global features, or a pure multilayer perceptron (MLP) architecture to obtain global perception by mixing information across different dimensions.

[0075] The convolutional backbone network is the main network structure in the image enhancement model used to extract local features from low-light surveillance images. Convolutional operations can effectively capture local patterns in images, such as textures, edges, corners, and other local details. Furthermore, the convolutional backbone network performs a series of convolutional operations on the input low-light surveillance image, progressively extracting local features at different levels, and finally outputting a local feature map.

[0076] Classic convolutional neural network (CNN) structures (such as VGG and ResNet) can be used as the convolutional backbone. For example, the residual block structure in ResNet can be used to solve the gradient vanishing problem in deep networks through skip connections, thus better extracting local features of the image. In practical applications, a ResNet structure of appropriate depth can be selected based on the image size and processing requirements.

[0077] Alternatively, you can design your own convolutional backbone network and extract multi-scale local features by stacking convolutional kernels of different sizes. For example, you can first use 3×3 convolutional kernels for preliminary feature extraction, then use 5×5 convolutional kernels to extract local features over a larger area, and finally fuse the features from different levels.

[0078] In this context, it can be understood that the local feature map is a feature tensor output by the convolutional backbone network. It is a three-dimensional data structure (number of channels C × height H × width W), where the vector at each spatial location (H, W) represents the pattern information of the corresponding local region in the input image. It retains strong spatial structure information and is sensitive to the geometric and texture details of the image. The global feature map, on the other hand, is a feature tensor obtained after processing by the global feature extraction branch and undergoing spatial dimension alignment. Its original form may be a one-dimensional class label vector, but in order to fuse with the local feature map, it is broadcast or reshaped into a spatial size that is the same as or matches the local feature map (such as C × H × W). It contains global information such as the overall content of the image, the categories of major objects, and the lighting distribution patterns of the scene.

[0079] The fusion process involves integrating information from local and global feature maps to form a fused feature map containing more comprehensive image information. Then, based on the fused feature information, enhancement parameters for each pixel are calculated, resulting in an image enhancement parameter map. In this embodiment, a specific fusion method (such as concatenation, element-wise addition, weighted summation, etc.) can be used to merge the local and global feature maps. Then, through a series of operations and processing, an image enhancement parameter map is generated from the fused feature map.

[0080] In one example, a simple concatenation method can be used to concatenate the local and global feature maps along the channel dimension to form a new feature map. For example, if the local feature map has C1 channels and the global feature map has C2 channels, the concatenated fused feature map will have C1 + C2 channels. Alternatively, a weighted fusion method can be used, assigning different weights to the local and global feature maps respectively, and then summing the weighted feature maps to obtain the fused feature map. For instance, by designing a lightweight neural network (such as a gating unit or attention module), the fusion weights of local and global features can be adjusted in real time based on the dynamic characteristics of the input image (such as scene complexity, dust concentration, and lighting distribution patterns). For example, when high dust concentration causes blurring of local details, the weights of global features are increased to compensate for the loss of local information; in dynamic scenes (such as fast-moving people), the weights of global features are reduced to avoid over-smoothing of motion-blurred regions.

[0081] Furthermore, fully connected layers or convolutional layers can be used to perform further transformations and calculations on the fused feature map to obtain enhancement parameters for each pixel location. For example, a fully connected layer can be used to map the fused feature map to a vector with the same dimension as the number of pixels, where each element of the vector is the enhancement parameter for the corresponding pixel, thus forming an image enhancement parameter map.

[0082] By clarifying the specific structure of the image enhancement model, the global feature extraction branch and the convolutional backbone network are used to accurately extract global and local features of the image, respectively. The convolutional backbone network focuses on capturing low-level textures and geometric structures that are sensitive to details, while the global feature extraction branch focuses on understanding high-level semantics and long-range dependencies that are not affected by local noise. The two are fused in multiple ways to generate an image enhancement parameter map, which further improves the accuracy and comprehensiveness of feature extraction from low-light monitoring images in coal mines. This makes the generated image enhancement parameter map more accurate, thereby effectively improving the quality of the target enhanced image, better solving the problem of over-enhancement or under-enhancement in local areas, and enhancing the applicability and stability of the image enhancement method in complex underground environments.

[0083] However, in actual underground coal mine monitoring scenarios, images typically contain a large amount of dust noise and irregular lighting. If the internal structure of the global branch is unclear, it may lead to unstable quality of the extracted global features. Therefore, based on the above embodiments, Figure 3 Schematic diagram of the global feature extraction branch provided in this application Figure 1 ,like Figure 3 As shown, the global feature extraction branch includes image patch embedding, category labeling, location embedding, Transformer encoder, linear layer, and broadcast module connected in sequence. Specifically: image patch embedding performs block embedding and dimensional transformation on the low-light surveillance image to obtain a sequence of image patch features; category labeling concatenates learnable category labels to the front of the image patch feature sequence to obtain a feature sequence with classification labels; location embedding adds location information to the feature sequence with classification labels to obtain a feature sequence with location awareness; the Transformer encoder performs global modeling and feature encoding on the location-aware feature sequence using a self-attention mechanism, outputting an encoded global feature sequence; the linear layer performs dimensional mapping on the category labels extracted from the encoded global feature sequence to obtain a global feature vector of a specified dimension; and the broadcast module expands the dimensions of the global feature vector and performs spatial broadcasting to obtain a global feature map of the low-light surveillance image.

[0084] In this embodiment, patch embedding is used to segment the input low-light monitoring image into multiple small image patches and map each image patch into a fixed-dimensional vector space to achieve image patch embedding and dimensional transformation processing, resulting in image patch features in sequence form.

[0085] In image patch embedding, the method of dividing the image patch is first determined (e.g., dividing it according to a certain size and stride). Then, each image patch is mapped to a specified dimension through linear transformation or convolution operation, forming a sequence of feature representations. In some examples, convolutional layers can be used to implement image patch embedding. For example, the size of the convolution kernel is set to the size of the image patch (e.g., 16×16), the stride of the convolution is the same as the size of the image patch, and the number of output channels of the convolution is the specified embedding dimension. Through such convolution operation, each image patch is mapped to the corresponding vector space, resulting in a sequence of image patch features. Alternatively, fully connected layers can be used. Each image patch is flattened into a one-dimensional vector and then input into the fully connected layer for dimensionality transformation to obtain a feature vector of the specified dimension, thus forming a sequence and ensuring the normalization of the input data.

[0086] A class token is a learnable vector that is concatenated to the beginning of an image patch feature sequence. It represents the category information or global semantic information of the entire image in subsequent processing. Through a self-attention mechanism, it interacts with the features of all image patches to obtain a feature sequence with a class token. Furthermore, a learnable class token vector can be created and concatenated with the image patch feature sequence in the sequence dimension. For example, concatenating a learnable class token vector (1×64 shape) results in a feature sequence with a ((N+1)×64) class token. Furthermore, the class token can be defined as a learnable parameter of the model during initialization and its value can be continuously updated during training.

[0087] Since the Transformer encoder itself is unordered, position embedding adds positional information to each element (including class labels and image patch features) in the feature sequence with classification labels, enabling the model to perceive the positional relationships of each feature in the image, resulting in a feature sequence with positional awareness. Further, based on the feature's position in the sequence, specific positional encoding methods (such as sine / cosine positional encoding, learnable positional encoding, etc.) are used to add positional information to each feature vector. For example, adding a set of positional embeddings (of shape (N+1)×64) matching the sequence length to the feature sequence obtained in the previous step yields a feature sequence with spatial positional awareness. Through the combination of positional embedding and the Transformer encoder, the model can stably extract robust global features reflecting the overall content and illumination distribution of the image even under dust interference and non-uniform lighting conditions in a coal mine.

[0088] Among them, the Transformer Encoder uses a self-attention mechanism to globally model and encode feature sequences with position awareness, which can capture long-range dependencies between features, fully explore the global information of the image, and output the encoded global feature sequence.

[0089] Furthermore, the location-aware feature sequence is input into the Transformer encoder, where it undergoes multiple calculations and transformations through components such as multi-head self-attention layers and feedforward neural network layers, ultimately outputting the encoded global feature sequence. For example, a pre-trained Transformer model (such as the Transformer encoder part in ViT) can be used as a base, fine-tuned according to the characteristics of low-light monitoring images in underground coal mines. This includes adjusting parameters such as the number of heads in the self-attention mechanism and the hidden layer dimension of the feedforward neural network to adapt to specific image enhancement tasks. Alternatively, a custom Transformer encoder can be built, following the standard Transformer structure by sequentially stacking components such as multi-head self-attention layers, layer normalization, residual connections, and feedforward neural network layers to achieve global modeling and encoding of the feature sequence.

[0090] Linear layers are used to perform dimension mapping on the class labels extracted from the encoded global feature sequence, mapping them from the current dimension space to a specified dimension space to obtain a global feature vector of the specified dimension. Further, feature vectors corresponding to the class labels are extracted from the encoded global feature sequence and input into the linear layer, where they are mapped to the specified dimension through linear transformation (matrix multiplication). For example, the first vector corresponding to the class label (shape 1×64) is specifically extracted from the encoded feature sequence and input into the linear layer for dimension mapping (e.g., mapping to 3D) to obtain the final global feature vector (shape 1×3). As another example, the functionality of a linear layer can be implemented using a fully connected layer. For instance, a fully connected layer can be defined with the input dimension being the dimension of the encoded class labels and the output dimension being the specified dimension. The class label vector is input into this fully connected layer to obtain a global feature vector of the specified dimension. Furthermore, the parameters of the linear layer, such as the weight matrix and bias vector, can be adjusted according to actual needs to optimize the feature mapping effect.

[0091] Optionally, a separate extra class token extraction layer can be set between the Transformer encoder and the linear layer to extract the corresponding class token from the encoded global feature sequence.

[0092] The broadcasting module is used to dimensionally expand and spatially broadcast the global feature vector, making it match the spatial dimensions of the image to obtain a global feature map with the same size as the low-light monitoring image. For example, the broadcasting module receives this 1×3 global feature vector and expands it in the height and width dimensions by copying it, generating a global feature map with a shape of (3×H×W). Here, H and W are usually consistent with the spatial dimensions of the local feature map.

[0093] In some embodiments, Figure 4 Schematic diagram of the global feature extraction branch provided in this application Figure 2 ,like Figure 4 As shown, the input size of the low-light monitoring image is 8×3×256×256 (batch size is 8, number of channels is 3, and image height and width are both 256). The global feature extraction branch completes the extraction from the original image to the global feature map through the sequentially connected image patch embedding, class token, position embedding, Transformer encoder, extra class token extraction, linear layer, and broadcasting module. The specific steps are as follows:

[0094] (1) Patch Embedding receives the input low-light monitoring image (8×3×256×256) and performs patch embedding and dimensionality transformation processing: First, a 16×16 convolution operation is performed on the low-light monitoring image ( The 256×256 image is divided into 16×16 image blocks, completing the image block mapping with channel dimensions to obtain intermediate features. Then, the intermediate features are unfolded to flatten the two-dimensional spatial features into one-dimensional sequence features. Finally, the feature dimension order is adjusted by the transpose operation, and the final output is the image block features in sequence form, with a data dimension of 8×256×64 (batch size 8, sequence length 256, feature dimension 64).

[0095] (2) The Class token calls the preset learnable category marker (dimension 8×1×64) and concatenates it to the front of the sequence of image patch features (8×256×64) output in step (1) to complete the construction of the feature sequence with classification marker. The dimension of the feature sequence after concatenation is 8×257×64 (the sequence length is expanded from 256 to 257, containing 256 image patch features and 1 category marker feature).

[0096] (3) Position Embedding adds position information to the feature sequence (8×257×64) with classification labels obtained in step (2): The preset position embedding (dimension matching 8×257×64) is added to the feature sequence element by element, and each element in the feature sequence is given a unique spatial position representation, and finally a feature sequence with position awareness is obtained, whose dimension is still 8×257×64.

[0097] (4) The Transformer Encoder performs global modeling and feature encoding on the position-aware feature sequence output in step (3) through a self-attention mechanism. The specific execution process is as follows: first, normalize the input feature sequence to obtain normalized features; perform multi-head attention on the normalized features to capture the global association information of each element in the feature sequence through the self-attention mechanism; perform residual connection (element-by-element addition) on the output of multi-head attention and the original input feature sequence in step (3) to retain the original feature information; perform normalization again on the residual connection feature and input it to the multilayer perceptron (MLP) for nonlinear feature transformation; perform secondary residual connection on the output of the multilayer perceptron and the normalized feature to finally output the encoded global feature sequence, whose dimension is maintained at 8×257×64.

[0098] (5) The Extra class token extracts the class label from the encoded global feature sequence (8×257×64) output in step (4), that is, it extracts the feature part of the corresponding class label in the sequence to obtain the class label feature with a dimension of 8×64; then the feature is input into Linear, and the dimension mapping is completed through linear transformation, mapping the feature dimension from 64 to 3, and finally obtaining the global feature vector of the specified dimension, whose data dimension is 8×3.

[0099] (6) Broadcasting performs dimensional expansion and spatial broadcasting operations on the global feature vector (8×3) output in step (5): the one-dimensional global feature vector is expanded along the height and width dimensions of the image so that its spatial size matches the 256×256 size of the original low-light monitoring image. Finally, a global feature map that perfectly matches the size and number of channels of the input image is obtained, with a data dimension of 8×3×256×256, thus completing the entire processing flow of the global feature extraction branch.

[0100] By clarifying the specific functions and implementation methods of components such as image patch embedding, category labeling, location embedding, Transformer encoder, linear layer, and broadcast module in the global feature extraction branch, more accurate and comprehensive global feature extraction of low-light monitoring images is achieved. The self-attention mechanism of the Transformer effectively captures long-range dependencies in the image, and location embedding preserves the spatial location information of the image. The resulting global feature map better represents the global features of the image, further improving the image enhancement model's ability to understand and process complex underground coal mine scenes, thereby enhancing the quality and accuracy of the target enhanced image.

[0101] Furthermore, it's important to consider that while convolutional backbone networks can be used to extract local features, for edge computing scenarios like underground coal mine monitoring that require real-time processing and are severely limited by hardware resources (computing power, memory, and power consumption), traditional standard convolutional layer stacking methods typically have a large number of parameters and high computational complexity, making it difficult to achieve real-time enhancement on edge devices such as underground explosion-proof cameras. Therefore, convolutional backbone networks need to efficiently extract local image features while reducing computational load and model parameter count.

[0102] Based on this, the convolutional backbone network can include multiple sequentially connected Ghost convolutional modules. The Ghost convolutional modules are used to extract features from low-light monitoring images based on a linear transformation mechanism to generate local feature maps of the low-light monitoring images. Specifically, each Ghost convolutional module is used to: perform a convolution operation on the input feature map to generate an intrinsic feature map corresponding to the input feature map; perform a linear transformation operation on the intrinsic feature map to generate the remaining feature maps corresponding to the intrinsic feature map; and concatenate the intrinsic feature map with the remaining feature maps to output a feature map.

[0103] In this embodiment, the Ghost convolution module is a lightweight convolution module. Unlike traditional convolution modules, it believes that there are a lot of redundancies ("ghost" features) in the feature maps output by traditional convolution. These redundant features can be efficiently generated by performing a simple linear transformation on a small number of inherent feature maps, thereby avoiding the use of a large number of convolution kernels to directly calculate all feature maps, and achieving the goal of significantly reducing the amount of computation and parameters.

[0104] Linear transformation operations refer to a series of linear mathematical operations performed on the intrinsic feature map to generate new feature maps. These linear operations can be simple matrix multiplication, offset, etc., with the aim of extracting more feature information from the existing intrinsic feature map. For example, the parameters of the linear transformation (such as the transformation matrix, offset vector, etc.) are determined, and then each element of the intrinsic feature map is calculated accordingly according to these parameters to obtain new feature map elements, thereby generating the remaining feature maps.

[0105] The input feature map refers to the feature data input to the current Ghost convolutional module. For the first module, it might be the preprocessed raw image; for subsequent modules, it is the output feature map of the previous Ghost convolutional module.

[0106] Intrinsic feature maps refer to a small subset of feature maps generated directly from the input feature maps through a few standard convolution operations. These feature maps are considered the core features of the image, containing the most essential expression of the input information. The remaining feature maps are generated by applying a series of computationally inexpensive linear transformations (e.g., depthwise convolution, affine transformation, etc.) to the intrinsic feature maps. These feature maps are considered "ghost copies" of the intrinsic feature maps, and together with the intrinsic feature maps, they constitute the complete set of output feature maps.

[0107] Concatenation refers to merging the inherent feature map and the remaining feature maps generated through linear transformation along the channel dimension to form a complete output feature map with more channels.

[0108] In some examples, the convolutional backbone network can be composed of Ghost convolutional modules connected sequentially with different numbers and parameter settings. For example, near the input layer, Ghost convolutional modules with smaller kernel sizes (e.g., 3×3) and fewer channels can be used to extract some basic local features; while near the output layer, Ghost convolutional modules with larger kernel sizes (e.g., 5×5) and more channels can be used to further extract more complex and higher-level local features.

[0109] For example, suppose the convolutional backbone network consists of three Ghost convolutional modules. The first module uses a 3×3 convolutional kernel with 32 output channels; the second module uses a 3×3 convolutional kernel, increasing the number of output channels to 64; and the third module uses a 5×5 convolutional kernel with 128 output channels. Gradually increasing the receptive field and number of channels of the convolutional kernels in this way allows for better extraction of local features at different levels.

[0110] In some embodiments, in order to introduce nonlinearity and enhance the expressive power of the model, a ReLU activation function can be followed immediately after the Ghost convolution of each Ghost convolution module to form a combined module of "Ghost convolution → ReLU".

[0111] Furthermore, in the Ghost convolution module, different linear transformations can be used to generate the remaining feature maps. Besides common matrix multiplication and offset operations, more complex combinations of linear transformations can be used, such as cascading or parallel operations of multiple linear transformations. For example, one linear transformation could be to first perform a learnable matrix multiplication transformation on the intrinsic feature map, and then add a learnable offset vector; another approach could be to process the intrinsic feature map using two different linear transformations simultaneously, and then fuse the results of the two transformations to obtain the remaining feature maps.

[0112] By introducing the Ghost convolution module to construct a convolutional backbone network, and generating and stitching feature maps step by step based on the linear transformation mechanism, the computational load and model parameter count of convolution operations can be reduced while ensuring the feature extraction effect. The model is more efficient in processing low-light monitoring images in coal mines, and can quickly and accurately extract local features of the image. Combined with the global feature extraction branch, it further improves the overall understanding of the image, thus balancing the image enhancement effect and computational efficiency, and is more suitable for scenarios such as coal mines where real-time performance is required.

[0113] However, relying solely on the local features extracted by the Ghost convolutional module may not provide a sufficient understanding of the overall semantics and global relationships of the image. When dealing with complex scenes such as low-light monitoring images from underground coal mines, the pure convolutional backbone network still has limited ability to perceive global illumination distribution and long-distance structural relationships in its intermediate layers. This may result in the extracted local feature maps lacking sufficient global context guidance, affecting the subsequent global feature fusion effect with the global feature extraction branch. Therefore, based on the above embodiments, the convolutional backbone network can also include at least one global context module. This global context module is connected after the preset Ghost convolutional module and is used to perform global context modeling on the feature maps output by the Ghost convolutional module, obtaining enhanced feature maps with global context information, and then outputting the enhanced feature maps to the next Ghost convolutional module.

[0114] In this embodiment, the Global Context Block (GC Block) is a lightweight neural network module used to capture global statistics and long-range dependencies in the feature map. While ordinary convolutional operations in convolutional neural networks primarily focus on local features, the GC Block can explore the relationships between features at different locations from the perspective of the entire feature map, thereby adding more comprehensive contextual information to the feature map.

[0115] In this process, after the pre-defined Ghost convolutional module outputs a feature map, the global context module receives this feature map. It first performs some form of global information aggregation operation on the feature map (such as global average pooling, global max pooling, etc.), compressing the information of the entire feature map into a fixed-length vector. Then, it processes this vector through some transformation operations (such as fully connected layers, convolutional layers, etc.), mapping it to the same channel dimension as the input feature map. Finally, it fuses the processed vector with the original input feature map (such as addition, concatenation, etc.) to obtain an enhanced feature map with global context information, which is then output to the next Ghost convolutional module.

[0116] In some examples, the global context module can employ different global information aggregation methods. Besides global average pooling and global max pooling mentioned above, more complex aggregation methods can be used, such as attention-based pooling. Attention-based pooling adaptively allocates weights based on the importance of different positions in the feature map, thus more effectively aggregating global information. For example, when using attention-based pooling, a convolutional layer first generates an attention map with the same number of channels as the input feature map, but with each position representing a weight. Then, the input feature map is multiplied by the attention map, and summation is performed to aggregate global information.

[0117] Furthermore, there are several methods available for fusing the processed global information vector with the original input feature map. Besides simple addition and concatenation, channel attention mechanisms can also be used. Channel attention mechanisms selectively fuse the processed vector and the original feature map based on the importance of different channels. For example, when using channel attention mechanisms, first, global average pooling is performed on both the processed vector and the original feature map to obtain two channel description vectors. Then, these two vectors are input into a shared fully connected layer network to generate weights for each channel. Finally, the processed vector and the original feature map are weighted and added together according to these weights to achieve fusion.

[0118] In this embodiment, it should be noted that the global context module can be connected after Ghost convolutional modules at different locations. Depending on the model's design requirements and actual performance, the global context module can be connected after shallow Ghost convolutional modules, deep Ghost convolutional modules, or multiple different locations. For example, in a convolutional backbone network composed of multiple Ghost convolutional modules, a global context module can be connected after two Ghost convolutional modules in the mid-to-deep layers of the network. In this case, the feature maps already possess some semantic information, but the demand for global context is higher.

[0119] In some examples, Figure 5A schematic diagram of the structure of the global context module provided in this application is shown below. Figure 5 As shown, the global context module includes context modeling and transformation. The execution flow of context modeling is as follows: a 1×1 convolution (Conv(1×1)) is performed on the input feature map (dimension C×H×W) to generate the key feature map W. k (Dimension 1×H×W); Flatten the key feature map to HW×1×1, and obtain spatial attention weights through Softmax activation; Multiply the attention weights element-wise with the original input feature map to obtain the weighted global context features. The execution flow of Transform is: Perform a 1×1 convolution (W) on the weighted features. v1 The number of channels is compressed to C / r; after layer normalization (LayerNorm) and ReLU activation, it is then passed through a 1×1 convolution (W). v2 The transformed feature map is restored to C×1×1 dimension; the transformed feature map is added element by element to the original input feature map to output an enhanced feature map with global context information.

[0120] By introducing a global context module into the convolutional backbone network and connecting it after a pre-defined Ghost convolutional module, effective global context modeling can be performed on the feature maps output by the Ghost convolutional module. This allows the model to fully explore the correlations between different locations in the feature maps, obtain more comprehensive image semantic information, and thus generate enhanced feature maps with global context information. Inputting these enhanced feature maps into the next Ghost convolutional module aids in subsequent feature extraction and image enhancement processing, further improving the model's understanding and processing capabilities for low-light monitoring images in coal mines, and enhancing the quality and accuracy of image enhancement.

[0121] Based on the above embodiments, there are 7 Ghost convolutional modules and 2 global context modules. Specifically, the first Ghost convolutional module extracts features from the low-light monitoring image and outputs a first feature map; the second Ghost convolutional module extracts features from the first feature map and outputs a second feature map; the third Ghost convolutional module extracts features from the second feature map and outputs a third feature map; the fourth Ghost convolutional module extracts features from the third feature map and outputs a fourth feature map; the first global context module performs global context modeling on the fourth feature map to obtain a first enhanced feature map; the fifth Ghost convolutional module extracts features from the fusion result of the first enhanced feature map and the third feature map and outputs a fifth feature map; the second global context module performs global context modeling on the fifth feature map to obtain a second enhanced feature map; the sixth Ghost convolutional module extracts features from the fusion result of the second enhanced feature map and the second feature map and outputs a sixth feature map; and the seventh Ghost convolutional module extracts features from the fusion result of the sixth feature map and the first feature map and outputs a seventh feature map, which is then used as a local feature map output by the convolutional backbone network.

[0122] In this embodiment, the first to seventh feature maps refer to the feature maps output sequentially by the seven Ghost convolutional modules in the network. They represent the network's representation of the input image at different levels of abstraction. From the first feature map (the shallowest layer, containing low-level features such as edges and textures) to the seventh feature map (the deepest layer, containing high-level semantic features), the level of abstraction gradually increases, while the spatial resolution typically decreases. The first and second enhanced feature maps refer to the feature maps processed by the first and second global context modules, respectively. They incorporate global statistical information and long-range dependencies based on the original input feature maps (the fourth and fifth feature maps), and are intermediate features with global context enhancement.

[0123] Feature map fusion refers to the operation of merging two different feature maps into a new feature map. The purpose of fusion is to integrate feature information from different levels or processing stages, enabling the model to obtain richer and more comprehensive feature representations. Feature map fusion methods can include channel concatenation, element-wise addition, etc., and the specific implementation depends on the actual network design. For example, if the first and third enhanced feature maps have the same number of channels, using the addition method allows the feature values ​​of the two to be directly added together, enhancing the strength of the feature; if the concatenation method is used, the channel information of the two will be integrated, enabling the subsequent Ghost convolutional module to learn more complex feature combination relationships.

[0124] It should be noted that although this embodiment specifies the number of global context modules and Ghost convolutional modules and their approximate connection order, the insertion position of the global context modules can be fine-tuned in practical applications. For example, an additional global context module can be added after the sixth Ghost convolutional module to process the relevant feature maps. When performing feature map fusion, different approaches can be taken to select which layer of feature maps to fuse with. For instance, at the input of the sixth Ghost convolutional module, in addition to fusing with the second feature map, fusing with the fourth feature map can also be considered.

[0125] In other words, the convolutional backbone network structure provided in this application embodiment is not limited to this embodiment. It can also use 9 or 11 Ghost convolutional modules and 3 or 4 global context modules to suit more complex scenarios or higher resolution inputs. Alternatively, for scenarios with extremely high real-time requirements, 5 Ghost convolutional modules and 1 global context module can be used, sacrificing a small amount of accuracy for faster processing speed. The pairing of skip connections is also not limited to the first enhanced feature map and the third feature map, the second enhanced feature map and the second feature map, or the sixth feature map and the first feature map; it can be other combinations, such as directly fusing the fourth enhanced feature map with the second feature map, or using denser connections.

[0126] By clearly defining the input source and processing object of each layer, the network performs downsampling encoding in the first three layers, introduces global context enhancement after the fourth and fifth layers, and then gradually fuses the enhanced deep semantics with the corresponding shallow details through three skip connections. This ensures that full-spectrum information from low-level details to high-level semantics is preserved and transmitted to the final output local feature map. Furthermore, the model can more effectively extract multi-level features from low-light monitoring images, and by fusing feature maps of different levels multiple times, it fully utilizes the local and global information of the image, helping the model to better understand the semantic content of low-light monitoring images, improving the accuracy and comprehensiveness of feature extraction, and thus enhancing the overall image enhancement method's processing effect on low-light monitoring images in coal mines.

[0127] Based on the above embodiments, in some examples... Figure 6 Schematic diagram of the image enhancement model provided in this application Figure 2 ,like Figure 6As shown, the input low-light monitoring image has dimensions of 3×H×W (i.e., 3 channels, height H, and width W). The image enhancement model includes a global feature extraction branch (Vision Transformer branch) and a convolutional backbone network (Adjustment based on backbone network). The convolutional backbone network includes 7 Ghost convolutional modules (Ghost-ReLU) and 2 global context modules (GC Block) as described in the previous embodiment, used to extract local feature maps. The global feature extraction branch includes image patch embedding, class labeling, location embedding, Transformer encoder, class label extraction, linear layers, and broadcasting modules as described in the previous embodiment, used to extract global feature maps. Furthermore, in some examples, the first six convolutional layers use the ReLU activation function, and the last convolutional layer uses a 10×Tanh activation function to enhance the output dynamic range.

[0128] Furthermore, in practical image enhancement processing, besides the image enhancement model, an effective pixel value mapping method is also needed to accurately adjust each pixel of the low-light monitoring image according to the image enhancement parameters, thereby achieving a better image enhancement effect. The Zero-DCE scheme uses a multi-iterative mapping method, that is, gradually adjusting the brightness by repeatedly applying a simple quadratic curve. While this method is effective, it has significant technical drawbacks: multiple iterations lead to complex calculations and high computational costs, and multiple passes may introduce accumulated errors or cause unstable enhancement results.

[0129] Based on this, and building upon the above embodiments, the method for determining a target enhanced image based on a low-light monitoring image and an image enhancement parameter map, as described in S103, may include: obtaining the target enhanced image based on a preset reciprocal mapping function, according to the low-light monitoring image and the image enhancement parameter map; wherein, the reciprocal mapping function satisfies:

[0130] ;

[0131] in, These are pixel coordinates; For low-light monitoring images Pixel value at; Enhance the image for the target Pixel value at; The corresponding parameters in the image enhancement diagram Enhanced parameters at that location.

[0132] In this embodiment, the reciprocal mapping function is a mathematical function used to convert the pixel values ​​of a low-light monitoring image into the pixel values ​​of the target enhanced image by combining the parameters in the image enhancement parameter map. It achieves the enhancement effect on the low-light image by performing a specific mathematical transformation on the pixel values ​​of the low-light monitoring image and adjusting the pixel values ​​according to the image enhancement parameters.

[0133] Furthermore, The input low-light monitoring image is in coordinates The pixel value at a given location is usually normalized to the range [0,1], and the function expression uses the normalized value for calculation by default. After enhancement, the enhanced image of the target is in the same coordinates. The output pixel value at that location has a range of values ​​similar to... Maintain consistency (i.e., if the input is normalized to [0,1], the output is also within [0,1]).

[0134] in, These are trainable, pixel-level parameters that control the enhancement intensity at that pixel. According to the function form, Different values ​​will result in different shapes of the mapping curve: when hour, That is, no enhancement; when hour, The denominator becomes smaller. This achieves increased brightness, and The larger the value, the greater the increase; when hour, The denominator becomes larger. This reduces brightness (which can be used to address locally overexposed areas). Optionally, its range can be set to... ,or, .

[0135] In one example, the input low-light monitoring image I (size H×W×C) and the model output image enhancement parameter map (also size H×W×C) are obtained. These two maps are in one-to-one correspondence in spatial location and channels; for each pixel location in the image... For each color channel, perform the above inverse mapping function calculation and output the pixel value; combine all pixel positions and the calculation results of all channels according to the original spatial order and channel order to form the final H×W×C dimension target enhancement image L; output the enhanced image, which can be used for subsequent display, storage or further video analysis.

[0136] Furthermore, in some embodiments, other simple mapping functions (such as histogram equalization, contrast stretching, etc.) can be used to perform preliminary processing on the low-light monitoring image before substituting the processed result into the reciprocal mapping function for further enhancement. Alternatively, the reciprocal mapping function can be combined with other functions to form a more complex composite function. For example, a linear mapping function can be used first. (k and b are adjustment parameters), perform preliminary brightness adjustment on the low-light monitoring image, and then substitute the obtained pixel values ​​into the reciprocal mapping function. A second enhancement is performed to obtain an image enhancement effect that better meets the requirements.

[0137] Additionally, the enhancement parameters in the image enhancement parameter diagram It can be dynamically adjusted according to different scenarios or needs. It can use fixed preset values, or it can automatically generate suitable values ​​based on certain features of the currently input low-light monitoring image (such as overall brightness and contrast). Value. For example, if the overall low-light monitoring image is very dark, the value can be increased appropriately. The value of makes the enhancement effect of the reciprocal mapping function on pixel values ​​more obvious; if the image itself has a certain brightness base, it can be appropriately reduced. The value should be adjusted to avoid over-enhancement that could lead to image distortion.

[0138] By using a pre-defined reciprocal mapping function instead of multiple iterative mappings, the computational overhead of the image enhancement process can be reduced, the processing speed can be improved, and the computational efficiency can be increased while ensuring the enhancement effect.

[0139] Based on the above embodiments, before inputting the low-light monitoring image into the image enhancement model as described in S102 to obtain the image enhancement parameter map corresponding to the low-light monitoring image, the method may further include: acquiring monitoring video from underground coal mines; extracting multiple image frames from the monitoring video based on a preset time interval to obtain model training data; and training the initial image enhancement model based on the model training data until the output information of the initial image enhancement model meets the preset training requirements to obtain the image enhancement model.

[0140] In this embodiment, it should be noted that the image enhancement model is built on Zero-DCE, as described in the above embodiment, including a global feature extraction branch (Vision Transformer branch) and a convolutional backbone network (Adjustment based on backbone network).

[0141] In this context, underground coal mine monitoring video refers to continuous, unenhanced raw video streams obtained from monitoring systems actually deployed underground in coal mines. These videos contain various characteristics of the underground environment: insufficient lighting, uneven lighting, dust interference, water mist, and movement of personnel and equipment.

[0142] The preset time interval can be a pre-defined duration, such as 1 second, 5 seconds, or 30 seconds, used to uniformly extract image frames from a continuous video stream. This avoids excessive similarity between adjacent frames (leading to data redundancy) while ensuring sufficient coverage of scene variations. Furthermore, in some examples, in addition to a fixed preset interval, the preset time interval can be dynamically adjusted based on the activity level of different time periods or areas within the coal mine. For example, during peak working hours in the coal mine (e.g., 8:00 AM to 10:00 AM), when personnel and equipment are frequently active, the time interval can be set shorter (e.g., 0.5 seconds) to obtain richer training data; while during off-peak hours (e.g., 2:00 PM to 4:00 AM), the time interval can be appropriately extended (e.g., 2 seconds).

[0143] The model training data consists of a dataset of multiple image frames extracted from surveillance videos. These image frames serve as input data to train the initial image enhancement model, enabling the model to learn the features of low-light surveillance images from underground coal mines and the corresponding enhancement methods. The initial image enhancement model is an untrained or incompletely trained model; it possesses the ability to process the input image to some extent but has not been optimized and cannot accurately output image enhancement parameter maps that meet the requirements. Preset training requirements can be pre-defined termination conditions used to determine whether the model training is complete. These could include a set of loss function values ​​falling below a certain threshold, the validation set metrics no longer improving, or reaching the preset maximum number of training epochs.

[0144] For a further example, the system accesses the actual surveillance cameras deployed underground in the coal mine to obtain the raw video stream, or reads the stored historical surveillance video files from the storage device of the underground monitoring system. At preset time intervals, the acquired surveillance video is processed by frame extraction, and the frames that meet the time interval requirements are extracted. These extracted image frames are then aggregated to form model training data. The images in the constructed training dataset are input into the initial image enhancement model, which propagates forward and outputs an enhancement parameter map (which can then generate an enhanced image). Then, a set of loss functions that do not require a reference image (such as exposure control loss, color constancy loss, spatial consistency loss, and illumination smoothing loss) are calculated. These loss functions together constitute a quantitative evaluation standard for whether the enhanced result is a good image.

[0145] Optionally, in some examples, in addition to common loss functions such as exposure control loss function, color constancy loss function, spatial consistency loss function, and illumination smoothing loss function, the loss function combination may also include a feedback loss term. The feedback loss term is constructed based on the difference in brightness gradient between the enhanced image and the original input image. It is used to constrain the model to avoid local over-enhancement during image enhancement, so that the model maintains the coordination between local and global features during the enhancement process.

[0146] Then, based on the calculated total loss value, the gradient is calculated using the backpropagation algorithm, and all parameters of the model are updated using an optimizer (such as Adam or SGD). The training steps are repeated, iterating repeatedly on the dataset. The performance of the model on the training or validation set is continuously monitored. When the output information of the model (mainly the various loss values) stably meets the preset training requirements (such as the total loss value no longer decreasing for 10 consecutive epochs), training is stopped, and the model parameters at this time are saved as the final usable image enhancement model.

[0147] Furthermore, before using the extracted image frames as model training data, preprocessing operations such as denoising and image enhancement can be performed on the image frames to improve the quality of the training data and thus enhance the model's training performance. For example, median filtering can be used to denoise the extracted image frames, removing interference factors such as salt-and-pepper noise from the images, and then the processed images can be used as training data to input into the initial image enhancement model for training.

[0148] In some embodiments, fusion training can be performed based on multi-source monitoring videos: multiple underground coal mine monitoring videos from different locations and angles are acquired, image frames are extracted from each video, and then these image frames from different sources are combined as model training data, enabling the model to learn more comprehensive underground coal mine image features. For example, monitoring videos from different locations such as the main roadway and various working faces in an underground coal mine are acquired, image frames are extracted according to their respective time intervals, and these image frames from different locations are combined into a large model training dataset for training the initial image enhancement model.

[0149] By employing a loss function that requires no reference images, the model can autonomously learn enhancement strategies solely from massive amounts of raw, unlabeled surveillance video. Training data comes directly from existing surveillance cameras in coal mines, eliminating the need for manual screening, labeling, or preprocessing. A large-scale training set containing hundreds of thousands or even millions of images can be automatically constructed by setting a simple time interval. This allows the model to be customized for specific mines, roadways, and lighting conditions, achieving optimal enhancement results. Furthermore, when the underground environment changes (e.g., new lighting equipment is installed, dust conditions change), new surveillance video can be collected for incremental training or fine-tuning, ensuring the model remains optimally adapted. Because the model is trained on real underground surveillance video containing various complex situations, it can better learn the characteristics and enhancement patterns of low-light images in coal mines, improving its accuracy and adaptability. This results in a final image enhancement model that can more effectively enhance low-light surveillance images in coal mines, better meeting the image quality requirements for safe coal mine production.

[0150] As can be seen from the above embodiments, the coal mine low-light image enhancement method provided in this application, without requiring paired labeled data, achieves coordinated enhancement of brightness, contrast, and color information of coal mine low-light monitoring images by integrating local and global feature modeling mechanisms, combining lightweight network structures and improved illumination mapping functions (such as reciprocal mapping functions), while taking into account both computational efficiency and global consistency. Its core lies in: (1) solving the problem of local enhancement inconsistency by introducing global features (such as the Vision Transformer branch, global feature extraction branch); (2) reducing computational complexity by using lightweight feature extraction structures (such as Ghost convolution); and (3) designing an improved illumination mapping function to replace the traditional iterative method, thereby improving computational efficiency.

[0151] This method is applicable to complex lighting scenarios in underground coal mines, adapting to dynamic environmental changes (such as dust interference and rapid personnel movement). It achieves an adaptive balance between local and global features through dynamic feature fusion strategies (such as gating units), and can be deployed in underground coal mine monitoring systems. For example, this image enhancement model can be an end-to-end model deployed in underground embedded devices (such as intelligent monitoring terminals and edge computing nodes). The model adopts a lightweight architecture (Ghost convolution + VisionTransformer branch) to adapt to low-power, low-computing-power hardware environments. Its input is a low-light image (H×W×3) captured by an underground monitoring camera, and the output is the enhanced image, which is directly connected to subsequent visual analysis modules (such as personnel behavior recognition and equipment status detection).

[0152] Figure 7A flowchart illustrating the low-light image enhancement method for underground coal mines provided in this application. Figure 2 ,like Figure 7 As shown, in this embodiment... Figure 1 Based on the examples, a method for enhancing low-light images in underground coal mines is described in detail. This method may include:

[0153] S701. Acquire low-light monitoring images of coal mines to be enhanced.

[0154] The aforementioned underground coal mine monitoring image data originated from actual underground coal mine operations and was collected by underground monitoring equipment. The images exhibit typical characteristics such as insufficient lighting, uneven brightness distribution, and dust interference. The images were obtained by extracting image frames from underground coal mine monitoring videos.

[0155] S702. Input the image to be enhanced into the Zero-DCE image enhancement model that fuses global information. The model includes a backbone network for extracting local image features and a global feature extraction branch for modeling global image information. By fusing local and global image features, the model adaptively enhances the image pixels based on an improved illumination mapping function.

[0156] The Zero-DCE image enhancement model is trained in the following way: using underground coal mine monitoring videos as the data source, image frames are extracted from the monitoring videos at fixed time intervals to construct training data.

[0157] In this step, the Zero-DCE image enhancement model that fuses global information may include:

[0158] A backbone network employing a Ghost convolutional structure (i.e., a convolutional backbone network) is used to extract local features from images. Furthermore, to reduce the computational cost and scale of the image enhancement model, Ghost convolutional modules replace traditional standard convolutional layers in the backbone network. These Ghost convolutional modules generate partial intrinsic feature maps by performing a small number of standard convolutional operations on the input features, and then generate the remaining feature maps using a low-computational-cost linear transformation based on these intrinsic feature maps. This significantly reduces computational cost and parameter size while maintaining feature representation capabilities.

[0159] A global context module, located in the middle layer of the backbone network, is used to perform global context modeling on the feature maps. Furthermore, a global context module is introduced into the backbone network to acquire global context features of the image and perform global modeling of the feature information, thereby enhancing the network's ability to perceive the overall brightness distribution and spatial relationships of the image.

[0160] A global feature extraction branch employing a Vision Transformer structure is used to globally model the input features, and the obtained global features are fused with the local features output by the backbone network. Furthermore, a Vision Transformer branch (i.e., the global feature extraction branch) is introduced into the image enhancement model, using a self-attention mechanism to model the global relationships between features, thereby obtaining feature representations with global semantic information. The global features output by the Vision Transformer branch are fused with the local features output by the backbone network, enabling the model to retain local details while incorporating global contextual information, thus enhancing the model's ability to model the overall image structure, brightness distribution, and long-distance spatial relationships.

[0161] Furthermore, it may also include using a lighting mapping curve based on a reciprocal mapping function to adjust image brightness, replacing the lighting mapping function based on multiple iterations. This reciprocal mapping function replaces the traditional multi-iteration mapping function as a new lighting enhancement curve function. By using this reciprocal mapping function, image brightness and contrast adjustments can be achieved while reducing the computational complexity of iterative mapping calculations, thereby reducing the computational overhead in the image enhancement process. For example, in practical applications, to reduce computational costs and parameter size, a reciprocal lighting mapping function is designed to replace the 8-iteration lighting mapping function. This reciprocal mapping function satisfies:

[0162] .

[0163] As a further example, addressing the common problems of low light, uneven light distribution, and dust interference in underground coal mine monitoring images, a reference-free image enhancement model structure integrating global information was designed based on the Zero-DCE low-light image enhancement framework. This model can be embedded as a plug-and-play enhancement module into the underground coal mine monitoring image processing workflow and supports end-to-end training. Its overall structure diagram is shown below. Figure 6 As shown:

[0164] (1) In the convolutional backbone network, the 7 standard convolutional layers are replaced with Ghost convolutions. The first six convolutional layers use the ReLU activation function, and the last convolutional layer uses the 10×Tanh activation function to enhance the output dynamic range. A global context block is added after the fourth and fifth Ghost convolutions, such as... Figure 5As shown, the global context module includes a context modeling module and a feature transformation module. The context modeling module is used to aggregate feature information from different spatial locations in the feature map to obtain global context features. The feature transformation module is used to perform channel-dimensional feature transformation on the global context features to characterize the correlation between channels. Finally, the feature-transformed global context features are fused with the original features at each spatial location to enhance the global expressive power of the features.

[0165] (2) To further extract global features, a Vision Transformer branch (i.e., a global feature extraction branch) is introduced, such as... Figure 4 As shown, during feature extraction, the Vision Transformer first segments the input image into fixed-size 16×16 pixel image patches. Each patch is then flattened and projected onto the feature vector space, forming a one-dimensional sequence. Class labels are added as additional sequence elements and fed into a Transformer encoder along with the image patch feature sequence to generate a global feature representation of the image. Furthermore, location embeddings are incorporated into the image patch sequence, enabling the model to recognize the spatial locations of different regions. This design enhances the model's location awareness, allowing it to effectively capture spatial and structural information within the image. After obtaining the sequence containing location information, the model feeds it into the Transformer encoder for feature extraction. After passing through a single Transformer encoder, class labels are extracted from the output as a global feature representation. This global feature is then broadcast to match the spatial dimension of the input image. Finally, it is added pixel-wise to the output of the backbone network, thus fusing global contextual information with local features.

[0166] S703. The input image is enhanced based on the model, and the enhanced underground monitoring image of the coal mine is output.

[0167] Figure 8 A schematic diagram of the multiple iteration mapping function provided in this application. Figure 1 , Figure 9 A schematic diagram of the multiple iteration mapping function provided in this application. Figure 2 ,like Figure 8 and Figure 9 As shown, this type of mapping function requires 4 or 8 iterations to generate the enhancement curve. The more iterations there are, the steeper the curve becomes. Although it can achieve a stronger effect of stretching dark areas or compressing bright areas, it also carries the risk of over-enhancement. Furthermore, multiple iterations lead to complex multi-parameter debugging and high computational costs. Figure 10 A schematic diagram of the reciprocal mapping function provided in this application is shown below. Figure 10As shown, the reciprocal mapping function used in this embodiment does not require iteration and operates through a single parameter. Generate a smooth mapping curve directly in one step, when It can achieve dark area stretching, Highlight compression can be achieved at this time. The image degradation is converted to a linear mapping, resulting in a balanced and natural enhancement effect, and parameter tuning is intuitive and simple. Therefore, the low-light image enhancement method for underground coal mines provided in this application achieves enhancement effects comparable to multi-iteration schemes, while replacing traditional multi-round iterations with a single direct mapping. This significantly reduces the algorithm's time complexity and computational cost, greatly improves computational efficiency, and is more suitable for latency-sensitive applications such as real-time monitoring in underground coal mines.

[0168] The low-light image enhancement method for underground coal mines provided in this application involves inputting the low-light monitoring image to be enhanced into a Zero-DCE image enhancement model that fuses global information for processing, thereby achieving adaptive enhancement of image pixels. This model uses Ghost convolutions to construct the backbone network to extract local image features, thus reducing computational load and parameter size. A global context module is introduced into the backbone network, and a global feature extraction branch based on the Vision Transformer structure is set to model global image information and fuse it with local features. Simultaneously, a lighting mapping curve based on the inverse mapping function is used instead of the traditional iterative mapping function to adjust image brightness. Through this method, effective enhancement of brightness, contrast, and color information in low-light monitoring images of underground coal mines is achieved without the need for paired labeled data. That is, while maintaining the advantages of Zero-DCE no-reference image enhancement, a global information modeling mechanism is introduced to effectively improve problems such as local over-enhancement and insufficient color recovery, while balancing enhancement effect and computational efficiency, thus improving the real-time processing capability and engineering applicability of this method in underground coal mine monitoring systems.

[0169] Figure 11 This is a schematic diagram of the structure of the coal mine underground low-light image enhancement device provided in this application, as shown below. Figure 11 As shown, the coal mine underground low-light image enhancement device 110 provided in this embodiment includes:

[0170] The acquisition module 1101 is used to acquire low-light monitoring images in coal mines;

[0171] The processing module 1102 is used to input the low-light monitoring image into the image enhancement model to obtain the image enhancement parameter map corresponding to the low-light monitoring image. The image enhancement model is a model obtained by training a dual-branch model based on a linear transformation mechanism and a self-attention mechanism to fuse local and global features of the image and generate the image enhancement parameter map. The image enhancement parameter map includes the enhancement parameters of the low-light monitoring image at each pixel position.

[0172] The determination module 1103 is used to determine the target enhanced image based on the low-light monitoring image and the image enhancement parameter map.

[0173] In one possible implementation, the processing module 1102 includes a global feature extraction branch and a convolutional backbone network. The processing module 1102 can also be used to: input the low-light monitoring image into the convolutional backbone network to extract local feature maps of the low-light monitoring image; input the low-light monitoring image into the global feature extraction branch to extract global feature maps of the low-light monitoring image; fuse the local feature maps and global feature maps to obtain a fused feature map; and generate an image enhancement parameter map corresponding to the low-light monitoring image based on the fused feature map.

[0174] In one possible implementation, the global feature extraction branch includes, in sequence, image patch embedding, category labeling, location embedding, a Transformer encoder, a linear layer, and a broadcast module; wherein: image patch embedding is used to perform block embedding and dimensional transformation processing on the low-light monitoring image to obtain image patch features in sequence form; category labeling is used to concatenate learnable category labels to the front of the sequence of image patch features to obtain a feature sequence with classification labels; location embedding is used to add location information to the feature sequence with classification labels to obtain a feature sequence with location awareness; the Transformer encoder is used to perform global modeling and feature encoding on the feature sequence with location awareness through a self-attention mechanism, and output the encoded global feature sequence; the linear layer is used to perform dimensional mapping on the category labels extracted from the encoded global feature sequence to obtain a global feature vector of a specified dimension; the broadcast module is used to perform dimensional expansion and spatial broadcasting on the global feature vector to obtain a global feature map of the low-light monitoring image.

[0175] In one possible implementation, the convolutional backbone network includes multiple sequentially connected Ghost convolutional modules. These Ghost convolutional modules are used to extract features from low-light surveillance images based on a linear transformation mechanism, generating local feature maps of the low-light surveillance images. Specifically, each Ghost convolutional module is used to: perform a convolution operation on the input feature map to generate an intrinsic feature map corresponding to the input feature map; perform a linear transformation operation on the intrinsic feature map to generate the remaining feature maps corresponding to the intrinsic feature map; and concatenate the intrinsic feature map with the remaining feature maps to output a feature map.

[0176] In one possible implementation, the convolutional backbone network also includes at least one global context module, which is connected after a pre-defined Ghost convolutional module. The global context module is used to perform global context modeling on the feature map output by the Ghost convolutional module to obtain an enhanced feature map with global context information, and then outputs the enhanced feature map to the next Ghost convolutional module.

[0177] In one possible implementation, there are 7 Ghost convolutional modules and 2 global context modules. Specifically, the first Ghost convolutional module extracts features from the low-light monitoring image and outputs a first feature map; the second Ghost convolutional module extracts features from the first feature map and outputs a second feature map; the third Ghost convolutional module extracts features from the second feature map and outputs a third feature map; the fourth Ghost convolutional module extracts features from the third feature map and outputs a fourth feature map; the first global context module performs global context modeling on the fourth feature map to obtain a first enhanced feature map; the fifth Ghost convolutional module extracts features from the fusion result of the first enhanced feature map and the third feature map and outputs a fifth feature map; the second global context module performs global context modeling on the fifth feature map to obtain a second enhanced feature map; the sixth Ghost convolutional module extracts features from the fusion result of the second enhanced feature map and the second feature map and outputs a sixth feature map; and the seventh Ghost convolutional module extracts features from the fusion result of the sixth feature map and the first feature map and outputs a seventh feature map, which is then used as a local feature map output by the convolutional backbone network.

[0178] In one possible implementation, the determining module 1103 can also be used to: obtain the target enhanced image based on a preset reciprocal mapping function, according to the low-light monitoring image and the image enhancement parameter map; wherein the reciprocal mapping function satisfies:

[0179] ;

[0180] in, These are pixel coordinates; For low-light monitoring images Pixel value at; Enhance the image for the target Pixel value at; The corresponding parameters in the image enhancement diagram Enhanced parameters at that location.

[0181] In one possible implementation, the processing module 1102 can also be used to: acquire monitoring videos from underground coal mines; extract multiple image frames from the monitoring videos based on a preset time interval to obtain model training data; and train an initial image enhancement model based on the model training data until the output information of the initial image enhancement model meets the preset training requirements to obtain an image enhancement model.

[0182] The coal mine low-light image enhancement device provided in this embodiment can perform the method provided in the above-mentioned method embodiment. Its implementation principle and technical effect are similar, and will not be described in detail here.

[0183] Figure 12 A schematic diagram of the structure of the electronic device provided in this application. Figure 12 As shown, the electronic device 120 provided in this embodiment includes at least one processor 1201 and a memory 1202. Optionally, the device 120 further includes a communication component 1203. The processor 1201, the memory 1202, and the communication component 1203 are connected via a bus 1204.

[0184] In a specific implementation, at least one processor 1201 executes computer execution instructions stored in memory 1202, causing at least one processor 1201 to perform the above-described method.

[0185] The specific implementation process of processor 1201 can be found in the above method embodiments, and its implementation principle and technical effect are similar. It will not be repeated here.

[0186] In the above embodiments, it should be understood that the processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), etc. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in this invention can be directly implemented by a hardware processor, or implemented by a combination of hardware and software modules within the processor.

[0187] The memory may include random access memory (RAM) and may also include non-volatile memory (NVM), such as at least one disk storage device.

[0188] The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of illustration, the buses shown in the accompanying drawings are not limited to a single bus or a single type of bus.

[0189] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the above-described method.

[0190] This application also provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the above-described method.

[0191] The aforementioned readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The readable storage medium can be any available medium accessible to a general-purpose or special-purpose computer.

[0192] An exemplary readable storage medium is coupled to a processor, enabling the processor to read information from and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can reside in an Application Specific Integrated Circuit (ASIC). Alternatively, the processor and the readable storage medium can exist as discrete components in the device.

[0193] The division of units is merely a logical functional division; in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be indirect coupling or communication connection through some interfaces, devices, or units, and may be electrical, mechanical, or other forms.

[0194] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0195] In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0196] If a function is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0197] Those skilled in the art will understand that all or part of the steps of the above-described method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When executed, the program performs the steps of the above-described method embodiments; and the aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.

[0198] It should be understood that the terms “comprising” and “having”, and any variations thereof, in the specification, claims, and accompanying drawings of this application are intended to cover but not exclude inclusion. For example, a product or device that includes a series of components is not necessarily limited to those components that are explicitly listed, but may include other components that are not explicitly listed or that are inherent to such product or device.

[0199] As used in this application, the term "module" means any known or subsequently developed hardware, software, firmware, artificial intelligence, fuzzy logic, or combination of hardware and / or software code capable of performing the functions associated with that element.

[0200] Finally, it should be noted that other embodiments of the invention will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This invention is intended to cover any variations, uses, or adaptations of the invention that follow the general principles of the invention and include common knowledge or customary techniques in the art not disclosed herein, and is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of the invention is limited only by the appended claims.

Claims

1. A method for enhancing low-light images in underground coal mines, characterized in that, include: Acquire low-light monitoring images from underground coal mines; The low-light monitoring image is input into the image enhancement model to obtain the image enhancement parameter map corresponding to the low-light monitoring image. The image enhancement model is a model for fusing local and global features of the image and generating the image enhancement parameter map by training a dual-branch model based on a linear transformation mechanism and a self-attention mechanism. The image enhancement parameter map includes the enhancement parameters of the low-light monitoring image at each pixel position. Based on the low-light monitoring image and the image enhancement parameter map, the target enhanced image is determined.

2. The method according to claim 1, characterized in that, The image enhancement model includes a global feature extraction branch and a convolutional backbone network; The step of inputting the low-light monitoring image into an image enhancement model to obtain an image enhancement parameter map corresponding to the low-light monitoring image includes: The low-light monitoring image is input into the convolutional backbone network to extract local feature maps of the low-light monitoring image; The low-light monitoring image is input into the global feature extraction branch to extract the global feature map of the low-light monitoring image; The local feature map and the global feature map are fused to obtain a fused feature map, and an image enhancement parameter map corresponding to the low-light monitoring image is generated based on the fused feature map.

3. The method according to claim 2, characterized in that, The global feature extraction branch includes, in sequence, image patch embedding, category labeling, location embedding, Transformer encoder, linear layer, and broadcast module; wherein: The image block embedding is used to perform block embedding and dimensional transformation processing on the low-light monitoring image to obtain image block features in sequence form; The category label is used to concatenate a learnable category label to the front of the sequence of image patch features to obtain a feature sequence with a classification label; The location embedding is used to add location information to the feature sequence with classification labels to obtain a feature sequence with location awareness capability. The Transformer encoder is used to perform global modeling and feature encoding on the position-aware feature sequence through a self-attention mechanism, and output the encoded global feature sequence. The linear layer is used to perform dimension mapping on the category labels extracted from the encoded global feature sequence to obtain a global feature vector of a specified dimension. The broadcast module is used to perform dimensional expansion and spatial broadcasting of the global feature vector to obtain the global feature map of the low-light monitoring image.

4. The method according to claim 2, characterized in that, The convolutional backbone network includes multiple Ghost convolutional modules connected in sequence. The Ghost convolutional modules are used to extract features from the low-light monitoring image based on a linear transformation mechanism to generate a local feature map of the low-light monitoring image. Each Ghost convolutional module is specifically used for: Perform a convolution operation on the input feature map to generate an intrinsic feature map corresponding to the input feature map; Perform a linear transformation operation on the intrinsic feature map to generate the remaining feature maps corresponding to the intrinsic feature map; The inherent feature map is concatenated with the remaining feature maps to output a feature map.

5. The method according to claim 4, characterized in that, The convolutional backbone network also includes at least one global context module, which is connected after a preset Ghost convolutional module. The global context module is used to perform global context modeling on the feature map output by the Ghost convolutional module to obtain an enhanced feature map with global context information, and output the enhanced feature map to the next Ghost convolutional module.

6. The method according to claim 5, characterized in that, The Ghost convolutional module consists of 7 modules, and the global context module consists of 2 modules. The first Ghost convolutional module is used to extract features from the low-light monitoring image and output a first feature map; the second Ghost convolutional module is used to extract features from the first feature map and output a second feature map; the third Ghost convolutional module is used to extract features from the second feature map and output a third feature map; and the fourth Ghost convolutional module is used to extract features from the third feature map and output a fourth feature map. The first global context module is used to perform global context modeling on the fourth feature map to obtain the first enhanced feature map; The fifth Ghost convolutional module is used to extract features from the fusion result of the first enhanced feature map and the third feature map, and output the fifth feature map; The second global context module is used to perform global context modeling on the fifth feature map to obtain the second enhanced feature map; The sixth Ghost convolutional module is used to extract features from the fusion result of the second enhanced feature map and the second feature map, and output the sixth feature map; The seventh Ghost convolutional module is used to extract features from the fusion result of the sixth feature map and the first feature map, output the seventh feature map, and use the seventh feature map as the local feature map output by the convolutional backbone network.

7. The method according to any one of claims 1-6, characterized in that, The step of determining the target enhanced image based on the low-light monitoring image and the image enhancement parameter map includes: Based on a preset reciprocal mapping function, the target enhanced image is obtained according to the low-light monitoring image and the image enhancement parameter map; Wherein, the reciprocal mapping function satisfies: ； in, These are pixel coordinates; For the low-light monitoring image in Pixel value at; Enhance the image for the target Pixel value at; The corresponding image enhancement parameters in the image diagram Enhanced parameters at that location.

8. The method according to any one of claims 1-6, characterized in that, Before inputting the low-light monitoring image into the image enhancement model to obtain the image enhancement parameter map corresponding to the low-light monitoring image, the method further includes: Obtain surveillance video from underground coal mines; Based on a preset time interval, multiple image frames are extracted from the surveillance video to obtain model training data; Based on the model training data, the initial image enhancement model is trained until the output information of the initial image enhancement model meets the preset training requirements, thus obtaining the image enhancement model.

9. A low-light image enhancement device for underground coal mines, characterized in that, include: The acquisition module is used to acquire low-light monitoring images from underground coal mines; The processing module is used to input the low-light monitoring image into the image enhancement model to obtain the image enhancement parameter map corresponding to the low-light monitoring image. The image enhancement model is a model obtained by training a dual-branch model based on a linear transformation mechanism and a self-attention mechanism to fuse local and global features of the image and generate the image enhancement parameter map. The image enhancement parameter map includes the enhancement parameters of the low-light monitoring image at each pixel position. The determination module is used to determine the target enhanced image based on the low-light monitoring image and the image enhancement parameter map.

10. An electronic device, characterized in that, include: Memory, processor; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory, causing the processor to perform the method as described in any one of claims 1-8.

Images