Low-light image enhancement method and system based on spatial-frequency domain feature resolution self-adjustment

Abstract

This invention belongs to the field of computer vision and image processing technology, and discloses a low-light image enhancement method and system based on self-adjusting spatial-frequency domain feature resolution. The enhancement method proposes a feature splitting and stitching process, a network architecture based on an encoder-decoder framework with self-adjusting spatial-frequency domain feature resolution, and a complete processing flow from low-light image input to normal-light image output. This invention further flattens multi-scale feature maps into two-dimensional matrices, and performs image restoration from features at different scales through linear layers and feedforward neural networks, further preserving the integrity of the receptive field. Furthermore, this invention extracts important spatial features and then performs frequency restoration on important regions, effectively solving the common problems of over-enhancement in dark areas and loss of detail in bright areas, resulting in superior performance in terms of color fidelity and structural integrity in low-light scenes.

Description

Technical Field

[0001] This invention belongs to the field of computer vision and image processing technology, and particularly relates to a low-light image enhancement method and system based on spatial frequency domain feature resolution self-adjustment. Background Technology

[0002] Low-light image enhancement, a crucial research area in computer vision and image processing, aims to address image quality degradation under low-light conditions, including noise interference, loss of detail, and color distortion. This technology can significantly improve human visual perception and provide high-quality input data for subsequent computer vision tasks such as object detection and face recognition. However, achieving high-quality image enhancement remains a significant challenge due to the complex lighting conditions and sensor noise interference in low-light environments.

[0003] Traditional enhancement methods primarily employ techniques such as histogram equalization and gamma correction based on image statistics, or decomposition methods based on Retinex theory. These methods achieve enhancement by adjusting the brightness distribution or assuming that the image can be decomposed into illumination and reflectance components. However, their core assumptions often ignore degradation factors in actual low-light images, leading to problems such as noise amplification and color distortion in the enhancement results, especially with limited detail recovery capabilities in complex scenes. With the development of deep learning technology, enhancement methods based on convolutional neural networks and Transformers have made significant progress. These methods use end-to-end network architectures, either combining Retinex theory to learn illumination and reflectance components or directly establishing a mapping relationship from low-light images to normal-light images. However, existing deep learning methods still have significant shortcomings: high sensitivity to noise and artifacts, insufficient cross-scene generalization ability, computational complexity limiting real-time applications, and a need for breakthroughs in balancing degradation modeling and detail recovery. These limitations severely affect the performance of low-light enhancement technology in practical applications, necessitating the development of more effective solutions.

[0004] The invention patent, "Neural Network for Enhancing Raw Low-Light Images" (Publication No. CN115004220B, Publication Date 20240820), proposes using a U-net convolutional neural network for downsampling and upsampling to restore images, directly generating full-color RGB images from the monochromatic pixels of the Bayer array of the image sensor and achieving low-light enhancement. However, this method is limited by inductive bias, making it difficult to effectively restore the illumination of certain images. Furthermore, its computational complexity is high, making it difficult to meet the stringent real-time requirements of application scenarios. In addition, its generalization ability to sensor noise patterns still needs further verification.

[0005] Invention Patent: A Low-Light Image Enhancement Method Based on Frequency Domain and Spatial Domain Dual-Domain Perception (Publication No. CN118674628A, Publication Date 20240920) proposes a low-light image enhancement method based on frequency domain and spatial domain dual-domain perception using U-net. However, the U-net method requires calling the dual-domain repair module multiple times, which greatly increases the computational power requirement. In addition, the spatial enhancement module cannot recover complex scenes using only a simple convolutional network.

[0006] Invention Patent: A Transformer-based method for enhancing low-light images in underground mines (Publication No. CN116152117A, Publication Date 20230721). This invention provides a Transformer-based method for enhancing low-light images in underground mines. It utilizes a dual-branch MobileViT module to predict the multiplication and addition graphs respectively, and combines this with a Cross Attention module to generate a 3×3 color matrix and parameters. Finally, it fuses the input image with the prediction results to achieve brightness enhancement, color preservation, and detail retention. Although this method has some effect in low-light underground scenes, it typically requires downsampling the image when calculating self-attention, which may lead to the loss of important information. Furthermore, existing methods mainly focus on the spatial domain, neglecting the use of spectral information in the frequency domain.

[0007] Invention Patent: A method and apparatus for low-light image enhancement based on a Fourier transform (Transformer) network (Publication No. CN117808710A, Publication Date 20250321). This invention provides a low-light image enhancement method based on a Fourier transform (Transformer) network. By feeding the frequency of the low-light image into the Transformer network for training, this method relies on skip connections across multiple Transformer layers, resulting in a very complex network structure and parameters. Furthermore, it only considers the frequency features of the image, neglecting its spatial features. Therefore, there is an urgent need for a method that can simultaneously utilize information from both the spatial and frequency domains and effectively capture key regions to further improve the performance and robustness of low-light image enhancement. Summary of the Invention

[0008] To overcome the problems existing in related technologies, the present invention discloses a low-light image enhancement method and system based on spatial and frequency domain feature resolution self-adjustment. This method can adaptively fuse spatial and frequency domain features at different resolution levels, significantly improving image quality in low-light environments.

[0009] The technical solution is as follows: a low-light image enhancement method based on spatial frequency domain feature resolution self-adjustment, comprising the following steps:

[0010] S1 employs an encoder-decoder architecture, using a hybrid module of ResNet and Transformer to perform convolution operations on the upper and lower half features of the image, thereby encoding the input dark light image and completing multi-scale feature learning.

[0011] S2 flattens the multi-scale features output by the encoder and applies the multi-head self-attention mechanism of the spatial enhancement module to obtain the spatial enhancement features of the image.

[0012] S3: Input the spatial enhancement features into the convolutional layer of the frequency enhancement module to extract high-frequency features. Then, use the extracted high-frequency features and spatial enhancement features to calculate the image frequency and feature frequency through fast Fourier transform to complete the extraction and enhancement of edge information.

[0013] S4, the output features are processed by the decoder. Decode the image to restore its resolution, convert the low-light image back to a normal image, and combine spatial and frequency domain loss optimization networks.

[0014] Step S1 specifically includes:

[0015] S101, the input image is fed into a 3×3 convolutional network to extract image features F;

[0016] S102, the image feature F is divided into average segments. and Two parts, The input is fed into a network consisting of four residual blocks, each containing two 3×3 convolutional layers and a ReLU activation function. The convolution with stride 2 performs a 1 / 2 downsampling, which is then input into the Transformer module to capture long-range dependencies. Next, bilinear interpolation upsampling restores the resolution, outputting features. and By stitching along the channel dimension, the fused features are obtained. ;

[0017] S103 will integrate features By inserting into ResBlock, we obtain a feature scale of 1 / 2. ,Will Inputting into ResBlock yields a feature scale of 1 / 4. .

[0018] Step S2 specifically includes:

[0019] S201, the obtained feature scale , , , Each feature matrix is ​​flattened into a 2D feature matrix using a linear neural network, and the number of channels is expanded using a linear layer.

[0020] S202, apply multi-head self-attention to the features obtained in step S201. The mechanism, through residual connections and layer normalization Retaining the original information, the expression is:

[0021] ;

[0022] In the formula, For normalization, Features are those processed by the attention mechanism. The features are those obtained after linear mapping. This is handled by a multi-head self-attention mechanism. The features are the result of a linear mapping to dimension 1. The features are obtained after a 2-dimensional linear mapping. Features are obtained after a linear mapping of dimension 3;

[0023] S203, input the features into the feedforward network It contains two linear layers with ReLU activation functions, and the output is connected via residuals;

[0024] S204 reshapes the features back to their original spatial dimensions. Spatial augmentation features are obtained. The expression is:

[0025] ;

[0026] In the formula, For linear mapping layer, To modify the activation function of the linear unit, These represent the height, width, and number of feature channels of the original feature map, respectively.

[0027] Step S3 specifically includes:

[0028] S301, Enhancement features for the output space High-frequency features are generated by applying concatenated 3×3 and 1×1 depthwise separable convolutions. ;

[0029] S302 will incorporate high-frequency characteristics and spatial enhancement features The dual-domain enhanced features are obtained by summing the results using Fast Fourier Transform. .

[0030] Step S4 specifically includes:

[0031] S401 enhances dual-domain features The image is upsampled sequentially using bilinear interpolation and then concatenated with encoder features at the corresponding scale. This concatenated feature is then processed by a ResBlock layer containing two 3×3 convolutional layers and residual connections, ensuring the number of output channels remains consistent with the encoder's corresponding scale. The ResBlock layer gradually restores the image resolution, ultimately outputting the enhanced image. ;

[0032] S402, constrains spatial features using the L1 absolute value loss function;

[0033] S403 performs a Fast Fourier Transform on the original image and the predicted image, and constrains the frequency features using the L1 absolute value loss function.

[0034] In step S402, the spatial features are constrained by the L1 absolute value loss function, expressed as:

[0035] ;

[0036] In the formula, For spatial feature loss, For absolute value operations, To enhance the image pixel matrix, This is the original image pixel matrix.

[0037] In step S403, the original image and the predicted image are subjected to Fast Fourier Transform, and the frequency features are constrained by the L1 absolute value loss function, the expression of which is:

[0038] ;

[0039] In the formula, For frequency feature loss, For Fast Fourier Transform.

[0040] Another object of the present invention is to provide a low-light image enhancement system based on self-adjustment of spatial frequency domain feature resolution, the system implementing the aforementioned low-light image enhancement method based on self-adjustment of spatial frequency domain feature resolution, the system comprising:

[0041] The feature resolution self-adjuster includes an encoder and a decoder. The encoder uses a hybrid module of ResNet and Transformer to perform convolution operations on the upper and lower half features of the image, respectively, to encode the input dark-light image and complete multi-scale feature learning. The decoder processes the output features... Decode the image to restore its resolution, restore the low-light image to a normal image, and combine spatial and frequency domain loss optimization networks.

[0042] The spatial enhancement module flattens the input feature map into a two-dimensional matrix, expands the number of channels through linear layers, applies a multi-head self-attention mechanism combined with residual connections, refines the features through layer normalization and feedforward networks, reshapes the processed features back to the original spatial dimension, and repairs them through the frequency enhancement module.

[0043] The frequency enhancement module applies deep convolution to the spatially enhanced features to generate high-frequency features, and generates the final output through element-wise multiplication and residual connections.

[0044] Furthermore, the system is applied to noise assessment in complex interference scenarios involving underground dust in coal and rock environments, and to the recovery of high-frequency edge information of safety signs.

[0045] Furthermore, the system is mounted on a computer-readable storage medium storing a computer program, which, when executed by a processor, can realize the functions of the aforementioned low-light image enhancement system based on spatial frequency domain feature resolution self-adjustment.

[0046] Combining all the above technical solutions, the beneficial effects of this invention are as follows:

[0047] First, compared to existing technologies that directly use Transformer or convolutional layers to extract features, this invention designs a feature resolution self-adjuster to replace simple feature stitching. It dynamically filters multi-scale features through channel-spatial dual attention. The feature resolution self-adjuster extracts features by fusing Resblock and Transformer blocks and extracting features through splitting and downsampling operations. This provides the ability to correlate distant pixels while maintaining low complexity. To simplify computation, the computational load is greatly reduced without sacrificing accuracy. For the spatial enhancement module, compared to directly repairing images from feature convolution, this invention provides a more efficient and efficient approach.

[0048] Secondly, this invention further flattens the multi-scale feature map into a two-dimensional matrix, and performs image restoration from features at different scales through a linear layer and a feedforward neural network, thereby further maintaining the integrity of the receptive field. In addition, this invention extracts important spatial features and then performs frequency restoration on important regions, effectively solving the common problems of over-enhancement of dark areas and loss of details in bright areas, and performs better in terms of color fidelity and structural integrity in low-light scenes.

[0049] Third, as a low-light image enhancement technology based on spatial frequency domain feature resolution self-adjustment, the inventive value of this invention is reflected in several important dimensions. In terms of technology transfer and commercial value, this solution has been successfully applied to practical scenarios such as underground safety monitoring in coal mines and low-light photography for smartphones, significantly improving the accuracy of safety sign recognition and image signal-to-noise ratio performance in extreme low-light environments. According to third-party market analysis data, this technology reduces hardware costs by optimizing algorithm efficiency.

[0050] Fourth, this invention achieves adaptive adjustment of spatial-frequency domain feature resolution for the first time, overcoming the inherent defects of traditional methods in spatial and frequency domain processing. Through innovative network architecture design, it achieves breakthrough progress in key performance indicators, outperforming existing mainstream methods in both objective evaluation metrics and real-time processing speed on standard test datasets, demonstrating significant technical advantages.

[0051] Fifth, this invention solves the long-standing technical problem of "overexposure in dark areas and distortion in bright areas" in the field of low-light image enhancement. Addressing the limitations of traditional Retinex theory in handling complex scenes, it achieves high-quality restoration of image details under non-uniform lighting conditions through the synergistic optimization of spatial attention and frequency domain convolution, resulting in significant performance improvements on professional test datasets.

[0052] Sixth, addressing the limitations of the Transformer architecture in real-time processing, computational efficiency was optimized through a feature splitting strategy. Simultaneously, the frequency domain processing flow was innovatively simplified by combining lightweight depthwise convolution with precise frequency domain loss constraints, significantly reducing computational resource consumption while ensuring enhancement effects. These technological innovations provide new technical routes and practical solutions for the development of low-light image processing. Attached Figure Description

[0053] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure;

[0054] Figure 1 This is a flowchart of the training process for a low-light image enhancement method based on spatial frequency domain feature resolution self-adjustment provided in an embodiment of the present invention.

[0055] Figure 2 This is a flowchart of a low-light image enhancement method based on spatial frequency domain feature resolution self-adjustment provided in an embodiment of the present invention;

[0056] Figure 3 This is a diagram of the feature resolution self-adjusting encoder network structure provided in an embodiment of the present invention;

[0057] Figure 4 This is a network structure diagram of the spatial enhancement module provided in an embodiment of the present invention;

[0058] Figure 5 This is a network structure diagram of the frequency enhancement module provided in an embodiment of the present invention;

[0059] Figure 6 This is a diagram of the feature resolution self-adjusting decoder network structure provided in an embodiment of the present invention;

[0060] Figure 7 This is an indoor building low-light enhancement image provided in an embodiment of the present invention; wherein, (a) is the input image and (b) is the enhanced image;

[0061] Figure 8 This is a low-light enhancement image of a ping-pong table provided in an embodiment of the present invention; wherein, (a) is the input image and (b) is the enhanced image;

[0062] Figure 9 This is a low-light enhancement image of a swimming pool provided in an embodiment of the present invention; wherein, (a) is the input image and (b) is the enhanced image. Detailed Implementation

[0063] To make the above-mentioned objects, features, and advantages of the present invention more apparent and understandable, specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Many specific details are set forth in the following description to provide a thorough understanding of the present invention. However, the present invention can be practiced in many other ways different from those described herein, and those skilled in the art can make similar modifications without departing from the spirit of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0064] The innovation of this invention lies in its proposed self-adjusting mechanism for spatial-frequency domain feature resolution. This mechanism achieves multi-scale feature adaptive fusion through a hybrid encoder combining ResNet and Transformer, and significantly improves low-light image enhancement by combining spatial domain attention enhancement and frequency domain depthwise convolution for dual-domain collaborative optimization. Compared to existing technologies, its breakthrough lies in: a pioneering feature splitting downsampling strategy that divides input features into spatial / semantic branches for parallel processing. While retaining high-resolution spatial information in 50% of the channels, the other half of the channels captures long-range dependencies through downsampling Transformer, balancing computational efficiency and feature integrity; and a cascaded spatial-frequency enhancement module design. The spatial enhancement module uses Transformer multi-head attention to locate degraded regions, while the frequency enhancement module employs lightweight depthwise separable convolution to enhance high-frequency edge information, avoiding the computational overhead of traditional FFT frequency domain transformation.

[0065] Example 1 discloses a low-light image enhancement method based on spatial-frequency domain feature resolution self-adjustment. By jointly utilizing spatial and frequency domain information and introducing a feature resolution self-adjuster, the enhancement effect of low-light images is significantly improved. This method solves the problem that traditional methods, which rely solely on spatial domain brightness adjustment, cannot effectively handle noise, artifacts, and chromatic aberration. See details... Figure 1 The training process of the low-light image enhancement method based on spatial frequency domain feature resolution self-adjustment provided in this embodiment of the invention.

[0066] For example, such as Figure 2As shown, this invention discloses a low-light image enhancement method based on self-adjusting spatial-frequency domain feature resolution. It proposes a feature splitting and stitching process, a network architecture based on an encoder-decoder framework with self-adjusting spatial-frequency domain feature resolution, and a complete processing flow from low-light image input to normal-light image output. Specifically, it includes the following steps:

[0067] S1 employs an encoder-decoder architecture, using a hybrid module of ResNet and Transformer to perform convolution operations on the upper and lower half features of the image, thereby encoding the input dark light image and completing multi-scale feature learning.

[0068] like Figure 3 This is a feature resolution self-adjusting encoder network structure diagram provided in an embodiment of the present invention. The specific process is as follows:

[0069] S101, the input image is fed into a 3×3 convolutional network to extract image features F;

[0070] S102, the image feature F is divided into average segments. and Two parts, The input is fed into a network consisting of four residual blocks, each containing two 3×3 convolutional layers and a ReLU activation function. The convolution with stride 2 performs a 1 / 2 downsampling, which is then input into the Transformer module to capture long-range dependencies. Next, bilinear interpolation upsampling restores the resolution, outputting features. and By stitching along the channel dimension, the fused features are obtained. ;

[0071] S103 will integrate features By inserting into ResBlock, we obtain a feature scale of 1 / 2. ,Will Inputting into ResBlock yields a feature scale of 1 / 4. .

[0072] S2 flattens the multi-scale features output by the encoder and applies the multi-head self-attention mechanism of the spatial enhancement module to obtain the spatial enhancement features of the image.

[0073] Figure 4 This is a network structure diagram of the spatial enhancement module provided in an embodiment of the present invention. The specific process is as follows:

[0074] S201, the obtained feature scale , , , Each feature matrix is ​​flattened into a 2D feature matrix using a linear neural network, and the number of channels is expanded using a linear layer.

[0075] S202, apply multi-head self-attention to the features obtained in step S201. The mechanism, through residual connections and layer normalization Retaining the original information, the expression is:

[0076] ;

[0077] In the formula, For normalization, Features are those processed by the attention mechanism. The features are those obtained after linear mapping. This is handled by a multi-head self-attention mechanism. The features are the result of a linear mapping to dimension 1. The features are obtained after a 2-dimensional linear mapping. Features are obtained after a linear mapping of dimension 3;

[0078] S203, input the features into the feedforward network It contains two linear layers with ReLU activation functions, and the output is connected via residuals;

[0079] S204 reshapes the features back to their original spatial dimensions. Spatial augmentation features are obtained. The expression is:

[0080] ;

[0081] In the formula, For linear mapping layer, To modify the activation function of the linear unit, These represent the height, width, and number of feature channels of the original feature map, respectively.

[0082] S3: Input the spatial enhancement features into the convolutional layer of the frequency enhancement module to extract high-frequency features. Then, use the extracted high-frequency features and spatial enhancement features to calculate the image frequency and feature frequency through fast Fourier transform to complete the extraction and enhancement of edge information.

[0083] Perform Fast Fourier Transform (FFT) on the two features respectively, and then process the image. (size The formula is as follows:

[0084] ;

[0085] In the formula, For spatial domain pixel values, For frequency domain complex values, These are the image height and width, respectively. For spatial coordinates, For frequency index, It is the imaginary unit.

[0086] Figure 5 This is a network structure diagram of the frequency enhancement module provided in an embodiment of the present invention, specifically including:

[0087] S301, Enhancement features for the output space High-frequency features are generated by applying concatenated 3×3 and 1×1 depthwise separable convolutions. ;

[0088] S302 will incorporate high-frequency characteristics and spatial enhancement features The dual-domain enhanced features are obtained by summing the results using Fast Fourier Transform. .

[0089] Dual-domain enhancement features The restored image features consist of high-frequency features. and spatial enhancement features The transformation and addition are used to obtain the enhanced features of the two domains. Decoding will yield the repaired image.

[0090] S4, the output features are processed by the decoder. Decode the image to restore its resolution, convert the low-light image back to a normal image, and combine spatial and frequency domain loss optimization networks.

[0091] Figure 6 This is a feature resolution self-adjusting decoder network structure diagram provided in an embodiment of the present invention; that is, a decoder containing two ResNets is used to decode the extracted features to restore the low-light image to a normal image;

[0092] S401 enhances dual-domain features The image is upsampled sequentially using bilinear interpolation and then concatenated with encoder features at the corresponding scale. This concatenated feature is then processed by a ResBlock layer containing two 3×3 convolutional layers and residual connections, ensuring the number of output channels remains consistent with the encoder's corresponding scale. The ResBlock layer gradually restores the image resolution, ultimately outputting the enhanced image. ;

[0093] S402, constraining spatial features using the L1 absolute value loss function; the expression is:

[0094] ;

[0095] In the formula, For spatial feature loss, For absolute value operations, To enhance the image pixel matrix, This is the original image pixel matrix.

[0096] S403 performs a Fast Fourier Transform on the original image and the predicted image, and constrains the frequency features using the L1 absolute value loss function, as expressed in:

[0097] ;

[0098] In the formula, For frequency feature loss, For Fast Fourier Transform.

[0099] As illustrated by the above embodiments, this invention provides a low-light image enhancement method based on self-adjustment of spatial and frequency domain feature resolution. This method restores low-light images to normal images by combining spatial and frequency domain information through feature resolution self-adjustment. Specifically, it focuses more on key regions of the low-light image, fully understanding the differences between low-light and normal regions in both the spatial and frequency domains. The spatial enhancement module takes different scale features of the image as input and repairs spatial regions through linear layers and feedforward neural networks. Subsequently, the frequency enhancement model enhances high-frequency signals or complex regions. To reduce computational overhead, this invention uses Transformer only for half of the features, while ResBlock is used for the other half of the features that do not concern spatial location, significantly reducing computational load without sacrificing accuracy. This method solves the problem that traditional methods rely solely on spatial domain brightness adjustment and cannot effectively handle noise, artifacts, and chromatic aberration. It overcomes the limitations of existing deep learning methods that struggle to fully recover image illumination and detail information due to induction bias or high computational complexity, while also compensating for the shortcomings of existing methods that neglect frequency domain information, resulting in unsatisfactory enhancement effects in complex scenes. This invention is applicable to image enhancement scenarios in various low-light environments, such as nighttime surveillance, low-light photography, and medical image processing. It can effectively restore the brightness, contrast, and detail information of images, suppress noise and artifacts, and improve the visual quality of images and the performance of subsequent computer vision tasks. Through the synergistic effect of spatial enhancement and frequency enhancement modules, and the introduction of a feature resolution self-adjuster, this invention achieves efficient enhancement of key regions in low-light images, significantly improving the enhancement effect and robustness.

[0100] Example 2: The low-light image enhancement system based on spatial-frequency domain feature resolution self-adjustment provided in this embodiment of the invention proposes a Transformer-based spatial enhancement module, a frequency enhancement module using depth convolution and their sequential combination, and a feature resolution self-adjuster; as follows: Figures 2-6 As shown;

[0101] The feature resolution self-adjuster includes an encoder and a decoder. The encoder uses a hybrid module of ResNet and Transformer to perform convolution operations on the upper and lower half features of the image, respectively, to encode the input dark-light image and complete multi-scale feature learning. The decoder processes the output features... Decode the image to restore its resolution, restore the low-light image to a normal image, and combine spatial and frequency domain loss optimization networks.

[0102] The spatial enhancement module flattens the input feature map into a two-dimensional matrix, expands the number of channels through linear layers, applies a multi-head self-attention mechanism combined with residual connections, further refines the features through layer normalization and feedforward networks, and finally reshapes the processed features back to the original spatial dimension; then, the frequency enhancement module performs further repair.

[0103] The frequency enhancement module applies deep convolution to the spatially enhanced features to generate high-frequency features, and then generates the final output through element-wise multiplication and residual connections.

[0104] Specifically, the feature resolution self-regulator: This invention achieves multi-scale, multi-level feature fusion and reuse by splitting high-resolution features and introducing multi-scale receptive fields, while simultaneously improving computational efficiency. Specifically, the entire encoder and decoder network comprises three scales, each corresponding to different resolution features. The encoder consists of a Transformer and a ResBlock. The feature resolution self-regulator divides the input features into two equal parts along the channel dimension, using a Transformer for the spatially important half and a ResBlock for the other half, which is less concerned with spatial location. The results are then concatenated to generate the final output. This is because the Transformer provides the ability to correlate distant pixels, but its computational complexity is extremely high. By combining the advantages of both networks, this invention enhances the overall network's receptive field and feature extraction capabilities.

[0105] Spatial Augmentation Module: Utilizing a Transformer-based multi-head self-attention mechanism and a multi-layer feedforward network, this module enhances degenerate regions in the spatial domain, providing location information for subsequent frequency domain augmentation. Specifically, the spatial augmentation module flattens the input feature map into a two-dimensional matrix, expands the number of channels through linear layers, applies a multi-head self-attention mechanism combined with residual connections to maintain information integrity, further refines the features through layer normalization and a feedforward network, and finally reshapes the processed features back to the original spatial dimension. This is then further repaired by the frequency augmentation module.

[0106] Frequency Enhancement Module: The frequency enhancement module processes the output of the spatial enhancement module. It extracts high-frequency signals from the image through deep convolution, highlighting edge information or complex regions in low-light images and further optimizing the enhancement effect. Specifically: First, deep convolution is applied to the spatially enhanced features to generate high-frequency features, and then element-wise multiplication and residual connections are used to generate the final output.

[0107] As can be seen from the above embodiments, the present invention makes full use of spatial and frequency domain information, and optimizes the entire enhancement process by using a multi-input and multi-output scale strategy, combined with loss functions in the spatial and frequency domains, to generate high-quality normal illumination images.

[0108] This invention employs an innovative low-light image enhancement method with self-adjusting spatial-frequency domain feature resolution, achieving deep collaborative optimization of spatial and frequency domain information in low-light image processing. By combining a feature resolution self-adjuster with the advantages of Transformer and ResBlock, the receptive field of the network is increased while considering computational complexity. It can dynamically capture the long-range spatial correlation of degraded regions, accurately locate dark areas and noise interference features, and enhance the overall illumination recovery capability of the image. In the spatial enhancement module, a spatial enhancement module based on an improved Transformer architecture is constructed. Through multi-scale two-dimensional combination and a feedforward network, the receptive field is expanded and the ability to understand context is enhanced. In the frequency enhancement module, a frequency enhancement module constructed through depthwise separable convolution effectively extracts high-frequency detail information, enhancing image edge and texture features while preserving low-frequency illumination distribution, significantly alleviating the contradiction between detail blurring and noise amplification in traditional methods.

[0109] This invention addresses noise misjudgment issues in complex environments such as underground dust obscuring the coal and rock landscape, while also recovering high-frequency edge information of safety signs. Its modular, lightweight architecture optimizes computational efficiency while ensuring enhanced performance across both domains. Its cross-scene adaptability is demonstrated by robust handling of natural low-light environments and special industrial scenarios (such as strong underground light interference and equipment vibration blurring). Through multi-scale, multi-level feature fusion and reuse, it achieves a balance between illumination recovery, detail preservation, and noise suppression, providing a high-quality image foundation for subsequent computer vision tasks.

[0110] To further illustrate the effects of the embodiments of the present invention, the following experiments were conducted: The evaluation metrics selected for the experiments were PSNR and SSIM, both of which can effectively evaluate the illumination restoration effect of images. Higher PSNR and SSIM values ​​indicate that the restored image is closer to the true value.

[0111] Table 1 Relevant Evaluation Indicators

[0112]

[0113] Analysis reveals that traditional methods perform the weakest in terms of PSNR and SSIM, primarily because their reliance on global adjustments makes them ill-suited to the complex local degradation characteristics of low-light images. The U-Net-based method shows significant improvement over traditional methods, demonstrating the advantages of deep learning in feature learning and nonlinear mapping; however, its PSNR remains below 20, indicating that single spatial domain processing is insufficient to fully restore image quality. The Transformer-based method achieves a PSNR of 23.649, showcasing the effectiveness of the attention mechanism in capturing long-range dependencies; however, its SSIM is slightly lower than the U-Net scheme, reflecting room for improvement in Transformer's ability to preserve structural similarity.

[0114] The dual-domain enhancement method of this invention achieves optimal results across all metrics, with a PSNR of 27.727 and an SSIM of 0.888, significantly outperforming other comparative methods. This advantage stems primarily from three aspects: First, the spatial-frequency dual-domain collaborative enhancement mechanism achieves the best balance between illumination restoration and detail preservation; second, multi-scale feature learning effectively handles degradation patterns at different scales; and finally, the improved Transformer architecture maintains strong feature representation capabilities while reducing computational complexity. Particularly noteworthy is the invention's outstanding performance on the SSIM metric (7.5% improvement over the suboptimal method), demonstrating that the restored image more closely approximates real-world lighting conditions in terms of structural similarity, which is particularly important for subsequent visual analysis tasks.

[0115] The image stitching results in the test set are shown as follows: Figures 7-9 As shown, the experimental results reveal that the finely carved patterns and paintings on the building facade are almost invisible under low light conditions. However, the dual-domain enhancement mechanism of this invention not only accurately restores the geometric features of the building structure, but also restores the details of the murals very well.

[0116] Under extreme sidelighting conditions, the metal components at the edge of the ping-pong table exhibit complex specular reflections and interlacing shadows. Other methods either overexpose these areas or lose detail. However, this invention, through precise control of the frequency domain enhancement module, successfully restores the three-dimensionality and material properties of mechanical structures such as axles while preserving detail in highlight areas.

[0117] Traditional methods are completely incapable of distinguishing small buoys at the far end of a swimming pool when the subject is at a distance. However, this invention accurately recovers the outlines of these tiny objects through multi-scale feature fusion and long-range attention mechanisms.

[0118] Case Study: In the unique application scenario of underground coal mines, low-light image enhancement technology faces numerous unique challenges. The underground environment not only suffers from severe insufficient lighting but also is accompanied by a large amount of dust, water mist, equipment vibration, and other interference factors, creating extremely complex imaging conditions. Traditional image enhancement methods are often ineffective in this environment—histogram equalization can lead to over-enhancing of dust particles, resulting in snowflake-like noise; methods based on Retinex theory struggle to handle the localized strong light caused by direct mine lamp illumination; and ordinary deep learning enhancement networks frequently misjudge dust as image details, leading to severe image distortion.

[0119] The shortcomings of existing methods in coal mine scenarios are mainly reflected in three dimensions: First, in spatial domain processing, most methods cannot distinguish between real equipment outlines and dust interference, and the enhanced image often mistake drifting coal dust for equipment parts; second, in terms of dynamic range control, traditional methods have difficulty simultaneously processing the highlight areas of miners' headlamps and the shadow areas deep in the roadway; most importantly, existing methods generally lack the ability to model specific noise patterns underground, resulting in important safety warning signs either being drowned out by noise or being incorrectly smoothed during the enhancement process.

[0120] In contrast, this invention is specifically designed to meet the unique needs of underground coal mines, demonstrating significant advantages. The spatial enhancement module of this invention, through prior knowledge of the underground environment, can accurately identify structural features that need to be preserved (such as hydraulic supports and conveyor belts), while recognizing drifting dust as interference that needs to be suppressed. The frequency enhancement module specifically strengthens the edge features of safety signs, ensuring that key warning messages such as "No Entry" are clearly legible. Particularly noteworthy is the invention's handling of areas directly illuminated by miner's lamps—preserving the brightness information of the light source itself while clearly revealing the status of equipment behind it through strong light. This characteristic plays a crucial role in predicting equipment malfunctions underground.

[0121] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any modifications, equivalent substitutions and improvements made by those skilled in the art within the scope of the technology disclosed in the present invention and within the spirit and principles of the present invention should be covered within the scope of protection of the present invention.

Claims

1. A low-light image enhancement method based on spatial frequency domain feature resolution self-adjustment, characterized in that, The method includes the following steps: S1 employs an encoder-decoder architecture, using a hybrid module of ResNet and Transformer to perform convolution operations on the upper and lower half features of the image, thereby encoding the input dark light image and completing multi-scale feature learning. S2 flattens the multi-scale features output by the encoder and applies the multi-head self-attention mechanism of the spatial enhancement module to obtain the spatial enhancement features of the image. S3: Input the spatial enhancement features into the convolutional layer of the frequency enhancement module to extract high-frequency features. Then, use the extracted high-frequency features and spatial enhancement features to calculate the image frequency and feature frequency through fast Fourier transform to complete the extraction and enhancement of edge information. S4, the features of the output are processed by the decoder. Decode the image to restore its resolution, restore the low-light image to a normal image, and combine spatial and frequency domain loss optimization networks. Step S1 specifically includes: S101, the input image is fed into a 3×3 convolutional network to extract image features F; S102, the image feature F is divided into average segments. and Two parts, The input is fed into a network consisting of four residual blocks, each containing two 3×3 convolutional layers and a ReLU activation function. The convolution with stride 2 performs a 1 / 2 downsampling, which is then input into the Transformer module to capture long-range dependencies. Next, bilinear interpolation upsampling restores the resolution, outputting features. and By stitching along the channel dimension, the fused features are obtained. ; S103 will integrate features By inserting into ResBlock, we obtain a feature scale of 1 / 2. ,Will Inputting into ResBlock yields a feature scale of 1 / 4. ; Step S2 specifically includes: S201, the obtained feature scale , , , Each feature matrix is ​​flattened into a 2D feature matrix using a linear neural network, and the number of channels is expanded using a linear layer. S202, apply multi-head self-attention to the features obtained in step S201. The mechanism, through residual connections and layer normalization Retaining the original information, the expression is: ； In the formula, For normalization, Features are those processed by the attention mechanism. The features after linear mapping, This is handled by a multi-head self-attention mechanism. The features are obtained after a linear mapping of dimension 1. The features are obtained after a 2-dimensional linear mapping. Features are obtained after a linear mapping of dimension 3; S203, input the features into the feedforward network It contains two linear layers with ReLU activation functions, and the output is connected via residuals; S204 reshapes the features back to their original spatial dimensions. Spatial augmentation features are obtained. The expression is: ； In the formula, For linear mapping layer, To modify the activation function of the linear unit, These represent the height, width, and number of feature channels of the original feature map, respectively. Step S3 specifically includes: S301, Enhancement features for the output space High-frequency features are generated by applying concatenated 3×3 and 1×1 depthwise separable convolutions. ; S302 will incorporate high-frequency characteristics and spatial enhancement features The dual-domain enhanced features are obtained by summing the results using Fast Fourier Transform. .

2. The low-light image enhancement method based on spatial frequency domain feature resolution self-adjustment according to claim 1, characterized in that, Step S4 specifically includes: S401 enhances dual-domain features The image is upsampled sequentially using bilinear interpolation and then concatenated with encoder features at the corresponding scale. This concatenated feature is then processed by a ResBlock layer containing two 3×3 convolutional layers and residual connections, ensuring the number of output channels remains consistent with the encoder's corresponding scale. The ResBlock layer gradually restores the image resolution, ultimately outputting the enhanced image. ; S402, constrains spatial features using the L1 absolute value loss function; S403 performs a Fast Fourier Transform on the original image and the predicted image, and constrains the frequency features using the L1 absolute value loss function.

3. The low-light image enhancement method based on spatial frequency domain feature resolution self-adjustment according to claim 2, characterized in that, In step S402, the spatial features are constrained using the L1 absolute value loss function, expressed as: ； In the formula, For spatial feature loss, For absolute value operations, To enhance the image pixel matrix, This is the original image pixel matrix.

4. The low-light image enhancement method based on spatial frequency domain feature resolution self-adjustment according to claim 2, characterized in that, In step S403, the original image and the predicted image are subjected to Fast Fourier Transform, and the frequency features are constrained by the L1 absolute value loss function, the expression of which is: ； In the formula, For frequency feature loss, For Fast Fourier Transform.

5. A low-light image enhancement system based on spatial frequency domain feature resolution self-adjustment, characterized in that, The system implements the low-light image enhancement method based on spatial frequency domain feature resolution self-adjustment as described in any one of claims 1-4, and the system includes: The feature resolution self-adjuster includes an encoder and a decoder. The encoder uses a hybrid module of ResNet and Transformer to perform convolution operations on the upper and lower half features of the image, respectively, to encode the input dark-light image and complete multi-scale feature learning. The decoder processes the output features... Decode the image to restore its resolution, restore the low-light image to a normal image, and combine spatial and frequency domain loss optimization networks. The spatial enhancement module flattens the input feature map into a two-dimensional matrix, expands the number of channels through linear layers, applies a multi-head self-attention mechanism combined with residual connections, refines the features through layer normalization and feedforward networks, reshapes the processed features back to the original spatial dimension, and repairs them through the frequency enhancement module. The frequency enhancement module applies deep convolution to the spatially enhanced features to generate high-frequency features, and generates the final output through element-wise multiplication and residual connections.

6. The low-light image enhancement system based on spatial frequency domain feature resolution self-adjustment according to claim 5, characterized in that, This system is used in coal and rock scenarios to handle noise assessment in complex interference scenarios with underground dust cover, and to recover high-frequency edge information of safety signs.

7. The low-light image enhancement system based on spatial frequency domain feature resolution self-adjustment according to claim 5, characterized in that, The system is mounted on a computer-readable storage medium that stores a computer program. When the computer program is executed by a processor, it can realize the functions of the low-light image enhancement system based on spatial frequency domain feature resolution self-adjustment described above.

Images