A spatial and wavelet domain dual-domain transformer low-illumination image enhancement method and system

Abstract

The application discloses a kind of airspace and wavelet domain dual-domain Transform low-illumination image enhancement method and system, belong to image processing technical field, method includes: low-illumination RGB image is converted to YCbCr color space, after data is carried out shallow convolution mapping, input wavelet Transform module, and the discrete wavelet transform is executed to query feature, key feature and value feature and wavelet domain self-attention calculation is carried out;Again through multi-head transpose self-attention Transform module, multi-scale channel attention module and decoder processing, obtain reconstruction feature;Feature is converted back to RGB color space by convolution, and obtain enhanced RGB image.The application can take into account the brightness enhancement of low-illumination image, noise control, detail reconstruction and computational efficiency.

Description

Technical Field

[0001] This invention belongs to the field of computer vision technology, specifically relating to a method and system for enhancing low-light images using a dual-domain Transformer architecture in both the spatial and wavelet domains. Background Technology

[0002] Low-light image enhancement is a crucial image preprocessing step in systems such as autonomous driving, security monitoring, and mobile computing photography. In low-light environments, images acquired by imaging devices generally suffer from low brightness, poor contrast, loss of details in dark areas, and significantly amplified noise. These issues can affect the accuracy and robustness of subsequent advanced vision tasks such as object detection, image segmentation, and image recognition.

[0003] Traditional low-light image enhancement methods mainly include histogram equalization, gamma correction, and decomposition enhancement based on Retinex theory. These methods typically rely on hand-designed priors and mapping rules, and are prone to defects such as over-enhancement, color shift, and halo artifacts in complex lighting scenes, making it difficult to stably adapt to lighting changes in different low-light images.

[0004] Low-light image enhancement methods based on convolutional neural networks (CNNs) have improved image enhancement to some extent due to their strong feature extraction capabilities. However, the inherent local receptive field of convolutional operations limits the model's ability to model the global illumination distribution. When faced with scenes with large dark areas or non-uniform illumination, CNN-based methods are still prone to problems such as uneven brightness, local overexposure, or underexposure.

[0005] In recent years, the Transformer, with its self-attention mechanism, has demonstrated certain advantages in low-light image enhancement by achieving global long-range dependency modeling. However, the standard Transformer still suffers from the following problems in low-light image enhancement tasks: First, the computational complexity of self-attention increases quadratically with feature resolution, resulting in high inference and training overhead. Secondly, when performing attention calculations directly on query features, key features, and value features in the spatial domain, noise and effective texture are easily amplified simultaneously. Third, the existing Transformer method lacks a frequency domain decoupling mechanism, making it difficult to distinguish between low-frequency global illumination and high-frequency details / noise, resulting in a difficulty in achieving both brightening and denoising.

[0006] Therefore, existing low-light image enhancement methods still struggle to achieve an effective balance between reducing computational load, suppressing noise, preserving details, and uniform brightening, failing to meet the dual demands of enhancement accuracy and processing efficiency in practical engineering scenarios. A key technical challenge is how to combine spatial domain feature modeling with wavelet domain frequency decoupling during low-light image enhancement, enabling the model to simultaneously process global illumination information and high-frequency detail / noise information. Summary of the Invention

[0007] The technical problem to be solved by this invention is to address the shortcomings of the prior art by providing a dual-domain Transformer low-light image enhancement method and system in both spatial and wavelet domains. This method solves the technical problems in existing low-light image enhancement methods, such as high computational complexity of self-attention, easy synchronous amplification of noise by direct spatial domain enhancement, and lack of frequency domain decoupling processing for low-frequency global illumination information and high-frequency detail / noise information. These problems make it difficult to balance brightness enhancement, noise control, detail restoration, and computational efficiency. The invention achieves uniform global illumination correction and accurate restoration of local detail textures.

[0008] The present invention adopts the following technical solution: A dual-domain (spatial and wavelet domain) Transformer low-light image enhancement method includes the following steps: S1. Obtain a low-light RGB image and convert the low-light RGB image to the YCbCr color space; S2. Perform shallow convolution mapping on the data to obtain initial features; S3. Input the initial features into the wavelet Transformer module. The wavelet Transformer module generates query features, key features and value features based on the initial features, and performs discrete wavelet transform on the query features, key features and value features respectively to obtain the low-frequency components and high-frequency components corresponding to the query features, key features and value features respectively. Based on the low-frequency components and the high-frequency components, wavelet domain self-attention calculation is performed to obtain the first coding feature. S4. In the deep layer of the encoder, the encoding features input to the multi-head transposed self-attention Transformer module are subjected to self-attention calculation in the channel dimension to obtain the second encoding feature. S5. Input the encoded features output from at least one resolution level during the encoding process into the multi-scale channel attention module, generate channel weights through the multi-scale channel attention module, and perform weighted processing on the encoded features of the corresponding resolution level according to the channel weights to obtain weighted encoded features. S6. Input the encoded features output from the last layer of the encoding process into the decoder, and input the encoded features with the same spatial resolution as the corresponding layer of the decoder through skip connections to the corresponding layer of the decoder. The decoder performs upsampling, feature fusion and feature reconstruction to obtain the reconstructed features. S7. Map the reconstructed features to enhanced data, and convolve the data to obtain a YCbCr enhanced image; S8. Convert the YCbCr enhanced image to the RGB color space to obtain an enhanced RGB image.

[0009] Preferably, in step S2, the shallow convolution mapping includes: The data is processed by convolutional layer 3×3 to map the data to a high-dimensional feature space to obtain the initial features.

[0010] Preferably, in step S3, the wavelet Transformer module processes the initial features as follows: The features input to the wavelet Transformer module are subjected to layer normalization; The query features, key features, and value features are generated through convolution and linear projection. Channel dimensionality reduction is performed on the query features, key features, and value features; Perform discrete wavelet transform on the query features, key features, and value features after channel dimensionality reduction to obtain the low-frequency and high-frequency components corresponding to each query feature, key feature, and value feature; Calculate the wavelet domain self-attention result based on the low-frequency component and the high-frequency component; Perform an inverse wavelet transform on the wavelet domain self-attention result to obtain the features after restoring spatial resolution; The features after restoring spatial resolution are input into a gated feedforward network, and the first encoded features are output through residual connections.

[0011] Preferably, the discrete wavelet transform includes: Perform a discrete wavelet transform on the query features to obtain the query low-frequency component and the query high-frequency component; Perform a discrete wavelet transform on the key features to obtain the low-frequency and high-frequency components of the key; Perform a discrete wavelet transform on the value features to obtain the low-frequency component and the high-frequency component of the value; The wavelet domain self-attention calculation is based on the query low-frequency component, query high-frequency component, key low-frequency component, key high-frequency component, value low-frequency component, and value high-frequency component.

[0012] Preferably, in step S4, the multi-head transposed self-attention Transformer module performs multi-head self-attention calculation on the encoded features input to the multi-head transposed self-attention Transformer module in the channel dimension, and outputs the second encoded feature.

[0013] Preferably, in step S5, the multi-scale channel attention module includes multi-level channel attention blocks, which are cascaded together. Each level of channel attention block performs global average pooling on the input encoded features to obtain the channel description vector; The channel description vectors are convolved and activated to obtain the channel weights. The input encoded features are weighted channel by channel according to the channel weights to obtain the weighted encoded features of the corresponding resolution level.

[0014] Preferably, the wavelet domain Transformer module, the multi-head transposed self-attention Transformer module, and the multi-scale channel attention module are disposed in the encoder, and the encoder adopts a step-by-step downsampling structure; In the stepwise downsampling structure, after each downsampling, the feature resolution is reduced to half of what it was before downsampling, and the number of channels doubles relative to what it was before downsampling.

[0015] Preferably, in step S6, the decoder adopts a step-by-step upsampling structure corresponding to the encoder; The decoder upsamples the encoded features output from the final layer of the encoding process level by level. During each upsampling process, the current upsampled feature is fused with the encoded features of the same spatial resolution in the encoding process through skip connections; The features after skip connection fusion are processed for feature reconstruction to obtain the decoded features of the corresponding level; The reconstructed features are obtained based on the last level of decoding features.

[0016] Preferably, when training the network for performing the spatial and wavelet domain dual-domain Transformer low-light image enhancement method, the network is trained using a total loss function, which is the sum of Charbonnier loss and SSIM loss.

[0017] Secondly, embodiments of the present invention provide a dual-domain Transformer low-light image enhancement system in both the spatial and wavelet domains, comprising: Image acquisition module, used to acquire low-light RGB images; The preprocessing module is used to convert the low-light RGB image to the YCbCr color space; The shallow convolutional mapping module is used to perform shallow convolutional mapping on the data to obtain initial features; The encoding module includes a wavelet Transformer module, a multi-head transposed self-attention Transformer module, and a multi-scale channel attention module; The wavelet Transformer module is used to generate query features, key features, and value features based on the initial features, and to perform discrete wavelet transform on the query features, key features, and value features respectively to obtain the low-frequency components and high-frequency components corresponding to the query features, key features, and value features. Based on the low-frequency components and the high-frequency components, wavelet domain self-attention calculation is performed to obtain the first coding feature. The multi-head transposed self-attention Transformer module is used to perform self-attention calculation on the channel dimension of the encoded features input to the multi-head transposed self-attention Transformer module at a deep layer of the encoder to obtain the second encoded features; The multi-scale channel attention module is used to generate channel weights for the encoded features output at at least one resolution level during the encoding process, and to perform weighted processing on the encoded features of the corresponding resolution level according to the channel weights to obtain weighted encoded features. The decoding module is used to receive the encoded features output from the last layer of the encoding process, and to receive the encoded features input through skip connections that have the same spatial resolution as the corresponding layer of the decoder during the encoding process. The received features are upsampled, fused, and reconstructed to obtain the reconstructed features. The fusion output module is used to map the reconstructed features into enhanced data, pass the data through convolution to obtain a YCbCr enhanced image, and convert the YCbCr enhanced image to the RGB color space to obtain an enhanced RGB image. The spatial and wavelet domain dual-domain Transformer low-light image enhancement system is used to execute the spatial and wavelet domain dual-domain Transformer low-light image enhancement method.

[0018] Thirdly, a computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the computer program, implements the steps of the above-described spatial and wavelet domain dual-domain Transformer low-light image enhancement method.

[0019] Fourthly, embodiments of the present invention provide a computer-readable storage medium including a computer program, which, when executed by a processor, implements the steps of the above-described spatial and wavelet domain dual-domain Transformer low-light image enhancement method.

[0020] Fifthly, a chip includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the computer program, implements the steps of the above-described spatial and wavelet domain dual-domain Transformer low-light image enhancement method.

[0021] In a sixth aspect, embodiments of the present invention provide an electronic device, including a computer program, which, when executed by the electronic device, implements the steps of the above-described spatial and wavelet domain dual-domain Transformer low-light image enhancement method.

[0022] Compared with the prior art, the present invention has at least the following beneficial effects: A dual-domain Transformer low-light image enhancement method, encompassing both spatial and wavelet domains, first transforms the low-light image to the YCbCr color space. Then, it performs feature encoding, dual-domain Transformer processing, channel attention weighting, decoding and reconstruction, and luminance / chrominance fusion. This approach transforms the low-light image enhancement task into the YCbCr space, avoiding color shifts caused by direct overall enhancement in the RGB space. A wavelet Transformer module performs discrete wavelet transforms on query features, key features, and value features, generating low-frequency and high-frequency components in the wavelet domain, thus distinguishing global illumination information from detail / noise information. A multi-head transpose self-attention Transformer module and a multi-scale channel attention module process deep-encoded features and resolution-level features separately. Finally, a decoder and skip connections reconstruct the features, ultimately obtaining the enhanced RGB image. This process forms a complete closed loop of input, encoding, fusion, reconstruction, and output.

[0023] Furthermore, before entering the wavelet Transformer module, single-channel or low-dimensional data is first converted into a feature representation suitable for subsequent encoding processing. A 3×3 convolutional layer can extract local gray-level changes, edges, textures, and basic structural information within a small receptive region, and completes channel dimension expansion through convolutional mapping, providing a unified feature input for subsequent wavelet domain self-attention computation. Through shallow convolutional mapping, data can be transformed from pixel-level data into high-dimensional initial features, providing more stable basic features for subsequent wavelet domain Transformer encoding, deep channel self-attention processing, and decoding reconstruction without significantly increasing structural complexity.

[0024] Furthermore, the self-attention computation of the Transformer is no longer directly limited to the original spatial domain features, but is first transformed to the wavelet domain through discrete wavelet transform before processing. Layer normalization is used to stabilize the input feature distribution; query features, key features, and value features are used to build the foundation for self-attention computation; channel dimensionality reduction is used to control the scale of subsequent computations; discrete wavelet transform is used to obtain low-frequency and high-frequency components; wavelet domain self-attention computation is used to model relationships on the frequency-decomposed features; inverse wavelet transform is used to restore spatial resolution; and gated feedforward networks and residual connections are used to complete feature output. While maintaining the Transformer's long-range modeling capabilities, a wavelet domain decomposition mechanism is introduced, enabling the feature encoding process to take into account both spatial structural relationships and frequency component representation.

[0025] Furthermore, it is clarified that the discrete wavelet transform does not only act on the input image or ordinary intermediate features, but directly on the Q, K, and V features required for self-attention computation, thus establishing the calculation of attention relationships based on features after frequency decoupling. After decomposing the query features, key features, and value features respectively, the low-frequency components can correspond to overall illumination and large-scale structural information, while the high-frequency components can correspond to edge, texture, and noise-related information. Using these components as the basis for wavelet domain self-attention computation helps avoid directly mixing low-frequency illumination changes and high-frequency details / noise in the spatial domain. This enables the wavelet domain Transformer module to introduce a frequency component differentiation mechanism during the attention computation stage, thereby providing more targeted encoded features for subsequent feature enhancement.

[0026] Furthermore, a feature modeling approach different from the shallow wavelet Transformer module is introduced at a deeper level in the encoder. This allows for further integration of deep encoded features through self-attention computation along the channel dimension. By performing multi-head self-attention computation along the channel dimension, the brightness, texture, structure, and contextual information expressed by different channels can interact, resulting in second encoded features. This enables a more comprehensive expression of channel relationships at the deep feature stage. When used in conjunction with the aforementioned wavelet Transformer module, the encoder simultaneously possesses wavelet domain frequency decomposition capabilities and deep channel dependency modeling capabilities, providing richer encoded inputs for subsequent feature reconstruction by the decoder.

[0027] Furthermore, low-light images contain structural, textural, and brightness information at different resolution levels, and different channels contribute differently to the final enhancement result. Global average pooling provides channel-level statistical information, convolution and activation functions generate channel weights, and these weights are then used to weight the encoded features channel by channel, allowing the network to selectively enhance effective feature channels at different resolution levels. Cascading multi-level channel attention blocks allows this weighting process to be performed step-by-step, resulting in more refined channel feature adjustments. This helps compensate for the shortcomings of the Transformer structure in representing local texture and edge details, enabling the subsequent decoder to obtain more discriminative weighted encoded features.

[0028] Furthermore, a hierarchical coding structure is constructed, enabling the network to extract and organize image features at different spatial resolutions. Progressive downsampling gradually reduces the spatial size of the feature map, facilitating the formation of a wider range of contextual representations at deeper levels; doubling the number of channels compensates for the change in feature representation capacity due to the reduction in spatial resolution. The wavelet Transformer module, the multi-head transposed self-attention Transformer module, and the multi-scale channel attention module are incorporated into the encoder, ensuring that wavelet domain decomposition, deep channel self-attention, and multi-scale channel weighting all occur during the encoding stage. The encoded features output by the encoder contain both hierarchical representations of the data and information processed by the wavelet domain and channel attention, providing structured input for the decoder to reconstruct features step by step.

[0029] Furthermore, the decoding stage can utilize features of the same scale retained in the encoding stage while restoring spatial resolution. Through progressive upsampling, the decoder gradually restores feature resolution; by fusing features of the same scale through skip connections, spatial structural information from the encoding stage can be introduced into the corresponding decoding level; and through feature reconstruction processing, a more complete feature representation is formed step by step. This ensures that the reconstructed features not only originate from deep encoding information but also combine encoding features from different resolution levels, which is beneficial for forming an enhanced representation corresponding to the spatial structure of the input image.

[0030] Furthermore, considering both pixel-level differences and structural similarity from a training constraint perspective, Charbonnier loss is typically used to measure the pixel reconstruction error between the enhanced and target images. Its form is smoother for anomalous differences compared to ordinary squared error, helping to constrain the overall pixel value recovery of the image. SSIM loss is used to evaluate image structural similarity, constraining the enhancement results in terms of brightness, contrast, and structural relationships. Combining Charbonnier and SSIM losses allows the network training process to focus not only on pixel-level reconstruction but also on image structure preservation. After network training is complete, the trained network performs low-light image enhancement processing, ensuring that the training objective corresponds to the pixel recovery and structure preservation requirements of the low-light image enhancement task.

[0031] It is understood that the beneficial effects of the second to sixth aspects mentioned above can be found in the relevant descriptions in the first aspect mentioned above, and will not be repeated here.

[0032] In summary, this invention achieves brightness enhancement, noise control, and detail reconstruction in low-light images through YCbCr spatial transformation, wavelet domain Transformer frequency decoupling, multi-head transpose self-attention modeling, multi-scale channel attention weighting, and decoder skip fusion. Its self-attention computational complexity can be reduced to 1 / 4 of the original, balancing enhanced quality and processing efficiency.

[0033] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0034] Figure 1 This is an overall flowchart of the method of the present invention; Figure 2 This is a structural diagram of the wavelet Transformer module of the present invention; Figure 3 This is a structural diagram of the multi-scale channel attention module of the present invention; Figure 4 A schematic diagram of a computer device provided in an embodiment of the present invention; Figure 5 This is a block diagram of a chip provided according to an embodiment of the present invention.

[0035] Among them, 60. Computer equipment; 61. Processor; 62. Memory; 63. Computer program; 600. Electronic device; 610. Processing unit; 620. Storage unit; 6201. Random access memory unit; 6202. Cache memory unit; 6203. Read-only memory unit; 6204. Program / utility; 6205. Program module; 630. Bus; 640. Display unit; 650. Input / output interface; 660. Network adapter; 700. External device. Detailed Implementation

[0036] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0037] In the description of this invention, it should be understood that the terms "comprising" and "including" indicate the presence of the described features, integrals, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or collections thereof.

[0038] It should also be understood that the terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the invention. As used in this specification and the appended claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise.

[0039] It should also be further understood that the term "and / or" as used in this specification and the appended claims refers to any combination and all possible combinations of one or more of the associated listed items, and includes such combinations. For example, A and / or B can represent three cases: A alone, A and B simultaneously, and B alone. Additionally, the character " / " in this invention generally indicates that the preceding and following objects have an "or" relationship.

[0040] It should be understood that although terms such as first, second, third, etc., may be used in the embodiments of the present invention to describe the preset range, these preset ranges should not be limited to these terms. These terms are only used to distinguish the preset ranges from one another. For example, without departing from the scope of the embodiments of the present invention, the first preset range may also be referred to as the second preset range, and similarly, the second preset range may also be referred to as the first preset range.

[0041] Depending on the context, the word "if" as used here can be interpreted as "when," "when," "in response to determination," or "in response to detection." Similarly, depending on the context, the phrase "if determination" or "if detection (of the stated condition or event)" can be interpreted as "when determination," "in response to determination," "when detection (of the stated condition or event)," or "in response to detection (of the stated condition or event)."

[0042] The accompanying drawings illustrate various structural schematic diagrams according to embodiments disclosed in this invention. These drawings are not to scale, and some details have been enlarged for clarity, and some details may have been omitted. The shapes of the various regions and layers shown in the drawings, as well as their relative sizes and positional relationships, are merely exemplary and may deviate from reality due to manufacturing tolerances or technical limitations. Furthermore, those skilled in the art can design regions / layers with different shapes, sizes, and relative positions as needed.

[0043] This invention provides a dual-domain Transformer low-light image enhancement method using both spatial and wavelet domains. For input low-light degraded images, it employs dual-domain feature collaborative modeling to output a high-quality enhanced image with uniform brightness, suppressed noise, and preserved details. Compared to traditional histogram equalization, gamma correction, and existing Transformer-based low-light enhancement methods, this method significantly reduces computational overhead while achieving synergistic optimization of noise suppression, brightness enhancement, and detail preservation. It can provide an efficient technical solution for low-light image preprocessing in scenarios such as autonomous driving, security monitoring, and mobile photography.

[0044] Please see Figure 1 This invention discloses a dual-domain (spatial and wavelet domain) Transformer low-light image enhancement method, comprising the following steps: S1. Obtain a low-light RGB image and convert the low-light RGB image to the YCbCr color space; Obtain the low-light RGB image to be enhanced, and perform color space conversion on the low-light RGB image, converting it from the RGB color space to the YCbCr color space.

[0045] S2. Perform shallow convolution mapping on the data to obtain initial features; Shallow feature extraction is performed on the data to obtain initial features.

[0046] In one implementation, a 3×3 convolutional layer is used to convolve the data, mapping the data to a high-dimensional feature space to obtain the initial features.

[0047] S3. Input the initial features into the wavelet Transformer module to perform wavelet domain self-attention calculation to obtain the first encoded features; Please see Figure 2The initial features are input into the wavelet Transformer module. The wavelet Transformer module generates query features Q, key features K, and value features V based on the initial features, and performs Discrete Wavelet Transform (DWT) on each of the query features Q, key features K, and value features V to obtain the low-frequency and high-frequency components corresponding to each feature. This achieves lossless downsampling and frequency domain decoupling, decomposing the features into low-frequency illumination components and high-frequency detail components. Relying on adaptive attention weight allocation, it completes global brightness reshaping of the low-frequency components and noise suppression of the high-frequency components, while reducing the computational overhead of self-attention. The original features are then recovered through inverse wavelet transform.

[0048] Furthermore, the wavelet Transformer module performs wavelet domain self-attention calculation based on the low-frequency component and the high-frequency component to obtain the first coding feature.

[0049] In one implementation, the wavelet Transformer module includes the following processing steps: The features input to the wavelet Transformer module are subjected to layer normalization; Query features, key features, and value features are generated through convolution and linear projection. Channel dimensionality reduction is performed on the query features, key features, and value features; Perform discrete wavelet transform on the query features, key features, and value features after dimensionality reduction of the channel to obtain low-frequency and high-frequency components; Calculate the wavelet domain self-attention result based on the low-frequency component and the high-frequency component; Perform an inverse wavelet transform on the wavelet domain self-attention result to obtain the features after restoring spatial resolution; The features after restoring spatial resolution are input into a gated feedforward network, and the first encoded features are output through residual connections.

[0050] S4. In the deep layer of the encoder, self-attention calculation is performed on the channel dimension through the multi-head transpose self-attention Transformer module to obtain the second coding feature; Deep within the encoder, the encoded features input to the multi-head transposed self-attention Transformer module are fed into the multi-head transposed self-attention Transformer module.

[0051] The encoder adopts a step-by-step downsampling structure with a downsampling factor of powers of 2, which simultaneously doubles the number of channels and reduces the resolution to half; the decoder adopts bilinear interpolation upsampling.

[0052] The multi-head transpose self-attention Transformer module performs self-attention calculation on the encoded features input to the module in the channel dimension to obtain the second encoded feature.

[0053] S5. Input the encoded features output from at least one resolution level during the encoding process into the multi-scale channel attention module (MCAB) to obtain weighted encoded features; Please see Figure 3 During the encoding process, the encoded features output from at least one resolution level are input into a multi-scale channel attention module. The multi-scale channel attention module generates channel weights based on the input encoded features, and performs weighted processing on the encoded features of the corresponding resolution level according to the channel weights to obtain weighted encoded features.

[0054] A cascaded multi-scale channel attention module is connected after each resolution level of the encoder to adaptively allocate channel weights to the spatial domain feature map, thereby enhancing the perception and extraction of local texture and edge details and making up for the shortcomings of Transformer global modeling in representing local details.

[0055] In one embodiment, the multi-scale channel attention module includes multi-level channel attention blocks, which are cascaded. Each level of channel attention block performs global average pooling on the input encoded features to obtain a channel description vector. The channel description vector is then subjected to convolution and activation function processing to obtain channel weights. The input encoded features are then weighted channel by channel according to the channel weights to obtain weighted encoded features at the corresponding resolution level.

[0056] S6. Input the encoded features output from the last layer of the encoding process into the decoder, and fuse the encoded features of the same resolution in the encoding process through skip connections to obtain the reconstructed features; The encoded features output from the final layer of the encoding process are input into the decoder. The decoder upsamples the encoded features output from the final layer of the encoding process.

[0057] A decoder structure symmetrical to the encoder is adopted, and a dual-domain Transformer is used to complete feature decoding; The decoder upsamples and restores the encoded features layer by layer. The top layer uses a wavelet Transformer module to accurately restore high-frequency spatial details, while the bottom layer uses a multi-head transposed self-attention Transformer to reconstruct global illumination semantics. Features from the same-scale encoder are fused through skip connections to alleviate the loss of downsampled spatial information.

[0058] In at least one level of the decoder, the currently upsampled features are fused with encoded features with the same spatial resolution during the encoding process through skip connections, and the fused features are then subjected to feature reconstruction processing to obtain the decoded features of the corresponding level.

[0059] After step-by-step upsampling, feature fusion and feature reconstruction, the reconstructed features are obtained based on the last level of decoded features.

[0060] The wavelet Transformer module includes: The input features are normalized layer by layer; Q, K, and V features are generated through convolution; channel dimensionality reduction and discrete wavelet transform are performed on Q, K, and V; self-attention is calculated in the frequency domain; resolution is restored through inverse wavelet transform, and the enhanced features are output through a gated feedforward network and residual connection.

[0061] S7. Map the reconstructed features to enhanced data, and convolve the data to obtain a YCbCr enhanced image; The reconstructed features are mapped to augmented data, and the data is convolved to obtain a YCbCr augmented image.

[0062] S8. Convert the YCbCr enhanced image to the RGB color space to obtain an enhanced RGB image. The YCbCr enhanced image obtained in step S7 is subjected to inverse color space conversion, which converts it from the YCbCr color space to the RGB color space to obtain an enhanced RGB image. Pixel-level fidelity and structural integrity preservation are achieved through joint loss constraints, and a low-light enhanced image with uniform brightness, natural color, clear details, and low noise is output.

[0063] In one implementation, when training the network for performing the above-described spatial and wavelet domain dual-domain Transformer low-light image enhancement method, the network is trained using a total loss function, wherein the total loss function... The sum of Charbonnier loss and SSIM loss is expressed as:

[0064] in, Indicates Charbonnier loss. This indicates SSIM loss.

[0065] The total loss function is used to constrain the training process of the network so that the trained network can process the low-light RGB image and output the enhanced RGB image according to steps S1 to S8.

[0066] In another embodiment of the present invention, a spatial and wavelet domain dual-domain Transformer low-light image enhancement system is provided. This system can be used to implement the above-mentioned spatial and wavelet domain dual-domain Transformer low-light image enhancement method. Specifically, the spatial and wavelet domain dual-domain Transformer low-light image enhancement system includes an image acquisition module, a preprocessing module, a shallow convolution mapping module, an encoding module, a decoding module, and a fusion output module.

[0067] The image acquisition module is used to acquire low-light RGB images. The preprocessing module is used to convert the low-light RGB image to the YCbCr color space; The shallow convolutional mapping module is used to perform shallow convolutional mapping on the data to obtain initial features; The encoding module includes a wavelet Transformer module, a multi-head transposed self-attention Transformer module, and a multi-scale channel attention module; The wavelet Transformer module is used to generate query features, key features, and value features based on the initial features, and to perform discrete wavelet transform on the query features, key features, and value features respectively to obtain the low-frequency components and high-frequency components corresponding to the query features, key features, and value features. Based on the low-frequency components and the high-frequency components, wavelet domain self-attention calculation is performed to obtain the first coding feature. The multi-head transposed self-attention Transformer module is used to perform self-attention calculation on the channel dimension of the encoded features input to the multi-head transposed self-attention Transformer module at a deep layer of the encoder to obtain the second encoded features; The multi-scale channel attention module is used to generate channel weights for the encoded features output at at least one resolution level during the encoding process, and to perform weighted processing on the encoded features of the corresponding resolution level according to the channel weights to obtain weighted encoded features. The decoding module is used to receive the encoded features output from the last layer of the encoding process, and to receive the encoded features input through skip connections that have the same spatial resolution as the corresponding layer of the decoder during the encoding process. The received features are upsampled, fused, and reconstructed to obtain the reconstructed features. The fusion output module is used to map the reconstructed features into enhanced data, perform convolution on the data to obtain a YCbCr enhanced image, and convert the YCbCr enhanced image to the RGB color space to obtain an enhanced RGB image.

[0068] This invention provides a terminal device comprising a processor and a memory. The memory stores a computer program, which includes program instructions. The processor executes the program instructions stored in the computer storage medium. The processor may be a Central Processing Unit (CPU), or other general-purpose processors, graphics processing units (GPUs), tensor processing units (TPUs), digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. It is the computing and control core of the terminal, suitable for implementing one or more instructions, specifically suitable for loading and executing one or more instructions to achieve a corresponding method flow or function. The processor described in this embodiment can be used for the operation of a dual-domain Transformer low-light image enhancement method in both the spatial and wavelet domains, including: A low-light RGB image is acquired and converted to the YCbCr color space. Shallow convolution mapping is performed on the data to obtain initial features. These initial features are then input into a wavelet Transformer module, which generates query features, key features, and value features based on the initial features. Discrete wavelet transforms are then performed on each of the query features, key features, and value features to obtain their respective low-frequency and high-frequency components. Wavelet domain self-attention calculation is performed based on these low-frequency and high-frequency components to obtain the first encoded features. In the deeper encoder layer, a multi-head transposed self-attention Transformer module is used to perform channel dimension calculations on the encoded features input to the multi-head transposed self-attention Transformer module. The self-attention calculation on the layer yields the second encoded feature. Encoded features output from at least one resolution level during the encoding process are input into a multi-scale channel attention module. Channel weights are generated through this module, and the encoded features at the corresponding resolution level are weighted according to these weights to obtain weighted encoded features. The encoded features output from the final layer of the encoding process are input into the decoder. Encoded features with the same spatial resolution as the corresponding layer in the decoder are input to the corresponding layer of the decoder via skip connections. The decoder performs upsampling, feature fusion, and feature reconstruction to obtain reconstructed features. These reconstructed features are mapped to enhanced data, and the data is convolved to obtain a YCbCr enhanced image. The YCbCr enhanced image is converted to the RGB color space to obtain an enhanced RGB image.

[0069] Please see Figure 4 The terminal device is a computer device. In this embodiment, the computer device 60 includes a processor 61, a memory 62, and a computer program 63 stored in the memory 62 and executable on the processor 61. When executed by the processor 61, the computer program 63 implements the spatial and wavelet domain dual-domain Transformer low-light image enhancement method of this embodiment. To avoid repetition, these details are not elaborated here. Alternatively, when executed by the processor 61, the computer program 63 implements the functions of each model / unit in the spatial and wavelet domain dual-domain Transformer low-light image enhancement system of this embodiment. To avoid repetition, these details are not elaborated here.

[0070] Computer device 60 can be a desktop computer, laptop, handheld computer, cloud server, or other computing device. Computer device 60 may include, but is not limited to, a processor 61 and a memory 62. Those skilled in the art will understand that... Figure 4This is merely an example of computer device 60 and does not constitute a limitation on computer device 60. It may include more or fewer components than shown, or combine certain components, or different components. For example, computer device may also include input / output devices, network access devices, buses, etc.

[0071] The processor 61 may be a Central Processing Unit (CPU), or other general-purpose processors, graphics processing units (GPUs), tensor processing units (TPUs), digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor.

[0072] The memory 62 can be an internal storage unit of the computer device 60, such as a hard disk or memory of the computer device 60. The memory 62 can also be an external storage device of the computer device 60, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc. equipped on the computer device 60.

[0073] Furthermore, the memory 62 may include both internal storage units of the computer device 60 and external storage devices. The memory 62 is used to store computer programs and other programs and data required by the computer device. The memory 62 can also be used to temporarily store data that has been output or will be output.

[0074] Please see Figure 5 The terminal device is an electronic device 600, which is manifested in the form of a general-purpose computing device. The components of the electronic device may include, but are not limited to: at least one processing unit 610, at least one storage unit 620, a bus 630 connecting different platform components (including storage unit 620 and processing unit 610), a display unit 640, etc.

[0075] The storage unit stores program code, which can be executed by the processing unit 610 to perform the steps described in the method section of this specification according to various exemplary embodiments of the present invention. For example, the processing unit 610 can perform actions such as... Figure 1 The steps are shown in the figure.

[0076] Storage unit 620 may include readable media in the form of volatile storage units, such as random access memory (RAM) 6201 and / or cache memory 6202, and may further include read-only memory (ROM) 6203.

[0077] Storage unit 620 may also include a program / utility 6204 having a set (at least one) program module 6205, such program module 6205 including but not limited to: operating system, one or more application programs, other program modules and program data, each or some combination of these examples may include an implementation of a network environment.

[0078] Bus 630 can represent one or more of several types of bus structures, including a memory cell bus or memory cell controller, a peripheral bus, a graphics acceleration port, a processing unit, or a local bus using any of the multiple bus structures.

[0079] Electronic device 600 can also communicate with one or more external devices 700 (e.g., keyboard, pointing device, Bluetooth device, etc.), and with one or more devices that enable a user to interact with electronic device 600, and / or with any device that enables electronic device 600 to communicate with one or more other computing devices (e.g., router, modem). This communication can be performed via input / output interface 650. Furthermore, electronic device 600 can also communicate with one or more networks (e.g., local area network, wide area network, and / or public network, such as the Internet) via network adapter 660. Network adapter 660 can communicate with other modules of electronic device 600 via bus 630. It should be understood that, although not shown in the figures, other hardware and / or software modules can be used in conjunction with electronic device 600, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage platforms.

[0080] This invention also provides a storage medium, specifically a computer-readable storage medium, which is a memory device in a terminal device for storing programs and data. It is understood that the computer-readable storage medium here can include both built-in storage media in the terminal device and extended storage media supported by the terminal device; it can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. The computer-readable storage medium provides storage space that stores the terminal's operating system. Furthermore, the storage space also stores one or more instructions suitable for loading and execution by a processor, which can be one or more computer programs (including program code). More specific examples of the computer-readable storage medium include: an electrical connection with one or more wires, a portable disk, a hard disk, random access memory, read-only memory, erasable programmable read-only memory, optical fiber, portable compact disk read-only memory, optical storage device, magnetic storage device, or any suitable combination thereof.

[0081] Computer-readable storage media also include data signals propagated in baseband or as part of a carrier wave, carrying readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A readable storage medium can also be any readable medium other than a readable storage medium that can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the readable storage medium can be transmitted using any suitable medium, including but not limited to wireless, wired, optical fiber, radio frequency, etc., or any suitable combination thereof.

[0082] Program code for performing the operations of this invention can be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java and C++, and conventional procedural programming languages ​​such as C or similar languages. The program code can execute entirely on the user's computing device, partially on the user's device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0083] One or more instructions stored in a computer-readable storage medium can be loaded and executed by a processor to implement the corresponding steps of the spatial and wavelet domain dual-domain Transformer low-light image enhancement method in the above embodiments; one or more instructions in the computer-readable storage medium are loaded and executed by the processor to perform the following steps: A low-light RGB image is acquired and converted to the YCbCr color space. Shallow convolution mapping is performed on the data to obtain initial features. These initial features are then input into a wavelet Transformer module, which generates query features, key features, and value features based on the initial features. Discrete wavelet transforms are then performed on each of the query features, key features, and value features to obtain their respective low-frequency and high-frequency components. Wavelet domain self-attention calculation is performed based on these low-frequency and high-frequency components to obtain the first encoded features. In the deeper encoder layer, a multi-head transposed self-attention Transformer module is used to perform channel dimension calculations on the encoded features input to the multi-head transposed self-attention Transformer module. The self-attention calculation on the layer yields the second encoded feature. Encoded features output from at least one resolution level during the encoding process are input into a multi-scale channel attention module. Channel weights are generated through this module, and the encoded features at the corresponding resolution level are weighted according to these weights to obtain weighted encoded features. The encoded features output from the final layer of the encoding process are input into the decoder. Encoded features with the same spatial resolution as the corresponding layer in the decoder are input to the corresponding layer of the decoder via skip connections. The decoder performs upsampling, feature fusion, and feature reconstruction to obtain reconstructed features. These reconstructed features are mapped to enhanced data, and the data is convolved to obtain a YCbCr enhanced image. The YCbCr enhanced image is converted to the RGB color space to obtain an enhanced RGB image.

[0084] The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0085] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.

[0086] To further illustrate the technical effects of the present invention, the computational complexity and image enhancement effect of the present invention are analyzed below in conjunction with the wavelet domain Transformer module, the deep multi-head transpose self-attention Transformer module, and the multi-scale channel attention module.

[0087] In existing spatial domain Transformer low-light image enhancement methods, self-attention computation typically operates directly on spatial domain features. As the resolution of input features increases, the computational complexity of self-attention increases significantly, leading to high computational overhead for the model during training and inference phases.

[0088] This invention performs discrete wavelet transform on the query features, key features, and value features in the self-attention computation within the wavelet domain Transformer module, decomposing the features into low-frequency and high-frequency components, and then performs self-attention computation in the wavelet domain. According to the structural configuration of this invention, embedding self-attention computation into the discrete wavelet transform enables lossless downsampling, reducing the feature resolution to half of its original value and the computational complexity of self-attention computation to one-quarter of its original value.

[0089] Therefore, compared with the Transformer structure that performs self-attention calculation directly in the spatial domain, the present invention reduces the computational load of the self-attention module by performing self-attention calculation in the wavelet domain, which is beneficial to reducing the computational overhead in the low-light image enhancement process.

[0090] Meanwhile, this invention decomposes features into low-frequency and high-frequency components in the wavelet domain Transformer module. The low-frequency components correspond to global illumination information, while the high-frequency components correspond to detail and noise information. By performing wavelet domain self-attention calculation on the low-frequency and high-frequency components, global illumination information and high-frequency detail / noise information can be distinguished during feature encoding, thereby avoiding the simultaneous amplification of noise and effective texture during spatial domain enhancement.

[0091] Furthermore, this invention incorporates a multi-head transposed self-attention Transformer module deep within the encoder, processing encoded features through self-attention calculations along the channel dimension. Additionally, during encoding, a multi-scale channel attention module is set at at least one resolution level to weight the encoded features at the corresponding resolution level based on channel weights. Through this structure, this invention can simultaneously obtain wavelet domain features, deep encoded features, and multi-scale channel weighted features during the encoding stage, and enhance the RGB image output through upsampling, skip connection fusion, and feature reconstruction in the decoder.

[0092] Based on the above structure, the present invention has at least the following advantages over low-light image enhancement methods that directly employ spatial domain self-attention: By implementing feature decomposition before self-attention computation through discrete wavelet transform, the computational complexity of self-attention is reduced to 1 / 4 of the original. By decomposing low-frequency and high-frequency components, global illumination information and high-frequency detail / noise information can be distinguished and processed in the wavelet domain. The channel-dimensional self-attention calculation is performed on deep encoded features using the multi-head transpose self-attention Transformer module; Channel weighting is applied to encoded features at different resolution levels using a multi-scale channel attention module; An enhanced RGB image is obtained through stepwise upsampling, skip connection fusion, and feature reconstruction by the decoder.

[0093] In summary, this invention presents a dual-domain (spatial and wavelet domain) Transformer method and system for low-light image enhancement. It converts low-light RGB images to the YCbCr color space, performing feature encoding and reconstruction. The wavelet domain Transformer module performs discrete wavelet transforms on query features, key features, and value features, and performs self-attention calculations in the wavelet domain, reducing feature resolution to half and self-attention computation complexity to one-quarter, thus lowering computational overhead. By decomposing low-frequency and high-frequency components, it can distinguish between global illumination information and detail / noise information, avoiding the simultaneous amplification of noise and effective texture caused by direct spatial domain enhancement. A deep multi-head transpose self-attention Transformer module is used for channel-dimensional self-attention calculations, while a multi-scale channel attention module is used for resolution-level feature weighting. Combined with decoder upsampling, skip connection fusion, and feature reconstruction, this achieves a good balance between brightness, noise, detail, and color representation in the enhanced image, making it suitable for low-light image enhancement scenarios.

[0094] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0095] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0096] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed in this invention can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0097] In the embodiments provided by this invention, it should be understood that the disclosed devices / terminals and methods can be implemented in other ways. For example, the device / terminal embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0098] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0099] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0100] If the integrated module / unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments of the present invention can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a portable hard drive, a magnetic disk, an optical disk, a computer memory, a read-only memory (ROM), a random-access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium, etc.

[0101] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus, and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0102] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0103] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0104] The above content is only for illustrating the technical concept of the present invention and should not be construed as limiting the scope of protection of the present invention. Any modifications made to the technical solution based on the technical concept proposed in this invention shall fall within the scope of protection of the claims of this invention.

Claims

1. A dual-domain (spatial and wavelet domain) Transformer low-light image enhancement method, characterized in that, Includes the following steps: S1. Obtain a low-light RGB image and convert the low-light RGB image to the YCbCr color space; S2. Perform shallow convolution mapping on the data to obtain initial features; S3. Input the initial features into the wavelet Transformer module. The wavelet Transformer module generates query features, key features and value features based on the initial features, and performs discrete wavelet transform on the query features, key features and value features respectively to obtain the low-frequency components and high-frequency components corresponding to the query features, key features and value features respectively. Based on the low-frequency components and the high-frequency components, wavelet domain self-attention calculation is performed to obtain the first coding feature. S4. In the deep layer of the encoder, the encoding features input to the multi-head transposed self-attention Transformer module are subjected to self-attention calculation in the channel dimension to obtain the second encoding feature. S5. Input the encoded features output from at least one resolution level during the encoding process into the multi-scale channel attention module, generate channel weights through the multi-scale channel attention module, and perform weighted processing on the encoded features of the corresponding resolution level according to the channel weights to obtain weighted encoded features. S6. Input the encoded features output from the last layer of the encoding process into the decoder, and input the encoded features with the same spatial resolution as the corresponding layer of the decoder through skip connections to the corresponding layer of the decoder. The decoder performs upsampling, feature fusion and feature reconstruction to obtain the reconstructed features. S7. Map the reconstructed features to enhanced data, and pass the data through a convolutional layer to obtain a YCbCr enhanced image; S8. Convert the YCbCr enhanced image to the RGB color space to obtain an enhanced RGB image.

2. The low-light image enhancement method using a dual-domain Transformer approach in both spatial and wavelet domains according to claim 1, characterized in that... In step S2, the shallow convolution mapping includes: The data is processed by convolutional layer 3×3 to map the data to a high-dimensional feature space to obtain the initial features.

3. The low-light image enhancement method using a dual-domain Transformer approach in both spatial and wavelet domains according to claim 1, characterized in that... In step S3, the wavelet Transformer module processes the initial features as follows: The features input to the wavelet Transformer module are subjected to layer normalization; The query features, key features, and value features are generated through convolution and linear projection. Channel dimensionality reduction is performed on the query features, key features, and value features; Perform discrete wavelet transform on the query features, key features, and value features after channel dimensionality reduction to obtain the low-frequency and high-frequency components corresponding to each query feature, key feature, and value feature; Calculate the wavelet domain self-attention result based on the low-frequency component and the high-frequency component; Perform an inverse wavelet transform on the wavelet domain self-attention result to obtain the features after restoring spatial resolution; The features after restoring spatial resolution are input into a gated feedforward network, and the first encoded features are output through residual connections.

4. The low-light image enhancement method using a dual-domain Transformer approach in both spatial and wavelet domains according to claim 3, characterized in that... The discrete wavelet transform includes: Perform a discrete wavelet transform on the query features to obtain the query low-frequency component and the query high-frequency component; Perform a discrete wavelet transform on the key features to obtain the low-frequency and high-frequency components of the key; Perform a discrete wavelet transform on the value features to obtain the low-frequency component and the high-frequency component of the value; The wavelet domain self-attention calculation is based on the query low-frequency component, query high-frequency component, key low-frequency component, key high-frequency component, value low-frequency component, and value high-frequency component.

5. The low-light image enhancement method using a dual-domain Transformer approach in both spatial and wavelet domains according to claim 1, characterized in that... In step S4, the multi-head transposed self-attention Transformer module performs multi-head self-attention calculation on the encoded features input to the multi-head transposed self-attention Transformer module in the channel dimension, and outputs the second encoded feature.

6. The low-light image enhancement method using a dual-domain Transformer approach in both spatial and wavelet domains according to claim 1, characterized in that... In step S5, the multi-scale channel attention module includes multi-level channel attention blocks, which are cascaded together. Each level of channel attention block performs global average pooling on the input encoded features to obtain the channel description vector; The channel description vectors are convolved and activated to obtain the channel weights. The input encoded features are weighted channel by channel according to the channel weights to obtain the weighted encoded features of the corresponding resolution level.

7. The low-light image enhancement method using a dual-domain Transformer approach in both spatial and wavelet domains according to claim 1, characterized in that... The wavelet domain Transformer module, the multi-head transposed self-attention Transformer module, and the multi-scale channel attention module are disposed in the encoder, and the encoder adopts a step-by-step downsampling structure; In the stepwise downsampling structure, after each downsampling, the feature resolution is reduced to half of what it was before downsampling, and the number of channels doubles relative to what it was before downsampling.

8. The low-light image enhancement method using a dual-domain Transformer approach in both spatial and wavelet domains according to claim 1, characterized in that, In step S6, the decoder adopts a step-by-step upsampling structure corresponding to the encoder; The decoder upsamples the encoded features output from the final layer of the encoding process level by level. During each upsampling process, the current upsampled feature is fused with the encoded features of the same spatial resolution in the encoding process through skip connections; The features after skip connection fusion are processed for feature reconstruction to obtain the decoded features of the corresponding level; The reconstructed features are obtained based on the last level of decoding features.

9. The low-light image enhancement method using a dual-domain Transformer approach in both spatial and wavelet domains according to claim 1, characterized in that... When training the network for performing the spatial and wavelet domain dual-domain Transformer low-light image enhancement method, the network is trained using a total loss function, which is the sum of the Charbonnier loss and the SSIM loss.

10. A dual-domain (spatial and wavelet domain) Transformer low-light image enhancement system, characterized in that, include: Image acquisition module, used to acquire low-light RGB images; The preprocessing module is used to convert the low-light RGB image to the YCbCr color space; The shallow convolutional mapping module is used to perform shallow convolutional mapping on the data to obtain initial features; The encoding module includes a wavelet Transformer module, a multi-head transposed self-attention Transformer module, and a multi-scale channel attention module; The wavelet Transformer module is used to generate query features, key features, and value features based on the initial features, and to perform discrete wavelet transform on the query features, key features, and value features respectively to obtain the low-frequency components and high-frequency components corresponding to the query features, key features, and value features. Based on the low-frequency components and the high-frequency components, wavelet domain self-attention calculation is performed to obtain the first coding feature. The multi-head transposed self-attention Transformer module is used to perform self-attention calculation on the channel dimension of the encoded features input to the multi-head transposed self-attention Transformer module at a deep layer of the encoder to obtain the second encoded features; The multi-scale channel attention module is used to generate channel weights for the encoded features output at at least one resolution level during the encoding process, and to perform weighted processing on the encoded features of the corresponding resolution level according to the channel weights to obtain weighted encoded features. The decoding module is used to receive the encoded features output from the last layer of the encoding process, and to receive the encoded features input through skip connections that have the same spatial resolution as the corresponding layer of the decoder during the encoding process. The received features are upsampled, fused, and reconstructed to obtain the reconstructed features. The fusion output module is used to map the reconstructed features into enhanced data, perform convolution on the data to obtain a YCbCr enhanced image, and convert the YCbCr enhanced image to the RGB color space to obtain an enhanced RGB image. The spatial and wavelet domain dual-domain Transformer low-light image enhancement system is used to perform the spatial and wavelet domain dual-domain Transformer low-light image enhancement method according to any one of claims 1 to 9.

Images