A low-light image enhancement method based on transformer

Abstract

A transformer-based low-light image enhancement method includes the following steps: constructing and training a network structure comprising a local enhancement module branch and a cascaded branch of a sliding window-based self-attention mechanism module; wherein the local enhancement module comprises symmetrical downsampling and upsampling layers, each of which is composed of a downsampling layer connected by a residual transformer module, and the upsampling and downsampling layers are connected by a residual layer, forming an hourglass residual enhancement module based on the transformer model, thereby obtaining both low-dimensional information and deeper feature information during local correction; wherein the WSAB module performs gradual global enhancement through cascading downsampling and then cascading WSAB modules, ultimately obtaining a trainable gamma parameter and a global joint correction matrix; and using this network structure to enhance low-light images. The network designed by this method can improve the ability to extract deep features, improve the ability to extract global feature parameters, and improve the low-light image enhancement effect.

Description

Technical Field

[0001] The present invention relates to image enhancement, and in particular to a transformer-based low-illumination image enhancement method. Background Art

[0002] Application of low-light image enhancement methods: With the development of internet technology and the upgrading of imaging equipment, people have more and more ways to obtain images, including still cameras, remote sensing satellites, and surveillance cameras. More and more tasks and daily life scenarios require image information. Due to equipment limitations and insufficient lighting conditions, the images often have uneven lighting, poor contrast, overall darkness, and loss of detail, which hinders the extraction of image content information. Low-light image enhancement has a wide range of applications, including national defense security, night rescue, community security monitoring, autonomous driving, and other fields.

[0003] Existing traditional methods mainly rely on nonlinear functions such as histogram equalization and gamma correction, as well as theoretical models based on illumination decomposition. These methods suffer from shortcomings such as oversimplification, significant loss of image detail, and poor visual effects. Furthermore, the complex parameter adjustment process may not necessarily achieve optimal results. Therefore, deep learning methods are now widely used for image enhancement. Based on end-to-end training models, the local perception and weight sharing of pure convolutional models enable the model to perceive the pixel values ​​of the entire image through local pixels. However, the convolutional structure loses valuable information in the image through pooling and ignores the connection between the local and the whole. Introducing the transformer model can establish long-range contextual connections, expand the image's receptive field, extract more features, and establish a connection between the global and local aspects of the image.

[0004] Gamma correction: The values ​​of dark pixels in an image are very low. Gamma correction performs a power-order nonlinear function operation on the pixels of the image to quickly increase the values ​​of the pixels in the dark area, making the dark image brighter.

[0005] IAT (Illumination-Adaptive-Transformer)-based methods

[0006] The main solutions are as follows:

[0007] First, based on attention memory, through QKV, randomly initialize the query value to obtain a 3*3 color matrix and a gamma value in the module.

[0008] Second, the local branch passes through a 3x3 convolution kernel, followed by a 1x1 convolution kernel, a 5x5 convolution kernel, and a 1x1 convolution kernel. The residual connection output is normalized, and a 1x1 convolution kernel Gaussian activation layer is followed by a 1x1 convolution kernel, followed by a residual connection to form a local adjustment module. Finally, these three modules are connected to form a local branch. The image generated by operating one branch with the color matrix is ​​added to the enhanced image of the other branch to obtain the overall enhanced image, which is finally exponentially operated with the gamma parameter to obtain the final enhanced image.

[0009] Its disadvantages are:

[0010] 1. The model is too simple and lacks downsampling, making it impossible to obtain deep features. The two local branches cannot extract high-dimensional features of the image, and the ability to extract high-dimensional information is weak, resulting in unsatisfactory enhancement results.

[0011] 2. Low-light image enhancement requires a global focus. However, the first technique uses only a 3x3 convolution kernel and relies solely on the translation invariance of the convolutional model to extract local feature information. The receptive field fails to cover the entire feature map, and the operation of extracting global dependencies fails, resulting in a loss of inter-pixel dependencies in the final enhancement result.

[0012] It should be noted that the information disclosed in the above background technology section is only used to understand the background of this application, and therefore may include information that does not constitute prior art known to ordinary technicians in this field. Summary of the Invention

[0013] The main purpose of the present invention is to overcome the defects of the above-mentioned background technology and provide a low-light image enhancement method based on transformer.

[0014] To achieve the above object, the present invention adopts the following technical solutions:

[0015] A transformer-based low-light image enhancement method includes the following steps:

[0016] Build and train a network structure consisting of a local enhancement block (LEB) branch and a sliding window-based self-attention block (WSAB) cascade branch.

[0017] The local enhancement module includes symmetrical downsampling and upsampling layers. Each level is composed of a downsampling layer and a residual transformer module (RTM). The upsampling and downsampling layers are connected by a residual layer, forming an hourglass residual enhancement module based on the transformer model. This allows for both low-dimensional information and deeper feature information to be obtained during local correction.

[0018] Among them, the WSAB module performs gradual global enhancement by cascading downsampling and then cascading WSAB modules, and finally obtains the trainable Gamma parameters and the global joint correction matrix;

[0019] The low-light image is enhanced using the network structure.

[0020] Further:

[0021] In the global component of the network structure, a self-attention mechanism module based on a sliding window module is used to extract the Gamma parameter and the color joint matrix in a downsampling cascade extraction manner to achieve global enhancement.

[0022] The local components of the network structure adopt an hourglass structure, which is globally strengthened during continuous downsampling operations to enhance the long-range dependency of local components. While obtaining depth information, it also retains low-dimensional information of the original scale through jump connections. By changing the residual order, the LEB module is made more scalable to the data set and has increased generalization.

[0023] The RTM residual transformer layer after the downsampling operation in the local enhancement module is used as an RTM local residual enhancement module to further enhance the global features of the downsampled image, thereby enhancing the enhancement effect of the subsequent introduction of global parameters.

[0024] In the WSAB module, the input undergoes layer normalization once and is connected to the window multi-head self-attention layer (W-MSA), which is then jump-connected to the input. The output undergoes layer normalization once more and then jump-connected to the output through a fully connected layer (FFN) to output the result.

[0025] The fully connected layer (FFN) consists of a 1*1 convolution kernel followed by a Gaussian activation function, a 3*3 depth-dimensional convolution kernel, a Gaussian activation function, and finally a 1*1 convolution kernel to adjust the number of channels. The WSAB module combines the Transformer and CNN for more comprehensive feature extraction.

[0026] The fully connected layer (FFN) consists of two 1*1 convolution kernels, each convolution kernel is followed by a Gaussian activation function and a depth-wise convolution (DWConvs) layer.

[0027] In the local enhancement module (LEB), the input undergoes a downsampling operation, passes through maximum pooling, and then passes through a residual transformer module (RTM). Part of the output feature map is jump-connected to the symmetrical upsampling layer output through the residual module, and the feature map pixel values ​​are added to enter the upsampling layer of the next layer; another part of the feature map is used for global pooling again, and then passes through RTM to enter the next and jump-connected with the corresponding upsampling layer; finally, after the last downsampling operation, three global feature extraction operations are performed, that is, three layers of RTM cascade, the output is added to the output of the previous layer, and the upsampling operation is continuously performed to finally obtain the feature map of the initial input size; preferably, the weight superposition training of the loss function is performed on the output map of each subsequent local enhancement module for 3D low-light image enhancement scenarios.

[0028] The RTM input passes through two layers of cascaded WSAB modules, then passes through a 3*3 convolution kernel to adjust the number of channels, and finally is added to the input of the other branch through a 1*1 convolution kernel; the RM layer input passes through a normalization layer and an activation function, and adjusts the number of channels through a 1*1 convolution kernel, and then passes through a normalization layer and an activation function, extracts features through a 3*3 convolution kernel, and then passes through a normalization layer and an activation function, and adjusts the number of channels through a 1*1 convolution kernel and is merged with the input of the other branch through a 1*1 convolution kernel, and finally outputs.

[0029] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the transformer-based low-light image enhancement method.

[0030] The present invention has the following beneficial effects:

[0031] In the transformer-based low-light image enhancement method provided by the present invention, a network structure that can be used for low-light image enhancement is designed and trained, which effectively improves the enhancement effect of low-light images. The global parameters can be trained through the transformer and are adaptive parameters; the local enhancement module has high scalability and enhances the ability to extract global information from the image.

[0032] Compared with the previous methods, the advantages of the embodiments of the present invention are mainly reflected in:

[0033] Improve the semantic expression ability of the model and improve the ability to extract deep features.

[0034] The LEB local enhancement module consists of symmetrical downsampling and upsampling layers. Each level consists of a downsampling layer connected by an RTM residual transformer layer, with a residual layer connecting the upsampling and downsampling layers. This hourglass structure benefits from capturing both low-dimensional information and deeper feature information during local correction, laying the foundation for future 3D low-light enhancement.

[0035] Improve the ability to extract global feature parameters.

[0036] Use the WSAB module to perform cascade downsampling and then cascade the WSAB module for gradual global enhancement to finally obtain the trainable parameter gamma and the global joint correction matrix.

[0037] The specific advantages of the embodiments of the present invention are:

[0038] 1. A transformer-based sliding window module is used in the global component to extract gamma parameters and color joint matrices. The extracted parameters are trainable and have more global information, making them more effective for global enhancement.

[0039] 2. The local component adopts an hourglass structure, which performs global enhancement during continuous downsampling operations, enhancing the long-range dependency of local components and obtaining depth information. At the same time, it also retains the low-dimensional information of the original scale through jump connections, ensuring that information of all dimensions of the image is taken into account during the training process.

[0040] 3. The LEB module can make it more scalable for various data sets and increase generalization by changing the residual order. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 A sliding window module diagram and a fully connected layer network diagram of an embodiment of the present invention.

[0042] Figure 2 This is a structural diagram of an LEB module according to an embodiment of the present invention.

[0043] Figure 3 This is a diagram of the RM and RTM modules of an embodiment of the present invention.

[0044] Figure 4 A schematic diagram of the overall network framework of an embodiment of the present invention. DETAILED DESCRIPTION

[0045] The following is a detailed description of the embodiments of the present invention. It should be emphasized that the following description is only exemplary and is not intended to limit the scope of the present invention and its application.

[0046] See Figures 1 to 4An embodiment of the present invention provides a transformer-based low-light image enhancement method, comprising the following steps: constructing and training a cascaded network structure including a local enhancement block (LEB) and a sliding window-based self-attention mechanism module (WSAB); wherein the local enhancement module includes symmetrical downsampling layers and upsampling layers, each of which is composed of a downsampling layer and a residual transformer module (RTM), and the upsampling and downsampling layers are connected by a residual layer to form an hourglass residual enhancement module based on the transformer model, thereby obtaining both low-dimensional information and deeper feature information during local correction; wherein the WSAB module performs gradual global enhancement by cascading downsampling and then cascading WSAB modules, ultimately obtaining a trainable gamma parameter and a global joint correction matrix; and using the network structure to enhance low-light images.

[0047] Specific embodiments of the present invention are further described below.

[0048] Sliding window transformer cascade module

[0049] Refer to the sliding window model of Swin-transformer, such as Figure 1 The embodiment of the present invention proposes a new module of multi-head self-attention mechanism based on sliding window transformer, which is used to extract long-range dependencies at the pixel level of the image. Among them, WSAB (Window-based Self-Attention Block) is connected by a layer-normalized multi-head self-attention layer, and at the same time, it is jump-connected with the input and finally outputs it through another layer normalization and a fully connected layer and jump-connected with the previous output, and finally outputs the result. The fully connected layer (FFN) here in the embodiment of the present invention is composed of a 1*1 convolution kernel followed by a Gaussian activation function, a 3*3 depth-dimensional convolution kernel, a Gaussian activation function, and finally a 1*1 convolution kernel to adjust the number of channels. Compared with CNN focusing on and learning local feature information, Transformer can better capture long-range correlations. WSAB combines Transformer and CNN, so that the module can perform feature extraction more comprehensively and improve the performance of the model. The FFN consists of two 1*1 convolutions, each followed by a Gaussian activation function and a DWConvs (depth-dimensional convolution) layer. Compared with the commonly used 3*3 convolution kernel, the DWConvs3*3 reduces the number of parameters and the amount of calculation by sharing weights.

[0050] Local enhancement module

[0051] like Figure 2 The embodiment of the present invention proposes a LEB (Local Enhancement Block) local enhancement module which is strictly symmetrical to ensure the superiority of the module performance. The input of the embodiment of the present invention undergoes a downsample operation using maximum pooling and then passes through an RTM (Resdual transformer module) residual transformer module to output the feature Figure 1 Part of it is connected to the symmetrical upsampling layer output through the residual module jump, and the feature map pixel values ​​are added to enter the upsampling layer of the next layer. Another part of the feature map is used for global pooling again, and then enters the next time through RTM and jumps to the corresponding upsampling layer. Finally, after the last downsampling operation, after three global feature extraction operations, that is, three layers of RTM cascade, the output is added to the output of the previous layer, and the upsampling operation is continuously performed to finally obtain the feature map of the initial input size. It not only retains the feature information of the original scale, but also extracts deep information through continuous downsampling, making the information obtained from the image richer. Figure 2 It represents a 4th-order LEB module. The model of the embodiment of the present invention can arbitrarily conduct experiments on multi-layer residual connections according to the size and characteristics of the data set, which greatly improves the generalization, accuracy and scalability of the model. At the same time, the weight superposition training of the loss function is performed in the output graph of each subsequent local enhancement module, which can be used in 3D low-light image enhancement scenarios.

[0052] like Figure 3 , where a and b are the RTM module and RM module respectively. The specific details are as follows: the RTM input passes through two layers of cascaded WSAB modules, and then passes through a 3*3 convolution kernel to adjust the number of channels, and finally is added to the input of the other branch through a 1*1 convolution kernel. This function is to adjust the value of each channel after downsampling and enhance the global feature information of the feature map after downsampling. The RM layer is the input that passes through a normalization layer and an activation function, adjusts the number of channels through a 1*1 convolution kernel, and then passes through a normalization layer and an activation function, extracts features through a 3*3 convolution kernel, and then passes through a normalization layer and an activation function, adjusts the number of channels through a 1*1 convolution kernel, and is merged with the input of the other branch through a 1*1 convolution kernel, and finally outputs.

[0053] Overall network model and experimental results

[0054] like Figure 4As shown, the algorithm of the embodiment of the present invention is generally composed of two parts. One part is the global branch, which is used to extract the white balance matrix, the color conversion joint matrix and the gamma parameters. The second part, the local branch, performs local enhancement on the low-light image for joint matrix operations and obtains the bias feature map. The white balance is used to estimate the gain of each channel and maintain the color constancy of the object under various lighting conditions. At the same time, after white balance, the camera color matrix converts the internal white balance data to the sRGB color space. The algorithm simultaneously completes the white balance and color conversion matrix to enhance each pixel. Finally, the obtained image is subjected to pixel-level gamma correction to match people's perception of dark areas.

[0055] In the embodiment of the present invention, the global component consists of an input projection layer, a local transformer layer (WSAB), a downsampling layer, and an output projection layer. It has been verified that the embodiment of the present invention uses a single downsampling, two local transformer layers, and finally a perceptron to obtain a 1*10 vector and reshape it, obtaining an adjustable parameter gamma and a joint adjustment matrix. The single downsampling operation here is experimentally derived. If a new dataset is added, two or more downsampling times may be used. In short, this is a structure that can be trained and adjusted.

[0056] In the local adjustment branch, this embodiment of the present invention maintains resolution and preserves detail. This embodiment first performs an input projection layer to widen the channel dimension, then stacks these two independent branches through a local enhancement module, where the local enhancement module operates at the pixel level. This embodiment of the present invention focuses on correcting for illumination effects through local, trainable components A and B. This embodiment avoids the adjustable baseline model because, while lightweight and requiring no resizing for images of any resolution, the algorithm still requires image cropping, which is not effective for the dataset proposed in this invention, as dark areas are not enhanced. The LEB (Local Enhancement Block) local enhancement block in this embodiment of the present invention demonstrates a four-stage local enhancement module consisting of symmetrical downsampling and upsampling layers. Each stage consists of a downsampling layer and an RTM residual transformer layer, with a residual layer connecting the upsampling and downsampling layers. The advantage of this hourglass structure is that it captures both low-dimensional information and deeper feature information during local correction, laying the foundation for future 3D dark light enhancement. Locally, long-range dependencies are established by adding transformer layers to improve local correction capabilities.

[0057] The main features of the embodiments of the present invention are:

[0058] First, in the field of low-light image enhancement, the design originality of the LEB module (hourglass residual enhancement module based on the transformer model).

[0059] Second, the downsampling cascade extraction method of the WSAB module of the gamma parameter and the color matrix.

[0060] Third, the RTM local residual enhancement module after the downsampling operation in the LEB module further enhances the global features of the downsampled image, thereby strengthening the subsequent enhancement effect of the global parameters.

[0061] The present invention uses a cascade of sliding window modules and selects adjustable adaptive parameters for downsampling. The local enhancement module introduces a transformer structure after downsampling to enhance global extraction, obtaining deep features while retaining the original feature information, thus improving the image enhancement effect.

[0062] The technical solution of this invention can address the model's poor global perception of low-light images. This method trains every parameter, resolving the untrainable nature of traditional methods. It also addresses the model's inability to account for both deep and shallow features.

[0063] A multi-stage hourglass structure, employing downsampling followed by a self-attention module and then upsampling, improves the model's ability to extract both deep and shallow features. The model's trainability is enhanced, and the number and order of local enhancement modules within each module can improve the model's scalability and generalization capabilities.

[0064] application

[0065] This invention has huge commercial prospects. The new energy vehicle industry is developing in full swing, and intelligent driving will also be the direction of future development. Night-time assisted driving relies on images captured by cameras. This algorithm can improve the quality of night-time images and enhance the safety of assisted driving.

[0066] The present invention can be used in the aerospace field. The operation of the flexible arm on the satellite relies on the auxiliary camera to take pictures. In space, there are completely dark situations and uneven exposure. This algorithm can enhance the information of the dark areas of the auxiliary image.

[0067] The present invention can be applied in the mineral mining industry. The mining environment is dark, and the pictures taken with the assistance of robots can be brightened after algorithm processing, thereby enhancing the details of dark areas and improving the efficiency of the industry.

[0068] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0069] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowcharts and / or block diagrams, as well as combinations of processes and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowcharts and / or block diagrams. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0070] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0071] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 The steps for the function specified in one or more boxes.

[0072] The background section of the present invention may contain background information about the problem or environment of the present invention, but does not necessarily describe the prior art. Therefore, the inclusion of content in the background section is not an admission by the applicant that the prior art is present.

[0073] The above description further details the present invention in conjunction with specific / preferred embodiments, and the specific implementation of the present invention should not be construed as being limited to these descriptions. Persons skilled in the art will appreciate that, without departing from the spirit of the present invention, they may make various substitutions or modifications to the described embodiments, and these substitutions or modifications should be considered to fall within the scope of protection of the present invention. Throughout this specification, reference to terms such as "one embodiment," "some embodiments," "preferred embodiments," "examples," "specific examples," or "some examples" indicates that the specific features, structures, materials, or characteristics described in conjunction with such embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of these terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples. Persons skilled in the art may combine and assemble the different embodiments or examples described in this specification, as well as features of different embodiments or examples, without conflicting opinions. Although the embodiments of the present invention and their advantages have been described in detail, it should be understood that various changes, substitutions, and modifications may be made within the scope of protection of the patent application.

Claims

1. A transformer-based low-light image enhancement method, characterized in that: The steps include: Construct and train a network structure consisting of a local enhancement module (LEB) branch and a sliding window-based self-attention mechanism module (WSAB) cascade branch; The local enhancement module includes symmetrical downsampling and upsampling layers. Each stage is composed of a downsampling layer and a residual transformer module (RTM). The upsampling and downsampling layers are connected by a residual layer (RM), forming an hourglass residual enhancement module based on the transformer model. This allows for both low-dimensional information and deeper feature information to be obtained during local correction. Among them, in the cascade branch of the sliding window self-attention mechanism module (WSAB), the WSAB module performs cascade downsampling and then cascades WSAB modules for gradual global enhancement, ultimately obtaining a trainable Gamma parameter and a global joint correction matrix. In the WSAB module, the input undergoes a layer normalization and is connected to the window multi-head self-attention layer (W-MSA). The window multi-head self-attention layer (W-MSA) is jump-connected to the input. The output undergoes another layer normalization and then jump-connected to the output through a fully connected layer (FFN) to finally output the result. Among them, the residual transformer module RTM input passes through two layers of cascaded WSAB modules, then passes through a 3*3 convolution kernel to adjust the number of channels, and finally adds the input of the other branch through a 1*1 convolution kernel; the RM layer input passes through a normalization layer and an activation function, and adjusts the number of channels through a 1*1 convolution kernel, and then passes through a normalization layer and an activation function, and extracts features through a 3*3 convolution kernel, and then passes through a normalization layer and an activation function, and adjusts the number of channels through a 1*1 convolution kernel and merges with the input of the other branch through a 1*1 convolution kernel, and finally outputs; The low-light image is enhanced using the network structure.

2. The transformer-based low-light image enhancement method according to claim 1, wherein: In the global component of the network structure, a self-attention mechanism module based on a sliding window module is used to extract the Gamma parameter and the color joint matrix in a downsampling cascade extraction manner to achieve global enhancement.

3. The transformer-based low-light image enhancement method according to claim 1, wherein: The local components of the network structure adopt an hourglass structure, which is globally strengthened during continuous downsampling operations to enhance the long-range dependency of local components. While obtaining depth information, it also retains low-dimensional information of the original scale through jump connections. By changing the residual order, the LEB module is made more scalable to the data set and has increased generalization.

4. The transformer-based low-light image enhancement method according to any one of claims 1 to 3, wherein: The RTM residual transformer layer after the downsampling operation in the local enhancement module is used as an RTM local residual enhancement module to further enhance the global features of the downsampled image, thereby enhancing the enhancement effect of the subsequent introduction of global parameters.

5. The transformer-based low-light image enhancement method according to any one of claims 1 to 3, wherein: The fully connected layer (FFN) consists of a 1*1 convolution kernel followed by a Gaussian activation function, a 3*3 depth-dimensional convolution kernel, a Gaussian activation function, and finally a 1*1 convolution kernel to adjust the number of channels. The WSAB module combines the Transformer and CNN for more comprehensive feature extraction.

6. The transformer-based low-light image enhancement method according to any one of claims 1 to 3, wherein: The fully connected layer (FFN) consists of two 1*1 convolution kernels, each of which is followed by a Gaussian activation function and a depth-wise convolution (DWConvs) layer.

7. The transformer-based low-light image enhancement method according to any one of claims 1 to 3, wherein: In the local enhancement module (LEB), the input undergoes a downsampling operation, passes through maximum pooling, and then passes through a residual transformer module (RTM). Part of the output feature map is jump-connected to the symmetrical upsampling layer output through the residual module, and the feature map pixel values ​​are added to enter the upsampling layer of the next layer; another part of the feature map is used for global pooling again, and then passes through the RTM to enter the next and jump-connected to the corresponding upsampling layer; finally, after the last downsampling operation, three global feature extraction operations are performed, that is, three layers of RTM cascade, the output is added to the output of the previous layer, and the upsampling operation is continuously performed to finally obtain the feature map of the initial input size.

8. The transformer-based low-light image enhancement method according to claim 7, wherein: The weighted superposition training of the loss function is performed on the output graph of each subsequent local enhancement module for 3D low-light image enhancement scenarios.

9. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the transformer-based low-light image enhancement method according to any one of claims 1 to 8 is implemented.

Images