A low-light image enhancement method, system, device and medium
Through a low-light image enhancement model based on grouping self-attention, combined with the autoencoder and denoising color tuning module, the problems of noise, color distortion and detail loss in the prior art are solved, and more efficient image enhancement effects are achieved. Reduces computational complexity and is suitable for embedded device deployment.
Patent Information
- Application Number
- CN202111544055.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-12-16
- Publication Date
- 2024-12-03
- Estimated Expiration
- 2041-12-16
AI Technical Summary
Existing low-light image enhancement methods cannot effectively deal with noise, color distortion and loss of details simultaneously, limiting their flexibility and effectiveness.
Using a low-light image enhancement model based on group self-attention, the image downsampling, group self-attention operation, upsampling, denoising and color adjustment are performed through the self-encoder, group self-attention module and denoising color adjustment module. , output the final enhanced image.
This method can solve the problems of noise, color distortion and detail loss at the same time, reduce time and space complexity, is suitable for embedded device deployment, and improves image enhancement effect.
Smart Images

Figure CN114359073B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of image data processing technology, and in particular to a low-light image enhancement method, system, device and medium. Background Art
[0002] Images taken under insufficient lighting conditions usually suffer from degradation problems such as loss of detail information, high noise, color distortion, etc. This not only affects people's visual experience, but is also not conducive to high-level visual tasks (such as classification, detection, segmentation, etc.).
[0003] Existing low-light image enhancement methods can be roughly divided into the following three categories according to different principles:
[0004] Histogram-based methods. This type of method roughly assumes that the grayscale values of a normal illumination image are evenly distributed on the histogram. By nonlinearly stretching the image, the histogram of the image is evenly distributed, thereby obtaining an enhanced image. This type of method is simple and easy to implement, but the enhanced image obtained usually has problems such as large noise, loss of details, insufficient enhancement or over-enhancement.
[0005] Methods based on Retinex theory. This method theory believes that images can be decomposed into reflection components and illumination components. The reflection component reflects the essence of the image, and the illumination component reflects the influence of illumination on the image. Based on this, this type of method estimates the illumination component of the image a priori, and then calculates the reflection component as the enhanced image according to the Retinex theory. Although this type of method overcomes the shortcomings of the small application range of the histogram-based method, the enhanced image still has problems such as large noise and color distortion.
[0006] Methods based on deep learning. This type of method mainly uses the powerful modeling ability of neural networks to learn the mapping from low-light images to normal-light images. Existing representative deep learning image enhancement algorithms include LLNet, RetinexNet, KinD, MIRNet, etc. Although this type of method is superior to traditional image enhancement methods in terms of running speed, denoising, color restoration, etc., it still cannot simultaneously handle degradation problems such as noise, color distortion, and detail loss in low-light images, thus limiting its flexibility and effectiveness. Summary of the invention
[0007] In order to solve at least one of the technical problems existing in the prior art to a certain extent, the object of the present invention is to provide a low-light image enhancement method, system, device and medium based on grouped self-attention.
[0008] The technical solution adopted by the present invention is:
[0009] A low-light image enhancement method comprises the following steps:
[0010] Get the training set;
[0011] Build a low-light image enhancement model based on grouped self-attention;
[0012] Using the training set to train the low-light image enhancement model;
[0013] Acquire a low-light image to be enhanced, input the low-light image into the trained low-light image enhancement model for processing, and output an enhanced image;
[0014] The low-light image enhancement model performs the following operations on the input low-light image:
[0015] After downsampling the low-light image, a grouped self-attention operation is performed on the feature map, and then upsampling is performed to output a preliminary enhanced map. Finally, denoising and color adjustment are performed on the preliminary enhanced map to obtain a final enhanced map.
[0016] Furthermore, the low-light image enhancement model includes an autoencoder, a grouped self-attention module, and a plurality of denoising and color adjustment modules;
[0017] The autoencoder includes an encoder and a decoder, and the decoding end and the encoding end adopt a symmetrical structure; the encoding end includes three downsampling layers, each of which consists of a convolution and three residual blocks with channel attention (i.e. Figure 1 The SE-ResNet block in is used to extract the semantic features of the image;
[0018] The grouped self-attention module is arranged between the encoding end and the decoding end to expand the receptive field of the network;
[0019] The denoising and color adjustment module is connected to the end of the decoder and is used for denoising and color adjustment of the preliminary enhanced image output by the decoder.
[0020] Furthermore, a jump operation is adopted between the encoding end and the decoding end to fuse information at different levels.
[0021] Furthermore, the residual block with channel attention performs the following operations on the input feature map X:
[0022] The input feature map X is subjected to two convolution operations to obtain the feature map X1;
[0023] Perform global average pooling on the feature map X1 to extract the global information of each channel;
[0024] The global information interaction of the channel is carried out through two fully connected layers with activation functions. The first fully connected layer adopts the ReLU activation function, and the second fully connected layer adopts the Sigmoid activation function.
[0025] The normalized weights are added to the features of each channel of the feature map X1, and then the weighted feature map is added to the input feature map X to obtain the output feature map.
[0026] Furthermore, the grouped self-attention module performs the following operations on the input feature map F:
[0027] Group the input feature map F to obtain G groups of feature maps;
[0028] The feature maps of different groups are spatially shuffled according to a preset rule, and the shuffled feature maps are regrouped to obtain G groups of new feature maps (F1, F2, ..., F G );
[0029] Perform a primitive non-local attention operation on each new set of feature maps to obtain G sets of output feature maps (R1, R2, ..., R G );
[0030] Among them, each spatial position of the output feature map contains the information of the feature vectors of all spatial positions of the input feature map F.
[0031] Furthermore, the original non-local attention operation includes:
[0032] Perform three linear transformations on the input feature map F to obtain feature maps Q, K and V;
[0033] The feature vector of each spatial position in the feature map Q is correlated with the feature vector in the feature map K, and an attention map S is obtained after normalization using the Softmax function;
[0034] After multiplying the attention map S with the feature map V, it is added to the input feature map F to obtain the output feature map R.
[0035] Further, the denoising and color adjustment module includes a denoising submodule and a color adjustment submodule;
[0036] The denoising submodule performs the following operations on the preliminary enhanced image R′:
[0037] The input preliminary enhanced map R′ is converted to the transform domain by three convolution operations to obtain the feature map L;
[0038] Perform two convolution operations on the feature map L to generate a soft attention map W;
[0039] Adding the soft attention map W to the feature map L to adaptively filter out noise in the feature map;
[0040] The feature map obtained by adding is inversely transformed through three convolution operations, and then added to the preliminary enhanced map R′ to output the noise-filtered image R d ;
[0041] The color adjustment submodule adjusts the image R d Do the following:
[0042] The input feature map R d Perform three convolution operations to convert to the transform domain and obtain the feature map T;
[0043] Perform two convolution operations on the feature map T to generate a soft attention map Z;
[0044] Multiplying the soft attention map Z, the feature map T and the self-learning parameter α to adaptively adjust the color of the feature map;
[0045] The feature map obtained by multiplication is inversely transformed through three convolution operations and then combined with the image R d Add and output the color-adjusted image R a Another technical solution adopted by the present invention is:
[0046] A low-light image enhancement system, comprising:
[0047] A data acquisition module, used to obtain a training set;
[0048] Model building module, used to build a low-light image enhancement model based on grouped self-attention;
[0049] A model training module, used to train the low-light image enhancement model using the training set;
[0050] An image enhancement module, used for acquiring a low-light image to be enhanced, inputting the low-light image into the trained low-light image enhancement model for processing, and outputting an enhanced image;
[0051] The low-light image enhancement model performs the following operations on the input low-light image:
[0052] After downsampling the low-light image, a grouped self-attention operation is performed on the feature map, and then upsampling is performed to output a preliminary enhanced map. Finally, denoising and color adjustment are performed on the preliminary enhanced map to obtain a final enhanced map.
[0053] Another technical solution adopted by the present invention is:
[0054] A low-light image enhancement device, comprising:
[0055] at least one processor;
[0056] at least one memory for storing at least one program;
[0057] When the at least one program is executed by the at least one processor, the at least one processor implements the above method.
[0058] Another technical solution adopted by the present invention is:
[0059] A computer-readable storage medium stores a program executable by a processor, wherein the program executable by the processor is used to execute the method described above when executed by the processor.
[0060] The beneficial effects of the present invention are as follows: the present invention proposes a low-light image enhancement solution based on grouped self-attention, which solves the degradation problems of noise, color distortion, detail loss, etc. through a single model. Compared with the original non-local attention module, the proposed grouped self-attention module has a time complexity of It has dropped to The space complexity is from It has dropped to It is more conducive to deploying the model to the embedded end. BRIEF DESCRIPTION OF THE DRAWINGS
[0061] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the embodiments of the present invention or the drawings of related technical solutions in the prior art are introduced below. It should be understood that the drawings introduced below are only for the convenience of clearly describing some embodiments of the technical solutions of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.
[0062] Figure 1 is a schematic diagram of the overall network framework of the low-light image enhancement model in an embodiment of the present invention;
[0063] Figure 2 is a structural diagram of a grouped self-attention module in an embodiment of the present invention;
[0064] Figure 3 is a structural diagram of a denoising and color adjustment module in an embodiment of the present invention;
[0065] Figure 4 It is a flowchart of the steps of a low-light image enhancement method in an embodiment of the present invention. DETAILED DESCRIPTION
[0066] Embodiments of the present invention are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and are not to be construed as limitations of the present invention. For the step numbers in the following embodiments, they are only provided for the convenience of explanation, and the order between the steps is not limited in any way, and the execution order of each step in the embodiment can be adaptively adjusted according to the understanding of those skilled in the art.
[0067] In the description of the present invention, it should be understood that descriptions involving orientations, such as up, down, front, back, left, right, etc., and orientations or positional relationships indicated are based on the orientations or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as a limitation on the present invention.
[0068] In the description of the present invention, "several" means one or more, "more" means more than two, "greater than", "less than", "exceed" etc. are understood as not including the number itself, and "above", "below", "within" etc. are understood as including the number itself. If there is a description of "first" or "second", it is only used for the purpose of distinguishing the technical features, and cannot be understood as indicating or implying the relative importance or implicitly indicating the number of the indicated technical features or implicitly indicating the order of the indicated technical features.
[0069] In the description of the present invention, unless otherwise clearly defined, terms such as setting, installing, connecting, etc. should be understood in a broad sense, and technicians in the relevant technical field can reasonably determine the specific meanings of the above terms in the present invention based on the specific content of the technical solution.
[0070] like Figure 4 As shown, this embodiment provides a low-light image enhancement method, comprising the following steps:
[0071] S1. Obtain the training set.
[0072] In this embodiment, the LOL dataset is used as a training set, and the dataset includes 500 pairs of pictures taken in real scenes, of which 485 pairs of pictures are used as a training set and 15 pairs of pictures are used as a verification set.
[0073] S2. Build a low-light image enhancement model based on grouped self-attention.
[0074] like Figure 1 As shown, Figure 1The overall network framework of the low-light image enhancement model is shown in Figure 1. The model first uses an autoencoder architecture to obtain a preliminary enhanced image. Then, three denoising and color adjustment modules (abbreviated as DeCA, hereinafter referred to as DeCA modules) are introduced after the autoencoder to solve the problems of excessive noise and color distortion in the preliminary enhanced image.
[0075] The autoencoder consists of an encoder and a decoder. The encoder consists of three downsampling layers, each of which consists of a convolution and three residual blocks with channel attention (i.e. Figure 1 The decoder uses a symmetrical structure with the encoder. To retain more information, the decoder uses transposed convolution for upsampling.
[0076] A skip operation (addition) is used at the encoding and decoding ends to help the network fuse information at different levels and solve the problem of detail information loss during the encoding process. In addition, a group self-attention module (GSA) is added between the encoding and decoding ends to expand the network's receptive field and reduce color distortion.
[0077] Residual block with channel attention: The deepening of the network will cause the loss of detail information, and the feature maps of different channels have different contributions to the entire network. Therefore, the network model proposed in this embodiment adopts a large number of residual blocks with channel attention. The residual block with channel attention first obtains the feature map X1 by performing two convolution operations on the input feature map X. Then the feature map X1 is globally averaged pooled to extract the global information of each channel separately, and then the channel information is interacted through two fully connected layers with activation functions. Among them, the first fully connected layer uses the ReLU activation function, and the second fully connected layer uses the Sigmoid activation function. Then a Scale operation is used to weight the normalized weights to the features of each channel of the feature map X1. Finally, the weighted feature map is added to the input feature map X.
[0078] Grouped self-attention module: The structure of the grouped self-attention module is as follows Figure 2As shown. A large number of studies have shown that a larger receptive field can effectively reduce color distortion; while the convolution operation in the convolutional neural network can only extract local information, and the receptive field is limited. The current mainstream approach is to use a non-local attention module. However, the time complexity and space complexity of this module have relatively high requirements on hardware equipment, which is not conducive to practical application. In response to this problem, this embodiment proposes a grouped self-attention module, which cleverly introduces the idea of grouping. In the original non-local attention operation, the input feature map F is first subjected to three linear transformations to obtain three feature maps Q, K and V, and then the feature vector of each spatial position in the feature map Q is correlated with the feature vector in K, and then Softmax is used to normalize it to obtain an attention map S. Then the attention map S is multiplied by V (that is, weighted summation is performed according to the degree of correlation between the feature vectors at different spatial positions and the current feature vector) to obtain the output feature map R. The present invention optimizes this module and first groups the input feature map F. Assume that it is divided into G groups (F1, F2,..., F G ), perform a single original non-local attention operation on each set of feature maps to obtain G sets of output feature maps (R1, R2, ..., R G ). Each spatial position of each group of output feature maps has information about other positions in the group, but there is still no interaction between different groups. In order to achieve cross-group information interaction, the spatial positions of feature maps of different groups are shuffled according to a certain rule (for details, please refer to Figure 2 ). Then the feature map is grouped, and the original non-local attention operation is performed on each group of features. Finally, each spatial position of the output feature map contains the information of all spatial position feature vectors of the input feature map F. This module greatly reduces the time complexity and space complexity of the original non-local attention while maintaining the performance of the original non-local attention module. Assume that the size of the input feature map F is H×W×C, where H is the height of the feature map F, W is the width of the feature map F, and C is the number of channels of the feature map F. Then, compared with the original non-local attention module, the time complexity of this module is reduced from It has dropped to The space complexity is from It has dropped to This greatly reduces the time and space complexity of the original non-local attention module, making it easier to deploy the model on the embedded side.
[0079] DeCA module: The structure of the DeCA module is as follows Figure 3 It includes two submodules, namely, a denoising submodule and a color adjustment submodule, which are used to further filter out the noise in the preliminary enhanced image and adjust its color.
[0080] Denoising submodule: This denoising submodule is used to effectively filter out noise in the image while retaining details. The denoising submodule first converts the input preliminary enhancement map R′ into the transform domain by three convolution operations to obtain the feature map L. Then, the feature map L is convolved twice to generate a soft attention map W, and then the soft attention map W and the feature map L are added to adaptively filter out the noise in the feature map. The obtained features are then inversely transformed through three convolution operations to output features that filter out useless information. Finally, in order to accelerate network convergence, residual learning is introduced in this module, and the denoised result map R is finally obtained. d .
[0081] Color adjustment submodule: Result after denoising R d There is still a certain deviation in color from the real image, so a color adjustment module is proposed to further improve the result image R d The overall structure of this module is similar to that of the denoising module, except that the role of the soft attention map is different. Existing methods usually use a single artificially set magnification factor to adjust the image color in the image domain, while the present invention chooses to adjust the color in the feature domain with stronger robustness. First, a learnable parameter α is used to adjust the features in a coarse-grained manner, and then the soft attention map generated by the soft attention module is used to adaptively adjust the features in a fine-grained manner, thereby obtaining more accurate transform domain features and finally restoring an image with more realistic colors.
[0082] S3. Use the training set to train the low-light image enhancement model.
[0083] The network model is trained using the sorted data set. In this embodiment, the loss function used in the network training is Charbonnier loss, the optimization algorithm is Adam Optimizer, and the learning rate is set to 0.0001. During the training process, the Batch size is set to 32 and the Patch size is 128×128.
[0084] S4. Obtain a low-light image to be enhanced, input the low-light image into a trained low-light image enhancement model for processing, and output an enhanced image.
[0085] The low-light image is enhanced using the trained low-light image enhancement model based on the grouped self-attention module, and the resulting image is superior to the existing algorithm.
[0086] In summary, compared with the prior art, the method of this embodiment has the following beneficial effects: the present invention proposes a low-light image enhancement solution based on grouped self-attention, which solves the degradation problems of noise, color distortion, detail loss, etc. through a single model. Among them, the proposed grouped self-attention module has a time complexity of 100% compared with the original non-local attention module. It has dropped to The space complexity is from It has dropped to It is more conducive to deploying the model to the embedded end.
[0087] This embodiment also provides a low-light image enhancement system, including:
[0088] A data acquisition module, used to obtain a training set;
[0089] Model building module, used to build a low-light image enhancement model based on grouped self-attention;
[0090] A model training module, used to train the low-light image enhancement model using the training set;
[0091] An image enhancement module, used for acquiring a low-light image to be enhanced, inputting the low-light image into the trained low-light image enhancement model for processing, and outputting an enhanced image;
[0092] The low-light image enhancement model performs the following operations on the input low-light image:
[0093] After downsampling the low-light image, a grouped self-attention operation is performed on the feature map, and then upsampling is performed to output a preliminary enhanced map. Finally, denoising and color adjustment are performed on the preliminary enhanced map to obtain a final enhanced map.
[0094] A low-light image enhancement system of this embodiment can execute a low-light image enhancement method provided by the method embodiment of the present invention, can execute any combination of implementation steps of the method embodiment, and has the corresponding functions and beneficial effects of the method.
[0095] This embodiment also provides a low-light image enhancement device, including:
[0096] at least one processor;
[0097] at least one memory for storing at least one program;
[0098] When the at least one program is executed by the at least one processor, the at least one processor implements the following Figure 4 The method shown.
[0099] A low-light image enhancement device of this embodiment can execute a low-light image enhancement method provided by the method embodiment of the present invention, can execute any combination of implementation steps of the method embodiment, and has the corresponding functions and beneficial effects of the method.
[0100] The present application also discloses a computer program product or a computer program, which includes a computer instruction stored in a computer-readable storage medium. A processor of a computer device can read the computer instruction from the computer-readable storage medium, and the processor executes the computer instruction, so that the computer device executes Figure 4 The method shown.
[0101] This embodiment also provides a storage medium, which stores instructions or programs that can execute a low-light image enhancement method provided by an embodiment of the method of the present invention. When the instructions or programs are run, any combination of implementation steps of the method embodiment can be executed, and the corresponding functions and beneficial effects of the method can be obtained.
[0102] In some selectable embodiments, the function / operation mentioned in the block diagram may not occur in the order mentioned in the operation diagram. For example, depending on the function / operation involved, the two boxes shown in succession can actually be executed substantially simultaneously or the boxes can sometimes be executed in reverse order. In addition, the embodiment presented and described in the flow chart of the present invention is provided by way of example, for the purpose of providing a more comprehensive understanding of technology. The disclosed method is not limited to the operation and logic flow presented herein. Selectable embodiments are expected, wherein the order of various operations is changed and the sub-operation of a part for which is described as a larger operation is performed independently.
[0103] In addition, although the present invention is described in the context of functional modules, it should be understood that, unless otherwise specified, one or more of the functions and / or features described may be integrated into a single physical device and / or software module, or one or more functions and / or features may be implemented in separate physical devices or software modules. It is also understood that a detailed discussion of the actual implementation of each module is unnecessary for understanding the present invention. More specifically, in view of the properties, functions, and internal relationships of the various functional modules in the device disclosed herein, the actual implementation of the module will be understood within the conventional skills of the engineer. Therefore, those skilled in the art can implement the present invention set forth in the claims without excessive experimentation using ordinary techniques. It is also understood that the specific concepts disclosed are merely illustrative and are not intended to limit the scope of the present invention, which is determined by the full scope of the appended claims and their equivalents.
[0104] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc., which can store program codes.
[0105] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in conjunction with such instruction execution systems, devices or apparatuses. For the purposes of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in conjunction with such instruction execution systems, devices or apparatuses.
[0106] More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or more wires (electronic device), a portable computer disk case (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be a paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, deciphering or, if necessary, processing in another suitable manner, and then stored in a computer memory.
[0107] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, a plurality of steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.
[0108] In the above description of this specification, the description with reference to the terms "one embodiment / example", "another embodiment / example" or "certain embodiments / examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.
[0109] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the claims and their equivalents.
[0110] The above is a specific description of the preferred implementation of the present invention, but the present invention is not limited to the above embodiments. Those skilled in the art may make various equivalent modifications or substitutions without violating the spirit of the present invention. These equivalent modifications or substitutions are all included in the scope defined by the claims of this application.
Claims
1. A low-light image enhancement method, It is characterized in that The following steps are involved: Get the training set; Build a low-light image enhancement model based on grouped self-attention; Using the training set to train the low-light image enhancement model; Acquire a low-light image to be enhanced, input the low-light image into the trained low-light image enhancement model for processing, and output an enhanced image; The low-light image enhancement model performs the following operations on the input low-light image: Downsampling the low-light image, performing a grouped self-attention operation on the feature map, upsampling the image, outputting a preliminary enhanced map, and finally denoising and color adjusting the preliminary enhanced map to obtain a final enhanced map; The low-light image enhancement model includes an autoencoder, a grouped self-attention module, and several denoising and color adjustment modules; The autoencoder includes an encoder and a decoder, and the decoding end and the encoding end adopt a symmetrical structure; the encoding end includes three downsampling layers, each of which is composed of a convolution and three residual blocks with channel attention, which are used to extract semantic features of the image; The grouped self-attention module is arranged between the encoding end and the decoding end to expand the receptive field of the network; The denoising and color adjustment module is connected to the end of the decoder and is used to perform denoising and color adjustment on the preliminary enhanced image output by the decoder; The grouped self-attention module performs the following operations on the input feature map F: Group the input feature map F to obtain G groups of feature maps; The feature maps of different groups are spatially shuffled according to the preset rules, and the shuffled feature maps are regrouped to obtain G groups of new feature maps (F 1 , F 2 , ..., F G ); Perform the original non-local attention operation on each new set of feature maps to obtain G sets of output feature maps (R 1 , R 2 , ..., R G ); Among them, each spatial position of the output feature map contains the information of all spatial position feature vectors of the input feature map F; The original non-local attention operation includes: Perform three linear transformations on the input feature map F to obtain feature maps Q, K and V; The feature vector of each spatial position in the feature map Q is correlated with the feature vector in the feature map K, and an attention map S is obtained after normalization using the Softmax function; After multiplying the attention map S with the feature map V, it is added to the input feature map F to obtain the output feature map R.
2. The low-light image enhancement method according to claim 1, It is characterized in that A jump operation is used between the encoding end and the decoding end to fuse information at different levels.
3. The low-light image enhancement method according to claim 1, It is characterized in that The residual block with channel attention performs the following operations on the input feature map X: The input feature map X is passed through two convolution operations to obtain the feature map X 1 ; For feature map X 1 Perform global average pooling to extract global information of each channel; The global information interaction of the channel is carried out through two fully connected layers with activation functions. The first fully connected layer adopts the ReLU activation function, and the second fully connected layer adopts the Sigmoid activation function. Add the normalized weights to the feature map X 1 For the features of each channel, the weighted feature map is added to the input feature map X to obtain the output feature map.
4. The low-light image enhancement method according to claim 1, It is characterized in that The denoising and color adjustment module includes a denoising submodule and a color adjustment submodule; The denoising submodule performs the following operations on the preliminary enhanced image R′: The input preliminary enhanced map R′ is converted to the transform domain by three convolution operations to obtain the feature map L; Perform two convolution operations on the feature map L to generate a soft attention map W; Adding the soft attention map W to the feature map L to adaptively filter out noise in the feature map; The feature map obtained by adding is inversely transformed through three convolution operations, and then added to the preliminary enhanced map R′ to output the noise-filtered image R d ; The color adjustment submodule adjusts the image R d Do the following: The input feature map R d Perform three convolution operations to convert to the transform domain and obtain the feature map T; Perform two convolution operations on the feature map T to generate a soft attention map Z; Multiplying the soft attention map Z, the feature map T and the self-learning parameter α to adaptively adjust the color of the feature map; The feature map obtained by multiplication is inversely transformed through three convolution operations and then combined with the image R d Add and output the color-adjusted image R a .
5. A low-light image enhancement system, applied to a low-light image enhancement method according to any one of claims 1 to 4, It is characterized in that include: A data acquisition module, used to obtain a training set; Model building module, used to build a low-light image enhancement model based on grouped self-attention; A model training module, used to train the low-light image enhancement model using the training set; An image enhancement module, used for acquiring a low-light image to be enhanced, inputting the low-light image into the trained low-light image enhancement model for processing, and outputting an enhanced image; The low-light image enhancement model performs the following operations on the input low-light image: After downsampling the low-light image, a grouped self-attention operation is performed on the feature map, and then upsampling is performed to output a preliminary enhanced map. Finally, denoising and color adjustment are performed on the preliminary enhanced map to obtain a final enhanced map.
6. A low-light image enhancement device, It is characterized in that include: at least one processor; at least one memory for storing at least one program; When the at least one program is executed by the at least one processor, the at least one processor implements the method according to any one of claims 1 to 4.
7. A computer-readable storage medium having a processor-executable program stored therein, It is characterized in that The processor-executable program is used to perform the method according to any one of claims 1 to 4 when executed by the processor.
