Image enhancement method based on global attention and Retinex theory
By introducing a global attention module into the image enhancement method and combining Retinex theory, the color distortion and halo problems during image adjustment in low-light environments are solved, and a higher quality image processing effect is achieved.
Patent Information
- Application Number
- CN202411704381.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-26
- Publication Date
- 2025-03-14
AI Technical Summary
The existing deep learning method based on Retinex theory has color distortion and halo when adjusting images in low-light environments, making it difficult to effectively deal with the diversity and uncertainty of low-light images.
A method of image enhancement based on global attention and Retinex theory is proposed. The global attention module is constructed and combined with Retinex theory. By learning the diversity and correlation between low-light images and real image features, it is embedded in the enhancement network model to enrich the detailed information of dark areas and solve the color distortion and halo problems.
Through the design of the global attention module, the spatial and channel information in the image can be extracted more effectively, the quality of the image can be improved, the color distortion and halo phenomena can be reduced, and the effect of image processing can be improved.
Smart Images
Figure CN119624807A_ABST
Abstract
Description
Technical Field
[0001] The invention belongs to the technical field of image processing, and in particular relates to an image enhancement method based on global attention and Retinex theory. Background Art
[0002] The datasets currently used in the field of computer vision are all high-quality images with clear content. However, images taken under natural conditions are affected by the environment, and most images will have partial exposure, uneven lighting, and overall darkness. This will cause the clarity and quality of the image to deteriorate, affecting the performance in the computer vision research process. Therefore, enhancing these images can not only effectively improve the quality and visual effects of the images, but also improve the performance of the methods, such as target detection, tracking, semantic segmentation, and recognition tasks.
[0003] The traditional Retinex method artificially decomposes the image into illumination component and reflection component. This decomposition method is relatively complex and cannot be applied to many scenarios. Deep learning is mainly data-driven for feature decomposition. Based on the learning of a large number of samples, a deep feature representation can be obtained. Its expression of the data set is more efficient and accurate, and the extracted abstract features are more robust and have better generalization ability. Therefore, a novel idea is to combine the Retinex theory with the advantages of deep learning. The low-light image degradation problem can be decomposed into two sub-problems in a Retinex way, namely the adjustment process of the reflection component and the illumination component. This can give full play to the advantages of both, and learn to map low-light images to normal images better and easier.
[0004] The RetinexNet method is based on the Retinex theory. It uses a convolutional neural network to decompose low-light images, uses reference images to adjust and eliminate noise, and generates better quality illumination and reflection components. The LightenNet method uses a convolutional neural network to learn the mapping from low-light images to illumination images and removes the illumination images, using the reflection images as the enhanced results. This will cause some images to be unnatural. The KinD++ method designs a reflection restoration network based on a multi-scale illumination attention module, which is more conducive to dealing with the degradation problem of the decomposed reflection image. However, due to the uncertainty and diversity of low-light environments, the deep learning method based on the Retinex theory still has certain color distortion and halo phenomena when adjusting images.
[0005] With the continuous development of deep learning, the attention mechanism has become popular in computer vision, and a variety of attention structures have been derived. In recent years, some low-light enhancement methods have also introduced the attention mechanism and achieved considerable results.
[0006] In view of the above problems, this paper proposes an image enhancement method based on global attention and Retinex theory. For the first time, an innovative method is proposed to construct a global attention module and integrate it with Retinex theory to learn the diversity and correlation between low-light images and real image features, and embed them into the enhancement network model to enrich the detail information of dark areas and solve the color distortion and halo problems of the Retinex method, thereby improving the effect of image processing. Summary of the invention
[0007] The present invention provides an image enhancement method based on global attention and Retinex theory. A global attention module is constructed to learn the diversity and correlation between low-light images and real image features, and is embedded into an enhancement network model to enrich the detail information of dark areas and solve the color distortion and halo phenomena of images in low-light environments.
[0008] In order to achieve the above object, the present invention solves the above technical problems through the following technical solutions:
[0009] An image enhancement method based on global attention and Retinex theory, including:
[0010] An image decomposition module inputs the image I into the image decomposition module and decomposes it into a reflection image R and an illumination image L;
[0011] The illumination image adjustment module inputs the illumination image L into the illumination image adjustment module for brightness adjustment processing to obtain the illumination image
[0012] The reflection image recovery module inputs the reflection image R into the reflection image recovery module for reflection recovery processing to obtain the reflection image
[0013] The adjusted illumination image and reflected image Multiply to get the enhanced image R E .
[0014] In the image decomposition module, the image classification decomposes a 3*3*32 volume into two branches, one branch contains a 5-layer codec U-net structure for calculating the reflection component image, and the other branch uses 3 layers of convolution and Sigmoid layer for estimating the illumination component. The U-net structure contains the Max pooling method, deconvolution upsampling and downsampling, and the cascade method is used before downsampling.
[0015] The loss function in the image decomposition is:
[0016]
[0017] The function is:
[0018]
[0019] The function is:
[0020] The function is:
[0021] The function is:
[0022] Input represents the input image, refl represents the reflection image, illu represents the illumination image, Represents the corresponding pixel multiplication, and are the gradients of the low-light and high-light illumination components, respectively.
[0023] When adjusting the brightness in the illumination image adjustment module, a lightweight model with four convolution layers and a Sigmoid activation function is set. The pixel value of the Sigmoid function is limited to between 0 and 1, and the dynamic range of the lower pixel value is expanded in a nonlinear manner.
[0024] The input part of the lightweight model mainly has two branches. The first branch is to input the low-light illumination image, and the second branch is based on the Ratio value tensor.
[0025] The Ratio value can be adjusted adaptively, or it can be determined during the model training process by calculating the ratio between the high light and low light illumination image pixels, and the formula is:
[0026]
[0027] The loss function of the adjusted network is:
[0028]
[0029] I h and I adjust represents the highlight and adjusted illumination image, The horizontal and vertical gradients are calculated symbolically, and the mean square error is used to calculate the loss function of the illumination adjustment network.
[0030] The main part of the reflection image restoration module is set up with a 10-layer encoding and decoding U-net structure. The global attention mechanism is calculated once after two convolutions, so that the feature map will amplify the global interaction representation after passing through the global attention mechanism, thereby improving the restoration effect of the model.
[0031] The global attention mechanism consists of two parts, channel attention and spatial attention, and the calculation formulas of the two parts are as follows:
[0032]
[0033] in represents the multiplication between pixels, F1, F2 and F3 represent the input feature map, the feature map after channel attention and the output feature map respectively, Mc and Ms are the channel attention map and the spatial attention map. For a given feature map of any size Channel attention mapped to three dimensions in sequence and 2D spatial attention
[0034] The feature map Merge the height and width into a two-dimensional feature vector After passing through a multilayer perceptron with two hidden layers, the two-dimensional feature vector is separated into a three-dimensional feature map. Finally, the channel feature map is obtained through the Relu function.
[0035] The calculation formula is:
[0036]
[0037] in is the Relu activation function, Separate and Merge are the separation and merging operations of feature vectors respectively.
[0038] The feature map Through two convolution modules, the first convolution module includes convolution layer, batch normalization and Relu activation function, while the second convolution module only contains convolution layer and batch normalization, and then the spatial attention feature map is obtained through the Sigmoid activation function.
[0039] The calculation formula of Mc is:
[0040] M s (F2) = σ(Conv2(Conv1(F2)))
[0041] Where σ is the Sigmoid activation function, and Conv1 and Conv2 are both convolution operations with a filter size of 8×8. The loss function in the reflection recovery network can be expressed as:
[0042]
[0043] in is the restored reflection image, R his the reflection image of the highlight, SSIM is the structural similarity calculation of the image, Calculates the mean square error.
[0044] Based on the implementation of the above technical solution, the present invention can achieve the following technical effects:
[0045] 1. The present invention provides an image enhancement method based on global attention and Retinex theory. By constructing a novel global attention module, the problem of weight reuse of channel weight feature maps at different positions in the same channel is solved. The feature map is adjusted by a transformation method and a fully connected layer. Figure 3 The weights of each dimension strengthen the correlation between the global space and the channel, which is more conducive to promoting reasoning learning.
[0046] 2. The present invention provides an image enhancement method based on global attention and Retinex theory. A reflection restoration network is designed and the global attention module is embedded into different layers of the network to extract richer shallow texture features and deep semantic features. It can not only learn the mapping relationship between the reflection map and the illumination map, but also help to reconstruct the detail information.
[0047] 3. The present invention provides an image enhancement method based on global attention and Retinex theory, which combines Retinex theory with deep learning methods. Compared with existing image processing methods, it effectively improves the quality of the image. However, color distortion and halo phenomena will occur in areas with large differences in image brightness. BRIEF DESCRIPTION OF THE DRAWINGS
[0048] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0049] Figure 1 It is a structural diagram of the method provided by the present invention;
[0050] Figure 2 is a brightness adjustment structure diagram provided by the present invention;
[0051] Figure 3 is a diagram of the global attention restoration network structure provided by the present invention;
[0052] Figure 4 is a global attention structure diagram provided by the present invention;
[0053] Figure 5 It is a channel attention structure diagram provided by the present invention;
[0054] Figure 6 It is a spatial attention structure diagram provided by the present invention;
[0055] Figure 7 This is a comparison chart of the experimental results of different methods in the LOL dataset;
[0056] Figure 8 This is a comparison chart of the experimental results of different methods on the SICE dataset;
[0057] Fig. 9 This is a comparison chart of the enhancement results of different methods under low-light conditions;
[0058] Fig.10 This is a comparison chart of the enhancement results of different methods under night exposure conditions;
[0059] Fig.11 This is a comparison chart of the enhancement results of different methods under extremely dark conditions. DETAILED DESCRIPTION
[0060] The following embodiments of the present invention are described in further detail in conjunction with the accompanying drawings and examples. The following examples are used to illustrate the present invention, but are not intended to limit the scope of the present invention.
[0061] In the description of the embodiments of the present invention, it should be noted that the terms "center", "longitudinal", "lateral", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside" and the like indicate positions or positional relationships based on the positions or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the embodiments of the present invention and simplifying the description, and do not indicate or imply that the devices or elements referred to must have a specific orientation, be constructed and operate in a specific orientation, and therefore cannot be understood as limitations on the embodiments of the present invention. In addition, the terms "first", "second", and "third" are used for descriptive purposes only and cannot be understood as indicating or implying relative importance.
[0062] In the description of the embodiments of the present invention, it should be noted that, unless otherwise clearly specified and limited, the terms "connected" and "connection" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium. For ordinary technicians in this field, the specific meanings of the above terms in the embodiments of the present invention can be understood according to specific circumstances.
[0063] In the embodiments of the present invention, unless otherwise clearly specified and limited, a first feature being "above" or "below" a second feature may mean that the first and second features are in direct contact, or the first and second features are in indirect contact through an intermediate medium. Moreover, a first feature being "above", "above" or "above" a second feature may mean that the first feature is directly above or obliquely above the second feature, or simply means that the first feature is higher in level than the second feature. A first feature being "below", "below" or "below" a second feature may mean that the first feature is directly below or obliquely below the second feature, or simply means that the first feature is lower in level than the second feature.
[0064] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the embodiment of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.
[0065] Combine the following Figure 1-Figure 8 An image enhancement method based on global attention and Retinex theory provided by the present invention is introduced in detail.
[0066] Combination Figure 1 As shown, in an embodiment of the present invention, the image enhancement method based on global attention and Retinex theory is divided into three parts: an image decomposition module, an illumination image adjustment module and a reflection image restoration module.
[0067] First, the input image I is decomposed into a reflection image R and an illumination image L through an image decomposition module;
[0068] Then the illumination image L is input into the illumination image adjustment module for brightness adjustment processing to obtain the illumination image
[0069] At the same time, the reflection image R is input into the reflection image recovery module for reflection recovery processing to obtain the reflection image
[0070] Finally, the adjusted illumination image With reflection image Multiply to get the enhanced result
[0071] The image decomposition module is based on the Retinex theory, which decomposes the input image I into a reflection image R reflecting the properties of the object and an illumination image L containing the illumination information. The reference image and the low-light image are used to calculate the decomposition process in a weight-sharing manner.
[0072] The illumination image adjustment module rebalances the illumination distribution of the illumination image;
[0073] The main purpose of the reflection image restoration module is to restore the detail information of the dark area of the reflection image.
[0074] The image enhancement method provided by the present invention comprises the following three steps:
[0075] 1. Image decomposition
[0076] First, a 3*3*32 convolution is used to divide the decomposition structure into two branches. One branch contains a 5-layer U-net structure for encoding and decoding to calculate the reflection component image, and the other branch uses 3 layers of convolution and Sigmoid layer to estimate the illumination component. The U-net structure contains Max pooling and deconvolution upsampling, which can effectively extract local and global information in the image. In order to avoid the loss of some detail information during the downsampling process, a cascade method is used before downsampling.
[0077] The image decomposition network layer structure parameters are shown in Table 1.
[0078] Table 1 Image decomposition network layer structure
[0079]
[0080]
[0081] The loss function used in the decomposition network is:
[0082]
[0083] The function is:
[0084]
[0085] The function is:
[0086] The function is:
[0087] Function
[0088] Input represents the input image, refl represents the reflection image, illu represents the illumination image, Represents the corresponding pixel multiplication, and The gradients of low-light and high-light illumination components are respectively. The illumination map components of high-light and low-light are consistent in structure. It is considered that the boundary should be retained only when the gradient changes between the two are large, otherwise it should be suppressed.
[0089] 2. Brightness adjustment
[0090] Combination Figure 2 As shown in the figure, the network only needs to adjust the brightness of the image and in order to reduce the computational complexity of the model, a lightweight model with four convolution layers and a Sigmoid activation function is designed. The use of the Sigmoid function not only limits the pixel value to between 0 and 1, but also expands the dynamic range of lower pixel values in a nonlinear manner. The input part of the model mainly has two branches, the first is the input low-light illumination image, and the second is based on the Ratio value tensor. This Ratio value can be adaptively adjusted, and can also be determined during the training process of the model by calculating the ratio between the pixels of the highlight and low-light illumination images. The formula is:
[0091]
[0092] The loss function of the adjusted network is:
[0093]
[0094] Ih and Iadjust represent the highlight and adjusted illumination images, The horizontal and vertical gradients are calculated symbolically, and the mean square error is used to calculate the loss function of the illumination adjustment network.
[0095] 3. Reflection recovery
[0096] Combination Figure 3 The global attention restoration network structure diagram is shown in Figure 1. The reflection restoration network based on the global attention mechanism provides powerful pixel adjustment capabilities, which helps to restore the detailed texture of the dark area of the image. The main part of the model uses a 10-layer encoding and decoding U-net structure. The global attention mechanism is calculated after every two convolutions, so that the feature map will amplify the global interaction representation after passing through the global attention mechanism to improve the restoration effect of the model.
[0097] The most important thing is to convert the illumination image With reflection image Before entering the network, the brightness image Perform a calculation of the spatial attention module. The number of channels of the illumination component is 1, so there is no need to calculate the channel attention module, which can reduce the amount of calculation of the model.
[0098] Then reflect the image Perform a global attention mechanism calculation. Increase the attention to areas with drastic brightness changes and areas with blurred details, and accelerate content learning in these areas. This can avoid adjusting the reflected image The halo and shadow phenomena appear, making the restored reflected image The details are richer and the colors are fuller.
[0099] Combination Figure 4 As shown in the figure, the global attention mechanism proposed in the present invention is derived from BAM, CBAM and GAM. The global attention mechanism mainly includes two parts, channel attention and spatial attention. The calculation formulas of the two parts are as follows:
[0100]
[0101] in represents the multiplication between pixels, F1, F2, and F3 represent the input feature map, the feature map after channel attention, and the output feature map, respectively, Mc and Ms are the channel attention map and the spatial attention map. For a given feature map of any size Channel attention mapped to three dimensions in sequence and 2D spatial attention
[0102] Combination Figure 5 As shown in the figure, the channel attention module, unlike the original method that only has one dimension of channel information, does not take into account the large difference in pixel values in low-light images, which leads to poor detail recovery and loses the interaction between global spatial channels. Therefore, the present invention proposes a new channel attention module to strengthen the interaction and correlation between channels and space, and improve the performance of detail recovery of the model. First, we will get the feature map Merge the height and width into a two-dimensional feature vector After passing through a multilayer perceptron with two hidden layers, the two-dimensional feature vector is separated into a three-dimensional feature map. Finally, the channel feature map is obtained through the Relu function. The calculation formula is:
[0103]
[0104] in is the Relu activation function, Separate and Merge are the separation and merging operations of feature vectors respectively.
[0105] Combination Figure 6 As shown in the figure, the spatial attention module is added in the present invention to capture the spatial relationship between any two pixels in the feature map, aggregate the features by weighted summation and generate a spatial attention map, where the weight is determined by the feature similarity between the two pixel values. Through two convolution modules, the first convolution module includes convolution layer, batch normalization and Relu activation function, while the second convolution module only has convolution layer and batch normalization. Finally, the spatial attention feature map is obtained through the Sigmoid activation function. The calculation formula of Mc is:
[0106] M s (F2) = σ(Conv2(Conv1(F2)))
[0107] Where σ is the Sigmoid activation function, and Conv1 and Conv2 are both convolution operations with a filter size of 8×8. The loss function in the reflection recovery network can be expressed as:
[0108]
[0109] in is the restored reflection image, R h is the reflection image of the highlight, SSIM is the structural similarity calculation of the image, Calculates the mean square error.
[0110] Example
[0111] In order to verify the superiority of the image enhancement method based on global attention and Retinex theory proposed in the present invention, this embodiment selected relatively advanced image processing methods in the past year, such as KinD, KinD++, RetinexNet and other methods, and further compared them with the method of the present invention on the LOL and SICE datasets, respectively, and conducted comparative analysis from the two dimensions of visual effect and quantitative evaluation, highlighting the significant beneficial effects of the present invention.
[0112] (1) Comparison of visual effects
[0113] KinD, KinD++, RetinexNet and the method of the present invention are used for image processing. One representative image is selected from the LOL dataset for display. The results are shown in Figure 2. Figure 7 In the LOL dataset, from Figure 7 (d) shows that the enhancement results of the RetinexNet method have color distortion and noise. The difference in enhancement effect between KinD and KinD++ and the method of the present invention is small, but it is mainly reflected in some details of the image. For example, Figure 7 (b) Figure 7 (c) and Figure 7 In the red box of (e), the method of the present invention restores more contour information of each seat, which is closer to the reference image. Figure 7 (f).
[0114] KinD, KinD++, RetinexNet and the method of the present invention are used for image processing. One representative image is selected from the SIEC dataset for display. The results are shown in Figure 2. Figure 8 In the SICE dataset, the brightness distribution of most images is uneven, such as Figure 8 (a) shows that the image enhanced by the RetinexNet method is unnatural and darkens the image color, for example Figure 8 (d) The white clouds. However, the images enhanced by KinD method have shadows, such as Figure 8 (b) The sky area. Compared with the KinD++ method, the image enhanced by the present invention is closer to the reference image not only in structure but also in color. Figure 8 (f) as shown.
[0115] In order to verify the superiority of the method proposed in this invention, three representative images under low light, night exposure and extreme dark conditions were selected from the test set for testing. The results are shown in the figure. Fig. 9 , Fig.10 and Fig.11 As shown. By observing Fig. 9 It can be found that compared with the RetinexNet method, KinD and KinD++ methods, the method provided by the present invention is closer to the reference image in terms of color and brightness processing. Fig. 9 (f) The present invention has certain advantages in processing image brightness and color, and can obtain better visual effects. At the same time, it can avoid the problem of over-enhancement, making the enhancement result more natural and realistic.
[0116] like Fig.10 In the partially exposed night scene, the brightness of the enhanced image generated by most methods is low, and the visibility of the image is also affected. Fig.10 As can be seen from the red box part and the enlarged part in the lower left corner, the present invention can show more texture features of leaves. In extremely dark environments, most methods are difficult to process effectively, making the brightness of the enhanced result similar to that of the original low-light image, such as Fig.11 However, even in such an extreme environment, the present invention can still restore some detail information without obvious distortion. The enhancement result is shown in Fig.11 (e) as shown.
[0117] 2. Quantitative evaluation
[0118] KinD, KinD++, RetinexNet and the method of the present invention are used for image processing. Tables 2 and 3 show the numerical results obtained by testing the above methods and the method of the present invention on the LOL and SICE datasets, respectively. In the field of signal processing, the Peak Signal Noise Ratio (PSNR) is usually used as an objective image quality indicator. It calculates the ratio between the maximum signal power and the noise power of the image, and the noise power of the image is calculated by the Mean Square Error (MSE). The higher the value of the Peak Signal Noise Ratio, the better the quality of the enhanced image, and the closer it is to the reference image. The value range of the structural similarity (Structural Similarity, SSIM) is between 0 and 1. The larger its value, the more similar the enhanced image is to the real reference image. Lightness Order Error (LOE) is an evaluation index for measuring the naturalness of image enhancement. It is mainly used to measure the closeness of the enhanced image to the reference image in terms of brightness order. The smaller the LOE value, the closer the brightness order of the enhanced image to the reference image, that is, the better the naturalness is maintained.
[0119] Table 2 Quantitative comparison of the SICE dataset in terms of PSNR, SSIM, NIQE, MSE and LOE (ref)
[0120]
[0121] Table 2 shows the evaluation results of the four methods on the SICE data. The method of the present invention still ranks first in the four evaluation indicators of PSNR, SSIM, NIQE and MSE. However, only when LOE is used as a reference indicator, the method of the present invention ranks second on the SICE dataset, lagging behind the KinD++ method.
[0122] Table 3 Quantitative comparison of the LOL dataset in terms of PSNR, SSIM, NIQE, MSE and LOE (ref)
[0123]
[0124] Table 3 shows the evaluation results of the four methods on LOL data. The method of the present invention ranks first in all indicators and is better than the other three methods, which verifies the superiority of the method of the present invention.
[0125] Table 4 Quantitative comparison of NIQE on LIME, NPE, MEF and DICM datasets
[0126]
[0127] Table 4 shows the results of the method proposed in this invention and three existing advanced methods on four different datasets. Since there are no corresponding reference images for the images in these datasets, we can only use the NIQE indicator to evaluate the performance differences between different methods. The smaller the NIQE value, the smaller the distortion of the enhanced image and the better the image quality. The method of this invention won the first place on four different datasets and is still better than other advanced low-light image enhancement methods, proving that our proposed method has strong generalization ability.
[0128] Based on the above comparative analysis, the image enhancement method based on global attention and Retinex theory of the present invention is significantly better than various existing image processing methods in terms of image enhancement effect, whether from the intuitive visual effect analysis or from the quantitative result analysis. The key factors for the success of this invention include: designing a global attention module to extract richer spatial and channel information, and expanding the information collected by the channel attention module to three dimensions; embedding the module in different layers of the network to better restore the texture and color details in the image; joint optimization based on neural networks, including Retinex decomposition, illumination adjustment and reflection recovery. A large number of experiments were carried out under different lighting conditions, especially under extremely dark light conditions, to prove the effectiveness of this method.
[0129] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims
1. An image enhancement method based on global attention and Retinex theory, including: An image decomposition module inputs the image I into the image decomposition module and decomposes it into a reflection image R and an illumination image L; The illumination image adjustment module inputs the illumination image L into the illumination image adjustment module for brightness adjustment processing to obtain the illumination image The reflection image recovery module inputs the reflection image R into the reflection image recovery module for reflection recovery processing to obtain the reflection image The adjusted illumination image and reflected image Multiply to get the enhanced image R E .
2. The image enhancement method according to claim 1, characterized in that: In the image decomposition module, the image classification decomposes a 3*3*32 volume into two branches, one branch contains a 5-layer codec U-net structure for calculating the reflection component image, and the other branch uses 3 layers of convolution and Sigmoid layer for estimating the illumination component. The U-net structure contains the Max pooling method, deconvolution upsampling and downsampling, and the cascade method is used before downsampling.
3. The image enhancement method according to claim 2, characterized in that: The loss function in the image decomposition is: The function is: The function is: The function is: The function is: Input represents the input image, refl represents the reflection image, illu represents the illumination image, Represents the corresponding pixel multiplication, and are the gradients of the low-light and high-light illumination components, respectively.
4. The image enhancement method according to claim 1, characterized in that: When adjusting the brightness in the illumination image adjustment module, a lightweight model with four convolution layers and a Sigmoid activation function is set. The pixel value of the Sigmoid function is limited to between 0 and 1, and the dynamic range of the lower pixel value is expanded in a nonlinear manner.
5. The image enhancement method according to claim 4, characterized in that: The input part of the lightweight model mainly has two branches. The first branch is to input the low-light illumination image, and the second branch is based on the Ratio value tensor.
6. The image enhancement method according to claim 5, characterized in that: The Ratio value can be adjusted adaptively, or it can be determined during the model training process by calculating the ratio between the high light and low light illumination image pixels, and the formula is: The loss function of the adjusted network is: I h and I adjust represents the highlight and adjusted illumination image, The horizontal and vertical gradients are calculated symbolically, and the mean square error is used to calculate the loss function of the illumination adjustment network.
7. The image enhancement method according to claim 1, characterized in that: The main part of the reflection image restoration module is set up with a 10-layer encoding and decoding U-net structure. The global attention mechanism is calculated once after two convolutions, so that the feature map will amplify the global interaction representation after passing through the global attention mechanism, thereby improving the restoration effect of the model.
8. The image enhancement method according to claim 7, characterized in that: The global attention mechanism consists of two parts, channel attention and spatial attention, and the calculation formulas of the two parts are as follows: in represents the multiplication between pixels, F1, F2 and F3 represent the input feature map, the feature map after channel attention and the output feature map respectively, Mc and Ms are the channel attention map and the spatial attention map. For a given feature map of any size Channel attention mapped to three dimensions in sequence and 2D spatial attention 9. The image enhancement method according to claim 8, characterized in that: The feature map Merge the height and width into a two-dimensional feature vector After passing through a multilayer perceptron with two hidden layers, the two-dimensional feature vector is separated into a three-dimensional feature map. Finally, the channel feature map is obtained through the Relu function. The calculation formula is: in is the Relu activation function, Separate and Merge are the separation and merging operations of feature vectors respectively.
10. The image enhancement method according to claim 9, characterized in that: The feature map Through two convolution modules, the first convolution module includes convolution layer, batch normalization and Relu activation function, while the second convolution module only includes convolution layer and batch normalization. Then the spatial attention feature map is obtained through the Sigmoid activation function. The calculation formula of Mc is: M s (F2)=σ(Conv2(Conv1(F2))) Where σ is the Sigmoid activation function, and both Conv1 and Conv2 are convolution operations with a filter size of 8×8. The loss function in the reflection recovery network can be expressed as: in is the restored reflection image, R h is the reflection image of the highlight, SSIM is the structural similarity calculation of the image, Calculates the mean square error.