Low-illumination image enhancement method and device based on improved normalized flow

Abstract

The embodiment of the application discloses a low-illumination image enhancement method and device based on an improved normalized flow, which comprises the following steps: acquiring a low-illumination image to be enhanced; performing normalization processing on the low-illumination image to be enhanced to obtain a normalized spectrum; performing edge extraction processing on the normalized spectrum to obtain an edge spectrum; performing feature fusion on the low-illumination image to be enhanced, the normalized spectrum and the edge spectrum to obtain a feature fusion image; performing feature extraction on the feature fusion image by using a feature extraction module to obtain a feature extraction image; performing feature mapping on the feature extraction image by using a feature mapping module to obtain a feature mapping image; and inputting the feature mapping image into a normalized flow reversible network to output a normal-illumination image. The low-illumination image is enhanced by the method, and the enhancement effect can be improved.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a low-light image enhancement method and apparatus based on improved normalized flow. Background Technology

[0002] Due to limitations in lighting conditions, low-light images often suffer from low contrast, high noise levels, and loss of detail. These problems severely impact the visual quality of the images and the accuracy of subsequent image analysis tasks. Therefore, it is necessary to enhance low-light images to improve their quality and usability.

[0003] Traditional image enhancement methods, such as histogram equalization and gamma correction, while improving the visual effect of images to some extent, often have many limitations. For example, histogram equalization may introduce artifacts, while simple gamma correction cannot effectively recover image details. Furthermore, these methods are mostly based on the global statistical properties of images, making it difficult to handle local details and regional features, especially when processing high dynamic range images, where it is difficult to balance highlight and shadow details. With the development of deep learning technology, methods based on deep neural networks have made significant progress in low-light image enhancement. However, existing deep learning-based methods, such as those based on convolutional neural networks (CNNs), while able to learn complex mapping relationships in images, still face some challenges. For example, CNNs may have limitations in processing detailed features, leading to loss of detail or increased artifacts in the enhanced image. Summary of the Invention

[0004] Therefore, it is necessary to propose a low-light image enhancement method and apparatus based on improved normalized flow to address the above problems.

[0005] First aspect

[0006] A low-light image enhancement method based on improved normalized flow, the method comprising:

[0007] Acquire the low-light image to be enhanced;

[0008] The low-light image to be enhanced is normalized to obtain a normalized spectrum;

[0009] The normalized spectrum is subjected to edge extraction processing to obtain the edge spectrum;

[0010] The low-light image to be enhanced, the normalized spectrum, and the edge spectrum are fused to obtain a feature-fused image;

[0011] The feature extraction module is used to extract features from the feature fusion image to obtain a feature-extracted image; wherein, the feature extraction module includes a two-dimensional convolutional layer, a max pooling layer, and residual dense blocks;

[0012] The feature extraction image is mapped using a feature mapping module to obtain a feature mapping image; wherein, the feature mapping module includes a two-dimensional convolutional layer and a grouped cross attention block, and the grouped cross attention block includes a grouping module, an average pooling layer, a feature fusion layer, a two-dimensional convolutional layer, a weight redistribution layer, and a group normalization layer;

[0013] The feature-mapped image is input into a normalized flow reversible network, which outputs a normalized illumination image; wherein the normalized flow reversible network includes multiple flow steps and a two-dimensional compression layer.

[0014] Preferably, the normalization process specifically includes:

[0015]

[0016] In the formula, Represents the normalized spectrum. This represents a low-light image that needs enhancement. This represents the 1-norm of the low-light image to be enhanced.

[0017] Preferably, the edge extraction process specifically includes:

[0018]

[0019] In the formula, Indicates the edge spectrum, This represents the gradient value of the normalized spectrum in the horizontal direction. This represents the gradient value of the normalized spectrum in the vertical direction.

[0020] Preferably, the step of using a feature extraction module to extract features from the feature fusion image to obtain a feature-extracted image specifically includes:

[0021] The feature fusion image is sequentially input into the first two-dimensional convolutional layer and the second two-dimensional convolutional layer, and the first convolutional image is output.

[0022] The first convolutional image is sequentially input into a max pooling layer and a residual dense block, and the first image is output.

[0023] The first convolutional image and the first image are summed to obtain the feature extraction image.

[0024] Preferably, the step of using a feature mapping module to perform feature mapping on the feature-extracted image to obtain a feature-mapped image specifically includes:

[0025] The feature-extracted image is upsampled to obtain the upsampled image;

[0026] The upsampled image is input into the third two-dimensional convolutional layer, which outputs the first feature map image;

[0027] The extracted feature image is input into the first grouped cross-attention block, and the second feature map image is output.

[0028] The feature-extracted image is downsampled to obtain a first downsampled image;

[0029] The first downsampled image is input into the fourth two-dimensional convolutional layer, and the second image is output.

[0030] The second image is input into the second grouped cross-attention block, and the third feature map image is output.

[0031] The second image is downsampled to obtain a second downsampled image;

[0032] The second downsampled image is sequentially input into the fifth two-dimensional convolutional layer and the third grouped cross attention block, and the fourth feature map image is output.

[0033] Preferably, the step of inputting the feature-extracted image into the first grouped cross-attention block and outputting the second feature map image specifically includes:

[0034] The feature extraction images are grouped to obtain the first group image, the second group image, and the third group image;

[0035] The second grouped image is subjected to horizontal average pooling to obtain the second pooled image;

[0036] The third grouped image is subjected to average pooling in the vertical direction to obtain the third pooled image;

[0037] The second pooling image and the third pooling image are stitched together to obtain a stitched image;

[0038] The stitched image is input into the sixth two-dimensional convolutional layer to obtain the third image;

[0039] The weights of the third image and the first group of images are redistributed to obtain the redistributed image;

[0040] The redistributed image is then subjected to group normalization to obtain a normalized image;

[0041] The normalized image is input into the first average pooling layer, and the fourth image is output.

[0042] The first group of images is input into the seventh two-dimensional convolutional layer to obtain the fifth image;

[0043] Multiply the fourth and fifth images to obtain the sixth image;

[0044] The fifth image is input into the second average pooling layer, and the seventh image is output.

[0045] Multiply the seventh image and the normalized image to obtain the eighth image;

[0046] The weights of the first grouped images, the sixth image, and the eighth image are redistributed to output the second feature map image.

[0047] Preferably, the flow step includes activation normalization, a 1×1 reversible convolutional layer, and an affine coupling transform layer.

[0048] Preferably, the normalized flow reversible network further includes a two-dimensional decompression layer.

[0049] Preferably, the step of inputting the feature mapping image into a normalized flow reversible network and outputting a normal illumination image specifically includes:

[0050] The feature map image is sequentially input into the streaming step and the two-dimensional compression layer, and the output is a normal illumination image.

[0051] Second aspect

[0052] A low-light image enhancement device based on improved normalized flow, the device comprising:

[0053] Low-light image acquisition module, used to acquire low-light images to be enhanced;

[0054] The normalized spectrum acquisition module is used to normalize the low-light image to be enhanced to obtain a normalized spectrum.

[0055] The edge spectrum acquisition module is used to perform edge extraction processing on the normalized spectrum to obtain the edge spectrum;

[0056] The feature fusion image acquisition module is used to perform feature fusion on the low-light image to be enhanced, the normalized spectrum, and the edge spectrum to obtain a feature fusion image;

[0057] The feature extraction image acquisition module is used to extract features from the feature fusion image using the feature extraction module to obtain a feature extraction image; wherein, the feature extraction module includes a two-dimensional convolutional layer, a max pooling layer, and residual dense blocks;

[0058] The feature mapping image acquisition module is used to perform feature mapping on the feature-extracted image using the feature mapping module to obtain a feature mapping image; wherein, the feature mapping module includes a two-dimensional convolutional layer and a grouped cross attention block, and the grouped cross attention block includes a grouping module, an average pooling layer, a feature fusion layer, a two-dimensional convolutional layer, a weight redistribution layer, and a group normalization layer;

[0059] A normal illumination image acquisition module is used to input the feature mapping image into a normalized flow reversible network and output a normal illumination image; wherein, the normalized flow reversible network includes multiple flow steps and a two-dimensional compression layer.

[0060] The embodiments of the present invention have the following beneficial effects:

[0061] This invention utilizes normalization and edge extraction techniques to preprocess and enhance input images. This step not only enhances the image's detail but also highlights its edge information, providing a solid foundation for subsequent image processing stages. In this way, the invention significantly improves the visual quality and structural integrity of images. Furthermore, this invention improves upon traditional flow-based generative models by introducing a multi-scale attention mechanism encoder, which effectively extracts depth features from low-light images. This feature extraction method captures essential details of the image at different scales, thereby enhancing image quality during image generation and achieving clearer, more detailed image output. Attached Figure Description

[0062] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0063] in:

[0064] Figure 1 This is a flowchart of a low-light image enhancement method based on improved normalized flow in one embodiment;

[0065] Figure 2 A diagram of an improved normalized flow low-light enhancement network structure in one embodiment;

[0066] Figure 3 This is a visualization of some intermediate results in one embodiment;

[0067] Figure 4 This is a block diagram of a low-light image enhancement device based on improved normalized flow in one embodiment. Detailed Implementation

[0068] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0069] Figure 1 This is a flowchart of a low-light image enhancement method based on improved normalized flow in one embodiment. Figure 1 As shown, in one embodiment, a low-light image enhancement method based on improved normalized flow is provided. The method specifically includes steps S1100~S1700:

[0070] S1100, acquire the low-light image to be enhanced.

[0071] Low-light image enhancement technology is an important research direction in the field of image processing, aiming to improve the image quality captured in low-light environments.

[0072] Image data is inherently discrete because it consists of a fixed number of pixels, each with a finite value, and can therefore be described using a discrete distribution. However, despite the discrete nature of images, it is generally more convenient and flexible to model image data using continuous distributions. It is a continuous probability distribution that can theoretically fit any distribution, where It is usually a standard Gaussian distribution, and It can be any conditional distribution.

[0073] In low-light image enhancement, our goal is to use a set of low-light... and normal illumination version Paired image set To estimate the conditional distribution parameters This estimation is typically performed through maximum likelihood estimation, which involves solving equations.

[0074]

[0075] However, due to continuous conditional distribution It is usually in integral form, and whether the above equation can be solved is unknown.

[0076] Normalized flow introduces an invertible neural network To parameterize , Map the image to an intermediate variable For each input In the potential space, there is a unique Conversely, the reverse is also true, which ensures that the network is strictly reversible, even under normal lighting conditions. Then it can be done (i.e., inverse network) generation. Through the latent space Define a simple distribution (Example: Standard Gaussian distribution), and using a reversible network. By mapping samples from this distribution to the data space, we can calculate using the following simple variable substitution. :

[0077]

[0078] Establish After calculating the formula, we introduce the negative log-likelihood as a model parameter. Loss function:

[0079]

[0080] Meanwhile, in order to further improve computational efficiency and stability, Decomposed into N One reversible layer Let the output of the nth layer be denoted as Considering the input and output, we can obtain , This transformation is written as:

[0081]

[0082]

[0083] After introducing the chain rule, the negative log-likelihood can be written as:

[0084]

[0085] Figure 2 This is a diagram of an improved normalized flow low-light enhancement network structure in one embodiment. The low-light enhancement network designed in this embodiment mainly consists of two main components: a conditional attention encoder, which first processes the low-light image... Processing is performed to produce enhanced intermediate results. And a reversible flow network that will process intermediate result images. Projected into potential space Then, guided by the Ground Truth image, a normal lighting image is reconstructed from this space. That is, the output.

[0086] The steps of this embodiment will be described in detail below with reference to the accompanying drawings:

[0087] S1200: Normalize the low-light image to be enhanced to obtain the normalized spectrum.

[0088] The input image is processed using a conditional attention encoder: such as Figure 2 As shown, the conditional attention encoder is primarily used to extract image features and generate multi-scale results to guide image generation by the flow network. The first step of this encoder is to process low-light images... Normalization is performed to ensure that pixel values ​​are within a continuous range. This step can be written as:

[0089]

[0090] In the formula, Represents the normalized spectrum. This represents a low-light image that needs enhancement. This represents the 1-norm of the low-light image to be enhanced.

[0091] S1300 performs edge extraction processing on the normalized spectrum to obtain the edge spectrum.

[0092] After image normalization, the next step is to calculate the edges within the image to highlight areas of rapid intensity change. This is done at each pixel by calculating its gradient values ​​in the horizontal and vertical directions, i.e., calculating the difference between adjacent pixels. The magnitude of the gradient represents the edge intensity, while the direction of the gradient indicates the edge orientation, expressed as:

[0093]

[0094] In the formula, Indicates the edge spectrum, This represents the gradient value of the normalized spectrum in the horizontal direction. This represents the gradient value of the normalized spectrum in the vertical direction.

[0095] Figure 3 This is a visualization of some intermediate results from one embodiment. The visualizations of the normalized spectrum and edge spectrum are as follows: Figure 3 As shown.

[0096] S1400 performs feature fusion on the low-light image to be enhanced, the normalized spectrum, and the edge spectrum to obtain a feature fused image.

[0097] After obtaining the normalized spectrum and edge spectrum, these features are concatenated with the original low-light image along the channel dimension to obtain a feature fusion image. In this way, the feature dimension of the image is increased, providing richer information for subsequent feature extraction and image enhancement.

[0098] S1500 uses a feature extraction module to extract features from the feature fusion image to obtain a feature-extracted image.

[0099] Next, the feature extraction process is further refined using Residual Dense Blocks (RRDBs), which enhance the model’s ability to capture complex image structures through dense connections and residual learning.

[0100] In some examples, the feature extraction module includes two-dimensional convolutional layers, max pooling layers, and residual dense blocks.

[0101] In some examples, S1500 uses a feature extraction module to extract features from the feature fusion image to obtain a feature-extracted image, specifically including S1501~S1503:

[0102] S1501, the feature fusion image is sequentially input into the first two-dimensional convolutional layer (1) and the second two-dimensional convolutional layer (2), and the first convolutional image (3) is output.

[0103] S1502, the first convolutional image (3) is sequentially input into the max pooling layer (4) and the residual dense block (5), and the first image (6) is output.

[0104] S1503, sum the first convolutional image (3) and the first image (6) to obtain the feature extraction image (7).

[0105] S1600 uses a feature mapping module to perform feature mapping on the feature-extracted image to obtain a feature-mapped image.

[0106] Grouped cross-attention (PCxA) refines feature maps by emphasizing prominent channels and suppressing less information-rich channels, thereby improving the encoder's sensitivity to relevant image attributes.

[0107] The conditional attention encoder finally outputs a processed intermediate result. and its multiple downsampling features , , These features are designed to serve different flow steps within the network, corresponding to their downsampling scales. This design ensures that each step of the network has access to the corresponding level of detail features to guide the generation of enhanced images. The four images, each corresponding to a different feature level and resolution, will be fed into different layers of the normalized flow reversible network, primarily for feature fusion to improve detail and quality.

[0108] In some examples, the feature mapping module includes a two-dimensional convolutional layer and a grouped cross attention block, which includes a grouping module, an average pooling layer, a feature fusion layer, a two-dimensional convolutional layer, a weight redistribution layer, and a group normalization layer.

[0109] The input feature map is divided into multiple groups along the channel dimension. The purpose of grouping is to apply different processing strategies to different channel groups to highlight important features and suppress unimportant information.

[0110] The feature maps of each group are processed through different paths. X-average pooling and Y-average pooling correspond to average pooling in the horizontal and vertical directions, respectively, and then they are concatenated.

[0111] After grouping, the weights of the feature maps are redistributed to adjust the contributions between different groups. This involves the learned weight matrices, which are adjusted according to the importance of the features. This is specifically implemented using torch.matmul and reshape.

[0112] In some examples, S1600 uses a feature mapping module to perform feature mapping on the feature-extracted image to obtain a feature-mapped image, specifically including S1601~S1608:

[0113] S1601, upsample the feature extraction image (7) to obtain the upsampled image (8).

[0114] S1602, input the upsampled image (8) into the third two-dimensional convolutional layer (9) and output the first feature map image (10).

[0115] S1603, input the feature extraction image (7) into the first grouped cross attention block (11) and output the second feature mapping image (12).

[0116] In S1603, the feature extraction image is input into the first grouped cross-attention block, and the second feature mapping image is output, specifically including S101~S113:

[0117] S101, the feature extraction image (7) is grouped to obtain the first group image (701), the second group image (702) and the third group image (703).

[0118] S102, perform horizontal average pooling on the second grouped image (702) to obtain the second pooled image (704).

[0119] S103, perform vertical average pooling on the third group image (703) to obtain the third pooled image (705).

[0120] S104, the second pooling image (704) and the third pooling image (705) are stitched together to obtain the stitched image (706).

[0121] S105, input the stitched image (706) into the sixth two-dimensional convolutional layer (707) to obtain the third image (708).

[0122] S106, the weights of the third image (708) and the first group image (701) are redistributed to obtain the redistributed image (709).

[0123] S107, the redistributed image (709) is subjected to group normalization processing to obtain the normalized image (710).

[0124] S108, the normalized image (710) is input into the first average pooling layer (711) and the fourth image (712) is output.

[0125] S109, input the first group image (701) into the seventh two-dimensional convolutional layer (713) to obtain the fifth image (714).

[0126] S110, multiply the fourth image (712) and the fifth image (714) to obtain the sixth image (715).

[0127] S111, the fifth image (714) is input into the second average pooling layer (716) and outputs the seventh image (717).

[0128] S112, multiply the seventh image (717) and the normalized image (710) to obtain the eighth image (718).

[0129] S113, the weights of the first group image (701), the sixth image (715) and the eighth image (718) are redistributed, and the second feature map image (12) is output.

[0130] S1604, the feature extraction image (7) is downsampled to obtain the first downsampled image (13).

[0131] S1605, input the first downsampled image (13) into the fourth two-dimensional convolutional layer (14) and output the second image (15).

[0132] S1606, input the second image (15) into the second group cross attention block (16) and output the third feature map image (17).

[0133] S1607, the second image (15) is downsampled to obtain the second downsampled image (18).

[0134] S1608, the second downsampled image (18) is sequentially input into the fifth two-dimensional convolutional layer (19) and the third grouped cross attention block (20), and the fourth feature map image (21) is output.

[0135] S1700 inputs the feature map image into the normalized flow reversible network and outputs a normal illumination image.

[0136] like Figure 2 As shown in the network on the right, the normalized flow reversible network consists of several flow steps and a compression (decompression) layer. The flow steps include activation normalization, 1×1 reversible convolution, and affine coupling transformation.

[0137] Activation normalization is used to ensure that the activation values ​​of each layer of a neural network maintain a standardized distribution. Specifically, the goal is to make the mean zero and the variance one. By stabilizing the distribution of activation values, the activation normalization layer reduces the possibility of gradient vanishing or gradient exploding.

[0138] 1×1 reversible convolution classes use 1×1 convolutions to implement permutations. In reversible neural networks, especially in normalized flow, a permutation refers to a data rearrangement that is both reversible and differentiable. Weight matrix use Decompose the initialization to ensure orthogonality. An orthogonal matrix has the property that its inverse is equal to its transpose, simplifying the inverse transformation process. In forward propagation, the input and weight matrix... Convolution; during backpropagation, The reverse It can be derived from the formula Calculation, where for The determinant of .

[0139] The affine coupling layer is a key component in the normalized flow architecture, facilitating the modeling of complex distributions through invertible transformations. This layer transforms the input data... Divided into two parts and Then to This is achieved using affine transformation, i.e. The transformation is by Determined. This transformation can be expressed mathematically as:

[0140]

[0141] in, Indicates the result of the transformation. It is a scaling factor. It is the translation factor. This is conditional information. The transformation is clearly reversible, expressed as:

[0142]

[0143] The affine coupling layer design ensures the invertibility and differentiability of the transformation, enabling end-to-end training using gradient descent-based optimization. It includes conditional information. It provides flexibility to adapt to different contexts, enhancing the model's ability to capture diverse data patterns.

[0144] Compression (decompression) layers serve as an effective dimensionality reduction and feature enhancement mechanism within the normalized flow framework. They are used to reduce the spatial dimension of input data while expanding its channel dimension. During forward propagation, this layer compresses the input using a squeeze2d function, while during backpropagation, it restores the original spatial dimension using an unsqueeze2d function. This helps the model capture and represent features more effectively when processing image data, while reducing computational burden.

[0145] The ground truth image serves as a reference standard to guide the network in generating images that more closely resemble real-world lighting conditions. Within the network, the ground truth image is used to evaluate the difference between the generated image and the real image. The specific process involves comparing the ground truth image with the network-generated image, calculating the loss, and then updating the network parameters through backpropagation to progressively reduce the difference between the generated and real images.

[0146] In S1700, the feature map image is input into a normalized flow reversible network, and the output is a normal illumination image, specifically including:

[0147] The feature map image is sequentially input into the streaming step and the two-dimensional compression layer, and the output is a normal illumination image.

[0148] This embodiment utilizes normalization and edge extraction techniques to preprocess and enhance the input image. This step not only enhances the image's detail but also highlights its edge information, providing a solid foundation for subsequent image processing stages. In this way, this embodiment can significantly improve the visual quality and structural integrity of the image. Furthermore, this embodiment improves upon traditional flow-based generative models by introducing a multi-scale attention mechanism encoder, which effectively extracts depth features from low-light images. This feature extraction method captures essential details of the image at different scales, thereby enhancing image quality during image generation and achieving a clearer, more detailed image output.

[0149] Figure 4 This is a structural block diagram of a low-light image enhancement device based on improved normalized flow in one embodiment. Figure 4 As shown, the device includes:

[0150] The low-light image acquisition module 4100 is used to acquire the low-light image to be enhanced.

[0151] The normalized spectrum acquisition module 4200 is used to normalize the low-light image to be enhanced to obtain the normalized spectrum.

[0152] The edge spectrum acquisition module 4300 is used to perform edge extraction processing on the normalized spectrum to obtain the edge spectrum.

[0153] The feature fusion image acquisition module 4400 is used to perform feature fusion on the low-light image to be enhanced, the normalized spectrum, and the edge spectrum to obtain a feature fusion image.

[0154] The feature extraction image acquisition module 4500 is used to extract features from the feature fusion image using the feature extraction module to obtain the feature extraction image; wherein, the feature extraction module includes a two-dimensional convolutional layer, a max pooling layer and residual dense blocks.

[0155] The feature mapping image acquisition module 4600 is used to perform feature mapping on the feature extraction image using the feature mapping module to obtain the feature mapping image; wherein, the feature mapping module includes a two-dimensional convolutional layer and a grouped cross attention block, and the grouped cross attention block includes a grouping module, an average pooling layer, a feature fusion layer, a two-dimensional convolutional layer, a weight redistribution layer and a group normalization layer.

[0156] A normal illumination image (4700) is acquired and used to input the feature map image into a normalized flow reversible network, which outputs a normal illumination image. The normalized flow reversible network includes multiple flow steps and a two-dimensional compression layer.

[0157] It should be noted that the above description of the low-light image enhancement device based on improved normalized flow is similar to the method description above and has the same beneficial effects as the method embodiments. For any technical details not disclosed in the embodiments of the low-light image enhancement device based on improved normalized flow of the present invention, those skilled in the art should refer to the description of the method embodiments of the present invention for understanding.

[0158] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments described above. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and RAMbus dynamic RAM (RDRAM), etc.

[0159] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0160] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. A low-light image enhancement method based on improved normalized flow, characterized in that, The method includes: Acquire the low-light image to be enhanced; The low-light image to be enhanced is normalized to obtain a normalized spectrum; the normalization process specifically includes: In the formula, Represents the normalized spectrum. This represents a low-light image that needs enhancement. Represents the 1-norm of the low-light image to be enhanced; The normalized spectrum is subjected to edge extraction processing to obtain the edge spectrum; the edge extraction processing specifically includes: In the formula, Indicates the edge spectrum, This represents the gradient value of the normalized spectrum in the horizontal direction. This represents the gradient value of the normalized spectrum in the vertical direction; The low-light image to be enhanced, the normalized spectrum, and the edge spectrum are subjected to channel-dimensional deep feature fusion to obtain a feature fusion image; The feature extraction module is used to extract features from the feature fusion image to obtain a feature-extracted image; wherein, the feature extraction module includes a two-dimensional convolutional layer, a max pooling layer, and residual dense blocks; The feature extraction image is mapped using a feature mapping module to obtain a feature mapping image; wherein, the feature mapping module includes a two-dimensional convolutional layer and a grouped cross attention block, and the grouped cross attention block includes a grouping module, an average pooling layer, a feature fusion layer, a two-dimensional convolutional layer, a weight redistribution layer, and a group normalization layer; The feature-mapped image is input into a normalized flow reversible network, which outputs a normal illumination image. The normalized flow reversible network includes multiple flow steps and a two-dimensional compression layer. The flow steps sequentially include activation normalization, a 1×1 reversible convolutional layer, and an affine coupling transformation layer. The normalized flow reversible network also includes a two-dimensional decompression layer, which works in conjunction with the two-dimensional decompression layer to achieve adaptive transformation between the feature dimension and the spatial dimension.

2. The low-light image enhancement method based on improved normalized flow according to claim 1, characterized in that, The step of using a feature extraction module to extract features from the feature fusion image to obtain a feature-extracted image specifically includes: The feature fusion image is sequentially input into the first two-dimensional convolutional layer and the second two-dimensional convolutional layer, and the first convolutional image is output. The first convolutional image is sequentially input into a max pooling layer and a residual dense block, and the first image is output. The first convolutional image and the first image are summed to obtain the feature extraction image.

3. The low-light image enhancement method based on improved normalized flow according to claim 1, characterized in that, The step of using a feature mapping module to perform feature mapping on the feature-extracted image to obtain a feature-mapped image specifically includes: The feature-extracted image is upsampled to obtain the upsampled image; The upsampled image is input into the third two-dimensional convolutional layer, which outputs the first feature map image; The extracted feature image is input into the first grouped cross-attention block, and the second feature map image is output. The feature-extracted image is downsampled to obtain a first downsampled image; The first downsampled image is input into the fourth two-dimensional convolutional layer, and the second image is output. The second image is input into the second grouped cross-attention block, and the third feature map image is output. The second image is downsampled to obtain a second downsampled image; The second downsampled image is sequentially input into the fifth two-dimensional convolutional layer and the third grouped cross attention block, and the fourth feature map image is output.

4. The low-light image enhancement method based on improved normalized flow according to claim 3, characterized in that, The step of inputting the feature-extracted image into the first grouped cross-attention block and outputting the second feature map image specifically includes: The feature extraction images are grouped to obtain the first group image, the second group image, and the third group image; The second grouped image is subjected to horizontal average pooling to obtain the second pooled image; The third grouped image is subjected to average pooling in the vertical direction to obtain the third pooled image; The second pooling image and the third pooling image are stitched together to obtain a stitched image; The stitched image is input into the sixth two-dimensional convolutional layer to obtain the third image; The weights of the third image and the first grouped images are redistributed to obtain the redistributed image; The redistributed image is then subjected to group normalization to obtain a normalized image; The normalized image is input into the first average pooling layer, and the fourth image is output. The first group of images is input into the seventh two-dimensional convolutional layer to obtain the fifth image; Multiply the fourth and fifth images to obtain the sixth image; The fifth image is input into the second average pooling layer, and the seventh image is output. Multiply the seventh image and the normalized image to obtain the eighth image; The weights of the first grouped images, the sixth image, and the eighth image are redistributed to output the second feature map image.

5. The low-light image enhancement method based on improved normalized flow according to claim 1, characterized in that, The step of inputting the feature-mapped image into a normalized flow reversible network and outputting a normal illumination image specifically includes: The feature map image is sequentially input into the streaming step and the two-dimensional compression layer, and the output is a normal illumination image.

6. A low-light image enhancement device based on improved normalized flow, characterized in that, The device includes: Low-light image acquisition module, used to acquire low-light images to be enhanced; The normalized spectrum acquisition module is used to normalize the low-light image to be enhanced to obtain a normalized spectrum; the normalization process specifically includes: In the formula, Represents the normalized spectrum. This represents a low-light image that needs enhancement. Represents the 1-norm of the low-light image to be enhanced; The edge spectrum acquisition module is used to perform edge extraction processing on the normalized spectrum to obtain the edge spectrum; the edge extraction processing specifically includes: In the formula, Indicates the edge spectrum, This represents the gradient value of the normalized spectrum in the horizontal direction. This represents the gradient value of the normalized spectrum in the vertical direction; The feature fusion image acquisition module is used to perform channel-dimensional deep feature fusion on the low-light image to be enhanced, the normalized spectrum, and the edge spectrum to obtain a feature fusion image. The feature extraction image acquisition module is used to extract features from the feature fusion image using the feature extraction module to obtain a feature extraction image; wherein, the feature extraction module includes a two-dimensional convolutional layer, a max pooling layer, and residual dense blocks; The feature mapping image acquisition module is used to perform feature mapping on the feature-extracted image using the feature mapping module to obtain a feature mapping image; wherein, the feature mapping module includes a two-dimensional convolutional layer and a grouped cross attention block, and the grouped cross attention block includes a grouping module, an average pooling layer, a feature fusion layer, a two-dimensional convolutional layer, a weight redistribution layer, and a group normalization layer; A normal illumination image acquisition module is used to input the feature mapping image into a normalized flow reversible network and output a normal illumination image. The normalized flow reversible network includes multiple flow steps and a two-dimensional compression layer. Each flow step sequentially includes activation normalization, a 1×1 reversible convolutional layer, and an affine coupling transformation layer. The normalized flow reversible network also includes a two-dimensional decompression layer, which, in conjunction with the two-dimensional decompression layer, achieves adaptive transformation between the feature dimension and the spatial dimension.

Images