Dark light image enhancement method and system based on normalized flow model

Abstract

The application provides a dark light image enhancement method and system based on a normalized flow model, and belongs to the technical field of computer vision.The application provides a conditional normalized flow network model based on a transformer, the model uses image structure information as a condition, and realizes image decomposition and reconstruction through a reversible normalized flow.The normal light image is decomposed into multi-scale conditional features and hidden features through the reversible conditional normalized flow network.The high-dimensional conditional features more fully express the image structure information irrelevant to light, and the hidden features conforming to a normal distribution express the light features of the normal light image in a simple form.The image is decomposed and recombined in the feature space, and the method can restore a more real image enhancement result in light and color.

Description

TECHNICAL FIELD

[0001] The application relates to a normalized flow model-based dark light image enhancement method and system and belongs to the technical field of computer vision. BACKGROUND

[0002] Dark light image enhancement is a very key task in the low-level vision field. Traditional dark light image enhancement methods include histogram equalization, gamma correction and methods based on the Retinex theory. These early-stage methods are difficult to handle the complex image degradation in the real-world low-light environment and produce unnatural artifacts. With the development of deep learning, more and more methods begin to use a data-driven way to learn dark light image enhancement in a supervised manner.

[0003] At present, supervised dark light enhancement methods are generally divided into two categories:

[0004] The first category is an end-to-end direct learning method. This kind of method designs different structures of neural networks to learn one-to-one mapping from dark light images to normal light images. However, due to the lack of light, there is serious noise and distortion of image color in the dark light image, which makes it difficult to learn the dark light enhancement task with a single model and it is difficult to recover multiple degradations in the dark light image.

[0005] The second category is a dark light image enhancement method based on image decomposition. This kind of method decomposes the image into image structure and light information, and designs different models to recover the two kinds of information. Among them, the mainstream method is to use a deep neural network to decompose the image into a reflection map and a light map based on the Retinex theory, and some are to decompose the structure and light of the image in the frequency domain. However, these decompositions do not utilize the multi-scale information in the image, making it difficult to balance the local texture information and the global semantic and light information in the image. In addition, this kind of method usually uses pixel-level constraints to explicitly decompose the image, which leads to unnatural artifacts in the recovered image, thereby affecting the visual effect. SUMMARY

[0006] The purpose of the present application is to effectively solve the technical problems of low contrast, color distortion and complex noise difficult to recover in dark light images, and unnatural artifacts produced by existing dark light enhancement methods, and creatively propose a normalized flow model-based dark light image enhancement method and system.

[0007] The present application decomposes normal light images into multi-scale conditional features and implicit features by reversible conditional normalization flow network. The high-dimensional conditional features more fully express the illumination-independent image structure information, and the implicit features conforming to normal distribution express the illumination features of normal light images in a simple form. The image is implicitly decomposed and recombined in the feature space, and the method can restore the image enhancement result with more realistic illumination and color.

[0008] The innovation of the present application is that a conditional normalization flow network model based on Transformer is first proposed, which uses image structure information as a condition and realizes image decomposition and reconstruction through reversible normalization flow. The network model includes two modules: a scale adaptive Transformer module and a fine-grained normalization flow module. The scale adaptive Transformer module combines window self-attention mechanism and transpose self-attention mechanism to fully extract multi-scale conditional features in dark light input. The conditional features aggregate global image content information and local image texture information in dark light input. The fine-grained normalization flow module learns the reversible transformation between normal light images and implicit states conforming to normal distribution, and the implicit states can be regarded as a simple and efficient representation of illumination information of normal light images.

[0009] To achieve the above purpose, the present application adopts the following technical solutions:

[0010] A dark light image enhancement method based on a normalization flow model includes a training phase and a use phase.

[0011] Step 1: training phase.

[0012] In the training phase, a pair of dark light / normal light RGB images are used to train the data set, and the conditional normalization flow network model parameters are continuously iteratively optimized. The optimal network model parameters are saved.

[0013] Specifically, step 1 includes the following steps:

[0014] Step 1.1: extract prior information with illumination invariance in the input dark light image, including two parts: dark light image histogram equalization result and color map.

[0015] The prior information and the dark light image are spliced in the channel dimension to obtain the dark light input I of the network model.

[0016] Step 1.2: use the scale adaptive Transformer module to extract multi-scale conditional features for representing illumination-independent structure information in the image.

[0017] Specifically, the method is as follows:

[0018] According to the dark input I obtained in step 1.1, first pass the input mapping, and convert it into a high-dimensional feature map F i . The input mapping consists of two layers of convolution, which converts the dark input I into a high-dimensional image feature F1 for the input of the first layer of scale adaptive Transformer module.

[0019] After that, the high-dimensional image feature passes through four layers of scale adaptive Transformer module in turn, and extracts multi-spatial resolution conditional features {L1, L2, L3, L4} at four scales.

[0020] Specifically, the high-dimensional image feature F i at the i-th layer is converted into the conditional feature L i by the scale adaptive Transformer module. The process is represented as:

[0021]

[0022] L i = GDFN(LN(F′ i ))+F′ i

[0023] Where F′ i represents the feature after multi-head self-attention mechanism, WMSA represents window self-attention mechanism, MDTA represents transposed self-attention mechanism, LN represents layer normalization, and GDFN represents gated deep separated convolution feedforward network. The layers are down-sampled by bilinear difference to obtain conditional features of different scales.

[0024] For the shallow high-dimensional image features of the first two layers, the scale adaptive Transformer module uses window self-attention mechanism to divide the input feature map into several w x w small windows, where w represents the window size, H x W is the spatial resolution of the feature map, H and W represent its width and height respectively, and C is the feature dimension defined on the real number field .

[0025] For each window feature, it is first unfolded into and mapped to three matrices by a linear layer, where X k represents the unfolded window feature, Q k represents the query (Query), K k represents the key (Key), and V k represents the value (Value), which are the three matrices required for calculating self-attention.

[0026] After that, these matrices are divided into h heads in the feature dimension to calculate window multi-head self-attention:

[0027]

[0028] wherein, represents the matrix of the jth head of the ith window. B represents the relative position encoding, represents the transpose of the key K i,j .

[0029] For the deep high-dimensional image features of the last two layers, the scale adaptive Transformer module adopts the transpose self-attention mechanism to transpose the input feature map to three matrices Q, K, and V by using a depthwise separable convolution, which represent the query (Query), key (Key), and value (Value) transposed to the feature dimension; then, in the feature dimension, the matrices are divided into h heads to calculate the transpose multi-head self-attention:

[0030]

[0031] wherein, represents the matrix of the jth head, and a is a learnable parameter.

[0032] Step 1.3: Use the fine-grained normalization flow module to convert the normal light RGB image y corresponding to the dark light input I into multi-scale hidden features for expressing the illumination information contained in the normal light image, using the multi-scale conditional features in step 1.2.

[0033] Specifically, the implementation method is as follows:

[0034] The fine-grained normalization flow module contains four layers, and each layer contains a series of reversible transformations that do not change the data dimension. Through multi-scale transformation, a reversible mapping between the normal light RGB image y and the hidden features is finally established.

[0035] wherein, in the first layer, the normal light image y is converted to the hidden feature space through hidden feature conversion and conditional feature injection, and the process is represented as:

[0036] h1=f1(y,L1)

[0037] wherein, h1 represents the hidden feature output of the first layer, f1 represents the hidden feature conversion and conditional feature injection normalization flow operation of the first layer, and L1 represents the conditional feature of the first layer.

[0038] In the normalization flow operations of the second and third layers, the hidden features of the previous layer are first compressed in space to obtain h′ i-1 ​denotes the spatially compressed latent feature. Then, the illumination information at this scale is extracted by alternating the fine-grained split operation and the normalization flow operation, and a part of the diverse latent features at this scale is reserved for representing the illumination information at this scale. This process is denoted as:

[0039]

[0040]

[0041]

[0042] where Split() denotes the split operation on the latent feature in the feature dimension; L i denotes the conditional feature of the i-th layer; h i-1 denotes the latent feature output by the first split operation; Z i denotes the reserved latent feature of the i-th layer; h i denotes the latent feature output to the next layer for further feature transformation; f i 1 and f i 2 denote the normalization flow operations such as latent feature transformation and conditional feature injection.

[0043] At the fourth layer, the latent feature h3 output by the third layer is transformed into the latent feature Z3 at the lowest resolution, and this process is denoted as:

[0044] Z3 = f4(h3, L4)

[0045] where L4 denotes the last conditional feature of the fourth layer.

[0046] Step 1.4: Based on the maximum likelihood estimation and the variable transformation theorem, the relationship between the conditional probability distribution of the normal illumination image and the normal distribution of the latent feature is established, and the loss function for training the constrained conditional normalization flow network model is constructed:

[0047]

[0048] where NLL denotes the negative log-likelihood loss, x = {L1, L2, L3, L4} denotes the conditional feature, y denotes the normal light image, and z = {Z1, Z2, Z3} denotes the latent feature; denotes the partial derivative of the i-th normalization flow operation; N denotes the total number of normalization flow operations; p y|x denotes the conditional probability distribution of the normal light image y with respect to the conditional feature x; p z|x denotes the conditional probability distribution of the latent feature with respect to the conditional feature x.

[0049] The network model parameters are iteratively optimized by repeating steps 1.1 to 1.3 by minimizing the loss function through the back propagation algorithm, and the optimal model is stored.

[0050] Step 2: Use stage.

[0051] First, the image structure features are extracted from the dark light image input, and the hidden features are sampled on the normal distribution to express the illumination information.

[0052] Then, using the optimal model parameters obtained in the step 1 training stage, the image structure information and illumination information are reorganized to generate enhanced normal light images.

[0053] The enhanced results are saved, and if there is a real normal light image as a reference, the generated results are evaluated with the real image.

[0054] Specifically, step 2 includes the following steps:

[0055] Step 2.1: Create a conditional normalization flow network model according to the model parameters saved in the training stage, and construct a dark light input I from the input dark light RGB image according to the method described in step 1.1. Use the trained scale adaptive Transformer module to extract conditional features from the dark light input.

[0056] Step 2.2: Randomly sample hidden features according to the normal distribution, and input them into the fine-grained normalization flow module. Through the inverse transformation process opposite to step 1.3, the normal light RGB image is generated from the hidden features by using the image structure information in the conditional features.

[0057] Step 2.3: The results obtained are stored, and if there is a real normal light image as a reference, the generated image is evaluated with the real image.

[0058] Among them, the evaluation criteria adopts two indexes of peak signal to noise ratio (Peak Signal to Noise Ratio, PSNR) and structural similarity (Structural Similarity, SSIM).

[0059] Further, based on the above method, the present application also proposes a dark light image enhancement system based on a normalization flow model, which includes a dark light image preprocessing subsystem, a conditional feature extraction subsystem, a hidden feature transformation subsystem, an enhanced image generation subsystem, a loss function calculation subsystem and a generated result evaluation subsystem.

[0060] The dark-light image preprocessing subsystem includes a dark-light image augmentation module and a dark-light input extraction module. The dark-light image augmentation module performs random cropping, rotation, flipping, etc. on the input dark-light image to increase the diversity of training samples and improve the generalization ability of the model. The dark-light input extraction module extracts illumination-independent prior information such as histogram equalization results and color maps in the dark-light image to provide guidance for extracting image structure information.

[0061] The conditional feature extraction subsystem includes an input mapping module, a scale-adaptive Transformer module, and a down-sampling module. The input mapping module maps the dark-light input to a high-dimensional image feature space. The scale-adaptive Transformer module uses window self-attention mechanisms and transpose self-attention mechanisms to extract image structure features at different scales. The down-sampling module is inserted between the scale-adaptive Transformer modules to generate multi-scale features.

[0062] The implicit feature transformation subsystem includes an implicit feature extraction module and a fine-grained separation module. This subsystem is applied in the training phase to transform normal-light images into implicit features. The implicit feature extraction module uses implicit feature conversion and conditional feature injection normalization flow operations to extract illumination information from normal-light images based on conditional features. The fine-grained separation module is used to preserve illumination information in implicit features at different scales.

[0063] The enhanced image generation subsystem is applied in the use phase. First, implicit features are sampled from the implicit state space according to a normal distribution, and the image structure information in the conditional features and the inverse transformation of the normalization flow are used. The normal-light image is generated from the implicit features.

[0064] The loss function calculation subsystem is applied in the training phase. The negative log-likelihood loss is calculated based on the implicit features generated by the conditional normalization flow network model, and the network model parameters are optimized using the backpropagation algorithm.

[0065] The generated result evaluation subsystem is applied in the use phase. The generated enhanced image result is compared with the real image, and the peak signal-to-noise ratio and structural similarity index are used to evaluate the quality of the enhanced image.

[0066] The connection relationship between the above-mentioned constituent systems is as follows:

[0067] The output end of the dark-light image preprocessing subsystem is connected to the input end of the conditional feature extraction subsystem, providing processed dark-light input. The output end of the conditional feature extraction subsystem is connected to the implicit feature transformation subsystem in the training phase. Further, the implicit feature transformation subsystem is connected to the loss function calculation subsystem. The output end of the conditional feature extraction subsystem is connected to the enhanced image generation subsystem in the use phase. Further, the enhanced image generation subsystem is connected to the generated result evaluation subsystem.

[0068] In the dark light image preprocessing subsystem, the output end of the dark light image augmentation module is connected with the input end of the dark light input extraction module, and the corresponding prior information is calculated for the augmented dark light image.

[0069] In the conditional feature extraction subsystem, the output end of the input mapping module is connected with the input end of the first layer scale adaptive Transformer module, and the output end of the first layer scale adaptive Transformer module is connected to the input end of the next layer down sampling module. The above two modules are alternately connected three layers, which are used for extracting multi-scale image conditional features.

[0070] The implicit feature transformation subsystem contains four layers of structure, the output end of the implicit feature extraction module of each layer is connected with the input end of the fine-grained separation module, and the output end of the fine-grained separation module is connected to the implicit feature extraction module of the next layer.

[0071] Advantages

[0072] Compared with the prior art, the present application has the following advantages:

[0073] 1. The dark light enhancement idea based on image decomposition in the present application decomposes the dark light image into multi-scale conditional features and implicit features, so as to fully express the structure information and illumination information in the image. The network model of the conditional normalization flow is used to implicitly decompose the image in the feature space, which can better process various degradation problems in the image taken in the dark light environment, and reduce the artifacts generated in the enhancement result, so as to achieve better visual effect.

[0074] 2. The scale adaptive Transformer module is designed for extracting the structure features of the dark light image, which combines the window self-attention mechanism and the transpose self-attention mechanism, and extracts local information and global information for image features of different scales, so as to improve the ability of the network model to capture the details of the dark light image and the ability to suppress image noise.

[0075] 3. The fine-grained multi-scale normalization flow module is designed, which establishes a reversible mapping from the normal light image to the implicit feature. By establishing a negative log likelihood constraint in the implicit space, the implicit feature conforming to the normal distribution accurately and efficiently describes the illumination information in the normal light image. In the use stage, the present application can generate diversified image enhancement results by sampling, and the enhanced image can restore the true color and illumination. BRIEF DESCRIPTION OF DRAWINGS

[0076] Figure 1 is the total flow chart of the dark light image enhancement method based on the normalization flow model in the present application.

[0077] Figure 2 is a schematic diagram of the conditional normalization flow network model described in the method of the present application.

[0078] Figure 3 is a structural schematic diagram of the dark light image enhancement system disclosed in the present application. DETAILED DESCRIPTION

[0079] For better illustration of the purposes and advantages of the present application, the method of the present application is further illustrated below in combination with the drawings and examples.

[0080] EMBODIMENT

[0081] The present embodiment discloses a dark light image enhancement method based on a normalization flow model. As shown in the figure, it includes two steps of training phase and use phase. Figure 1

[0082] Step 1: training phase, using paired dark light / normal light RGB image training dataset, constantly iterating and optimizing the parameters of the conditional normalization flow network model, and saving the optimal network model parameters.

[0083] Specifically, step 1 includes the following steps:

[0084] Step 1.1: extract prior information with illumination invariance in the input dark light image. It includes two parts: the first part is the dark light image histogram equalization result; the second part is the color map. These prior information and dark light image are spliced in the channel dimension to obtain the dark light input I of the network model.

[0085] Step 1.2: use the scale adaptive Transformer module to extract multi-scale conditional features for representing illumination-independent structural information in the image, as shown in the figure. Figure 2

[0086] Specific methods are as follows:

[0087] According to the dark light input I obtained in step 1.1, it is first converted into high-dimensional image features through the input mapping module. The input mapping module is composed of two layers of convolution, which converts the dark light input I into the input high-dimensional image features F1 of the first layer scale adaptive Transformer module.

[0088] After that, the high-dimensional image features pass through four layers of scale adaptive Transformer modules in turn, extracting multi-spatial resolution conditional features {L1, L2, L3, L4} at four scales. Specifically, the high-dimensional image features F i are converted into conditional features L i by the scale adaptive Transformer module. This process is represented as: ​​

[0089]

[0090] L i =GDFN(LN(F′) i ))+F′ i

[0091] Where WMSA represents window self-attention mechanism, MDTA represents transposed self-attention mechanism, LN represents layer normalization, and GDFN represents gated depthwise segregating convolutional feedforward network. Conditional features at different scales are obtained by downsampling between layers using bilinear interpolation.

[0092] For the shallow high-dimensional image features in the first two layers, the scale-adaptive Transformer module employs a window self-attention mechanism to process the input feature map. The space is divided into several w×w small windows, where H×W is the spatial resolution and C is the feature dimension. For each window feature, it is first expanded into... And map it to a linear layer On three matrices, these matrices are then divided into h heads along the feature dimension to compute window multi-head self-attention:

[0093]

[0094] in, Let B be the matrix representing the j-th head of the i-th window. B represents the relative position encoding.

[0095] For the deep, high-dimensional image features in the latter two layers, the scale-adaptive Transformer module employs a transposed self-attention mechanism to transform the input feature map... Extract and transpose using depthwise separable convolution. Three matrices are then divided into h heads along the feature dimension to compute the transposed multi-head self-attention:

[0096]

[0097] in Let represent the matrix of the j-th head, and α be a learnable parameter.

[0098] Using a fine-grained normalized flow module, and leveraging the multi-scale conditional features from step 1.2, the normal light RGB image y corresponding to the dark light input I is transformed into multi-scale latent features to express the illumination information contained in the normal light image. The specific method is as follows:

[0099] Step 1.3: The fine-grained normalization flow module contains four layers, each of which contains a series of reversible transformations that do not change the dimensionality of the data. Through multi-scale transformation, a reversible mapping between the normal light RGB image y and the latent feature is finally established.

[0100] In the first layer, the normal light image y is transformed into the latent feature space through latent feature transformation and conditional feature injection. This process can be represented as:

[0101] h1 = f1(y, L1)

[0102] where f1 represents the normalization flow operations such as latent feature transformation and conditional feature injection in the first layer.

[0103] In the normalization flow operations of the second and third layers, the latent feature h After spatial compression, we get Then, fine-grained separation and normalization flow operations are alternately performed to extract the illumination information at this scale and retain a portion of the diversified latent features at this scale to represent the illumination information at this scale. This process can be represented as:

[0104]

[0105]

[0106]

[0107] where Z i is the retained latent feature at the i-th layer, h i is the input latent feature to the next layer for further feature transformation, and represent the normalization flow operations such as latent feature transformation and conditional feature injection.

[0108] In the fourth layer, the latent feature h3 output by the third layer is transformed into the lowest resolution latent feature Z3. This process can be represented as:

[0109] Z3 = f4(h3, L4)

[0110] Step 1.4: Based on the maximum likelihood estimation and the variable transformation theorem, the relationship between the conditional probability distribution of the normal light image and the normal distribution of the latent feature is established, and the loss function for training the constrained conditional normalization flow network model is constructed:

[0111]

[0112] where NLL denotes the negative log-likelihood loss, x = {L1, L2, L3, L4} denotes the conditional features, y denotes the normal light image, and z = {Z1, Z2, Z3} denotes the latent features. By minimizing the loss function through the backpropagation algorithm, the network model parameters are iteratively optimized by repeating steps 1.1 to 1.3, and the optimal model is stored.

[0113] Step 2: Use phase, extract image structure features from dark light image input, and sample latent features on normal distribution for expressing illumination information. Then, using the optimal model parameters obtained in step 1 training phase, recombine image structure information and illumination information to generate enhanced normal light images. The enhanced results are further saved, and if there is a real normal light image as a reference, the generated results are evaluated with the real image.

[0114] Specifically, step 2 includes the following steps:

[0115] Step 2.1: Create a conditional normalization flow network model according to the model parameters saved in the training phase, and construct dark light input I from the input dark light RGB image using the method described in step 1.1. Use the trained scale adaptive Transformer module to extract conditional features from the dark light input.

[0116] Step 2.2: Randomly sample latent features according to normal distribution and input them into the fine-grained normalization flow module. Through the inverse transformation process opposite to step 1.3, use the image structure information in the conditional features to generate normal light RGB images from the latent features.

[0117] Step 2.3: Store the resulting images, and if there is a real normal light image as a reference, evaluate the generated images with the real image. The evaluation criteria use two indicators: Peak Signal to Noise Ratio (PSNR) and Structural Similarity (SSIM).

[0118] As shown in Figure 3 A dark light image enhancement system based on a normalization flow model includes a dark light image preprocessing subsystem, a conditional feature extraction subsystem, a latent feature transformation subsystem, an enhanced image generation subsystem, a loss function calculation subsystem, and a generated result evaluation subsystem.

[0119] The dark-light image preprocessing subsystem includes a dark-light image augmentation module and a dark-light input extraction module. The dark-light image augmentation module performs random cropping, rotation, flipping, etc. on the input dark-light image to increase the diversity of training samples and improve the generalization ability of the model. The dark-light input extraction module extracts illumination-independent prior information such as histogram equalization results and color maps in the dark-light image to provide guidance for extracting image structure information.

[0120] The conditional feature extraction subsystem includes an input mapping module, a scale-adaptive Transformer module, and a down-sampling module. The input mapping module maps the dark-light input to a high-dimensional image feature space. The scale-adaptive Transformer module uses window self-attention mechanisms and transpose self-attention mechanisms to extract image structure features at different scales. The down-sampling module is inserted between the scale-adaptive Transformer modules to generate multi-scale features.

[0121] The implicit feature transformation subsystem includes an implicit feature extraction module and a fine-grained separation module. This subsystem is applied in the training phase to transform normal-light images into implicit features. The implicit feature extraction module uses implicit feature conversion and conditional feature injection normalization flow operations to extract illumination information from normal-light images based on conditional features. The fine-grained separation module is used to preserve illumination information in implicit features at different scales.

[0122] The enhanced image generation subsystem is applied in the use phase. First, implicit features are sampled from the implicit state space according to a normal distribution, and the image structure information in the conditional features and the inverse transformation of the normalization flow are used. The normal-light image is generated from the implicit features.

[0123] The loss function calculation subsystem is applied in the training phase. The negative log-likelihood loss is calculated based on the implicit features generated by the conditional normalization flow network model, and the network model parameters are optimized using the backpropagation algorithm.

[0124] The generated result evaluation subsystem is applied in the use phase. The generated enhanced image result is compared with the real image, and the peak signal-to-noise ratio and structural similarity index are used to evaluate the quality of the enhanced image.

[0125] The connection relationship between the above-mentioned constituent systems is as follows:

[0126] The output end of the dark-light image preprocessing subsystem is connected to the input end of the conditional feature extraction subsystem, providing processed dark-light input. The output end of the conditional feature extraction subsystem is connected to the implicit feature transformation subsystem in the training phase. Further, the implicit feature transformation subsystem is connected to the loss function calculation subsystem. The output end of the conditional feature extraction subsystem is connected to the enhanced image generation subsystem in the use phase. Further, the enhanced image generation subsystem is connected to the generated result evaluation subsystem.

[0127] In the dark light image preprocessing subsystem, the output end of the dark light image augmentation module is connected with the input end of the dark light input extraction module, and the corresponding prior information is calculated for the augmented dark light image. In the conditional feature extraction subsystem, the output end of the input mapping module is connected with the input end of the first layer scale adaptive Transformer module, and the output end of the first layer scale adaptive Transformer module is further connected with the input end of the next layer downsampling module. The above two modules are alternately connected with three layers for extracting multi-scale image conditional features. The implicit feature transformation subsystem contains four layers of structure, and the output end of the implicit feature extraction module of each layer is connected with the input end of the fine-grained separation module, and the output end of the fine-grained separation module is connected with the implicit feature extraction module of the next layer.

[0128] The above specific description further details the purpose, technical scheme and beneficial effects of the application. It should be understood that the above description is only a specific embodiment of the application and is not used to limit the protection scope of the application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principles of the application shall be included in the protection scope of the application.

Claims

1. A dark-light image enhancement method based on normalized flow model, comprising a training stage and a use stage, characterized in that: Step 1: In the training stage, a paired dark-light / normal-light RGB image is used to train the data set, and the conditional normalization flow network model parameters are iteratively optimized, and the optimal network model parameters are saved; Comprising the following steps: Step 1.1: Extracting prior information with illumination invariance in the input dark-light image, including dark-light image histogram equalization results, color maps; Concatenate these prior information with the dark-light image in the channel dimension to obtain the dark-light input of the network model; Step 1.2: Use the scale adaptive Transformer module to extract multi-scale conditional features for representing the illumination-independent structural information in the image; Step 1.3: Use the fine-grained normalization flow module to convert the corresponding normal-light RGB image of the dark-light input into multi-scale hidden features for expressing the illumination information contained in the normal-light image; Step 1.3 implementation method as follows: The fine-grained normalization flow module contains four layers, each layer contains a series of reversible transformations without changing the data dimension; Through multi-scale transformation, a reversible mapping between normal-light RGB image y and hidden feature is finally established; Among them, in the first layer, the normal-light image y is converted to the hidden feature space through the hidden feature conversion and the conditional feature injection, and the process is represented as: h1=f1(y,L1) Where h1 represents the hidden feature output of the first layer, f1 represents the hidden feature conversion and conditional feature injection of the first layer, and L1 represents the conditional feature of the first layer; In the normalization flow operation of the second and third layers, the hidden features of the previous layer are first spatially compressed to obtain h′ i-1 representing the spatially compressed hidden features; then, the fine-grained separation operation and the normalization flow operation are alternately performed to extract the illumination information at this scale and to retain a part of the diversified hidden features at this scale for representing the illumination information at this scale; this process is represented as where Split() represents a split operation on the hidden features in the feature dimension; L i represents the conditional feature of the i-th layer; h" i-1 represents the hidden feature output by the first split operation; Z i is the hidden feature reserved for the i-th layer; h i is the hidden feature output to the next layer for further feature transformation; and represents a normalization flow operation such as hidden feature conversion and conditional feature injection; In the fourth layer, the hidden feature h3 output by the third layer is converted into the lowest resolution hidden feature Z3, and the process is represented as: Z3=f4(h3,L4) Where L4 represents the conditional feature of the last 4th layer; Step 1.4: Based on the maximum likelihood estimation and the variable transformation theorem, the relationship between the conditional probability distribution of the normal-light image and the normal distribution of the hidden feature is established, and the loss function for training the constrained conditional normalization flow network model is constructed; Minimize the loss function by back propagation algorithm, repeat steps 1.1 to 1.3 to iteratively optimize the network model parameters, and store the optimal model; Step 2: Use stage; First, extract the image structure features from the dark-light image input, and sample the hidden features on the normal distribution to express the illumination information; Then, using the optimal model parameters obtained in step 1 training stage, recombine the image structure information and illumination information to generate the enhanced normal-light image; The enhanced result is saved, and if there is a real normal-light image as a reference, the generated result is evaluated with the real image.

2. The dark light image enhancement method based on normalized flow model according to claim 1, comprising a training stage and a using stage, characterized in that, Step 1.2 implementation method as follows: According to the dark light input I obtained in step 1.1, first pass input mapping, it is converted into a high-dimensional feature map F i ; the input mapping is composed of two layers of convolution, which converts the dark light input I into a high-dimensional image feature F1 input to the first layer scale adaptive Transformer module; After that, the high-dimensional image features pass through the four layers of scale adaptive Transformer module in turn, and the multi-space resolution conditional features {L1, L2, L3, L4} are extracted at four scales; High-dimensional image feature F at the i-th layer i Transformed into conditional feature L by the scale adaptive Transformer module i This process is represented as: L i = GDFN(LN(F′ i ))+ F′ i wherein F' = F + F i wherein F' = F + F i wherein F' = F + F i wherein F' = F + F i wherein F' = F + F i wherein F' = F + F i wherein F' = F + F i wherein F' = F + F i wherein F' = F + F i wherein F' = F + F i wherein F' = F + F i wherein F' = F + F i wherein F' = F + F i wherein F' = F For the shallow high-dimensional image features in the first two layers, the scale-adaptive Transformer module employs a window self-attention mechanism to process the input feature map. The feature map is divided into several w×w small windows, where w represents the window size, H×W is the spatial resolution of the feature map, and H and W represent its width and height, respectively; C is the feature dimension, defined in the real number domain. superior; For each window feature, it is first unfolded as and mapped to three matrices, X k representing the unfolded window feature, Q k representing the Query, K k representing the Key, and V k representing the Value, which are the three matrices required for computing self-attention. After that, these matrices are divided into h heads in the feature dimension to calculate window multi-head self-attention: wherein, represents the matrix of the jth header of the ith window; B represents the relative position encoding, represents the transpose of the key K i,j ; For the deep high-dimensional image features of the last two layers, the scale adaptive Transformer module adopts the transpose self-attention mechanism to transpose the input feature map extracted by the depth separation convolution and transposed into Three matrices, Q, K, and V, represent the query Query, the key Key, and the value Value transposed to the feature dimension; then, in the feature dimension, these matrices are divided into h heads to calculate the transpose multi-head self-attention: wherein, represents the matrix of the jth head, and a is a learnable parameter.

3. The normalized flow model based dim light image enhancement method of claim 1, comprising a training phase and a usage phase, wherein, In step 1.4, the loss function for training the conditional normalizing flow network model is constructed as follows: where NLL denotes the negative log-likelihood loss, x = {L1, L2, L3, L4} denotes the conditional features, y denotes the normal light image, and z = {Z1, Z2, Z3} denotes the latent features; denotes the partial derivative of the i-th normalization flow operation; N denotes the total number of normalization flow operations; p y|x denotes the conditional probability distribution of the normal light image y given the conditional features x; p z|x denotes the conditional probability distribution of the latent features given the conditional features x.

4. The dark light image enhancement method based on normalized flow model according to claim 1, comprising a training stage and a using stage, characterized in that, Step 2 includes the following steps: Step 2.1: Create a conditional normalizing flow network model according to the model parameters saved during the training phase, construct the dark-light input I from the input dark-light RGB image using the method of step 1.1; extract the conditional features from the dark-light input using the trained scale adaptive Transformer module; Step 2.2: Randomly sample the hidden features according to the normal distribution, and input them into the fine-grained normalizing flow module; by the inverse transformation process opposite to step 1.3, use the image structure information in the conditional features to generate normal-light RGB images from the hidden features; Step 2.3: Store the results, and if there is a real normal-light image as a reference, evaluate the generated image with the real image; The evaluation criteria use the peak signal-to-noise ratio and structural similarity.

5. A dim light image enhancement system based on a normalized flow model, characterized in that, The system includes a dark-light image preprocessing subsystem, a conditional feature extraction subsystem, a hidden feature transformation subsystem, an enhanced image generation subsystem, a loss function calculation subsystem, and a generated result evaluation subsystem; The dark-light image preprocessing subsystem includes a dark-light image augmentation module and a dark-light input extraction module; the dark-light image augmentation module operates on the input dark-light image, including random cropping, rotation, and flipping; the dark-light input extraction module extracts prior information such as histogram equalization results and color maps that are independent of illumination from the dark-light image; The conditional feature extraction subsystem includes an input mapping module, a scale adaptive Transformer module, and a down-sampling module; the input mapping module maps the dark-light input to a high-dimensional image feature space, the scale adaptive Transformer module uses window self-attention mechanisms and transpose self-attention mechanisms to extract image structure features at different scales; the down-sampling module is inserted between the scale adaptive Transformer modules to generate multi-scale features; The hidden feature transformation subsystem includes a hidden feature extraction module and a fine-grained separation module; this subsystem is applied in the training phase to transform normal-light images into hidden features; the hidden feature extraction module uses normalizing flow operations to extract illumination information from normal-light images based on conditional features; the fine-grained separation module is used to preserve illumination information in hidden features at different scales; The enhanced image generation subsystem is applied in the use phase; first, sample hidden features from the hidden state space according to the normal distribution, use the image structure information in the conditional features and the inverse transformation of the normalizing flow; generate normal-light images from the hidden features; The loss function calculation subsystem is applied in the training phase; calculate the negative log-likelihood loss based on the hidden features generated by the conditional normalizing flow network model, and optimize the network model parameters using the backpropagation algorithm; The generated result evaluation subsystem is applied in the use phase; compare the generated enhanced image results with the real image, and use the peak signal-to-noise ratio and structural similarity indicators to evaluate the quality of the enhanced image; The connection relationship between the above subsystems is as follows: The output end of the dark-light image preprocessing subsystem is connected with the input end of the conditional feature extraction subsystem to provide the processed dark-light input for the conditional feature extraction subsystem; the output end of the conditional feature extraction subsystem is connected with the implicit feature transformation subsystem in the training stage; the implicit feature transformation subsystem is further connected with the loss function calculation subsystem; the output end of the conditional feature extraction subsystem is connected with the enhanced image generation subsystem in the use stage; and the enhanced image generation subsystem is further connected with the generation result evaluation subsystem; In the dark-light image preprocessing subsystem, the output end of the dark-light image augmentation module is connected with the input end of the dark-light input extraction module to calculate the corresponding prior information for the dark-light image after the dark-light image is augmented; In the conditional feature extraction subsystem, the output end of the input mapping module is connected with the input end of the first layer scale adaptive Transformer module; the output end of the first layer scale adaptive Transformer module is connected with the input end of the next layer downsampling module; the above two modules are alternately connected with three layers to extract the multi-scale image conditional features; The implicit feature transformation subsystem comprises four layers of structures; the output end of the implicit feature extraction module of each layer is connected with the input end of the fine-grained separation module; and the output end of the fine-grained separation module is connected with the implicit feature extraction module of the next layer.

Images