A low-illumination video frequency enhancement method based on laplace decomposition

Abstract

The application discloses a low-illumination video frequency enhancement method based on Laplace decomposition. The method comprises the following steps: establishing a low-illumination video enhancement model comprising an image multi-scale decomposition operator and a reconstruction operator based on a Laplace pyramid and a multi-scale enhancement network, and constructing an overall loss function of the model comprising a reconstruction loss function and a time domain consistency loss function; training the model until the overall loss function converges and the training is completed; and inputting video frames of a low-illumination video to be enhanced into the model for processing and outputting frame-by-frame enhanced video frames. The method can solve the problems of low video visibility and low signal-to-noise ratio. The model and the loss function constructed by the method can decouple the low-frequency signal and the high-frequency details of an image during frame-by-frame enhancement, can restore and enhance a low-illumination image, and finally reconstruct an original size image. The method can enhance the low-illumination video frame by frame, effectively improves the detail reconstruction quality and color accuracy of the low-illumination video enhancement, and can achieve good visual effects.

Description

Technical Field

[0001] This invention relates to a video frequency division enhancement method, which relates to the field of image processing, and specifically to a low-light video frequency division enhancement method based on Laplacian decomposition. Background Technology

[0002] Low-light video enhancement techniques are typically used to address low visibility and low signal-to-noise ratio issues caused by insufficient lighting, uneven illumination, backlighting, and other environmental conditions. Traditional methods based on histogram equalization and image decomposition models amplify noise while improving overall image brightness. With the development of deep learning technology, convolutional neural network-based low-light enhancement methods can improve image visibility while suppressing noise. Since frame-by-frame paired low-light-normal-light video is difficult to obtain, early low-light video enhancement tasks were often accomplished using single-image enhancement networks. However, these methods often exhibit flickering issues in video enhancement tasks, affecting visual experience.

[0003] To address these issues, several datasets and simulation methods for training low-light video enhancement network models have been proposed. Examples include the first publicly available low-light video enhancement dataset, "Dark Raw Video (DRV)," which contains low-light and regular-light video pairs of static scene videos, and the proposed dynamic low-light video enhancement dataset based on a sliding rail system, "Seeing Dynamic Scenes in the Dark (SDSD)." Furthermore, methods for generating simulated dynamic low-light videos from regular video datasets have also been gradually proposed. The emergence of low-light video datasets and simulation methods has prompted researchers to develop low-light video enhancement network models, such as SMID, SDSDNet, and the LVE-S2D model. However, existing methods suffer from varying degrees of over-smoothing of details and color cast, severely impacting video image quality and visual experience. Summary of the Invention

[0004] To address the problems existing in the background art, the present invention provides a low-light video frequency division enhancement method based on Laplacian decomposition.

[0005] The technical solution adopted in this invention is:

[0006] The low-light video frequency division enhancement method based on Laplacian decomposition of the present invention includes:

[0007] S1: Establish a low-light video enhancement model that includes a Laplacian pyramid-based image multi-scale decomposition operator, a multi-scale enhancement network, and a Laplacian pyramid-based image multi-scale reconstruction operator connected in sequence. Construct an overall loss function for the low-light video enhancement model that includes a reconstruction loss function and a temporal consistency loss function. Ambient light intensity below 0.5 lux is defined as low light intensity.

[0008] S2: In different scenarios, collect several low-light RGB videos and their reference RGB videos and divide them into video frames. Input the video frames of each low-light RGB video and its reference RGB video into the low-light video enhancement model for training until the overall loss function converges, and obtain the trained low-light video enhancement model.

[0009] S3: Input the video frames of the low-light RGB video to be enhanced into the trained low-light video enhancement model for processing. After processing, the trained low-light video enhancement model outputs video frames that are enhanced frame by frame, thereby realizing the frequency division enhancement of low-light video.

[0010] The low-light video enhancement model utilizes an image multi-scale decomposition operator to decompose the image into low-frequency components and multi-scale detail components. These low-frequency and detail components are then input into a multi-scale enhancement network for frame-by-frame enhancement. Finally, an image reconstruction operator is used to reconstruct a high-quality enhancement result of the original size. This invention utilizes the Laplacian pyramid to perform multi-scale decomposition of the image's low-frequency and detail components, suppressing noise and improving the detail reconstruction effect of the video.

[0011] In step 1), the image multi-scale decomposition operator based on the Laplacian pyramid includes N layers of Laplacian pyramids, each layer of the Laplacian pyramid including two Gaussian low-pass filters G and a 2×2 mean downsampling layer D. 2×2 and 2×2 bilinear interpolation upsampling layer U 2×2 The input of the i-th layer of the Laplace pyramid passes sequentially through its own first Gaussian low-pass filter and a 2×2 mean downsampling layer D. 2×2 The processed output is used as the i-th intermediate result, which is then passed through its own 2×2 bilinear interpolation upsampling layer U. 2×2The output after processing by the second Gaussian low-pass filter G is used as the i-th decomposition result. The i-th intermediate result and the (i+1)-th decomposition result are processed together using pixel-wise subtraction, and the output is used as the i-th detail component. N = 1, 2, ..., i, ..., N-1. The input of the first layer Laplacian pyramid is the low-light video frame input to the low-light video enhancement model. The low-light video frame and the first decomposition result are processed together using pixel-wise subtraction, and the output is used as the bottom detail component. The i-th intermediate result of the i-th layer Laplacian pyramid is directly used as the top-level low-frequency component. Finally, the obtained top-level low-frequency component, N-1 i-th detail components and bottom detail components are used together as the low-light image multi-scale decomposition result and input into the multi-scale enhancement network for multi-scale enhancement processing.

[0012] In step 1), the multi-scale enhancement network includes N+1 layers of encoder-decoder network layers. Each encoder-decoder network layer includes three sequentially connected encoder blocks, intermediate blocks, and three decoder blocks. The inputs of each encoder-decoder network layer are sequentially input to the three encoder blocks, intermediate blocks, and three decoder blocks for processing. The output of the first encoder block is skipped to the third decoder block and processed together with the output of the second decoder block. The output of the second encoder block is skipped to the second decoder block and processed together with the output of the first decoder block. The output of the third encoder block is skipped to the first decoder block and processed together with the output of the intermediate blocks. The output of the third decoder block is used as the output of the multi-scale enhancement network. The top-level low-frequency component, the N-1 i-th detail components, and the bottom-level detail components in the multi-scale decomposition result of the low-light image are respectively input to their respective encoder-decoder network layers for processing, and the processed results are output as their respective multi-scale enhancement results. The multi-scale decomposition ground truth video frames are used to supervise each multi-scale enhancement result, and the supervision signal is the loss function.

[0013] The encoding network block EB consists of two 3×3 convolutional layers with residual connections and a 2×2 convolutional downsampling layer. The input of the encoding network block EB is sequentially fed into the two 3×3 convolutional layers with residual connections for processing to obtain the convolution result. The convolution result and the input of the encoding network block EB are added together element-wise and then processed by the 2×2 convolutional downsampling layer before being output as the output of the encoding network block EB.

[0014] The intermediate network block (MB) consists of two 3×3 convolutional layers with residual connections, a fully connected layer, and two channel attention modules. The input of the intermediate network block (MB) is sequentially fed into the two 3×3 convolutional layers with residual connections for processing. The processed outputs are then fed into the first channel attention module along with the input of the intermediate network block (MB) for processing. The output of the first channel attention module is processed by the fully connected layer and then by the channel weight vector. The output of the first channel attention module is then multiplied by the channel dimension. The result of the multiplication is fed into the second channel attention module for processing, and the output is used as the output of the intermediate network block (MB).

[0015] The decoder network block DB consists of four 3×3 convolutional layers with residual connections, a 1×1 convolutional layer, a feature concatenation operation Cat, and an upsampling operation pixel shuffle. The input of the decoder network block DB is sequentially fed into two 3×3 convolutional layers with residual connections, a 1×1 convolutional layer, and two more 3×3 convolutional layers with residual connections for processing. The output of the 1×1 convolutional layer and the output of the fourth 3×3 convolutional layer with residual connections are combined and then subjected to the feature concatenation operation Cat, followed by the upsampling operation pixel shuffle. The final output is the output of the decoder network block DB.

[0016] In step 1), the Laplacian pyramid-based image multi-scale reconstruction operator includes N Laplacian pyramid reconstruction layers, each of which includes a Gaussian low-pass filter and a 2×2 bilinear interpolation upsampling layer U connected in sequence. 2×2 The output of the i-th detail component after multi-scale enhancement and the output of the (i+1)-th Laplacian pyramid reconstruction layer are processed together using pixel-wise addition and used as the input of the i-th Laplacian pyramid reconstruction layer. The low-frequency component of the top layer after multi-scale enhancement is used as the input of the i-th Laplacian pyramid reconstruction layer. The output of the bottom detail component after multi-scale enhancement and the output of the first Laplacian pyramid reconstruction layer are processed together using pixel-wise addition and used as the output of the Laplacian pyramid-based image multi-scale reconstruction operator, which is the original size enhancement result of the low-light video enhancement model.

[0017] In step 1), the reconstruction loss function L rec Specifically as follows:

[0018]

[0019] in, and L represents the i-th multi-scale enhancement result after the multi-scale enhancement network processes the video frames of the low-light RGB video, and the i-th multi-scale decomposition ground truth video frame of the reference RGB video; perλ represents the perceived loss. per L represents perceived loss per The weighting coefficients.

[0020] The perceived loss L per Specifically as follows:

[0021]

[0022] Among them, F VGG (·) indicates that the video frame is extracted using a pre-trained deep convolutional neural network VGG (Very Deep Convolutional Networks for Large-scale Image Recognition); and These represent the i-th multi-scale enhancement result after the multi-scale enhancement network processes the video frames of the low-light RGB video, and the i-th multi-scale decomposition ground truth video frame of the reference RGB video, respectively. It represents the 1-norm.

[0023] In step 1), the temporal consistency loss function L tc Specifically as follows:

[0024]

[0025] Where, N p This represents the total number of image blocks of size k×k that are segmented from the multi-scale enhancement results of two consecutive video frames; and Let represent the pixel mean of the j-th k×k image block; It represents the 1-norm.

[0026] This invention first starts with the frequency and characteristics of the neural network model, employing a multi-scale decomposition enhancement approach to enhance low-light video frame by frame. By enhancing multiple scales (base and detail) through multiple paths, the network is encouraged to fit high-frequency details and low-frequency content (such as color signals) separately, thus improving both over-smoothing and color cast. Considering the large optical flow estimation error in low-light images, this method further suppresses flicker through dynamic temporal consistency loss, achieving explicit temporal consistency constraints without optical flow on dynamic datasets. Compared to existing algorithms, this method effectively improves the spatial reconstruction quality and color accuracy of low-light video enhancement, achieving better visual effects.

[0027] The beneficial effects of this invention are:

[0028] This invention addresses the problems of low visibility and low signal-to-noise ratio in video shooting caused by environmental factors such as insufficient lighting, uneven illumination, and backlighting. Considering the varying fitting speeds of neural networks for signals of different frequencies, this method constructs a low-light video enhancement model based on Laplacian pyramid decomposition. During frame-by-frame enhancement, it decouples low-frequency signals and high-frequency details from the image, constructs a multi-scale enhancement network to recover and enhance low-light images, and finally reconstructs the original-size image signal. The network is trained using a reconstruction loss function and a temporal consistency loss function. Compared to existing methods, this method can enhance low-light videos frame-by-frame, effectively improving the detail reconstruction quality and color accuracy of low-light video enhancement, achieving better visual effects. Attached Figure Description

[0029] Figure 1 This is a schematic diagram of the image multi-scale decomposition operator of the method of the present invention;

[0030] Figure 2 This is a schematic diagram of the multi-scale enhancement network of the method of the present invention;

[0031] Figure 3 This is a schematic diagram of the image multi-scale reconstruction operator of the method of the present invention;

[0032] Figure 4 This is a schematic diagram of the encoding network block and decoding network block structure of the method of the present invention;

[0033] Figure 5 This is a schematic diagram of the intermediate network block structure of the method of the present invention;

[0034] Figure 6 These are illustrations of the low-light video frame input, the enhancement result of the method of the present invention, the enhancement and magnification result of the method of the present invention, and the enhancement result of the ChebyLighter model, according to embodiments of the present invention. Figure 6 (a) is a diagram illustrating the low-light video frame input of Scenario 1 of this embodiment, the enhancement result of the method of this invention, the enhancement and magnification result of the method of this invention, and the enhancement result using the ChebyLighter model. Figure 6 (b) is a diagram illustrating the low-light video frame input of Scenario 2 of this embodiment, the enhancement result of the method of this invention, the enhancement and magnification result of the method of this invention, and the enhancement result using the ChebyLighter model. Figure 6 (c) is a diagram showing the low-light video frame input of scenario three in this embodiment of the invention, the enhancement result of the method of the present invention, the enhancement and magnification result of the method of the present invention, and the enhancement result using the ChebyLighter model. Detailed Implementation

[0035] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0036] The low-light video frequency division enhancement method based on Laplacian decomposition of the present invention is as follows:

[0037] S1: Establish a low-light video enhancement model that includes a Laplacian pyramid-based image multi-scale decomposition operator, a multi-scale enhancement network, and a Laplacian pyramid-based image multi-scale reconstruction operator connected in sequence. Construct an overall loss function for the low-light video enhancement model that includes a reconstruction loss function and a temporal consistency loss function. Ambient light intensity below 0.5 lux is defined as low light intensity.

[0038] like Figure 1 As shown, the image multi-scale decomposition operator based on the Laplacian pyramid consists of N Laplacian pyramids, each of which includes two Gaussian low-pass filters G and a 2×2 mean downsampling layer D. 2×2 and 2×2 bilinear interpolation upsampling layer U 2×2 The input of the i-th layer of the Laplace pyramid passes sequentially through its own first Gaussian low-pass filter and a 2×2 mean downsampling layer D. 2×2 The processed output is used as the i-th intermediate result, which is then passed through its own 2×2 bilinear interpolation upsampling layer U. 2×2 The output of the second Gaussian low-pass filter G is used as the i-th decomposition result. The i-th intermediate result and the (i+1)-th decomposition result are processed together using pixel-wise subtraction, and the output is used as the i-th detail component. N = 1, 2, ..., i, ..., N-1. The input of the first-layer Laplacian pyramid is the low-light video frame input to the low-light video enhancement model. The low-light video frame and the first decomposition result are processed together using pixel-wise subtraction, and the output is used as the bottom-level detail component. The i-th intermediate result of the i-th layer Laplacian pyramid is directly used as the top-level low-frequency component. Finally, the obtained top-level low-frequency component, N-1 i-th detail components, and bottom-level detail components are used together as the low-light image multi-scale decomposition result and input into the multi-scale enhancement network for multi-scale enhancement processing. In specific implementation, a two-layer Laplacian pyramid can be used.

[0039] like Figure 2As shown, the multi-scale enhancement network comprises N+1 layers of encoder-decoder networks. Each layer consists of three sequentially connected encoder blocks, intermediate blocks, and three decoder blocks. The inputs of each encoder-decoder layer are sequentially fed into these blocks for processing. The output of the first encoder block is fed into the third decoder block and processed together with the output of the second decoder block. Similarly, the output of the second encoder block is fed into the second decoder block and processed together with the output of the first decoder block. The output of the third encoder block is fed into the first decoder block and processed together with the output of the intermediate blocks. The output of the third decoder block serves as the output of the multi-scale enhancement network. The top-level low-frequency component, the N-1 i-th detail components, and the bottom-level detail components from the multi-scale decomposition result of the low-light image are each fed into their respective encoder-decoder layers for processing, and the resulting multi-scale enhancement results are output. Ground-value video frames from the multi-scale decomposition are used to supervise each multi-scale enhancement result, with the supervision signal being the loss function. In a specific implementation, a three-layer encoder-decoder network can be used.

[0040] like Figure 4 As shown, the encoding network block EB includes two 3×3 convolutional layers with residual connections and a 2×2 convolutional downsampling layer. The input of the encoding network block EB is sequentially fed into the two 3×3 convolutional layers with residual connections for processing to obtain the convolution result. The convolution result and the input of the encoding network block EB are added together element-wise and then processed by the 2×2 convolutional downsampling layer before being output as the output of the encoding network block EB.

[0041] like Figure 5 As shown, the intermediate network block (MB) includes two 3×3 convolutional layers with residual connections, a fully connected layer, and two channel attention modules. The input of the intermediate network block (MB) is sequentially fed into the two 3×3 convolutional layers with residual connections for processing. The processed outputs are then fed into the first channel attention module along with the input of the intermediate network block (MB) for processing. The output of the first channel attention module is processed by the fully connected layer and then by the channel weight vector. The output of the first channel attention module is then multiplied by the channel dimension. The result of the multiplication is fed into the second channel attention module for processing, and the output is used as the output of the intermediate network block (MB).

[0042] like Figure 4As shown, the decoder network block DB includes four 3×3 convolutional layers with residual connections, a 1×1 convolutional layer, a feature concatenation operation Cat, and an upsampling operation pixel shuffle. The input of the decoder network block DB is sequentially fed into two 3×3 convolutional layers with residual connections, a 1×1 convolutional layer, and two more 3×3 convolutional layers with residual connections for processing. The output of the 1×1 convolutional layer and the output of the fourth 3×3 convolutional layer with residual connections are combined and then subjected to the feature concatenation operation Cat, followed by the upsampling operation pixel shuffle. The final output is the output of the decoder network block DB.

[0043] like Figure 3 As shown, the Laplacian pyramid-based image multi-scale reconstruction operator consists of N Laplacian pyramid reconstruction layers. Each Laplacian pyramid reconstruction layer includes a Gaussian low-pass filter and a 2×2 bilinear interpolation upsampling layer U connected in sequence. 2×2 The i-th detail component after multi-scale enhancement and the output of the (i+1)-th Laplacian pyramid reconstruction layer are processed together using pixel-wise addition, and the resulting output is used as the input to the i-th Laplacian pyramid reconstruction layer. The low-frequency component of the top layer after multi-scale enhancement is used as the input to the i-th Laplacian pyramid reconstruction layer. The bottom detail component after multi-scale enhancement and the output of the first Laplacian pyramid reconstruction layer are processed together using pixel-wise addition, and the resulting output is used as the output of the Laplacian pyramid-based image multi-scale reconstruction operator, which serves as the original size enhancement result of the low-light video enhancement model. In practice, two Laplacian pyramid reconstruction layers can be used.

[0044] Reconstruction loss function L rec Specifically as follows:

[0045]

[0046] in, and L represents the i-th multi-scale enhancement result after the multi-scale enhancement network processes the video frames of the low-light RGB video, and the i-th multi-scale decomposition ground truth video frame of the reference RGB video; per λ represents the perceived loss. per L represents perceived loss per The weighting coefficients.

[0047] Perceived loss L per Specifically as follows:

[0048]

[0049] Among them, F VGG (·) indicates that the video frame uses multi-layer features extracted using a pre-trained deep convolutional neural network (VGG); and These represent the i-th multi-scale enhancement result after the multi-scale enhancement network processes the video frames of the low-light RGB video, and the i-th multi-scale decomposition ground truth video frame of the reference RGB video, respectively. It represents the 1-norm.

[0050] Temporal consistency loss function L tc Specifically as follows:

[0051]

[0052] Where Np represents the total number of image blocks of size k×k that divide the multi-scale enhancement results of two consecutive video frames; and Let represent the pixel mean of the j-th k×k image block; It represents the 1-norm.

[0053] S2: In different scenarios, collect several low-light RGB videos and their reference RGB videos and divide them into video frames. Input the video frames of each low-light RGB video and its reference RGB video into the low-light video enhancement model for training until the overall loss function converges, and obtain the trained low-light video enhancement model.

[0054] S3: Input the video frames of the low-light RGB video to be enhanced into the trained low-light video enhancement model for processing. After processing, the trained low-light video enhancement model outputs video frames that are enhanced frame by frame, thereby realizing the frequency division enhancement of low-light video.

[0055] The low-light video enhancement model utilizes an image multi-scale decomposition operator to decompose the image into low-frequency components and multi-scale detail components. These low-frequency and detail components are then input into a multi-scale enhancement network for frame-by-frame enhancement. Finally, an image reconstruction operator is used to reconstruct a high-quality enhancement result of the original size. This invention utilizes the Laplacian pyramid to perform multi-scale decomposition of the image's low-frequency and detail components, suppressing noise and improving the detail reconstruction effect of the video.

[0056] This invention constructs a low-light video enhancement model based on Laplacian pyramid decomposition. During frame-by-frame enhancement, it decouples low-frequency signals and high-frequency details of the image, constructs a multi-scale enhancement network to restore and enhance low-light images, and finally reconstructs the original-size image signal.

[0057] like Figure 6 of (a), Figure 6 (b) and Figure 6As shown in (c), this is the running result of the method and ChebyLighter model based on the present invention on the DRV dataset in a specific embodiment of the present invention. It can be seen that the present method starts from the frequency and characteristics of the neural network model, and uses a multi-scale decomposition enhancement method to enhance low-light video frame by frame. By enhancing multi-scale base and detail through multiple paths, the network is promoted to fit high-frequency details and low-frequency content (such as color signals) respectively, which effectively improves the spatial reconstruction quality and color accuracy of low-light video enhancement and achieves better visual effects.

Claims

1. A low-light video frequency division enhancement method based on Laplacian decomposition, characterized in that, include: S1: Establish a low-light video enhancement model that includes a Laplacian pyramid-based image multi-scale decomposition operator, a multi-scale enhancement network, and a Laplacian pyramid-based image multi-scale reconstruction operator connected in sequence, and construct an overall loss function for the low-light video enhancement model that includes a reconstruction loss function and a temporal consistency loss function. S2: In different scenarios, collect several low-light RGB videos and their reference RGB videos and divide them into video frames. Input the video frames of each low-light RGB video and its reference RGB video into the low-light video enhancement model for training until the overall loss function converges and the trained low-light video enhancement model is obtained. S3: Input the video frames of the low-light RGB video to be enhanced into the trained low-light video enhancement model for processing. After processing, the trained low-light video enhancement model outputs video frames that are enhanced frame by frame, thereby realizing the frequency division enhancement of low-light video. In step 1), the multi-scale enhancement network includes N+1 layers of encoder-decoder network layers. Each encoder-decoder network layer includes three encoder network blocks, an intermediate network block, and three decoder network blocks connected in sequence. The input of each encoder-decoder network layer is sequentially input to the three encoder network blocks, the intermediate network block, and the three decoder network blocks for processing. The output of the first encoder network block is skipped to the third decoder network block and processed together with the output of the second decoder network block. The output of the second encoder network block is skipped to the second decoder network block and processed together with the output of the first decoder network block. The output of the third encoder network block is skipped to the first decoder network block and processed together with the output of the intermediate network block. The output of the third decoder network block is used as the output of the multi-scale enhancement network. The top low-frequency component, the N-1 i-th detail components, and the bottom detail components in the multi-scale decomposition result of the low-light image are respectively input to their respective encoder-decoder network layers for processing, and their respective multi-scale enhancement results are output after processing. In step 1), the temporal consistency loss function Specifically as follows: in, This indicates that the multi-scale enhancement result of two consecutive video frames is divided into segments of size . The total number of image blocks; and They represent the j-th The average pixel value of an image patch; It represents the 1-norm.

2. The low-light video frequency division enhancement method based on Laplacian decomposition according to claim 1, characterized in that: In step 1), the image multi-scale decomposition operator based on the Laplacian pyramid includes N layers of Laplacian pyramids, each layer of the Laplacian pyramid including two Gaussian low-pass filters G and a 2×2 mean downsampling layer D. 2×2 and 2×2 bilinear interpolation upsampling layer U 2×2 The input of the i-th layer of the Laplace pyramid passes sequentially through its own first Gaussian low-pass filter and a 2×2 mean downsampling layer D. 2×2 The processed output is used as the i-th intermediate result, which is then passed through its own 2×2 bilinear interpolation upsampling layer U. 2×2 The output after processing by the second Gaussian low-pass filter G is used as the i-th decomposition result. The i-th intermediate result and the (i+1)-th decomposition result are processed together using pixel-wise subtraction, and the output is used as the i-th detail component. N=1, 2, ..., i, ..., N-1. The input of the first layer Laplacian pyramid is the low-light video frame input to the low-light video enhancement model. The low-light video frame and the first decomposition result are processed together using pixel-wise subtraction, and the output is used as the bottom detail component. The i-th intermediate result of the i-th layer Laplacian pyramid is directly used as the top-level low-frequency component. Finally, the obtained top-level low-frequency component, N-1 i-th detail components and bottom detail components are used together as the low-light image multi-scale decomposition result and input into the multi-scale enhancement network for multi-scale enhancement processing.

3. The low-light video frequency division enhancement method based on Laplacian decomposition according to claim 2, characterized in that: In step 1), the Laplacian pyramid-based image multi-scale reconstruction operator includes N Laplacian pyramid reconstruction layers, each of which includes a Gaussian low-pass filter and a 2×2 bilinear interpolation upsampling layer U connected in sequence. 2×2 The output of the i-th detail component after multi-scale enhancement and the output of the (i+1)-th Laplacian pyramid reconstruction layer are processed together using pixel-wise addition and used as the input of the i-th Laplacian pyramid reconstruction layer. The low-frequency component of the top layer after multi-scale enhancement is used as the input of the i-th Laplacian pyramid reconstruction layer. The output of the bottom detail component after multi-scale enhancement and the output of the first Laplacian pyramid reconstruction layer are processed together using pixel-wise addition and used as the output of the Laplacian pyramid-based image multi-scale reconstruction operator, which is the original size enhancement result of the low-light video enhancement model.

4. The low-light video frequency division enhancement method based on Laplacian decomposition according to claim 3, characterized in that: In step 1), the reconstruction loss function is described. Specifically as follows: in, and These represent the i-th multi-scale enhancement result after the multi-scale enhancement network processes the video frames of the low-light RGB video, and the i-th multi-scale decomposition ground truth video frame of the reference RGB video, respectively. Indicates perceived loss. Indicates perceived loss The weighting coefficients.

5. The low-light video frequency division enhancement method based on Laplacian decomposition according to claim 4, characterized in that: The perceived loss Specifically as follows: in, This indicates that the video frame uses multi-layer features extracted using a pre-trained deep convolutional neural network (VGG). and These represent the i-th multi-scale enhancement result after the multi-scale enhancement network processes the video frames of the low-light RGB video, and the i-th multi-scale decomposition ground truth video frame of the reference RGB video, respectively. It represents the 1-norm.

Images