Low-illumination video enhancement method and system based on fast lookup table

Abstract

The invention discloses a low-illumination video enhancement method and system based on a fast lookup table, and the method specifically comprises the steps: obtaining an initial low-illumination video, carrying out the frame-by-frame splitting of the video, and obtaining an initial low-illumination image; processing the low-illumination image through a lightweight encoder to obtain potential features of the image; the potential features of the image are converted into compact potential feature vectors, and an intensity graph of each frame of low-illumination image is generated; forming lookup table mapping through a weight predictor, and constructing a lookup table model adaptively enhanced according to the video; and processing the enhanced image by using a de-noising module, and outputting an enhanced video. According to the method, the intensity graph and the fast self-adaptive lookup table are combined, the problems of low video visibility and low signal-to-noise ratio in the low-illumination environment are solved, the low-illumination video can be enhanced frame by frame, the calculation cost is low, the calculation efficiency is high, the real-time performance is high, and the real-time performance requirement of video image processing is met.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image low-illumination enhancement, and in particular to a low-illumination video enhancement method and system based on a fast lookup table. BACKGROUND

[0002] Under weak light conditions, videos often have problems of texture deterioration and low contrast, resulting in poor visibility and significant performance degradation of high-level visual tasks. Low-light video enhancement aims to reverse the degradation domain and improve the visibility and visual quality of videos captured under low-light conditions, providing good video data for various downstream tasks, such as autonomous driving, unmanned platform navigation, text detection, and photography on mobile devices. In hardware-based methods, video enhancement is usually performed using high ISO, long exposure time, and large aperture. However, these methods have limitations, such as high ISO amplifying noise, long exposure time causing motion blur, and large aperture causing overexposure and loss of details.

[0003] Existing low-light image enhancement methods are mainly based on traditional optimization methods such as histogram equalization and Retinex theory, using information in the image itself for processing to produce manual priors. However, due to the presence of numerous unknown degradation factors, processing methods that highly depend on artificial priors cannot accurately achieve image optimization. In recent years, data-driven methods based on deep learning have shown better generalization ability and effectiveness. For example, Zero-DCE designs a lightweight deep network that learns a set of image-specific, pixel-level high-order curves to iteratively adjust the pixel values of the input image, achieving end-to-end enhancement without paired training data, and can preserve the details and structures of the original image. However, in extremely dark or severely noisy areas, iterative curve adjustment can significantly amplify noise and artifacts, and the preset curve form cannot fit all complex illumination degradation patterns. SNRNet proposes a dual-branch network structure, one branch focuses on modeling long-range dependencies and context information to better understand image content, and the other branch learns spatially varying operations to dynamically adjust processing strength, achieving a good balance between brightness enhancement, detail restoration, and noise suppression. However, the dual-branch structure significantly increases the parameter quantity and computational overhead of the model, and the inference speed is relatively slow, requiring high hardware requirements. Moreover, high-quality paired training data is needed for complex mapping and estimation during training. NeRco proposes a neural normalization module that maps the feature representation of the input low-light image to a standardized intermediate state, and uses a relatively simple and general enhancer to further map to high-quality enhancement results, effectively reducing the difficulty of image enhancement and improving generalization ability. However, the normalization module needs to be run first, and the overall computational load may be greater than that of a single-stage model. Moreover, the original image information may be lost during the normalization to the intermediate state.

[0004] However, unlike image tasks, video tasks have strong correlation in time and space, and effective stability is needed to ensure the consistency of color brightness over time. Therefore, directly applying image-based methods in video tasks will cause video flicker problems and increase the consumption of computing resources to some extent. At present, low-light video enhancement can reduce the inter-frame brightness change by exploring the spatiotemporal information of the video. For example, SMOID directly uses a 3D convolutional neural network to process the spatiotemporal cube of the video, and slides in the spatial dimension and the time dimension to implicitly capture and aggregate the spatiotemporal features and motion information between adjacent frames, realize consistent enhancement, effectively reduce the inter-frame flicker problem, and enhance the perception of context. However, the expansion of 3D convolution operation in the time dimension leads to exponential growth of computational complexity and memory consumption, which is much higher than 2D convolution, making the model training and inference very expensive and slow, and difficult to meet the real-time video processing requirements. StableLLVE uses optical flow estimation to calculate the pixel-level motion vector between adjacent frames, aligns the features of adjacent frames to the coordinate system of the reference frame for fusion and enhancement, which can improve the quality of the reference frame, improve the computing efficiency and ensure the temporal stability. However, optical flow estimation is prone to errors in low-light, high-noise, motion blur, occlusion or texture missing areas, resulting in cumulative alignment errors, which will cause serious artifacts, ghosting and breaking in the enhanced results. In contrast, RVRT uses a recurrent neural network RNN to dynamically focus on and aggregate useful information in related time-series frames for the pixels of the reference frame, achieving a better balance between computational complexity and effectiveness. It is theoretically more robust than alignment-based methods in motion blur, occlusion and other scenes, but the computational complexity increases sharply with the increase of frame number and spatial resolution, still not meeting the strict real-time requirements, and without explicit motion guidance, it cannot accurately aggregate time-series information and maintain spatial detail clarity, which will result in insufficient details in the enhanced results. These methods do not balance performance and efficiency, increasing the challenges of real-time applications in the real world. SUMMARY

[0005] The purpose of the present application is to provide a low-light video enhancement method that can adaptively enhance low-light images in different light environments, reduce noise in the processed video, improve contrast, and increase the detail processing capability of the enhanced video.

[0006] The technical solution for achieving the purpose of the present application is: a low-light video enhancement method based on a fast lookup table, comprising the following steps:

[0007] Step 1, obtaining an initial low-light video, splitting the video frame by frame to obtain an initial low-light image;

[0008] Step 2, processing the low-light image through a lightweight encoder to obtain the latent features of the image;

[0009] Step 3: Convert the latent features of the image into a compact latent feature vector to generate an intensity map for each frame of the low-light image;

[0010] Step 4: Construct a lookup table mapping using a weight predictor and build a lookup table model that is adaptively enhanced based on the video.

[0011] Step 5: Use the noise reduction module to process the enhanced image and output the enhanced video.

[0012] Further, in step 1, the initial low-light video is obtained by splitting the video frame by frame to obtain the initial low-light image, as follows:

[0013] An initial, unprocessed low-light video is obtained and split into multiple initial low-light image groups with the same resolution based on the video shooting frame rate. The resolution is in pixels, and each initial low-light image group contains multiple consecutive low-light video frames.

[0014] Furthermore, step 2 involves processing the low-light image using a lightweight encoder to obtain the image's latent features, as detailed below:

[0015] The obtained resolution is The low-light image of the frame is used as input, through a process with five... A lightweight encoder for convolutional layers, with one for each layer. The convolutional kernel performs spatial upsampling, increasing the height and width of the feature map. At the same time, each layer performs feature fusion and channel compression. When the convolutional kernel slides on the input video block, it calculates the weighted sum of all pixel values ​​in the local spatiotemporal region, learns and extracts spatiotemporal patterns, including temporal and spatial features. The learned convolutional kernel is used to integrate the multi-channel information of the input and reduce the channel dimension.

[0016] A series of convolutional layers form a hierarchical structure. Shallow layers use a spatial kernel smaller than a set value and a temporal range shorter than a set value to extract basic spatiotemporal gradients, local motion, and simple textures. Deeper layers capture more complex spatiotemporal structures, semantic information, and global illumination dynamics by increasing the receptive field. Each layer builds upon the features extracted by the previous layer to achieve a higher level of understanding. These convolutional layers work together to capture the temporal continuity and spatial visual features of video frames, ultimately encoding the input low-light video frames into a set of latent feature maps.

[0017] Furthermore, the latent feature map output by the lightweight encoder is a set of Tensor; each time step Corresponding to one channel Feature map, spatial size is The resolution is smaller than the input resolution; each channel represents a specific spatiotemporal feature learned by the network, and the feature vector at each spatial location incorporates information from the corresponding spatial region and its neighboring time frames in the original input.

[0018] Furthermore, step 3 involves converting the latent features of the image into a compact latent feature vector to generate an intensity map for each frame of the low-light image, as detailed below:

[0019] Step 3.1: Output a set of data from the lightweight encoder. The tensor is transformed into a compact feature vector. Size is hyperparameters and These represent the number of pixels and the number of channels in the image, respectively.

[0020] Step 3.2: Convert the feature vector The data is fed into the decoder, where the shallow layer receives the feature vector. The system uses smaller convolutional kernels for initial spatial upsampling and feature channel integration to restore basic spatial structure and local illumination trends. As the layers deepen, the deeper layers of the decoder utilize a larger receptive field, combining feature maps with gradually increasing resolution from shallower upsampling layers. This first refines spatial details and then further upsamples to the target resolution. Simultaneously, it performs complex illumination modeling, fusing feature information at different scales to learn more complex global and local illumination distribution patterns. Subsequently, feature channel compression is performed, gradually reducing the number of feature channels. Finally, the output layer compresses the feature channels to a single channel, removing redundant single-channel dimensions to obtain the intensity map of each low-light image frame.

[0021] Furthermore, step 4 involves constructing a lookup table mapping using a weighted predictor and building a lookup table model that is adaptively enhanced based on the video, as detailed below:

[0022] Master Mapping It is implemented by two fully connected layers, namely and Cascade:

[0023]

[0024] The first fully connected layer Its function is to compress the feature vector output by the encoder. Mapped to a lower-dimensional space for fusing the basic strength lookup table:

[0025]

[0026] in, The feature vector is transformed by matrix multiplication and non-linear activation function operations, mapping to a weight vector of dimension λ. The weight vector Include Each element represents the relative importance of a base lookup table in the final fusion; weights are learned. The model can dynamically adjust the fusion method of the lookup table according to the video content;

[0027] The second fully connected Its function is to take from Obtained video-related weights Mapping to the final lookup table element:

[0028]

[0029] in, Will come from the first fully connected layer The output weight vector Through another matrix multiplication and nonlinear activation function operation, it is mapped to a dimension of lookup table elements This generates a lookup table that adapts to the input video content, where each element directly corresponds to the color adjustment value of a pixel in the video frame, thereby achieving color correction of the video frame.

[0030] Furthermore, step 5 involves processing the enhanced image using a denoising module to output the enhanced video, as detailed below:

[0031] Step 5.1: Receive the enhanced video sequence mapped by the intensity lookup table, and extract the spatiotemporal features of adjacent frames using spatiotemporal separable convolution, where the spatial depth separable convolution... , is represented as:

[0032]

[0033] In the formula Using the input feature map, spatially depthwise separable convolution decomposes the standard convolution into depthwise convolution and pointwise convolution, thus completing spatially separable convolution; Represents spatial feature maps; This indicates a depthwise separable convolution with a 3×3 kernel; This represents pointwise convolution;

[0034] Depthwise separable convolution , is represented as:

[0035]

[0036] Temporally depth-separable convolutions aggregate features along the temporal dimension; Represents a spatiotemporal feature map;

[0037] Step 5.2: Perform multi-scale noise modeling using an axial attention mechanism. Long-range dependencies are modeled by independently processing different axes of the feature map. The attention for the spatial axes is as follows:

[0038]

[0039] Spatial axial attention, attention-weighted feature map Specifically, it is expressed as follows:

[0040]

[0041] In the formula , , These are the query, key, and value matrices, respectively. This represents the normalized exponential function;

[0042] Step 5.3: Optical flow-guided inter-frame feature propagation ensures temporal consistency, and the output is a denoised and enhanced video with the same resolution as the input;

[0043] optical flow The propagation formula is used to guide feature propagation between adjacent frames:

[0044]

[0045] In the formula It is a weighted aggregation function; Represents the feature map after propagation;

[0046] Adjust feature weights based on optical flow:

[0047]

[0048] In the formula Indicates time The propagation characteristics at the location; Indicates time Attentional characteristics at the location; It is a Gaussian kernel weighting function based on optical flow; From time Time Optical flow;

[0049] Step 5.4: Combining the joint optimization loss function, the pairwise loss between the denoised result and the true situation is included in the loss function, defined as:

[0050]

[0051] in This represents the reconstruction loss between the transformed normal optical video and the ground truth image. This represents the denoising loss between the final denoised result and the ground truth image. and All use Loss; Smoothing Regularization Loss It consists of two parts, corresponding to the video adaptive lookup table and video-related weights, respectively, to prevent artifacts caused by extreme color changes in the lookup table; monotonicity regularization loss. Ensure robustness during the enhancement process; , These are the smoothing regularization losses. Monotonicity regularization loss Weighting coefficients; This indicates noise reduction.

[0052] A low-light video enhancement system based on a fast lookup table is disclosed. This system implements the aforementioned low-light video enhancement method based on a fast lookup table. The system includes a segmentation module, a latent feature acquisition module, an intensity map generation module, a lookup table model construction module, and a denoising module, wherein:

[0053] The module is split to obtain the initial low-light video. The video is then split frame by frame to obtain the initial low-light image.

[0054] The latent feature acquisition module processes low-light images using a lightweight encoder to obtain the latent features of the images.

[0055] The intensity map generation module converts the latent features of the image into a compact latent feature vector to generate an intensity map for each frame of low-light image.

[0056] The lookup table model building module constructs a lookup table mapping through a weight predictor and builds a lookup table model that is adaptively enhanced based on the video.

[0057] The noise reduction module processes the enhanced image and outputs the enhanced video.

[0058] A mobile terminal includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the program, implements the low-light video enhancement method based on a fast lookup table.

[0059] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps in the low-light video enhancement method based on a fast lookup table.

[0060] Compared with the prior art, the present invention has the following significant advantages: (1) It uses a stable and adaptive intensity-aware lookup table to convert RGB colors from one color space to another, which solves the common one-to-many mapping problem in low-light video enhancement. At the same time, the pixel-based lookup table transformation is consistent with all pixels with the same RGB value and enhancement intensity, which can maintain the brightness consistency between frames. It can adaptively enhance low-light images according to different lighting environments, reduce the noise of the processed video, and improve the contrast of the video; (2) It improves the problem of texture deterioration and low contrast in low-light video enhancement tasks, supplements image details, can be adaptively processed, and can achieve significant results in various lighting environments. It increases the detail processing capability after video enhancement and provides good video data for various downstream tasks; (3) It has higher computational efficiency and stronger real-time performance, which meets the real-time requirements of video image processing. Attached Figure Description

[0061] Figure 1 This is a flowchart illustrating a low-light video enhancement method based on a fast lookup table according to the present invention.

[0062] Figure 2 These are images of different low-light conditions and their corresponding intensity maps in embodiments of the present invention.

[0063] Figure 3 This is a schematic diagram illustrating the principle of the four-linear interpolation method in an embodiment of the present invention.

[0064] Figure 4 This is a schematic diagram of an initial low-light video screenshot in an embodiment of the present invention.

[0065] Figure 5 This is a schematic diagram of a video screenshot after enhancing a low-light image in an embodiment of the present invention. Detailed Implementation

[0066] The present invention will now be described in further detail with reference to the accompanying drawings and specific embodiments.

[0067] like Figure 1 As shown, the present invention provides a low-light video enhancement method based on a fast lookup table, comprising the following steps:

[0068] Step 1: Obtain the initial low-light video, and split the video frame by frame to obtain the initial low-light image;

[0069] Step 2: Process the low-light image using a lightweight encoder to obtain the image's latent features;

[0070] Step 3: Convert the latent features of the image into a compact latent feature vector to generate an intensity map for each frame of the low-light image;

[0071] Step 4: Construct a lookup table mapping using a weight predictor and build a lookup table model that is adaptively enhanced based on the video.

[0072] Step 5: Use the noise reduction module to process the enhanced image and output the enhanced video.

[0073] As a specific example, step 1 involves obtaining the initial low-light video, splitting the video frame by frame to obtain the initial low-light image, as detailed below:

[0074] An initial, unprocessed low-light video is obtained and split into multiple initial low-light image groups with the same resolution based on the video shooting frame rate. The resolution is in pixels, and each initial low-light image group contains multiple consecutive low-light video frames.

[0075] The initial low-light video required can be obtained by adjusting the ambient light or recording the video using the acquisition device under different lighting conditions, or by using an existing low-light video dataset. This invention does not limit the content of the low-light video. For example, the original video can be furniture or objects filmed indoors, or vehicles or buildings filmed outdoors.

[0076] As a specific example, step 2 involves processing the low-light image using a lightweight encoder to obtain the image's latent features, as detailed below:

[0077] Input low-light video sequence The resolution is , represented as a low-light video of frame 1, with data dimensions of 1. The obtained resolution is The low-light image of the frame is used as input, through a process with five... A lightweight encoder for each convolutional layer, using... Based on the kernel size, perform spatial upsampling (usually using a step size). The process involves increasing the height and width of the feature map to approximate the target resolution. Simultaneously, each layer performs feature fusion and channel compression, integrating multi-channel information from the input using learned convolutional kernels and significantly reducing its channel dimension (gradually decreasing from an initial high number of channels to smaller values). This is ultimately used to capture a rough understanding of the input image and some global attributes. As the convolutional kernel slides across the input video patch, it calculates a weighted sum (plus a bias) of all pixel values ​​within that local spatiotemporal region, enabling it to learn and extract specific spatiotemporal patterns, including temporal features (motion of objects between adjacent frames (optical flow), scene changes, continuity of brightness / noise) and spatial features (visual elements such as edges, textures, and object shapes within a single frame).

[0078] A series of convolutional layers form a hierarchical structure. Shallow layers use a spatial kernel smaller than a set value and a temporal range shorter than a set value to extract basic spatiotemporal gradients, local motion, and simple textures. Deeper layers capture more complex spatiotemporal structures, semantic information (such as the temporal coherence of object contours), and global illumination dynamics by increasing the receptive field. Each layer builds upon the features extracted by the previous layer to achieve a higher level of understanding. These convolutional layers work together to capture the temporal continuity and spatial visual features of video frames, ultimately encoding the input low-light video frames into a compact and information-rich set of latent feature maps.

[0079] As a specific example, the latent feature map output by the lightweight encoder is a set of Tensor; each time step Corresponding to one channel Feature map, spatial size is The resolution is significantly smaller than the input resolution, resulting in highly compressed information. Each channel represents a specific spatiotemporal feature learned by the network, and the feature vector at each spatial location incorporates information from the corresponding spatial region and its neighboring time frames in the original input.

[0080] As a specific example, step 3 involves converting the latent features of the image into a compact latent feature vector to generate an intensity map for each frame of the low-light image: the fifth deconvolution layer precisely upsamples the feature map to the target size and compresses the number of channels to 1, generating the raw intensity value for each pixel location. Finally, an activation function (such as...) is typically applied. Constrain these original values ​​to the required strength range (e.g.) Then, output the final intensity map of each low-light image frame. The specific steps are as follows:

[0081] Step 3.1: Output a set of data from the lightweight encoder. The tensor is transformed into a compact feature vector. Size is hyperparameters and These represent the number of pixels and the number of channels in the image, respectively.

[0082] Step 3.2: Convert the feature vector The data is fed into the decoder, where the shallow layer receives the feature vector. Use a smaller convolution kernel (such as Initial spatial upsampling and feature channel integration are performed to restore the basic spatial structure and local illumination trends. As the layers deepen, the deeper layers of the decoder utilize a larger receptive field (achieved through stacked layers or larger convolutional kernels), combining feature maps from shallower upsampled layers with gradually increasing resolution. First, spatial details are refined, and further upsampling is performed to the target resolution. Simultaneously, complex illumination modeling is performed, fusing feature information at different scales to learn more complex global and local illumination distribution patterns. Then, feature channel compression is performed, gradually reducing the number of feature channels, ultimately compressing it to a single channel at the output layer. Redundant single-channel dimensions are removed to obtain the intensity map of each frame of low-light image, such as... Figure 2 As shown.

[0083] As a specific example, step 4, which involves constructing a lookup table mapping using a weight predictor and building a lookup table model that is adaptively enhanced based on the video, is detailed below:

[0084] The encoder output is transformed into a compact latent feature vector. Size is determined by hyperparameters and The decision, representing the number of pixels and the number of channels respectively, is passed through a weighted predictor, and two fully connected layers together form an efficient mapping process.

[0085] Master Mapping It is implemented by two fully connected layers, namely and The cascading of these parameters, a hierarchical mapping method, helps reduce the number of parameters, thereby lowering computational costs.

[0086]

[0087] The first fully connected layer The first fully connected layer is responsible for learning the weight representation of the video content. Its function is to compress the feature vector output by the encoder. Mapped to a lower-dimensional space for fusing the basic strength lookup table:

[0088]

[0089] in, The feature vector is transformed by matrix multiplication and non-linear activation function operations, mapping to a weight vector of dimension λ. The weight vector Include Each element represents the relative importance of a base lookup table in the final fusion; weights are learned. The model can dynamically adjust the fusion method of the lookup table according to the video content;

[0090] The second fully connected Its function is to take from Obtained video-related weights Mapping to the final lookup table element:

[0091]

[0092] in, Will come from the first fully connected layer The output weight vector Through another matrix multiplication and nonlinear activation function operation, it is mapped to a dimension of lookup table elements This generates a lookup table that adapts to the input video content, where each element directly corresponds to the color adjustment value of a pixel in the video frame, thereby achieving color correction of the video frame.

[0093] As a specific example, such as Figure 3 As shown, for sparse discrete In terms of the input space, this mapping is implemented using quadlinear interpolation, as follows:

[0094] make Then we have:

[0095]

[0096] in This indicates the input red, green, and blue colors and the enhancement intensity. It is the mapped color value.

[0097] make To find the number of grid points in each dimension of the table;

[0098] Representing grid points index

[0099] in .

[0100] For this grid point The stored value used for color mapping in the lookup table is represented as follows:

[0101]

[0102] If you enter the index If it cannot be mapped to any grid point, quadlinear interpolation will be applied in the nearest unit lattice.

[0103] The grid points are:

[0104]

[0105] Among them are:

[0106]

[0107] Furthermore, we can obtain:

[0108]

[0109] So, unit lattice The quadlinear interpolation process in [the context] can be described as follows:

[0110]

[0111] Where the coefficient Indicates the input index relative to the matrix. The offset of the most recent 24 sampled grids.

[0112] The mapping can now be represented as:

[0113]

[0114] As a specific example, step 5 describes processing the enhanced image using a denoising module to output an enhanced video: A lightweight temporal-spatial joint filtering module is designed, constructing a short temporal window (e.g., 3-5 frames), utilizing... Convolutional processing extracts spatiotemporal features from adjacent frames, and deformable convolutions adapt to moving scenes to separate noise from details. Simultaneously, a noise estimation sub-network is introduced to dynamically adjust the filtering intensity based on the estimated noise level, avoiding uniform smoothing. Finally, cross-frame feature fusion (such as an attention mechanism) can further suppress residual flicker noise. The steps are as follows:

[0115] Step 5.1: Receive the enhanced video sequence mapped by the intensity lookup table, and extract the spatiotemporal features of adjacent frames using spatiotemporal separable convolution, where the spatial depth separable convolution... , is represented as:

[0116]

[0117] In the formula Using the input feature map, spatially depthwise separable convolution decomposes the standard convolution into depthwise convolution and pointwise convolution, thus completing spatially separable convolution; Represents spatial feature maps; This indicates a depthwise separable convolution with a 3×3 kernel; This represents pointwise convolution;

[0118] Depthwise separable convolution , is represented as:

[0119]

[0120] Temporally depth-separable convolutions aggregate features along the temporal dimension; Represents a spatiotemporal feature map;

[0121] Step 5.2: Perform multi-scale noise modeling using an axial attention mechanism. Long-range dependencies are modeled by independently processing different axes of the feature map. The attention for the spatial axes is as follows:

[0122]

[0123] Spatial axial attention, attention-weighted feature map Specifically, it is expressed as follows:

[0124]

[0125] In the formula , , These are the query, key, and value matrices, respectively. This represents the normalized exponential function;

[0126] Step 5.3: Optical flow-guided inter-frame feature propagation ensures temporal consistency, and the output is a denoised and enhanced video with the same resolution as the input;

[0127] optical flow The propagation formula is used to guide feature propagation between adjacent frames:

[0128]

[0129] In the formula It is a weighted aggregation function; Represents the feature map after propagation;

[0130] Adjust feature weights based on optical flow:

[0131]

[0132] In the formula Indicates time The propagation characteristics at the location; Indicates time Attentional characteristics at the location; It is a Gaussian kernel weighting function based on optical flow; From time Time Optical flow;

[0133] Step 5.4: Combining the joint optimization loss function, the pairwise loss between the denoised result and the true situation is included in the loss function, defined as:

[0134]

[0135] in This represents the reconstruction loss between the transformed normal optical video and the ground truth image. This represents the denoising loss between the final denoised result and the ground truth image. and All use Loss; Smoothing Regularization Loss It consists of two parts, corresponding to the video adaptive lookup table and video-related weights, respectively, to prevent artifacts caused by extreme color changes in the lookup table; monotonicity regularization loss. Ensure robustness during the enhancement process; , These are the smoothing regularization losses. Monotonicity regularization loss Weighting coefficients; This indicates noise reduction.

[0136] This invention also provides a low-light video enhancement system based on a fast lookup table. This system implements the aforementioned low-light video enhancement method based on a fast lookup table. The system includes a splitting module, a latent feature acquisition module, an intensity map generation module, a lookup table model construction module, and a denoising module, wherein:

[0137] The module is split to obtain the initial low-light video. The video is then split frame by frame to obtain the initial low-light image.

[0138] The latent feature acquisition module processes low-light images using a lightweight encoder to obtain the latent features of the images.

[0139] The intensity map generation module converts the latent features of the image into a compact latent feature vector to generate an intensity map for each frame of low-light image.

[0140] The lookup table model building module constructs a lookup table mapping through a weight predictor and builds a lookup table model that is adaptively enhanced based on the video.

[0141] The noise reduction module processes the enhanced image and outputs the enhanced video.

[0142] The present invention also provides a mobile terminal, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the low-light video enhancement method based on a fast lookup table.

[0143] The present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps in the low-light video enhancement method based on a fast lookup table.

[0144] Example

[0145] This embodiment employs a low-light video enhancement method based on a fast lookup table, as described in this invention, to enhance low-light video. Figure 4 The following images show initial low-light videos of different scenarios undergoing video enhancement. Figure 4 (a), (b), and (c) in the image correspond to three different low-light scenes; the enhanced video screenshots are as follows: Figure 5 As shown, Figure 5 (a), (b), and (c) in the text correspond to respectively Figure 4 Three different low-light scenarios were presented. It can be seen that the method of this invention solves the common one-to-many mapping problem in low-light video enhancement. Simultaneously, the pixel-based lookup table transformation is consistent with all pixels having the same RGB values ​​and enhancement intensity, maintaining inter-frame brightness consistency. It can adaptively enhance low-light images according to different lighting environments, reduce noise in the processed video, improve video contrast, and alleviate the problems of texture degradation and low contrast in low-light video enhancement tasks. It also replenishes image details, can adaptively process, and achieves significant results in various lighting environments, increasing the detail processing capability of the enhanced video and providing excellent video data for various downstream tasks.

[0146] The above are merely preferred embodiments of the present invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A low-light video enhancement method based on a fast lookup table, characterized in that, Includes the following steps: Step 1: Obtain the initial low-light video, and split the video frame by frame to obtain the initial low-light image; Step 2: Process the low-light image using a lightweight encoder to obtain the image's latent features; Step 3: Convert the latent features of the image into a compact latent feature vector to generate an intensity map for each frame of the low-light image; Step 4: Construct a lookup table mapping using a weight predictor and build a lookup table model that is adaptively enhanced based on the video. Step 5: Use the noise reduction module to process the enhanced image and output the enhanced video.

2. The low-light video enhancement method based on a fast lookup table according to claim 1, characterized in that, Step 1 involves obtaining the initial low-light video, which is then split frame by frame to obtain the initial low-light image, as detailed below: An initial, unprocessed low-light video is obtained and split into multiple initial low-light image groups with the same resolution based on the video shooting frame rate. The resolution is in pixels, and each initial low-light image group contains multiple consecutive low-light video frames.

3. The low-light video enhancement method based on a fast lookup table according to claim 2, characterized in that, Step 2 describes processing the low-light image using a lightweight encoder to obtain the image's latent features, as detailed below: The obtained resolution is The low-light image of the frame is used as input, through a process with five... A lightweight encoder for convolutional layers, with one for each layer. The convolutional kernel performs spatial upsampling to increase the height and width of the feature map; at the same time, each layer performs feature fusion and channel compression. When the convolutional kernel slides on the input video block, it calculates the weighted sum of all pixel values ​​in the local spatiotemporal region, learns and extracts spatiotemporal patterns, including temporal and spatial features. By utilizing learned convolutional kernels, multi-channel information from the input is integrated, and channel dimensionality is reduced; A series of convolutional layers form a hierarchical structure. Shallow layers use a spatial kernel smaller than a set value and a temporal range shorter than a set value to extract basic spatiotemporal gradients, local motion, and simple textures. Deeper layers capture more complex spatiotemporal structures, semantic information, and global illumination dynamics by increasing the receptive field. Each layer builds upon the features extracted by the previous layer to achieve a higher level of understanding. These convolutional layers work together to capture the temporal continuity and spatial visual features of video frames, ultimately encoding the input low-light video frames into a set of latent feature maps.

4. The low-light video enhancement method based on a fast lookup table according to claim 3, characterized in that, The latent feature map output by the lightweight encoder is a set Tensor; each time step Corresponding to one channel Feature map, spatial size is The resolution is smaller than the input resolution; each channel represents a specific spatiotemporal feature learned by the network, and the feature vector at each spatial location incorporates information from the corresponding spatial region and its neighboring time frames in the original input.

5. The low-light video enhancement method based on a fast lookup table according to claim 4, characterized in that, Step 3 involves converting the latent features of the image into a compact latent feature vector to generate an intensity map for each frame of the low-light image, as detailed below: Step 3.1: Output a set of data from the lightweight encoder. The tensor is transformed into a compact feature vector. Size is hyperparameters and These represent the number of pixels and the number of channels in the image, respectively. Step 3.2: Convert the feature vector The data is fed into the decoder, where the shallow layer receives the feature vector. Using smaller convolution kernels, preliminary spatial upsampling and feature channel integration are performed to restore the basic spatial structure and local illumination trends. As the layers deepen, the deeper layers of the decoder utilize a larger receptive field, combining feature maps from shallower layers with gradually increasing resolution. First, spatial details are refined, and then further upsampled to the target resolution. Simultaneously, complex lighting modeling is performed, fusing feature information at different scales to learn more complex global and local lighting distribution patterns. Subsequently, feature channel compression is performed, gradually reducing the number of feature channels, and finally compressing it into a single channel at the output layer. Redundant single-channel dimensions are removed to obtain the intensity map of each frame of low-light image.

6. The low-light video enhancement method based on a fast lookup table according to claim 5, characterized in that, Step 4 describes constructing a lookup table mapping using a weighted predictor and building a lookup table model that is adaptively enhanced based on the video, as detailed below: Master Mapping It is implemented by two fully connected layers, namely and Cascade: ； The first fully connected layer Its function is to compress the feature vector output by the encoder. Mapped to a lower-dimensional space for fusing the basic strength lookup table: ； in, The feature vector is transformed by matrix multiplication and non-linear activation function operations, mapping to a weight vector of dimension λ. The weight vector Include Each element represents the relative importance of a base lookup table in the final fusion; weights are learned. The model can dynamically adjust the fusion method of the lookup table according to the video content; The second fully connected Its function is to take from Obtained video-related weights Mapping to the final lookup table element: ； in, Will come from the first fully connected layer The output weight vector Through another matrix multiplication and nonlinear activation function operation, it is mapped to a dimension of lookup table elements This generates a lookup table that adapts to the input video content, where each element directly corresponds to the color adjustment value of a pixel in the video frame, thereby achieving color correction of the video frame.

7. The low-light video enhancement method based on a fast lookup table according to claim 6, characterized in that, Step 5 describes using a denoising module to process the enhanced image and output the enhanced video, as follows: Step 5.1: Receive the enhanced video sequence mapped by the intensity lookup table, and extract the spatiotemporal features of adjacent frames using spatiotemporal separable convolution, where the spatial depth separable convolution... , represented as: ； In the formula Using the input feature map, spatially depthwise separable convolution decomposes the standard convolution into depthwise convolution and pointwise convolution, thus completing spatially separable convolution; Represents spatial feature maps; This indicates a depthwise separable convolution with a 3×3 kernel; This represents pointwise convolution; Depthwise separable convolution , represented as: ； Temporally depth-separable convolutions aggregate features along the temporal dimension; Represents a spatiotemporal feature map; Step 5.2: Perform multi-scale noise modeling using an axial attention mechanism. Long-range dependencies are modeled by independently processing different axes of the feature map. The attention for the spatial axes is as follows: ； Spatial axial attention, attention-weighted feature map Specifically, it is expressed as follows: ； In the formula , , These are the query, key, and value matrices, respectively. This represents the normalized exponential function; Step 5.3: Optical flow-guided inter-frame feature propagation ensures temporal consistency, and the output is a denoised and enhanced video with the same resolution as the input; optical flow The propagation formula is used to guide feature propagation between adjacent frames: ； In the formula It is a weighted aggregation function; Represents the feature map after propagation; Adjust feature weights based on optical flow: ； In the formula Indicates time The propagation characteristics at the location; Indicates time Attentional characteristics at the location; It is a Gaussian kernel weighting function based on optical flow; From time Time Optical flow; Step 5.4: Combining the joint optimization loss function, the pairwise loss between the denoised result and the true situation is included in the loss function, defined as: ； in This represents the reconstruction loss between the transformed normal optical video and the ground truth image. This represents the denoising loss between the final denoised result and the ground truth image. and All use Loss; Smoothing Regularization Loss It consists of two parts, corresponding to the video adaptive lookup table and video-related weights, respectively, to prevent artifacts caused by extreme color changes in the lookup table; monotonicity regularization loss. Ensure robustness during the enhancement process; , These are the smoothing regularization losses. Monotonicity regularization loss Weighting coefficients; This indicates noise reduction.

8. A low-light video enhancement system based on a fast lookup table, characterized in that, This system is used to implement the low-light video enhancement method based on a fast lookup table as described in any one of claims 1 to 7. The system includes a splitting module, a latent feature acquisition module, an intensity map generation module, a lookup table model construction module, and a denoising module, wherein: The module is split to obtain the initial low-light video. The video is then split frame by frame to obtain the initial low-light image. The latent feature acquisition module processes low-light images using a lightweight encoder to obtain the latent features of the images. The intensity map generation module converts the latent features of the image into a compact latent feature vector to generate an intensity map for each frame of low-light image. The lookup table model building module constructs a lookup table mapping through a weight predictor and builds a lookup table model that is adaptively enhanced based on the video. The noise reduction module processes the enhanced image and outputs the enhanced video.

9. A mobile terminal, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the low-light video enhancement method based on a fast lookup table as described in any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the steps in the low-light video enhancement method based on a fast lookup table as described in any one of claims 1 to 7.

Images