Low-illumination video quality measurement system based on deep learning

Abstract

The invention relates to the technical field of quality measurement, in particular to a low-illumination video quality measurement system based on deep learning. The system comprises a low-illumination video frame processing module, a video quality dimension measurement labeling module, a video quality model prediction module and a prediction error loss optimization module, and can utilize video acquisition equipment to acquire corresponding low-illumination video data in real time and perform video frame processing. Carrying out video index measurement and quality dimension measurement labeling at the same time to obtain a low-illumination video quality labeling score; constructing a video quality measurement model architecture by using a convolutional neural network, and carrying out model training optimization and quality weighted summation to obtain a low-illumination video quality prediction score; and error loss reinforcement learning and video quality evaluation measurement are carried out based on the low-illumination video quality annotation score and the low-illumination video quality prediction score so as to output a corresponding low-illumination video quality comprehensive measurement score. According to the invention, accurate evaluation and measurement of the low-illumination video quality can be realized.

Description

Technical Field

[0001] The present invention relates to the technical field of quality measurement, and in particular to a low-illumination video quality measurement system based on deep learning. Background Art

[0002] During the video acquisition process, low-light environment is a common and challenging scenario. Low light can cause video quality problems such as increased noise, reduced contrast, color distortion, and loss of details, which seriously affects the video viewing experience and subsequent analysis and utilization.

[0003] Traditional video quality measurement methods are mainly based on manual features and statistical models, such as pixel-based quality assessment indicators (such as mean square error, peak signal-to-noise ratio) and structure-based assessment indicators (such as structural similarity index). However, these methods have obvious shortcomings when processing low-light videos. Manual features are difficult to fully capture the complex visual characteristics of low-light videos, and statistical models are unable to adapt to the diversity of low-light videos in different scenarios. Moreover, traditional methods often only focus on a single quality variable and cannot achieve a comprehensive and accurate evaluation of the overall quality of low-light videos, thereby reducing the accuracy of low-light video quality measurement. Summary of the invention

[0004] Based on this, it is necessary for the present invention to provide a low-illumination video quality measurement system based on deep learning to solve at least one of the above technical problems.

[0005] To achieve the above objectives, a low-light video quality measurement system based on deep learning includes the following modules: A low-illumination video frame processing module is used to collect corresponding low-illumination video data in real time by using a video acquisition device in different low-illumination scenes, and perform video frame processing on the low-illumination video data to obtain a low-illumination video standard frame image set; The video quality dimension measurement and annotation module is used to measure the video index of each frame of the low-light video standard frame image set to obtain the video clarity, video noise level and video color restoration corresponding to each low-light video frame; based on the video clarity, video noise level and video color restoration corresponding to each low-light video frame, the corresponding low-light video is measured and annotated in terms of quality dimensions to obtain a low-light video quality annotation score; A video quality model prediction module is used to construct a video quality measurement model architecture using a convolutional neural network, and to optimize the video quality measurement model architecture using a low-light video standard frame image set to generate a low-light video measurement model based on deep learning, and output a prediction score corresponding to each video frame quality dimension; based on the prediction score corresponding to each video frame quality dimension, a quality-weighted sum is performed on the corresponding low-light video to obtain a low-light video quality prediction score; The prediction error loss optimization module is used to perform error loss reinforcement learning on the deep learning-based low-illumination video measurement model based on the low-illumination video quality annotation score and the low-illumination video quality prediction score to generate a low-illumination video measurement comprehensive model; the low-illumination video standard frame image set is re-input into the low-illumination video measurement comprehensive model for video quality assessment measurement to output the corresponding low-illumination video quality comprehensive measurement score.

[0006] Furthermore, the low illumination video frame processing module includes the following functions: By using video acquisition equipment in different low-light scenes to collect corresponding low-light video data in real time; The low-illumination video data is subjected to video single-frame extraction at a fixed frame rate of 24fps to obtain a low-illumination video single-frame image set; The frame image blurriness of a single frame image set of a low-illumination video is measured to obtain the frame image blurriness of the low-illumination video; Performing video frame image filtering on each low-illumination video frame in a low-illumination video single frame image set based on the low-illumination video frame image blurriness to obtain a low-illumination video frame filtered image set; The pixel values ​​corresponding to each frame of the low-illumination video image in the low-illumination video frame filtering image set are normalized to obtain a low-illumination video standard frame image set.

[0007] Furthermore, the different low-light scenes are specifically outdoor scenes at night, underground parking lots, and indoor low-light environment scenes.

[0008] Furthermore, the video quality dimension measurement and annotation module includes the following functions: Perform multi-channel decomposition on each frame of video image in the low-illumination video standard frame image set to obtain R, G, and B channel images corresponding to each frame of video image; obtain the grayscale mean and grayscale variance corresponding to each channel through the R, G, and B channel images corresponding to each frame of video image, and measure the channel energy ratio of each frame of video image in the low-illumination video standard frame image set based on the grayscale mean and grayscale variance corresponding to each channel to obtain the channel image energy ratio corresponding to each frame of video image; perform video clarity analysis on each corresponding frame of video image according to the channel image energy ratio corresponding to each frame of video image to obtain the video clarity corresponding to each low-illumination video frame; Measuring the noise level of each frame of video image in the low-illumination video standard frame image set to obtain the video noise level corresponding to each low-illumination video frame; The color restoration degree of each frame of the video image in the low-illumination video standard frame image set is measured to obtain the video color restoration degree corresponding to each low-illumination video frame; Based on the video clarity, video noise level and video color restoration corresponding to each low-illumination video frame, a video quality dimension measurement calculation formula is used to perform quality dimension measurement calculation on the corresponding low-illumination video to obtain the low-illumination video quality dimension; Based on the low-illumination video quality dimension, the corresponding low-illumination video is labeled with a quality dimension system, so that the quality dimension of 0%-20% is labeled as 1 point, 21%-40% is labeled as 2 points, 41%-60% is labeled as 3 points, 61%-80% is labeled as 4 points, and 81%-100% is labeled as 5 points, so as to obtain the low-illumination video quality labeling score.

[0009] Furthermore, the measuring of the noise level of each frame of video image in the low-illumination video standard frame image set includes: Dividing each frame of video image in the low-illumination video standard frame image set into a local image region to obtain a local video sub-region corresponding to each frame of video image; Calculating the texture fractal dimension of the local sub-region of the video corresponding to each frame of the video image to obtain the fractal dimension of the local region corresponding to each frame of the video image; Calculating the local pixel variance of the local sub-region of the video corresponding to each frame of the video image to obtain the local region pixel variance corresponding to each frame of the video image; The noise level of each frame of video image in the low-illumination video standard frame image set is measured based on the local area fractal dimension and the local area pixel variance corresponding to each frame of video image, so as to obtain the video noise level corresponding to each low-illumination video frame.

[0010] Furthermore, the color restoration measurement of each frame of the video image in the low-illumination video standard frame image set includes: Calculating the chromaticity coordinates of different color regions corresponding to each frame of the video image in the low-illumination video standard frame image set to obtain the chromaticity coordinates corresponding to the different color regions in each frame of the video image; Based on the chromaticity coordinates corresponding to the different color areas in each frame of the video image, the color saturation of the corresponding different color areas in each frame of the video image in the low-illumination video standard frame image set is measured to obtain the color saturation corresponding to the different color areas in each frame of the video image; Performing hue distribution statistics on different color regions corresponding to each frame of video image in the low-illumination video standard frame image set to obtain the hue distribution corresponding to different color regions in each frame of video image; The color restoration degree of each video frame in the low-illumination video standard frame image set is measured based on the color saturation and hue distribution corresponding to different color areas in each video frame to obtain the video color restoration degree corresponding to each low-illumination video frame.

[0011] Furthermore, the video quality dimension measurement calculation formula is specifically as follows: ; In the formula, is the low-light video quality dimension, is the total number of low-light video frames, For the Low-light video frames, For the The video clarity corresponding to the low-light video frame, For the The video noise level corresponding to a low-light video frame, For the The video color restoration degree corresponding to the low-light video frame, is the video frame time interval, specifically from arrive time range, For in time The corresponding low-light video frame is as follows: is the time frame parameter, is the reference video frame, is the noise penalty factor.

[0012] Furthermore, the video quality model prediction module includes the following functions: Construct a video quality measurement model architecture using convolutional neural networks, which includes a sharp feature extraction branch, a noise feature extraction branch, and a fully connected neural network layer; The low-illumination video standard frame image set is divided into a training set, a validation set, and a test set according to the division ratio of 7:2:1; The training set is input into the video quality measurement model architecture for model training, and stochastic gradient descent is used as an optimization algorithm combined with the validation set to continuously adjust the corresponding hyperparameters of the model, including learning rate, batch size, and number of network layers. At the same time, the test set is used to optimize the model performance to generate a low-light video measurement model based on deep learning, and the prediction score corresponding to each video frame quality dimension is outputted at each fully connected neural network layer; Based on the prediction score corresponding to each video frame quality dimension, the quality weighted sum of the corresponding low-light video is performed to obtain the low-light video quality prediction score.

[0013] Furthermore, the video quality measurement model architecture is specifically composed of a clear feature extraction branch, a noise feature extraction branch and multiple fully connected neural network layers, wherein the clear feature extraction branch is composed of 5 5x5 convolutional layers and 5 1x1 pooling layers, the noise feature extraction branch is composed of 5 3x3 convolutional layers and 5 1x1 pooling layers, and the fully connected neural network layer is determined by the total number corresponding to the quality dimension, so as to weightedly fuse the video frame features extracted by each branch into a comprehensive feature vector, and input the comprehensive feature vector into each fully connected neural network layer for nonlinear transformation to output the prediction score corresponding to each quality dimension.

[0014] Furthermore, the prediction error loss optimization module includes the following functions: The quality mean square error loss is calculated between the low-illumination video quality annotation score and the low-illumination video quality prediction score to obtain the video quality error loss between the model prediction score and the annotation score; Based on the video quality error loss between the model prediction score and the labeled score, the deep learning-based low-light video measurement model is subjected to error loss reinforcement learning, and the hyperparameters corresponding to the low-light video measurement model are constrained from overfitting by introducing the L2 regularization term to generate a comprehensive low-light video measurement model. The low-illumination video standard frame image set is re-input into the low-illumination video measurement comprehensive model for video quality assessment measurement to output the corresponding low-illumination video quality comprehensive measurement score.

[0015] Beneficial effects of the present invention: The low-illumination video quality measurement system based on deep learning proposed in the present invention is composed of a low-illumination video frame processing module, a video quality dimension measurement and annotation module, a video quality model prediction module and a prediction error loss optimization module. Compared with the prior art, the beneficial effect of the present application is that by using a video acquisition device to collect low-illumination video data in different low-illumination scenes, it can ensure that the clarity, noise and color reproduction of the video data are accurately captured. In a low-illumination environment, the video acquisition device needs to have higher sensitivity and performance to cope with the noise and detail loss problems caused by insufficient light. The video frame processing technology further optimizes the quality of each frame of the image. By denoising, enhancing contrast, adjusting exposure, etc., the quality of the low-illumination video can be made more stable and controllable. This processing process not only makes the video image clearer, but also improves the reliability and accuracy of the video data in subsequent analysis, thereby ensuring the accuracy and effectiveness of subsequent measurement and analysis. Secondly, by measuring the video indicators of each frame of low-light video image to obtain video clarity, noise level and color reproduction, this process is an important part of evaluating video quality. By performing in-depth image analysis on each frame of low-light video, the performance of video images in different quality dimensions can be quantified. The measurement of video clarity helps to determine the degree of detail and blur in the image; the noise level measurement evaluates the degree of noise interference in the image. Excessive noise will affect the viewing experience of the video; the color reproduction measurement determines the accuracy and reproduction of the color in the video image. Color distortion will greatly affect the realism and visual effects of the video. Through these measurement indicators, a quality annotation score can be given to each video frame. The key to this step is that through comprehensive indicator analysis, it can not only accurately reflect the quality characteristics of each frame, but also provide real and high-quality annotation data for subsequent deep learning model training, effectively supporting the precise learning and optimization of the model. Then, by constructing a video quality measurement model based on convolutional neural network (CNN), and using the low-light video standard frame image set for model training and optimization, the deep learning model can automatically extract complex feature information in the video frame, such as image clarity, noise, color and other quality indicators through self-learning and optimization. The convolutional neural network has powerful image processing capabilities and can automatically identify and analyze subtle differences and details in videos under low-light environments. It is particularly suitable for processing noise and blur problems in low-light videos. Through multi-layer network structure and a large amount of low-light video data training, the deep learning model can learn the complex relationship between image quality and video frames, and give accurate quality prediction scores. By weighted prediction of the quality dimension of each video frame, the low-light video quality prediction score output by the model can provide an accurate basis for the final video quality evaluation, thereby achieving a comprehensive and accurate evaluation of the overall quality of the low-light video.Finally, error loss reinforcement learning is performed based on the quality annotation scores of low-light videos and the model prediction scores. The prediction accuracy and robustness of the model can be significantly improved through the reinforcement learning method. The error loss function plays an important role in deep learning and can guide the model to adjust according to the difference between the predicted results and the true annotations. Through reinforcement learning, the model can better adapt to changes in different environments when facing low-light videos, reduce overfitting, and improve the model's adaptability to complex scenes. The comprehensive model after reinforcement learning can not only provide accurate quality prediction scores for each video frame, but also combine multiple quality dimensions for comprehensive evaluation, thereby obtaining the overall quality score of the low-light video, thereby ensuring that the final model can provide high-precision and reliable video quality measurement results in practical applications. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Other features, objects and advantages of the present invention will become more apparent from the detailed description of non-limiting embodiments thereof made with reference to the following drawings: Figure 1 It is a module schematic diagram of a low-illumination video quality measurement system based on deep learning of the present invention; Figure 2 for Figure 1 Schematic diagram of the functional flow of the low-light video frame processing module; Figure 3 for Figure 1 Schematic diagram of the functional flow of the video quality dimension measurement and annotation module. DETAILED DESCRIPTION

[0017] The technical system of the present invention is described clearly and completely below in conjunction with the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by technicians in this field without creative work are within the scope of protection of the present invention.

[0018] In addition, the accompanying drawings are only schematic illustrations of the present invention and are not necessarily drawn to scale. The same reference numerals in the figures represent the same or similar parts, and their repeated description will be omitted. Some of the block diagrams shown in the accompanying drawings are functional entities and do not necessarily correspond to physically or logically independent entities. The functional entities can be implemented in software form, or implemented in one or more hardware modules or integrated circuits, or implemented in different networks and / or processor systems and / or microcontroller systems.

[0019] It should be understood that, although the terms "first", "second", etc. may be used herein to describe various units, these units should not be limited by these terms. These terms are used only to distinguish one unit from another unit. For example, without departing from the scope of the exemplary embodiments, the first unit may be referred to as the second unit, and similarly the second unit may be referred to as the first unit. The term "and / or" used herein includes any and all combinations of one or more of the listed associated items.

[0020] To achieve this, please refer to Figures 1 to 3 The present invention provides a low-illumination video quality measurement system based on deep learning, and the system includes the following modules: A low-illumination video frame processing module is used to collect corresponding low-illumination video data in real time by using a video acquisition device in different low-illumination scenes, and perform video frame processing on the low-illumination video data to obtain a low-illumination video standard frame image set; The video quality dimension measurement and annotation module is used to measure the video index of each frame of the low-light video standard frame image set to obtain the video clarity, video noise level and video color restoration corresponding to each low-light video frame; based on the video clarity, video noise level and video color restoration corresponding to each low-light video frame, the corresponding low-light video is measured and annotated in terms of quality dimensions to obtain a low-light video quality annotation score; A video quality model prediction module is used to construct a video quality measurement model architecture using a convolutional neural network, and to optimize the video quality measurement model architecture using a low-light video standard frame image set to generate a low-light video measurement model based on deep learning, and output a prediction score corresponding to each video frame quality dimension; based on the prediction score corresponding to each video frame quality dimension, a quality-weighted sum is performed on the corresponding low-light video to obtain a low-light video quality prediction score; The prediction error loss optimization module is used to perform error loss reinforcement learning on the deep learning-based low-illumination video measurement model based on the low-illumination video quality annotation score and the low-illumination video quality prediction score to generate a low-illumination video measurement comprehensive model; the low-illumination video standard frame image set is re-input into the low-illumination video measurement comprehensive model for video quality assessment measurement to output the corresponding low-illumination video quality comprehensive measurement score.

[0021] In the embodiment of the present invention, please refer to Figure 1 FIG. 1 is a schematic diagram of a module of a low-light video quality measurement system based on deep learning according to the present invention. In this example, the low-light video quality measurement system based on deep learning includes the following modules: S1: a low illumination video frame processing module, used to collect corresponding low illumination video data in real time by using a video acquisition device in different low illumination scenes, and perform video frame processing on the low illumination video data to obtain a low illumination video standard frame image set; In an embodiment of the present invention, when performing video acquisition in a low-light environment, first select a high-sensitivity video acquisition device suitable for working under low-light conditions, such as a high-performance CMOS sensor or an image sensor based on back-illuminated technology. The acquisition device should be able to perform stable video recording in a low-light environment to avoid video blur or motion blur caused by excessive exposure time. In different low-light scenes, the acquisition device continues to record video to ensure that sufficient low-light video data is captured. The ambient light conditions during the video acquisition process should be fully considered to ensure data diversity, especially at night or in a dim indoor environment. The acquired data should cover different low-light levels. Next, use image processing software to perform frame processing on the recorded video, extract each frame of image data, and retain representative standard frame images by removing redundant or damaged frames to ultimately form a low-light video standard frame image set.

[0022] S2: Video quality dimension measurement and annotation module, used to measure the video index of each frame of video image in the low-illumination video standard frame image set to obtain the video clarity, video noise level and video color restoration corresponding to each low-illumination video frame; based on the video clarity, video noise level and video color restoration corresponding to each low-illumination video frame, perform quality dimension measurement and annotation on the corresponding low-illumination video to obtain the low-illumination video quality annotation score; In an embodiment of the present invention, after obtaining a set of low-illumination video standard frame images, a video index measurement is performed for each frame of video image. First, a clarity measurement algorithm is used to analyze each frame of video image. Common clarity evaluation indicators include edge strength, contrast, texture features, etc. By calculating the values ​​of these features, a clarity score of each frame of video image can be obtained. Then, a noise detection method is used to evaluate the noise level of each frame of image. For example, image noise features are extracted by image gradient analysis or wavelet transform method. The measurement result of the noise level can be reflected by calculating the signal-to-noise ratio (SNR) value of the image. In addition, the color reproduction of the video image can be measured by comparing it with a standard color card. A color difference measurement method, such as CIEDE2000 or Delta E formula, is used to compare the deviation between the video frame and the standard color model to obtain a color reproduction score. By combining the clarity, noise level and color reproduction of each frame of image, a comprehensive analysis is performed to generate a corresponding low-illumination video quality annotation score for each frame of video image, and finally a low-illumination video quality annotation score is obtained.

[0023] S3: Video quality model prediction module, used to construct a video quality measurement model architecture using a convolutional neural network, and use a low-illumination video standard frame image set to perform model training and optimization on the video quality measurement model architecture to generate a low-illumination video measurement model based on deep learning, and output a prediction score corresponding to each video frame quality dimension; based on the prediction score corresponding to each video frame quality dimension, perform quality-weighted summation on the corresponding low-illumination video to obtain a low-illumination video quality prediction score; In an embodiment of the present invention, when constructing a framework of a low-illumination video quality measurement model based on deep learning technology, a convolutional neural network (CNN) model is first designed. The model can extract spatial features in video images through multiple convolutional layers and predict quality indicators through fully connected layers. To achieve quality measurement of low-illumination videos, the input of CNN is a set of standard frame images of low-illumination videos. Each frame image is preprocessed, such as grayscale, normalization, and other operations before being input into the network. The output of the network is a predicted score of clarity, noise level, and color restoration of each frame of video image. During network training, the model parameters are optimized by a back propagation algorithm to minimize the error between the predicted value and the true annotation score. In order to ensure that the model has good generalization ability, data enhancement technology is used, such as image rotation, translation, scaling, etc. to expand the training data set and reduce overfitting. By continuously adjusting the model structure and hyperparameters during the training process, the low-illumination video measurement model based on deep learning can output the predicted score corresponding to each video frame quality dimension, perform quality-weighted summation on the entire low-illumination video, and finally obtain the overall quality prediction score of the low-illumination video.

[0024] S4: A prediction error loss optimization module, which is used to perform error loss reinforcement learning on a deep learning-based low-illumination video measurement model based on a low-illumination video quality annotation score and a low-illumination video quality prediction score, so as to generate a low-illumination video measurement comprehensive model; and re-input the low-illumination video standard frame image set into the low-illumination video measurement comprehensive model for video quality assessment measurement, so as to output the corresponding low-illumination video quality comprehensive measurement score.

[0025] In an embodiment of the present invention, after obtaining the low-illumination video quality annotation score and the low-illumination video quality prediction score, the error loss reinforcement learning technology is used to further optimize the low-illumination video measurement model based on deep learning. The core of the error loss reinforcement learning process is to use the low-illumination video quality annotation score as a supervisory signal to dynamically adjust the difference between the low-illumination video quality prediction score and the annotation score. First, based on the difference between the prediction score and the annotation score output by the current model, the loss function value is calculated. The loss function value reflects the prediction error of the model in each quality dimension (clarity, noise level, color reproduction). Then, the error loss reinforcement learning technology is used to further optimize the low-illumination video measurement model based on deep learning. The learning algorithm is optimized, and training is focused on places with large errors. The weights of the model are adjusted to increase the model's learning ability for frames with large errors, thereby achieving fine optimization of the video quality. This process is carried out in continuous iterations until the model can predict the quality of low-light videos with higher accuracy. Finally, the low-light video standard frame image set is re-input into the optimized low-light video measurement comprehensive model for final video quality assessment and measurement. The final measurement outputs a comprehensive low-light video quality measurement score, which comprehensively reflects the comprehensive quality of the low-light video in terms of clarity, noise level, and color reproduction, and can provide a basis for video quality optimization or enhancement processing.

[0026] Furthermore, the low illumination video frame processing module includes the following functions: By using video acquisition equipment in different low-light scenes to collect corresponding low-light video data in real time; The low-illumination video data is subjected to video single-frame extraction at a fixed frame rate of 24fps to obtain a low-illumination video single-frame image set; Measuring the blurriness of a frame image of a low-illumination video single-frame image set to obtain the blurriness of the frame image of the low-illumination video; Performing video frame image filtering on each low-illumination video frame in a low-illumination video single frame image set based on the low-illumination video frame image blurriness to obtain a low-illumination video frame filtered image set; The pixel values ​​corresponding to each frame of the low-illumination video image in the low-illumination video frame filtered image set are normalized to obtain a low-illumination video standard frame image set.

[0027] As an embodiment of the present invention, refer to Figure 2 As shown, Figure 1 A schematic diagram of the functional flow of a low-light video frame processing module. In this embodiment, the low-light video frame processing module includes the following functions: S11: using a video acquisition device to collect corresponding low-illumination video data in real time in different low-illumination scenes; In an embodiment of the present invention, the collection of low-light video data is performed in different low-light environments through dedicated video acquisition equipment (such as night vision cameras, low-light cameras, etc.). In outdoor scenes at night, different time periods (such as 22:00 in the evening to 2:00 in the morning) are selected for collection to ensure that the collected scenes are representative and low-light. The collection of underground parking lots should be selected in areas with different light intensities. Low-light parking lot images are obtained through video acquisition equipment, especially places far away from lighting equipment. In indoor low-light environments, the acquisition equipment can be placed in insufficiently lit rooms (such as windowless or dimly lit rooms) to obtain low-light video data. In each scenario, the acquisition equipment needs to be able to maintain a stable frame rate and clarity to ensure that the data quality in different scenarios is consistent and can cover typical situations in low-light scenes, and finally obtain corresponding low-light video data.

[0028] S12: extracting single frames of the low-illumination video data at a fixed frame rate of 24 fps to obtain a single-frame image set of the low-illumination video; In an embodiment of the present invention, after the low-light video data is collected, a video processing tool is used to process the collected video to ensure that the frame rate of the video is 24fps (24 frames per second). A separate frame image per second is extracted and saved through an automated script. For each low-light video, all frame data is extracted to obtain a single-frame image set of the low-light video. This process can be done by writing a script in Python to call a video processing library (such as OpenCV), loading the video file through the cv2.VideoCapture method in the code, and using cv2.imwrite to save each frame image as a separate file for subsequent analysis and processing, thereby finally obtaining a single-frame image set of the low-light video.

[0029] S13: measuring the frame image blurriness of a single frame image set of the low illumination video to obtain the frame image blurriness of the low illumination video; In an embodiment of the present invention, blurriness measurement is performed on each frame in a single-frame image set of a low-illumination video to analyze whether the video frame is blurred. The blurriness measurement may adopt an algorithm based on image gradient, such as a Laplacian operator (Laplacian) or an algorithm based on edge detection. In a specific implementation, the cv2.Laplacian() function in OpenCV may be used to calculate the gradient of each frame image, and then obtain the blurriness value of the image. A threshold is set to determine whether the image is a blurred image. When the blurriness is greater than a set threshold, it is considered that the image is strongly blurred. In a specific implementation, an automated processing code may be written to calculate the blurriness of each frame image, and record the blurriness of each frame image, and finally obtain the blurriness of the low-illumination video frame image.

[0030] S14: performing video frame image filtering on each low-illumination video frame in the low-illumination video single frame image set based on the low-illumination video frame image blurriness to obtain a low-illumination video frame filtered image set; In an embodiment of the present invention, filtering is performed on each frame image based on a calculation result of the blurriness of a low-illumination video frame image to reduce blurring. The filtering operation may use image filtering algorithms such as Gaussian Blur and Median Filter. In specific implementation, the cv2.GaussianBlur() function in OpenCV may be called in the code to process images with higher blurriness, and the size of the Gaussian kernel may be adjusted to adapt to different degrees of blurriness. In the filtering process, a stronger filtering process is adopted for frame images with higher blurriness; a weaker filtering intensity is used for frame images with lower blurriness. The obtained low-illumination video frame filtered image set will include processed images, reduce blurring, and thus improve the clarity of the image, and finally obtain a low-illumination video frame filtered image set.

[0031] S15: performing video frame normalization on the pixel values ​​corresponding to each frame of the low-illumination video image in the low-illumination video frame filtered image set to obtain a low-illumination video standard frame image set.

[0032] In the embodiment of the present invention, the pixel values ​​of each frame of the image are normalized based on the low-illumination video frame filter image set to ensure that the pixel value range of the image is unified within a standard range (usually 0 to 255). The normalization of the image can be performed by analyzing its pixel distribution frame by frame and performing linear or nonlinear normalization on it. In specific implementation, the maximum pixel value and the minimum pixel value of each frame of the image are first calculated, and the pixel values ​​are normalized by a formula so that the values ​​are mapped to the standard range. For example, the following formula is used for normalization: ,in, is the original image pixel, and They are the minimum and maximum values ​​of the image respectively. This step ensures that the visual characteristics such as brightness and contrast of each frame of the image are within the same standard range, which is convenient for subsequent deep learning model analysis and processing, and finally generates a low-light video standard frame image set with high image quality and consistency.

[0033] Furthermore, the different low-light scenes are specifically outdoor scenes at night, underground parking lots, and indoor low-light environment scenes.

[0034] Furthermore, the video quality dimension measurement and annotation module includes the following functions: Perform multi-channel decomposition on each frame of video image in the low-illumination video standard frame image set to obtain R, G, and B channel images corresponding to each frame of video image; obtain the grayscale mean and grayscale variance corresponding to each channel through the R, G, and B channel images corresponding to each frame of video image, and measure the channel energy ratio of each frame of video image in the low-illumination video standard frame image set based on the grayscale mean and grayscale variance corresponding to each channel to obtain the channel image energy ratio corresponding to each frame of video image; perform video clarity analysis on each corresponding frame of video image according to the channel image energy ratio corresponding to each frame of video image to obtain the video clarity corresponding to each low-illumination video frame; Measuring the noise level of each frame of video image in the low-illumination video standard frame image set to obtain the video noise level corresponding to each low-illumination video frame; The color restoration degree of each frame of the video image in the low-illumination video standard frame image set is measured to obtain the video color restoration degree corresponding to each low-illumination video frame; Based on the video clarity, video noise level and video color restoration corresponding to each low-illumination video frame, a video quality dimension measurement calculation formula is used to perform quality dimension measurement calculation on the corresponding low-illumination video to obtain the low-illumination video quality dimension; Based on the low-illumination video quality dimension, the corresponding low-illumination video is labeled with a quality dimension system, so that the quality dimension of 0%-20% is labeled as 1 point, 21%-40% is labeled as 2 points, 41%-60% is labeled as 3 points, 61%-80% is labeled as 4 points, and 81%-100% is labeled as 5 points, so as to obtain the low-illumination video quality labeling score.

[0035] As an embodiment of the present invention, refer to Figure 2 As shown, Figure 1 Schematic diagram of the functional flow of the video quality dimension measurement and annotation module in the embodiment. In this embodiment, the video quality dimension measurement and annotation module includes the following functions: S21: Perform multi-channel decomposition on each frame of video image in the low-illumination video standard frame image set to obtain R, G, and B channel images corresponding to each frame of video image; obtain the grayscale mean and grayscale variance corresponding to each channel through the R, G, and B channel images corresponding to each frame of video image, and measure the channel energy ratio of each frame of video image in the low-illumination video standard frame image set based on the grayscale mean and grayscale variance corresponding to each channel to obtain the channel image energy ratio corresponding to each frame of video image; perform video clarity analysis on each corresponding frame of video image according to the channel image energy ratio corresponding to each frame of video image to obtain the video clarity corresponding to each low-illumination video frame; In an embodiment of the present invention, by splitting the RGB (red, green, blue) channels of each frame in the low-illumination video standard frame image set, in specific implementation, a computer vision library, such as OpenCV or PIL (Python Imaging Library), is used to perform color space separation on each frame of the image. The RGB information of each frame of the image can be obtained by separating each color channel of the image, wherein the R channel contains the pixel value of the red channel, the G channel contains the pixel value of the green channel, and the B channel contains the pixel value of the blue channel. The split result is three single-channel grayscale images, corresponding to the images of the red, green and blue channels, respectively. In each single-channel image, the pixel value reflects the brightness information of the channel, and each pixel value range of the R, G, and B channels is usually 0 to 255. This process is performed on all video frames to obtain the RGB channel images corresponding to each frame. At the same time, by calculating the grayscale mean and grayscale variance of the R, G, and B channel images of each video frame, the grayscale mean reflects the brightness concentration of the image on the channel, while the grayscale variance measures whether the brightness distribution of the image is uniform. In specific implementation, for the image of each channel, first extract the values ​​of all pixels, calculate the mean (that is, the sum of all pixel values ​​divided by the total number of pixels) and variance (that is, the average of the square of the difference between each pixel value and the mean) of the channel, then calculate the energy share of the channel based on the mean and variance of each channel. The energy share calculation formula can be performed by standardizing the relationship between the variance and the mean of each channel, so as to obtain the energy distribution of each frame of the image on each color channel. Finally, the energy share reflects the different color channels in the video frame. In addition, the clarity of each frame of video image is analyzed mainly based on the aforementioned channel energy ratio results. In specific implementation, based on the channel energy ratio, a weighted clarity evaluation model can be constructed. This model combines the contribution of different channels in the video image. First, by combining the energy ratio of each channel with the global clarity index of the image (such as image gradient, edge information, etc.), the clarity score of each frame of the image is calculated. Through these calculations, the clarity analysis of each frame of the low-light video can be performed to evaluate the clarity level of each frame of the image. The results of the clarity analysis help to understand the visual sharpness and detail presentation of the low-light video image, and finally the video clarity corresponding to each low-light video frame is obtained.

[0036] S22: measuring the noise level of each frame of the video image in the low-illumination video standard frame image set to obtain a video noise level corresponding to each low-illumination video frame; In an embodiment of the present invention, by measuring the noise level of each frame of video image, the calculation of noise usually involves comparing the difference between the real signal and the noise in the image. First, it is necessary to extract the background information of the image from each frame of image, and assume that the background information is the part with less noise. The image can be smoothed by using image filtering technology (such as Gaussian blur), and the difference between the original image and the smoothed image is calculated to obtain the noise. The difference is the noise information. Subsequently, the standard deviation, mean and energy value of the noise can be calculated to measure the noise level of each frame of video image. The higher the noise level, the worse the image quality, which affects the clarity and visual effect of the image. By quantifying the noise level, the noise problem caused by insufficient light in low-light video can be effectively evaluated, and finally the video noise level corresponding to each low-light video frame is obtained.

[0037] S23: measuring the color restoration degree of each frame of the video image in the low-illumination video standard frame image set to obtain the video color restoration degree corresponding to each low-illumination video frame; In an embodiment of the present invention, the color reproduction degree of each frame of video image is measured to evaluate the color reproduction accuracy in low-light video. First, the R, G, and B channels of each frame of image are extracted and compared with an ideal standard color image. The standard color image can be obtained through a set of standard test images (such as a standard color card or an artificially synthesized image). The color difference between each color channel of each frame of image and the corresponding channel of the standard image is calculated using the color difference measurement of the image (such as the CIEDE2000 color difference formula). The smaller the color difference, the higher the color reproduction degree of the video image. The color reproduction degree score of each frame is obtained by calculating the color difference of each frame of video image and weighting it, and then the color accuracy of the low-light video is evaluated, so as to obtain the color reproduction degree of each frame of video based on the color difference score, and finally obtain the video color reproduction degree corresponding to each low-light video frame.

[0038] S24: performing quality dimension measurement calculation on the corresponding low-illumination video based on the video clarity, video noise level, and video color restoration corresponding to each low-illumination video frame using a video quality dimension measurement calculation formula to obtain a low-illumination video quality dimension; In an embodiment of the present invention, a suitable video quality dimension measurement calculation formula is formed by combining the total number of low-illumination video frames, the video clarity corresponding to the low-illumination video frames, the video noise level, the video color restoration, the video frame time interval, the time frame parameters, the reference video frame, the noise penalty factor and related parameters to perform quality dimension measurement calculation to quantitatively calculate the quality dimension of each frame of video. In order to improve the calculation accuracy, a deep learning model can be used to automatically evaluate the quality of each video frame to finally obtain the low-illumination video quality dimension.

[0039] S25: Based on the low-illumination video quality dimension, the corresponding low-illumination video is labeled with a quality dimension system, so that the quality dimension of 0%-20% is labeled as 1 point, 21%-40% is labeled as 2 points, 41%-60% is labeled as 3 points, 61%-80% is labeled as 4 points, and 81%-100% is labeled as 5 points, so as to obtain the low-illumination video quality labeling score.

[0040] In an embodiment of the present invention, by implementing video quality score annotation according to the quality dimension of the low-light video, first, ensure that the calculation result of the quality dimension is within the range of 0%-100%, and then formulate a score annotation rule according to the numerical range of the quality dimension. For videos with a quality dimension between 0%-20%, it is marked as 1 point; between 21%-40%, it is marked as 2 points; between 41%-60%, it is marked as 3 points; between 61%-80%, it is marked as 4 points; and between 81%-100%, it is marked as 5 points. In the specific implementation, according to the numerical range of the quality dimension, the score annotation rule is formulated. According to the obtained video quality dimension value, it is compared with the preset score range. For example, for a frame of video, its quality dimension is 35%, then the video frame can be assigned 2 points according to the labeling rules. In order to ensure the efficiency of the labeling process, an automated scoring system can be developed to automatically assign scores to each frame of video based on the calculation results of the quality dimension, thereby completing the quality labeling of low-light videos. The automated system can be trained using deep learning technology to make the scoring process more accurate and faster, and ultimately obtain the low-light video quality labeling score.

[0041] Furthermore, the measuring of the noise level of each frame of video image in the low-illumination video standard frame image set includes: Dividing each frame of video image in the low-illumination video standard frame image set into a local image region to obtain a local video sub-region corresponding to each frame of video image; In an embodiment of the present invention, each frame of video image in a low-illumination video standard frame image set is divided into a local area. When implementing this step, each frame of image needs to be divided into several small areas, each of which is called a local area. The specific operation is to use a sliding window of a fixed size to scan the video frame, and divide the video frame into several sub-areas that are uniform or based on image content features. For each frame of image, the size and step size of the division window are first determined. Generally, common image block sizes such as 16×16 and 32×32 can be selected. The image is divided into multiple overlapping or non-overlapping small blocks by means of a sliding window, ensuring that each local area can effectively represent a part of the image. During the division process, the size of each local area can be adjusted according to the spatial resolution of the image and the demand for computing resources. This operation provides basic data support for subsequent texture analysis and noise level calculation, and finally obtains the video local sub-area corresponding to each frame of video image.

[0042] Preferably, the texture fractal dimension of the local sub-region of the video corresponding to each frame of the video image is calculated to obtain the fractal dimension of the local region corresponding to each frame of the video image; In an embodiment of the present invention, by extracting texture features from each local area, the box-counting method can be used to calculate the fractal dimension of each local area through fractal theory. Specifically, the core idea of ​​the box-counting method is to cover the local area with grids (boxes) of different sizes, and then calculate the minimum number of grids that can cover the local area as the grid size changes. In this way, the complexity of the local area at different scales can be obtained. Finally, the fractal dimension of the local area is obtained by logarithmic regression analysis. For low-illumination video, the details of the texture are affected by noise. Therefore, the fractal dimension calculated in this way can reflect the complexity of the image texture. The fractal dimension of each local area is closely related to the noise level of the area and the structural characteristics of the image. Finally, the fractal dimension of the local area corresponding to each frame of the video image is obtained.

[0043] Preferably, a local pixel variance calculation is performed on a local sub-region of the video corresponding to each frame of the video image to obtain a local region pixel variance corresponding to each frame of the video image; In an embodiment of the present invention, local pixel variance is calculated for a local sub-region of the video corresponding to each frame of video image. When implementing this step, for each local region, all pixel values ​​in the region are first extracted, and then the variance of the pixel values ​​in the local region is calculated. The specific calculation formula is: local variance=Σ((x_i -μ)^2) / N, wherein x_i is each pixel value in the region, μ is the average pixel value of the local region, and N is the total number of pixels in the region. The variance can reflect the degree of change of the pixel values ​​in the local region. Therefore, a large variance means that the pixel values ​​in the region change more dramatically and there is a high noise level. By calculating the variance of each local region, the pixel distribution of each region in the video frame can be effectively evaluated, and then the degree of influence of noise can be inferred, and finally the pixel variance of the local region corresponding to each frame of video image is obtained.

[0044] Preferably, the noise level of each frame of video image in the low-illumination video standard frame image set is measured based on the local area fractal dimension and the local area pixel variance corresponding to each frame of video image to obtain the video noise level corresponding to each low-illumination video frame.

[0045] In an embodiment of the present invention, the noise level of each frame of video image in the low-illuminance video standard frame image set is measured based on the local region fractal dimension and the local region pixel variance corresponding to each frame of video image. When implementing this step, the noise level measurement combines the fractal dimension and pixel variance obtained in the first two steps. Specifically, using the fractal dimension and variance information of each local region, firstly, the features of all local regions are weighted and summarized to obtain the comprehensive features of each frame of video image. These comprehensive features can effectively describe the overall noise level of the image. Then, by establishing a mapping relationship between the noise level and the local region features, regression analysis or machine learning models (such as support vector machines, random forests, etc.) are used to predict the noise level. Images with high noise levels usually have lower fractal dimensions and higher pixel variances, while images with low noise levels have the opposite. Through this method, the noise level of each frame of video image can be accurately evaluated, and finally the video noise level corresponding to each low-illuminance video frame is obtained.

[0046] Furthermore, the color restoration measurement of each frame of the video image in the low-illumination video standard frame image set includes: Calculating the chromaticity coordinates of different color regions corresponding to each frame of the video image in the low-illumination video standard frame image set to obtain the chromaticity coordinates corresponding to the different color regions in each frame of the video image; In the embodiment of the present invention, the low-light video standard frame image set is composed of multiple frames of video images, each frame of the image contains multiple different color regions. For each frame of the video image, a color segmentation algorithm (such as a threshold-based segmentation method) is first used to divide the image into different color regions. For each pixel in the color region, it is converted from the RGB color space to the CIE XYZ color space. The conversion formula is: ,in is the RGB value of the pixel, ranging from [0,255]. Then, the chromaticity coordinates (x, y) are calculated based on the CIE XYZ value. The calculation formula is , After calculating the chromaticity coordinates of all pixels in each color area, the average value is taken as the chromaticity coordinate corresponding to the color area. Through such operation, the chromaticity coordinates corresponding to different color areas in each frame of video image are finally obtained.

[0047] Preferably, color saturation of the corresponding different color areas in each frame of the video image in the low-illumination video standard frame image set is measured based on the chromaticity coordinates corresponding to the different color areas in each frame of the video image, so as to obtain the color saturation corresponding to the different color areas in each frame of the video image; In the embodiment of the present invention, after obtaining the chromaticity coordinates corresponding to different color regions in each frame of video image, the color saturation is measured. First, the chromaticity coordinates (x, y) are converted back to the CIE XYZ color space, and then converted from the CIE XYZ color space to the HSV color space. After conversion to the HSV color space, for each pixel in the color region, its saturation value S is extracted. The saturation S reflects the vividness of the color, and the value range is [0, 1]. For each color region, the average saturation value of all pixels in the region is calculated, and this is used as the color saturation corresponding to the color region. Specifically, suppose there are n pixels in a certain color region, and their saturation values ​​are respectively , then the color saturation of the color area By performing such calculations on different color areas of each frame of video image in the low-illumination video standard frame image set, the color saturation corresponding to different color areas in each frame of video image is finally obtained.

[0048] Preferably, the hue distribution statistics are performed on the corresponding different color areas in each frame of the video image in the low-illumination video standard frame image set to obtain the hue distribution corresponding to the different color areas in each frame of the video image; In the embodiment of the present invention, after converting the color region of each frame of video image from RGB to HSV color space, the hue value H is focused on. The hue value H represents the type of color, and the value range is usually [0,360). For each color region in each frame of video image, its hue value is divided into several intervals, for example, divided into 10 intervals, each interval is 36°, and the number of hue values ​​of pixels in each color region falling in each interval is counted. Suppose there are n pixels in a certain color region, and its hue value is counted into k intervals. The number of pixels falling in the jth interval is , then the frequency of this interval The frequency distribution of these intervals constitutes the hue distribution corresponding to the color area. By performing such statistics on the different color areas of each frame of video image in the low-illumination video standard frame image set, the hue distribution corresponding to the different color areas in each frame of video image is finally obtained.

[0049] Preferably, the color restoration degree of each video frame in the low-illumination video standard frame image set is measured based on the color saturation and hue distribution corresponding to different color areas in each video frame to obtain the video color restoration degree corresponding to each low-illumination video frame.

[0050] In the embodiment of the present invention, for each frame of video image in the low-light video standard frame image set, the color saturation and hue distribution corresponding to different color regions are known. First, weights are set for the color saturation and hue distribution, respectively. and ,and For each color region, the difference between the ideal value and the actual measured value of its color saturation is calculated. The difference can be expressed by the absolute value of the difference between the two. At the same time, the difference between the ideal distribution and the actual distribution of its hue distribution is calculated. The Kullback-Leibler divergence can be used to measure the difference between the two distributions. Suppose the ideal color saturation value of a color region is , the actual value is , saturation difference ; The ideal distribution of hue distribution is , the actual distribution is , the difference in hue distribution , the color restoration degree of all color areas in each frame of video image is weighted averaged, and the weight can be determined according to the size of the color area , and finally the video color restoration degree corresponding to each low-illumination video frame is obtained.

[0051] Furthermore, the video quality dimension measurement calculation formula is specifically as follows: ; In the formula, is the low-light video quality dimension, is the total number of low-light video frames, For the Low-light video frames, For the The video clarity corresponding to the low-light video frame, For the The video noise level corresponding to a low-light video frame, For the The video color restoration degree corresponding to the low-light video frame, is the video frame time interval, specifically from arrive time range, For in time The corresponding low-light video frame is as follows: is the time frame parameter, is the reference video frame, is the noise penalty factor.

[0052] The present invention obtains a video quality dimension measurement calculation formula by using a specific mathematical model and verifying it, which is used to measure the quality dimension of the corresponding low-light video. The formula fully considers the quality dimension of the low-light video. , the total number of low-light video frames , No. Low light video frames , No. The video clarity corresponding to the low-light video frames , No. The video noise level corresponding to low-light video frames , No. The video color restoration degree corresponding to the low-light video frame , video frame time interval , specifically from arrive time range, in time The corresponding low-light video frame , time frame parameters , reference video frame , noise penalty factor , according to the low-light video quality dimension The correlation between the above parameters constitutes a functional relationship , the formula can realize the quality dimension measurement and calculation process of the corresponding low-light video. At the same time, the formula takes into account multiple quality dimensions of the video, including clarity, noise level and color reproduction. These dimensions are the core factors for measuring the quality of low-light videos. The impact of each dimension will be comprehensively calculated into the quality evaluation, making the evaluation results more comprehensive and accurate. By introducing the time interval of the video frame and the difference calculation between frames, the formula can effectively capture the temporal change trend of the video. This means that the video quality is not just an evaluation of a single frame, but a measurement of the quality change of the entire video sequence in the temporal dimension, which helps to identify the overall quality of the video. By introducing the noise penalty factor, the weight adjustment can be made for the impact of noise. If the noise level of a certain frame of video is high, the introduction of the penalty factor makes the quality of this part of the video appropriately weakened, thereby avoiding the excessive impact of noise on the quality score and improving the accuracy of the evaluation results. By comparing the current frame with the reference video frame, the formula can be evaluated on a relatively standardized basis, reducing the deviation caused by different video sources or shooting conditions. The reference frame can be a standard definition frame, making the evaluation more relative and comparable. Each factor in the formula (clarity, noise level, color reproduction) is weighted to ensure that the contribution of each dimension to the final quality score is reasonable. This weighted calculation can reflect the relative importance of different quality features, thereby obtaining a more accurate quality assessment result. The final low-light video quality dimension scoring system maps the quality dimension to a specific score range (1-5 points). This annotation system not only provides a simple and easy-to-understand scoring standard to help users or systems quickly evaluate video quality, but also provides a basis for subsequent optimization and processing. The division of each score segment makes the evaluation results highly operational and facilitates further quality control and improvement. Since the quality of low-light videos is affected by many factors (such as noise, brightness, clarity, etc.), standardized quality dimension measurement formulas can reduce subjective evaluation differences, improve the consistency of low-light video quality assessment, and make quality scores reproducible.

[0053] Furthermore, the video quality model prediction module includes the following functions: Construct a video quality measurement model architecture using convolutional neural networks, which includes a sharp feature extraction branch, a noise feature extraction branch, and a fully connected neural network layer; In an embodiment of the present invention, a deep learning architecture is constructed to measure the quality of low-light video, specifically including a clear feature extraction branch, a noise feature extraction branch and a plurality of fully connected neural network layers. The clear feature extraction branch is used to extract the clarity information of the video frame, and the noise feature extraction branch is used to extract the noise information in the video frame. The clear feature extraction branch is composed of 5 5x5 convolutional layers and 5 1x1 pooling layers. The convolutional layer is used to extract the spatial features in the image, and the pooling layer is used for dimensionality reduction and feature compression. The noise feature extraction branch is composed of 5 3x3 convolutional layers and 5 1x1 pooling layers. The size of the convolutional layer is small, and the purpose is to improve the sensitivity to noise details. The number of fully connected neural network layers is determined according to the number of quality dimensions that need to be predicted, and each dimension corresponds to a prediction score. The design of the architecture is to weight the extraction results of the clear features and the noise features into a comprehensive feature vector, and input it into the fully connected neural network layer for nonlinear transformation, so as to output the prediction scores of each quality dimension. The purpose of this process is to quantitatively evaluate the video quality through the deep learning model to ensure that the prediction score can fully reflect the quality changes of the video, especially the performance under low-light conditions, and finally construct and generate the corresponding video quality measurement model architecture.

[0054] Preferably, the low-illumination video standard frame image set is divided into a training set, a verification set and a test set according to a division ratio corresponding to 7:2:1; In an embodiment of the present invention, a standard frame image set of a low-illumination video is obtained, and the data set is divided into a training set, a validation set and a test set according to the requirements of a video quality measurement model. The data set is divided in a ratio of 7:2:1, wherein 70% is used as a training set for model training, 20% is used as a validation set for adjusting hyperparameters and verifying model performance, and 10% is used as a test set for evaluating the performance of the final model. When dividing the data set, it is ensured that the data distribution of each subset is similar to that of the original data set, and that video frames of different qualities are evenly distributed in each subset to avoid data bias affecting the effect of model training. In the division process, a random sampling method is used to ensure the fairness of the division, while ensuring the representativeness and diversity of the video frames in each subset. The data set obtained by this step can provide a reliable data basis for subsequent model training and evaluation.

[0055] Preferably, the training set is input into the video quality measurement model architecture for model training, and the stochastic gradient descent is used as an optimization algorithm in combination with the validation set to continuously adjust the hyperparameters corresponding to the model, wherein the hyperparameters include learning rate, batch size, and number of network layers, and the test set is used to optimize the model performance, so as to generate a low-light video measurement model based on deep learning, and output the prediction score corresponding to each video frame quality dimension at each fully connected neural network layer; In an embodiment of the present invention, a training set is input into a designed deep learning model for training. The training goal of the model is to minimize the error between the prediction result and the actual quality dimension. Stochastic gradient descent (SGD) is used as an optimization algorithm. The weight of the model is continuously adjusted in combination with the training set data. In order to ensure the adaptability and generalization ability of the model in different situations, a validation set is used to tune the hyperparameters of the model. Specifically, the adjusted hyperparameters include learning rate, batch size and number of network layers. The learning rate controls the step size of each gradient update, the batch size determines the number of samples used for each training, and the number of network layers affects the complexity of the model. Through experiments on the validation set, the optimal hyperparameter combination is found to improve the training effect and stability of the model. During the training process, a cross entropy loss function is used to quantify the error between the prediction result and the actual label. The parameters of the model are updated through the back propagation algorithm. The low-illumination video quality measurement model obtained through training can effectively learn video frame features and generate reasonable prediction scores in different quality dimensions, and finally output the prediction score corresponding to each video frame quality dimension.

[0056] Preferably, based on the prediction score corresponding to each video frame quality dimension, a quality-weighted sum is performed on the corresponding low-illumination video to obtain a low-illumination video quality prediction score.

[0057] In an embodiment of the present invention, the quality of the low-light video is weightedly summed by using the prediction score corresponding to the quality dimension output by the model at each fully connected neural network layer. Specifically, the prediction score of each video frame represents the performance of the frame in a certain quality dimension. The prediction scores of all frames are weighted and aggregated to obtain the quality prediction score of the entire video. In the weighted summation process, it is necessary to set appropriate weighting coefficients based on the timing, features and performance of the video frames in different quality dimensions. This method can ensure that the overall quality prediction of the video can comprehensively consider the quality performance of each frame, while emphasizing the impact of specific dimensions (such as clarity, noise, etc.) on the video quality. The obtained quality prediction score can provide a comprehensive quantitative indicator for the quality evaluation of the low-light video, help further quality optimization and tuning, and finally predict and measure to obtain the low-light video quality prediction score.

[0058] Furthermore, the video quality measurement model architecture is specifically composed of a clear feature extraction branch, a noise feature extraction branch and multiple fully connected neural network layers, wherein the clear feature extraction branch is composed of 5 5x5 convolutional layers and 5 1x1 pooling layers, the noise feature extraction branch is composed of 5 3x3 convolutional layers and 5 1x1 pooling layers, and the fully connected neural network layer is determined by the total number corresponding to the quality dimension, so as to weightedly fuse the video frame features extracted by each branch into a comprehensive feature vector, and input the comprehensive feature vector into each fully connected neural network layer for nonlinear transformation to output the prediction score corresponding to each quality dimension.

[0059] Furthermore, the prediction error loss optimization module includes the following functions: The quality mean square error loss is calculated between the low-illumination video quality annotation score and the low-illumination video quality prediction score to obtain the video quality error loss between the model prediction score and the annotation score; In an embodiment of the present invention, the quality of the low-light video is labeled and scored, which can be achieved through human eye scoring or relying on existing video quality evaluation methods. These labeled scores will be used as standards for comparison with the scores predicted by the model. The low-light video quality prediction score is obtained by training a deep learning-based model on video frame image data. The quality mean square error loss (MSE) is an effective method for calculating the difference between the predicted score and the labeled score. Specifically, the difference between the labeled quality score of each low-light video frame and the quality score predicted by the model is first calculated, and then the squares of these differences are averaged to obtain the mean square error loss. This loss function can quantify the gap between the model prediction and the actual annotation, and serve as the basis for subsequent optimization, indicating the deviation and error of the model, so as to optimize the low-light video quality prediction model, and finally obtain the video quality error loss between the model prediction score and the labeled score.

[0060] Preferably, based on the video quality error loss between the model prediction score and the labeled score, the low illumination video measurement model based on deep learning is subjected to error loss reinforcement learning, and the hyperparameters corresponding to the low illumination video measurement model are overfitted constrained by introducing an L2 regularization term to generate a comprehensive low illumination video measurement model; In an embodiment of the present invention, the low-light video measurement model based on deep learning is subjected to error loss reinforcement learning by utilizing the mean square error loss calculated previously. In order to further improve the model performance, the model needs to be trained by a reinforcement learning strategy, and the model gradually improves its prediction ability under the guidance of the loss function. In the specific operation, the parameters of the model are back-propagated based on the MSE loss, and the network weights are adjusted by an optimization algorithm (such as gradient descent) to minimize the loss function. At this time, in order to avoid the overfitting problem, it is also necessary to introduce an L2 regularization term. The L2 regularization term increases the sum of squares of the model parameters in the loss function, so that the model tends to select a smaller parameter value during optimization, thereby effectively reducing the occurrence of overfitting. In this way, the trained low-light video measurement model will be able to be more stable and accurate in the process of video quality evaluation, reduce excessive dependence on specific data, and after multiple rounds of training optimization, obtain a comprehensive model that can accurately evaluate the quality of low-light videos, and finally generate a low-light video measurement comprehensive model.

[0061] Preferably, the low-illumination video standard frame image set is re-input into the low-illumination video measurement comprehensive model for video quality assessment measurement to output a corresponding low-illumination video quality comprehensive measurement score.

[0062] In an embodiment of the present invention, a standard frame image set of a low-light video will be re-input into a low-light video measurement comprehensive model that has undergone reinforcement learning and regularization processing for quality assessment. The standard frame image set consists of a set of representative video frames, covering different low-light scenes and lighting conditions. These images will be passed as input data to the trained low-light video measurement model. The model will analyze according to the characteristics of the image and the video quality prediction mechanism. Specifically, the model will extract key features in each frame of the video image, such as noise level, detail loss, color distortion, etc., and then calculate the comprehensive video quality score. The output low-light video quality comprehensive measurement score will reflect the overall quality of the video, including clarity, noise suppression effect, brightness adaptation and other factors. This score can be used as the basis for subsequent low-light video quality optimization, video content analysis and other tasks, that is, the corresponding scores in each fully connected layer are weighted and summed, and finally the corresponding low-light video quality comprehensive measurement score is output.

[0063] Therefore, the embodiments should be regarded as illustrative and non-restrictive from all points, and the scope of the present invention is limited by the appended claims rather than the above description, and it is therefore intended that all changes falling within the meaning and range of equivalent elements of the application documents are included in the present invention.

[0064] The above description is only a specific embodiment of the present invention, so that those skilled in the art can understand or implement the present invention. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but should conform to the widest scope consistent with the principles and novel features invented herein.

Claims

1. A low-light video quality measurement system based on deep learning, characterized in that: Includes the following modules: A low-illumination video frame processing module is used to collect corresponding low-illumination video data in real time by using a video acquisition device in different low-illumination scenes, and perform video frame processing on the low-illumination video data to obtain a low-illumination video standard frame image set; The video quality dimension measurement and annotation module is used to measure the video index of each frame of the low-light video standard frame image set to obtain the video clarity, video noise level and video color restoration corresponding to each low-light video frame; Based on the video clarity, video noise level and video color restoration corresponding to each low-light video frame, the corresponding low-light video is measured and annotated in terms of quality dimensions to obtain a low-light video quality annotation score; A video quality model prediction module is used to construct a video quality measurement model architecture using a convolutional neural network, and to optimize the video quality measurement model architecture using a low-light video standard frame image set to generate a low-light video measurement model based on deep learning, and output a prediction score corresponding to each video frame quality dimension; based on the prediction score corresponding to each video frame quality dimension, a quality-weighted sum is performed on the corresponding low-light video to obtain a low-light video quality prediction score; The prediction error loss optimization module is used to perform error loss reinforcement learning on the deep learning-based low-illumination video measurement model based on the low-illumination video quality annotation score and the low-illumination video quality prediction score to generate a low-illumination video measurement comprehensive model; the low-illumination video standard frame image set is re-input into the low-illumination video measurement comprehensive model for video quality assessment measurement to output the corresponding low-illumination video quality comprehensive measurement score.

2. The low-light video quality measurement system based on deep learning according to claim 1, characterized in that: The low illumination video frame processing module includes the following functions: By using video acquisition equipment in different low-light scenes to collect corresponding low-light video data in real time; The low-illumination video data is subjected to video single-frame extraction at a fixed frame rate of 24fps to obtain a low-illumination video single-frame image set; Measuring the blurriness of a frame image of a low-illumination video single-frame image set to obtain the blurriness of the frame image of the low-illumination video; Performing video frame image filtering on each low-illumination video frame in a low-illumination video single frame image set based on the low-illumination video frame image blurriness to obtain a low-illumination video frame filtered image set; The pixel values ​​corresponding to each frame of the low-illumination video image in the low-illumination video frame filtered image set are normalized to obtain a low-illumination video standard frame image set.

3. The low-light video quality measurement system based on deep learning according to claim 2, characterized in that: The different low-light scenes specifically include outdoor scenes at night, underground parking lots, and indoor low-light environment scenes.

4. The low-light video quality measurement system based on deep learning according to claim 1, characterized in that: The video quality dimension measurement and annotation module includes the following functions: Perform multi-channel decomposition on each frame of video image in the low-illumination video standard frame image set to obtain R, G, and B channel images corresponding to each frame of video image; The grayscale mean and grayscale variance corresponding to each channel are obtained through the R, G, and B channel images corresponding to each frame of the video image, and the channel energy ratio of each frame of the video image in the low-illumination video standard frame image set is measured based on the grayscale mean and grayscale variance corresponding to each channel to obtain the channel image energy ratio corresponding to each frame of the video image; Performing video clarity analysis on each corresponding frame of video image according to the channel image energy ratio corresponding to each frame of video image, so as to obtain the video clarity corresponding to each low-illumination video frame; Measuring the noise level of each frame of video image in the low-illumination video standard frame image set to obtain the video noise level corresponding to each low-illumination video frame; The color restoration degree of each frame of the video image in the low-illumination video standard frame image set is measured to obtain the video color restoration degree corresponding to each low-illumination video frame; Based on the video clarity, video noise level and video color restoration corresponding to each low-illumination video frame, a video quality dimension measurement calculation formula is used to perform quality dimension measurement calculation on the corresponding low-illumination video to obtain the low-illumination video quality dimension; Based on the low-illumination video quality dimension, the corresponding low-illumination video is labeled with a quality dimension system, so that the quality dimension of 0%-20% is labeled as 1 point, 21%-40% is labeled as 2 points, 41%-60% is labeled as 3 points, 61%-80% is labeled as 4 points, and 81%-100% is labeled as 5 points, so as to obtain the low-illumination video quality labeling score.

5. The low-light video quality measurement system based on deep learning according to claim 4, characterized in that: The measuring of the noise level of each frame of video image in the low-illumination video standard frame image set comprises: Dividing each frame of video image in the low-illumination video standard frame image set into a local image region to obtain a local video sub-region corresponding to each frame of video image; Calculating the texture fractal dimension of the local sub-region of the video corresponding to each frame of the video image to obtain the fractal dimension of the local region corresponding to each frame of the video image; Calculating the local pixel variance of the local sub-region of the video corresponding to each frame of the video image to obtain the local region pixel variance corresponding to each frame of the video image; The noise level of each frame of video image in the low-illumination video standard frame image set is measured based on the local area fractal dimension and the local area pixel variance corresponding to each frame of video image, so as to obtain the video noise level corresponding to each low-illumination video frame.

6. The low-light video quality measurement system based on deep learning according to claim 4, characterized in that: The color restoration measurement of each frame of video image in the low-illumination video standard frame image set includes: Calculating the chromaticity coordinates of different color regions corresponding to each frame of the video image in the low-illumination video standard frame image set to obtain the chromaticity coordinates corresponding to the different color regions in each frame of the video image; Based on the chromaticity coordinates corresponding to the different color areas in each frame of the video image, the color saturation of the corresponding different color areas in each frame of the video image in the low-illumination video standard frame image set is measured to obtain the color saturation corresponding to the different color areas in each frame of the video image; Performing hue distribution statistics on different color regions corresponding to each frame of video image in the low-illumination video standard frame image set to obtain the hue distribution corresponding to different color regions in each frame of video image; The color restoration degree of each video frame in the low-illumination video standard frame image set is measured based on the color saturation and hue distribution corresponding to different color areas in each video frame to obtain the video color restoration degree corresponding to each low-illumination video frame.

7. The low-light video quality measurement system based on deep learning according to claim 4, characterized in that: The video quality dimension measurement calculation formula is specifically: ; In the formula, is the low-light video quality dimension, is the total number of low-light video frames, For the Low-light video frames, For the The video clarity corresponding to the low-light video frame, For the The video noise level corresponding to a low-light video frame, For the The video color restoration degree corresponding to the low-light video frame, is the video frame time interval, specifically from arrive time range, For in time The corresponding low-light video frame is as follows: is the time frame parameter, is the reference video frame, is the noise penalty factor.

8. The low-light video quality measurement system based on deep learning according to claim 1, characterized in that: The video quality model prediction module includes the following functions: Construct a video quality measurement model architecture using convolutional neural networks, which includes a sharp feature extraction branch, a noise feature extraction branch, and a fully connected neural network layer; The low-light video standard frame image set is divided into a training set, a validation set, and a test set according to the division ratio of 7:2:1; The training set is input into the video quality measurement model architecture for model training, and stochastic gradient descent is used as an optimization algorithm combined with the validation set to continuously adjust the corresponding hyperparameters of the model, including learning rate, batch size, and number of network layers. At the same time, the test set is used to optimize the model performance to generate a low-light video measurement model based on deep learning, and the prediction score corresponding to each video frame quality dimension is outputted at each fully connected neural network layer; Based on the prediction score corresponding to each video frame quality dimension, the quality weighted sum of the corresponding low-light video is performed to obtain the low-light video quality prediction score.

9. The low-light video quality measurement system based on deep learning according to claim 8, characterized in that: The video quality measurement model architecture specifically consists of a clear feature extraction branch, a noise feature extraction branch, and multiple fully connected neural network layers, wherein the clear feature extraction branch consists of 5 5x5 convolutional layers and 5 1x1 pooling layers, the noise feature extraction branch consists of 5 3x3 convolutional layers and 5 1x1 pooling layers, and the fully connected neural network layer is determined by the total number corresponding to the quality dimension, so as to weightedly fuse the video frame features extracted by each branch into a comprehensive feature vector, and input the comprehensive feature vector into each fully connected neural network layer for nonlinear transformation to output the prediction score corresponding to each quality dimension.

10. The low-light video quality measurement system based on deep learning according to claim 1, characterized in that: The prediction error loss optimization module includes the following functions: The quality mean square error loss is calculated between the low-illumination video quality annotation score and the low-illumination video quality prediction score to obtain the video quality error loss between the model prediction score and the annotation score; Based on the video quality error loss between the model prediction score and the labeled score, the error loss reinforcement learning of the low-light video measurement model based on deep learning is performed, and the over-fitting constraints of the hyperparameters corresponding to the low-light video measurement model are imposed by introducing the L2 regularization term to generate a comprehensive low-light video measurement model. The low-illumination video standard frame image set is re-input into the low-illumination video measurement comprehensive model for video quality assessment measurement to output the corresponding low-illumination video quality comprehensive measurement score.

Images