A video color enhancement method and system based on UV histogram features

Abstract

This invention provides a method and system for video color enhancement based on UV histogram features. By adjusting the parameters of the UV component of an image using a deep learning algorithm, the invention can achieve real-time video color enhancement on a CPU, making the video colors more vivid and bright. The algorithm is based on the YUV color space, avoiding the time-consuming operations caused by color space conversion. Furthermore, by leveraging open-source datasets, the algorithm parameters can be adaptively adjusted according to different scenarios without human intervention, making the algorithm particularly user-friendly.

Description

Technical Field

[0001] The present invention relates to a video processing technology, and in particular to a video color enhancement method and system based on UV histogram features. Background Art

[0002] Existing algorithms for video color adjustment primarily enhance image color by manually adjusting parameters in the image's RGB or HSV color spaces. On the one hand, whether based on RGB or HSV color space, videos require conversion from the YUV color space to RGB or HSV color space. This color space conversion itself consumes a significant amount of time, hindering real-time processing. On the other hand, algorithms based on RGB or HSV color space often rely on manual parameter settings, making them unsuitable for various scenarios and unfriendly to use.

[0003] Color enhancement algorithm for RGB color space: Convert the video from YUV color space to RGB color space and adjust the saturation in RGB color space:

[0004]

[0005] Set saturation The value range is [-100, 100]. , the values of the RGB components of the image after saturation adjustment are deduced to be 、 、 , and then convert from RGB color space to YUV color space.

[0006]

[0007]

[0008]

[0009] Color enhancement algorithm in HSV color space: Convert the video from YUV color space to HSV color space and adjust the saturation in HSV color space:

[0010]

[0011] According to the saturation coefficient set , get the value of the S component after adjusting the saturation , and then convert from HSV color space to YUV color space

[0012] The above algorithms all require human intervention to set coefficients, which is very unfriendly and has algorithmic uncertainty. Summary of the Invention

[0013] The present invention provides a video color enhancement method based on UV histogram features to solve the technical problems in the prior art that the method cannot be well applied to various scenes and has unfriendly algorithm usage.

[0014] A video color enhancement method based on UV histogram features, comprising:

[0015] S1: Get model weights through deep neural network learning ;

[0016] S2: Learn the parameters of the UV component: convert the RGB image to the YUV color space and calculate the histogram on the UV component , the histogram and the model weights Multiply and sum to get the final parameters and :

[0017]

[0018] S3: Calculate the color enhancement data of the UV component of the image

[0019]

[0020] in 、 Represents the U and V component data of the input video, 、 Indicates the data after the U and V components are enhanced.

[0021] The method of the present invention further comprises:

[0022] S4: Adoption Conduct effective supervised learning

[0023]

[0024] Where N represents the number of pixels in a single channel of the image; 、 Represents the U and V component data of GT in the fiveK dataset.

[0025] Input video further includes converting RGB image into YCbCr data.

[0026] A video color enhancement system based on UV histogram features, comprising:

[0027] Model weight training model: used to obtain model weights through deep neural network learning ;

[0028] Learning UV component parameter calculation unit: used to convert RGB images to YUV color space and calculate histogram on UV components , the histogram and the model weights Multiply and sum to get the final parameters and :

[0029]

[0030] Computational unit: used to calculate the color enhanced data of the UV component of the image

[0031]

[0032] in 、 Represents the U and V component data of the input video, 、 Indicates the data after the U and V components are enhanced.

[0033] The system may also include:

[0034] Supervised Learning Processing Unit: used to adopt Conduct effective supervised learning

[0035]

[0036] Where N represents the number of pixels in a single channel of the image; 、 Represents the U and V component data of GT in the fiveK dataset.

[0037] This invention is a video color enhancement algorithm based on UV histogram features. By adjusting the parameters of the UV component of an image using a deep learning algorithm, the invention can achieve real-time video color enhancement on a CPU, making the video colors more vivid and bright. This algorithm is based on the YUV color space, avoiding the time-consuming operations caused by color space conversion. Furthermore, by leveraging open source datasets, the algorithm parameters can be adaptively adjusted according to different scenarios without human intervention, making the algorithm particularly user-friendly. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 This is a principle flow chart of a video color enhancement algorithm based on UV histogram features. DETAILED DESCRIPTION

[0039] The present invention is described in detail below with reference to the accompanying drawings.

[0040] The applicant has found that histogram equalization is to expand the dynamic range of an image with a small dynamic range of grayscale value distribution (for example, the grayscale values are concentrated in the right part of the histogram, and the image is too bright). The grayscale level of the changed image may be reduced. The known conditions in Table 1 are: the total number of grayscale levels is 8, and the distribution probability corresponding to each level of the original image is , after histogram equalization, gray level 0 is mapped to 1, gray level 1 is mapped to 3, gray level 2 is mapped to 5, gray levels 3 and 4 are mapped to 6, and gray levels 5, 6, and 7 are mapped to 7. It can be seen that the histogram equalization algorithm reduces the gray levels of the image from 8 gray levels of 0, 1, 2, 3, 4, 5, 6, and 7 to 5 gray levels of 1, 3, 5, 6, and 7. From the numerical value, it is not difficult to see that the contrast of some parts of the image must have been enhanced.

[0041] A video color enhancement method based on UV histogram features, comprising:

[0042] S110: Obtaining model weights through deep neural network learning ;

[0043] S120: Learn the parameters of the UV component: convert the RGB image to the YUV color space and calculate the histogram on the UV component , the histogram and the model weights Multiply and sum to get the final parameters and :

[0044]

[0045] S130: Calculate the color enhanced data of the UV component of the image

[0046]

[0047] in 、 Represents the U and V component data of the input video, 、 Indicates the data after the U and V components are enhanced.

[0048] The present invention can also be used Conduct effective supervised learning

[0049]

[0050] Where N represents the number of pixels in a single channel of the image; 、 Represents the U and V component data of GT in the fiveK dataset.

[0051] Input video further includes converting RGB image into YCbCr data.

[0052] Application Examples

[0053] Step 0: The first step is machine learning. The way to evaluate the effectiveness of machine learning is , below is The indicator calculation formula is:

[0054]

[0055] Where N represents the number of pixels in a single channel of the image; Represents the U component data of GT in the fiveK open source dataset; Represents the V component data of GT in the fiveK open source dataset; Represents the U component in the prediction dataset output by the model training process; Represents the V component in the prediction dataset output by the model training process.

[0056] This is the outermost framework of model training, used to determine whether the training results have achieved optimal results. The model's predicted data is used here to calculate the L1 Loss between the predicted data and the source data. When the L1 Loss changes remain stable at a minimal level, we consider the training to have achieved optimal results. The so-called predicted data here refers to the enhanced U and V components calculated using the U and V component enhancement formulas described in Step 1 below.

[0057] Step 1: The process of machine learning is called model training, which mainly includes model algorithm logic, input training data, and output prediction set.

[0058] The input data for model training is the open source REDS deblur dataset. The trained model is a linear model, which is trained by applying pytorch.nn.linear. Its working principle is to input a 256x256 dimension UV component histogram data: , through the following linear change formula, we can get the 256x256 dimension weights of the U and V components respectively: .

[0059] Linear transformation formula: y = xA + b

[0060] After obtaining the above weight data, we can transpose it in the actual operation and perform matrix multiplication with x.

[0061] The principle of using histogram here is that the histogram can more comprehensively reflect the overall characteristics of the image. 256x256 means that U has 256 values and v has 256 values. The combination of the two has a total of 256x256 eigenvalues, which reflects all the characteristics of the RGB image color. After inputting these eigenvalues, pytorch.nn.linear performs linear regression on these features through data learning and obtains the slope of the linear function, which is the weight data mentioned above. Then we can enhance the U and V of the image.

[0062] U component enhancement formula:

[0063] V component enhancement formula:

[0064] We can see that the U and V component enhancement formulas here are very similar to the linear transformation formulas above, and they are actually the same.

[0065] The linear transformation performed during model training above is intended to obtain Pu and Pv. UV enhancement is essentially a linear function. The key is the coefficients of this linear function and where they are derived. Some methods are manually set, such as the eq filter in FFmpeg, which enhances color saturation with the following coefficient: eq = saturation = 1.3. This enhances the entire image uniformly by a factor of 1.3. However, our method, trained through a linear model, takes into account the weighted average of each image feature to determine the enhancement coefficient. See the actual color enhancement process below.

[0066] Step 2: The above Step 0 and Step 1 are all about model training. The purpose is to obtain the enhancement coefficient of UV linear enhancement. Through the above steps, we finally get a 256x256 dimension weight on the U and V components: .

[0067] Then we will apply the above weight data in the actual image enhancement application. This step is about the actual image enhancement processing process.

[0068] In the actual enhancement process, similar to the calculation steps of the model algorithm, we first calculate the histogram of the UV component of the current image data and obtain Because when we process image data, the data is processed frame by frame, so when calculating the data of each frame, the UV component histogram of this frame will be recalculated;

[0069] Then, by and the above model weights Multiply and sum to get the final parameters and :

[0070]

[0071] This enhancement factor and , is an enhancement coefficient that takes into account all features of the current image frame data. It is an enhancement coefficient for the current image frame. The UV features of different image frames, represented by histUV, are weighted and then the enhancement coefficient of the current image frame is calculated.

[0072] Finally, the above enhancement coefficient is applied to the UV component color enhancement calculation of the image data. The calculation formula is as follows:

[0073]

[0074] in 、 Represents the U and V component data of the input video, 、 Indicates the data after the U and V components are enhanced.

[0075] A video color enhancement system based on UV histogram features, comprising:

[0076] Model weight training model: used to obtain model weights through deep neural network learning ;

[0077] Learning UV component parameter calculation unit: used to convert RGB images to YUV color space and calculate histogram on UV components , the histogram and the model weights Multiply and sum to get the final parameters and :

[0078]

[0079] Computational unit: used to calculate the color enhanced data of the UV component of the image

[0080]

[0081] in 、 Represents the U and V component data of the input video, 、 Indicates the data after the U and V components are enhanced.

[0082] The system also includes:

[0083] Supervised Learning Processing Unit: used to adopt Conduct effective supervised learning

[0084]

[0085] Where N represents the number of pixels in a single channel of the image; 、 Represents the U and V component data of GT in the fiveK dataset.

[0086] The embodiments of the present invention have been described in detail above with reference to the accompanying drawings, but the present invention is not limited to the above embodiments. Even if various changes are made to the present invention, if these changes fall within the scope of the claims of the present invention and their equivalents, they still fall within the scope of protection of the present invention.

Claims

1. A video color enhancement method based on UV histogram features, characterized by: include: S1: Get model weights through deep neural network learning ; S2: Learn the parameters of the UV component: convert the RGB image to the YUV color space and calculate the histogram on the UV component , the histogram and the model weights Multiply and sum to get the final parameters and : S3: Calculate the color enhancement data of the UV component of the image in 、 Represents the U and V component data of the input video, 、 Indicates the data after the U and V components are enhanced.

2. The method according to claim 1, wherein Also includes: S4: Adoption Conduct effective supervised learning Where N represents the number of pixels in a single channel of the image; 、 Represents the U and V component data of GT in the fiveK dataset.

3. The method according to claim 1, wherein Before step S1, the following steps are also included: Machine learning, the way to evaluate the effectiveness of machine learning is , below is The indicator calculation formula is: Where N represents the number of pixels in a single channel of the image; Represents the U component data of GT in the fiveK dataset; Represents the V component data of GT in the fiveK dataset; Represents the U component in the prediction dataset output by the model training process; Represents the V component in the prediction dataset output by the model training process; This part is the outermost framework of model training, which is used to judge whether the training results have achieved the best results. The predicted data of the model is used here, and the U component data and the V component data of GT in the fiveK dataset are used to compare the predicted data. A calculation of The changes have been stable within the preset range, and it is believed that the training has achieved its effect.

4. The method according to claim 3, wherein The process of machine learning is model training, which mainly includes model algorithm logic, input training data, and output prediction set; The input data for model training is the open source REDS deblur dataset. The trained model is a linear model, which is trained by applying pytorch.nn.linear. Its working principle is to input a 256x256 dimension UV component histogram data: , through the following linear change formula, the weights of the 256x256 dimensions of the U and V components are obtained respectively: ; Linear transformation formula: y = xA + b; After obtaining the above weight data, we can transpose it in actual calculations and perform matrix multiplication with x; because the histogram can more comprehensively reflect the overall characteristics of the image, 256x256 means that U has 256 values and V has 256 values. The combination of the two has a total of 256x256 eigenvalues, which reflects all the characteristics of the RGB image color. After inputting these eigenvalues, pytorch.nn.linear performs linear regression on these features through data learning to obtain the slope of the linear function, which is the mentioned weight data.

5. A video color enhancement system based on UV histogram features, characterized by: include: Model weight training unit: used to obtain model weights through deep neural network learning ; Learning UV component parameter calculation unit: used to convert RGB images to YUV color space and calculate histogram on UV components , the histogram and the model weights Multiply and sum to get the final parameters and : Computational unit: used to calculate the color enhanced data of the UV component of the image in 、 Represents the U and V component data of the input video, 、 Indicates the data after the U and V components are enhanced.

6. The system according to claim 5, wherein: Also includes: Supervised Learning Processing Unit: used to adopt Conduct effective supervised learning Where N represents the number of pixels in a single channel of the image; 、 Represents the U and V component data of GT in the fiveK dataset.

Images