A video noise reduction method, system, device and storage medium

Abstract

A video noise reduction method, characterized in comprising the following steps: extracting each image frame in a video to be denoised; obtaining image information in each image frame, and obtaining metric index information corresponding to the image information respectively through the image information; obtaining the metric index density of each image frame by substituting the metric index information of each image frame into a preset metric index formula respectively; obtaining the image quality score of each image frame by substituting the metric index density and a preset weight value, and obtaining a sequence of denoised image frames; updating each labeled category image frame set according to the sequence of denoised image frames, obtaining a new image frame set, and obtaining a denoised video according to the new image frame set. The application extracts representative image frames to be denoised from a large number of image frames, greatly reduces the number of image frames that need to be denoised, thereby reducing the operation time and calculation resources, and improving the efficiency of video noise reduction.

Description

Technical Field

[0001] This invention relates to the field of video noise reduction technology, and in particular to a video noise reduction method, system, device and storage medium. Background Technology

[0002] In daily life, due to limitations in shooting conditions and the influence of sending, transmitting, and receiving equipment, videos are often subject to noise interference, which degrades video quality, affects visual effects, and hinders further video processing. Therefore, video noise reduction is necessary to improve video quality.

[0003] Currently, in order to improve the quality of video noise reduction, the common method is to reduce noise for each image frame in the video to obtain a noise-reduced video. This noise reduction process requires processing each frame, which consumes a lot of computing time and resources, thus reducing the efficiency of video noise reduction. Summary of the Invention

[0004] To address the aforementioned shortcomings, the present invention aims to provide a video noise reduction method, system, device, and storage medium to improve the efficiency of video noise reduction in the prior art, which requires processing each frame and consumes a large amount of computing time and resources.

[0005] To achieve this objective, the present invention adopts the following technical solution: a video noise reduction method, comprising the following steps:

[0006] Extract individual image frames from the video to be denoised;

[0007] Image information from each image frame is acquired, and measurement index information corresponding to the image information is obtained through the image information. The image information includes brightness, saturation and gradient value, and the measurement index information includes average brightness vector, average saturation vector and average gradient vector.

[0008] The measurement index density of each image frame is obtained by substituting the measurement index information of each image frame into the preset measurement index formula.

[0009] The image quality score of the image frame is obtained by using the density of the metric index and the preset weight value.

[0010] The image frames are classified into categories to obtain multiple sets of image frames with different labeled categories, wherein the classification involves labeling multiple frames with similarity higher than a threshold.

[0011] Extract the image frame with the highest image quality score from the image frame set of each labeled category, and combine all the image frames with the highest image quality scores into a sequence of image frames to be denoised;

[0012] The image frame sequence to be denoised is input into the image frame denoising model to obtain the denoised image frame sequence;

[0013] The image frame sets of each labeled category are updated according to the denoised image frame sequence to obtain a new image frame set, and the denoised video is obtained based on the new image frame set.

[0014] Preferably, the process of obtaining the measurement index information is as follows:

[0015] Extract the mean value of the image brightness component in the image frame, and use the mean value of the image brightness component as the average brightness vector L;

[0016] Extract the mean value of the saturation components in the image frame, and use the mean value of the saturation components as the average saturation vector S;

[0017] The Laplace operator is used to calculate the gradient value components of the image frame, and the mean of the gradient value components is calculated. The mean of the gradient value components is used as the average gradient vector G.

[0018] Preferably, the specific process for calculating the density of the metric index is as follows:

[0019] Calculate the mean and variance of the average brightness vector L, average saturation vector S, and average gradient vector G respectively;

[0020] The average brightness vector L, average saturation vector S, average gradient vector G, and their corresponding mean and variance are substituted into the measurement index formula to obtain the measurement index density corresponding to the measurement index information.

[0021] The formula for the metric is as follows: Where x takes values ​​from the average brightness vector L, average saturation vector S, and average gradient vector G, σ x With μ x These are the mean and variance corresponding to the measurement indicators, respectively.

[0022] Preferably, the formula for obtaining the image quality score is as follows: Y = A × Y1 + B × Y2 + C × Y3; where A, B, and C are preset weight values, and Y1, Y2, and Y3 are the density vectors corresponding to average brightness, average saturation, and average gradient, respectively.

[0023] The preferred method, and the specific process of further classification, is as follows:

[0024] Each image frame is matched for similarity with other image frames. If the similarity between two image frames is greater than a threshold, the two image frames are labeled as the same category. If the similarity between two image frames is less than the threshold, the two image frames are labeled as different categories.

[0025] Preferably, the similarity matching process is as follows:

[0026] Each image frame is then subjected to depth autoencoding to obtain the autoencoder for each image frame.

[0027] Calculate the Euclidean distance between the autoencoders of different image frames respectively;

[0028] The similarity between different image frames is obtained by calculating the Euclidean distance between the autoencoders of different image frames.

[0029] Preferably, the step of updating each labeled category image frame set according to the denoised image frame sequence to obtain a new image frame set includes the following steps: replacing each image frame in the corresponding labeled category image frame set with each denoised image frame in the denoised image frame sequence to update each labeled category image frame set and obtain a new image frame set.

[0030] A video noise reduction system, using the aforementioned video noise reduction method, is characterized by comprising: an image frame extraction module, a metric calculation module, a metric density calculation module, an image quality score calculation module, an image frame annotation module, a denoising image frame extraction module, an image noise reduction module, a noise-reduced video module, and an image frame update module.

[0031] The image frame extraction module is used to extract each image frame from the video to be denoised.

[0032] The metric calculation module is used to obtain image information in each image frame, and to obtain the metric information corresponding to the image information through the image information respectively;

[0033] The metric density calculation module is used to obtain the metric density of each image frame by substituting the metric information of each image frame into a preset metric formula.

[0034] The image quality scoring calculation module is used to obtain the image quality score of the image frame by using the density of the metric index and the preset weight value.

[0035] The image frame annotation module is used to classify the image frames into categories, resulting in multiple sets of image frames with different annotation categories;

[0036] The image frame extraction module is used to extract the image frame with the highest image quality score from each labeled category image frame set, and combine all the image frames with the highest image quality scores into a sequence of image frames to be denoised.

[0037] The image denoising module is used to input the image frame sequence to be denoised into a preset image frame denoising model to obtain a denoised image frame sequence.

[0038] The image frame update module is used to update the image frame set of each labeled category according to the denoised image frame sequence to obtain a new image frame set;

[0039] The noise reduction video module is used to obtain noise reduction video based on the new set of image frames.

[0040] An apparatus includes a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor causes the processor to implement the video noise reduction method described above.

[0041] A storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the aforementioned video noise reduction method.

[0042] One of the above technical solutions has the following advantages or beneficial effects: By extracting representative image frames to be denoised from a large number of image frames, the present invention greatly reduces the number of image frames that need to be denoised, thereby reducing computing time and computing resources and improving the efficiency of video denoising. Attached Figure Description

[0043] Figure 1 This is a flowchart of a video noise reduction method provided in an embodiment of the present invention.

[0044] Figure 2 This is a structural block diagram of a video noise reduction system provided in an embodiment of the present invention.

[0045] Figure 3 This is a structural block diagram of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0046] Embodiments of the present invention are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and should not be construed as limiting the present invention.

[0047] In the description of embodiments of the present invention, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the stated features. In the description of embodiments of the present invention, "a plurality of" means two or more, unless otherwise explicitly specified.

[0048] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this invention, unless otherwise stated, "a plurality of" means two or more. Those skilled in the art will understand the specific meaning of the above terms in this invention based on the specific circumstances. Figures 1-3 As shown, a video noise reduction method includes the following steps:

[0049] Extract individual image frames from the video to be denoised;

[0050] The video to be denoised can be captured by a camera or obtained from existing video through data transmission or cloud download.

[0051] Image information from each image frame is acquired, and measurement index information corresponding to the image information is obtained through the image information. The image information includes brightness, saturation and gradient value, and the measurement index information includes average brightness vector, average saturation vector and average gradient vector.

[0052] The measurement index density of each image frame is obtained by substituting the measurement index information of each image frame into the preset measurement index formula.

[0053] The image quality score of the image frame is obtained by using the density of the metric index and the preset weight value.

[0054] The image frames are classified into categories to obtain multiple sets of image frames with different labeled categories, wherein the classification involves labeling multiple frames with similarity higher than a threshold.

[0055] Extract the image frame with the highest image quality score from the image frame set of each labeled category, and combine all the image frames with the highest image quality scores into a sequence of image frames to be denoised;

[0056] The image frame sequence to be denoised is input into the image frame denoising model to obtain the denoised image frame sequence;

[0057] The aforementioned preset image frame denoising model can be obtained by training a convolutional neural network (CNN). A CNN is a type of feedforward neural network that includes convolutional computation and has a deep structure; it is one of the representative algorithms of deep learning. CNNs have representation learning capabilities and can perform translation-invariant classification of input information according to their hierarchical structure. By using a CNN to denoise each image frame in the sequence of image frames to be denoised, the processing efficiency for each image frame can be improved, thereby efficiently denoising the video. The image frame denoising model described is an existing technique and will not be described in detail here.

[0058] The image frame sets of each labeled category are updated according to the denoised image frame sequence to obtain a new image frame set, and the denoised video is obtained based on the new image frame set.

[0059] The new set of image frames is obtained by replacing the denoised image frames, thus denoising each frame in the video to be denoised, and thus obtaining the denoised video.

[0060] This invention extracts representative image frames to be denoised from a large number of image frames, which greatly reduces the number of image frames that need to be denoised, thereby reducing computation time and computing resources and improving the efficiency of video denoising.

[0061] Preferably, the process of obtaining the measurement index information is as follows:

[0062] Extract the mean value of the image brightness component in the image frame, and use the mean value of the image brightness component as the average brightness vector L;

[0063] Extract the mean value of the saturation components in the image frame, and use the mean value of the saturation components as the average saturation vector S;

[0064] The Laplace operator is used to calculate the gradient value components of the image frame, and the mean of the gradient value components is calculated. The mean of the gradient value components is used as the average gradient vector G.

[0065] First, the mean value of the image brightness component of each image frame is extracted and calculated to obtain the average brightness vector L. The calculation process is to first extract the brightness component of the image, and then calculate the mean value. To facilitate the subsequent calculation, the calculated mean value can be normalized according to the gray level to normalize the mean value to the interval [0, 1] to obtain the average brightness vector L.

[0066] Then, the mean value of the saturation component of each image frame is extracted and calculated to obtain the average saturation vector S. The calculation method is to first extract the saturation component of the image, and then calculate the mean value. To facilitate the subsequent calculation, the calculated mean value can be normalized to the interval [0, 1] to obtain the average saturation vector S.

[0067] Finally, the Laplace operator is used to calculate the gradient components, and the mean of the gradient components is calculated to obtain the average gradient vector G, which forms the metric information for each image frame. The gradient function can be calculated using the Laplace operator, and then the mean is calculated. To facilitate subsequent calculations, the calculated mean can be normalized to the interval [0, 1] to obtain the average gradient vector G.

[0068] Preferably, the specific process for calculating the density of the metric index is as follows:

[0069] Calculate the mean and variance of the average brightness vector L, average saturation vector S, and average gradient vector G respectively;

[0070] The average brightness vector L, average saturation vector S, average gradient vector G, and their corresponding mean and variance are substituted into the measurement index formula to obtain the measurement index density corresponding to the measurement index information.

[0071] The formula for the metric is as follows: Where x takes values ​​from the average brightness vector L, average saturation vector S, and average gradient vector G, σ x With μ x These are the mean and variance corresponding to the measurement indicators, respectively.

[0072] For ease of calculation, the metric density O(L) corresponding to the average brightness vector, the metric density O(S) corresponding to the average saturation vector, and the metric density O(G) corresponding to the average gradient vector can be normalized to the interval [0, 1] to obtain the density vector Y1 corresponding to the average brightness, the density vector Y2 corresponding to the average saturation, and the density vector Y3 corresponding to the average gradient.

[0073] Preferably, the formula for obtaining the image quality score is as follows: Y = A × Y1 + B × Y2 + C × Y3; where A, B, and C are preset weight values, and Y1, Y2, and Y3 are the density vectors corresponding to average brightness, average saturation, and average gradient, respectively.

[0074] The preferred method for category division is as follows:

[0075] Each image frame is matched for similarity with other image frames. If the similarity between two image frames is greater than a threshold, the two image frames are labeled as the same category. If the similarity between two image frames is less than the threshold, the two image frames are labeled as different categories.

[0076] The following example illustrates this: If the similarity between image frame 1 and image frame 2 is determined to be similar, then both image frame 1 and image frame 2 can be labeled as category 1. If the similarity between image frame 3 and image frame 4 is determined to be dissimilar, then image frame 3 and image frame 4 can be labeled as category 3 and category 4 respectively. If the similarity between image frame 1 and image frame 5 is determined to be similar, then both image frame 1 and image frame 5 can be labeled as category 1.

[0077] Preferably, the similarity matching process is as follows:

[0078] Perform depth autoencoding on each image frame to obtain the autoencoding of each image frame; calculate the Euclidean distance between the autoencodings of different image frames; and obtain the similarity between each image frame based on the Euclidean distance between the autoencodings of different image frames.

[0079] In calculating Euclidean distance, a smaller Euclidean distance between two image frames indicates a higher similarity between them. Conversely, a larger Euclidean distance indicates a lower similarity. The threshold for determining similarity can be set based on the processor's processing power. A more powerful processor allows for a larger threshold, resulting in more image frames within a category, and vice versa. This makes the invention applicable to processors with varying processing capabilities.

[0080] Preferably, the step of updating each labeled category image frame set according to the denoised image frame sequence to obtain a new image frame set includes the following steps: replacing each image frame in the corresponding labeled category image frame set with each denoised image frame in the denoised image frame sequence to update each labeled category image frame set and obtain a new image frame set.

[0081] In the above example, if denoised image frame 1 in the denoised image frame sequence corresponds to category 1, then denoised image frame 1, image frame 2, and image frame 5 in category 1 are replaced with denoised image frame 1. If denoised image frame 3 corresponds to category 3, then image frame 3 in category 3 is replaced with denoised image frame 3. If denoised image frame 4 corresponds to category 4, then image frame 4 in category 4 is replaced with denoised image frame 4.

[0082] A video noise reduction system, using the aforementioned video noise reduction method, is characterized by comprising: an image frame extraction module, a metric calculation module, a metric density calculation module, an image quality score calculation module, an image frame annotation module, a denoising image frame extraction module, an image noise reduction module, a noise-reduced video module, and an image frame update module.

[0083] The image frame extraction module is used to extract each image frame from the video to be denoised.

[0084] The metric calculation module is used to obtain image information in each image frame, and to obtain the metric information corresponding to the image information through the image information respectively;

[0085] The metric density calculation module is used to obtain the metric density of each image frame by substituting the metric information of each image frame into a preset metric formula.

[0086] The image quality scoring calculation module is used to obtain the image quality score of the image frame by using the density of the metric index and the preset weight value.

[0087] The image frame annotation module is used to classify the image frames into categories, resulting in multiple sets of image frames with different annotation categories;

[0088] The image frame extraction module is used to extract the image frame with the highest image quality score from each labeled category image frame set, and combine all the image frames with the highest image quality scores into a sequence of image frames to be denoised.

[0089] The image denoising module is used to input the image frame sequence to be denoised into a preset image frame denoising model to obtain a denoised image frame sequence.

[0090] The image frame update module is used to update the image frame set of each labeled category according to the denoised image frame sequence to obtain a new image frame set;

[0091] The noise reduction video module is used to obtain noise reduction video based on the new set of image frames.

[0092] An apparatus includes a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor causes the processor to implement the video noise reduction method described above.

[0093] A storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the aforementioned video noise reduction method.

[0094] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "illustrative embodiment," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0095] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.

Claims

1. A video noise reduction method, characterized in that, Includes the following steps: Extract individual image frames from the video to be denoised; Image information from each image frame is acquired, and measurement index information corresponding to the image information is obtained through the image information. The image information includes brightness, saturation and gradient value, and the measurement index information includes average brightness vector, average saturation vector and average gradient vector. The measurement index density of each image frame is obtained by substituting the measurement index information of each image frame into the preset measurement index formula. The image quality score of the image frame is obtained by using the density of the metric index and the preset weight value. The image frames are classified into categories to obtain multiple sets of image frames with different labeled categories, wherein the classification involves labeling multiple frames with similarity higher than a threshold. Extract the image frame with the highest image quality score from the image frame set of each labeled category, and combine all the image frames with the highest image quality scores into a sequence of image frames to be denoised; The image frame sequence to be denoised is input into the image frame denoising model to obtain the denoised image frame sequence; The image frame sets of each labeled category are updated according to the denoised image frame sequence to obtain a new image frame set, and the denoised video is obtained according to the new image frame set. The process of updating the image frame sets of each labeled category according to the denoised image frame sequence to obtain a new image frame set includes the following steps: replacing each image frame in the corresponding labeled category image frame set with each denoised image frame in the denoised image frame sequence to update each labeled category image frame set and obtain a new image frame set.

2. The video noise reduction method according to claim 1, characterized in that, The process of obtaining the measurement index information is as follows: Extract the mean value of the image brightness component in the image frame, and use the mean value of the image brightness component as the average brightness vector L; Extract the mean value of the saturation components in the image frame, and use the mean value of the saturation components as the average saturation vector S; The Laplace operator is used to calculate the gradient value components of the image frame, and the mean of the gradient value components is calculated. The mean of the gradient value components is used as the average gradient vector G.

3. The video noise reduction method according to claim 2, characterized in that, The specific process for calculating the density of the metric is as follows: Calculate the mean and variance of the average brightness vector L, average saturation vector S, and average gradient vector G respectively; The average brightness vector L, average saturation vector S, average gradient vector G, and their corresponding mean and variance are substituted into the measurement index formula to obtain the measurement index density corresponding to the measurement index information. The formula for the metric is as follows: Where x takes values ​​from the average brightness vector L, average saturation vector S, and average gradient vector G. and These are the mean and variance corresponding to the measurement indicators, respectively.

4. The video noise reduction method according to claim 3, characterized in that, The formula for obtaining the image quality score is as follows: Where A, B, and C are preset weight values, , , These are the density vectors corresponding to average brightness, average saturation, and average gradient, respectively.

5. The video noise reduction method according to claim 1, characterized in that, The specific process of category division is as follows: Each image frame is matched for similarity with other image frames. If the similarity between two image frames is greater than a threshold, the two image frames are labeled as the same category. If the similarity between two image frames is less than the threshold, the two image frames are labeled as different categories.

6. The video noise reduction method according to claim 5, characterized in that, The similarity matching process is as follows: Each image frame is then subjected to depth autoencoding to obtain the autoencoder for each image frame. Calculate the Euclidean distance between the autoencoders of different image frames respectively; The similarity between different image frames is obtained by calculating the Euclidean distance between the autoencoders of different image frames.

7. A video noise reduction system, using the video noise reduction method according to any one of claims 1 to 6, characterized in that, include: The system includes an image frame extraction module, a metric calculation module, a metric density calculation module, an image quality score calculation module, an image frame annotation module, a denoising image frame extraction module, an image denoising module, a denoised video module, and an image frame update module. The image frame extraction module is used to extract each image frame from the video to be denoised. The metric calculation module is used to obtain image information in each image frame, and to obtain the metric information corresponding to the image information through the image information respectively; The metric density calculation module is used to obtain the metric density of each image frame by substituting the metric information of each image frame into a preset metric formula. The image quality scoring calculation module is used to obtain the image quality score of the image frame by using the density of the metric index and the preset weight value. The image frame annotation module is used to classify the image frames into categories, resulting in multiple sets of image frames with different annotation categories; The image frame extraction module is used to extract the image frame with the highest image quality score from each labeled category image frame set, and combine all the image frames with the highest image quality scores into a sequence of image frames to be denoised. The image denoising module is used to input the image frame sequence to be denoised into a preset image frame denoising model to obtain a denoised image frame sequence. The image frame update module is used to update the image frame set of each labeled category according to the denoised image frame sequence to obtain a new image frame set; The noise reduction video module is used to obtain noise reduction video based on the new set of image frames.

8. A video noise reduction device, characterized in that, The device includes a memory and a processor. The memory stores a computer program, which, when executed by the processor, causes the processor to implement a video noise reduction method as described in any one of claims 1 to 6.

9. A storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements a video noise reduction method as described in any one of claims 1 to 6.

Images