Noise estimation method and device, computer equipment, storage medium and program product

Through motion compensation and difference calculation of adjacent video frames, the problem of low accuracy of traditional noise estimation methods is solved, and more efficient video noise level estimation is achieved, and the performance of the video link system is improved.

CN118945280APending Publication Date: 2024-11-12XIAOHONGSHU TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202410945351.3
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-15
Publication Date
2024-11-12

AI Technical Summary

Technical Problem

The traditional noise estimation method has a low accuracy in estimating the noise level of video, which affects the calculation complexity and image quality processing efficiency of the video link system.

Method used

By obtaining adjacent decoded video frames, motion compensation is performed based on the pixel point position offset, the compensated frame is obtained, and the airspace noise is calculated by compensating the difference between the compensation frame and the previous frame, thereby accurately estimating the noise level of the video.

Benefits of technology

It improves the accuracy of noise estimation, reduces the computational complexity in the video link system, and improves the video image quality processing efficiency.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118945280A_ABST
    Figure CN118945280A_ABST
Patent Text Reader

Abstract

The invention relates to a noise estimation method and device, computer equipment, a storage medium and a program product. The noise estimation method comprises the steps that a previous frame in a video and a current frame in the video are acquired, the previous frame and the current frame are adjacent decoded video frames, and the decoding sequence of the previous frame is before the current frame; on the basis of the position offset of the pixel point of the current frame relative to the pixel point of the previous frame, predicting the position of the pixel point of the previous frame to obtain a compensation frame of the previous frame; obtaining first spatial domain noise of the previous frame based on the difference between the pixel points of the compensation frame and the pixel points of the previous frame; and obtaining the noise level of the video based on the first spatial domain noise. By adopting the noise estimation method, the noise level of the video can be estimated more accurately.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of video processing, and in particular to a noise estimation method and device, computer equipment, storage medium and program product. Background Art

[0002] Noise level estimation is a common video processing algorithm. Accurately estimating the noise level of a video is helpful for better designing downstream noise reduction algorithms and more accurately determining whether a video needs to be processed in a noise reduction module, thereby reducing the number of video noise reduction operations in the video link system, thereby reducing the computational complexity in the video link system and improving the video quality processing efficiency. However, the traditional noise estimation method has the disadvantage of low accuracy in estimating the noise level of a video. Summary of the invention

[0003] Based on this, it is necessary to provide a noise estimation method and apparatus, computer equipment, storage medium and program product that can more accurately estimate the noise level of a video in response to the above technical problems.

[0004] In a first aspect, the present application provides a noise estimation method, the method comprising:

[0005] Obtaining a previous frame in a video and a current frame in the video, wherein the previous frame and the current frame are adjacent decoded video frames, and a decoding order of the previous frame is before that of the current frame;

[0006] Predicting the positions of the pixels of the previous frame based on the position offsets of the pixels of the current frame compared to the pixels of the previous frame to obtain a compensated frame of the previous frame;

[0007] Based on the difference between the pixel points of the compensation frame and the pixel points of the previous frame, obtaining the first spatial noise of the previous frame;

[0008] A noise level of the video is obtained based on the first spatial domain noise.

[0009] The noise estimation method provided in the first aspect is adopted to obtain the previous frame and the current frame of the adjacent decoding in the video, and the position of the pixel points of the previous frame is predicted based on the position offset of the pixel points of the current frame compared with the pixel points of the previous frame. Based on the predicted positions of each pixel point, a compensation frame for predicting the previous frame is obtained. This noise estimation method introduces video motion estimation, performs motion compensation based on the previous frame, and obtains a compensation frame after motion compensation. If the motion compensation is successful, the previous frame is highly similar to the compensation frame, and there is almost no difference between the pixel points of the previous frame and the pixel points of the compensation frame. If there is a difference between the pixel points of the previous frame and the pixel points of the compensation frame, it indicates that the motion compensation is not fully successful, and the incomplete compensation is affected by the first spatial noise contained in the previous frame. Therefore, the difference between the pixel points of the compensation frame and the pixel points of the previous frame, that is, the difference between the predicted pixel points of the previous frame and the actual pixel points of the previous frame, can be used to characterize the information change caused by the first spatial noise contained in the previous frame. Based on the difference between the pixels of the compensation frame and the pixels of the previous frame, the first spatial noise of the previous frame can be accurately obtained. Furthermore, since the noise level of the video can be obtained based on the first spatial noise of the previous frame of the adjacent decoded video, when the first spatial noise of the previous frame is accurately obtained, the noise level of the video can be obtained more accurately.

[0010] In one of the embodiments, obtaining the first spatial noise of the previous frame based on the difference between the pixel points of the compensation frame and the pixel points of the previous frame includes:

[0011] Determine the difference frame between the compensation frame and the previous frame, the brightness information of the difference pixel of the difference frame is the absolute value of the difference between the brightness information of the first pixel of the previous frame and the brightness information of the second pixel of the compensation frame, the position of the difference pixel in the difference frame is the same as the position of the first pixel in the previous frame and the position of the second pixel in the compensation frame; the noise level of the difference frame is used to represent the first spatial domain noise of the previous frame;

[0012] A difference feature value of the difference frame is extracted, where the difference feature value is used to represent a noise level of the difference frame.

[0013] In one of the embodiments, the difference frame includes at least one difference image block; and extracting the difference feature value of the difference frame includes:

[0014] Performing a first target processing on the at least one difference image block of the difference frame to obtain at least one first processing parameter of the at least one difference image block, wherein the difference image block corresponds to the first processing parameter in a one-to-one manner; the first processing parameter is used to indicate a smoothness degree of the difference image block;

[0015] Determining at least one first target parameter less than a first threshold value from the at least one first processing parameter; the smoothness of the difference image block corresponding to the first target parameter matches the noise of the difference frame;

[0016] The difference feature value of the difference frame is determined from the at least one first target parameter.

[0017] In one of the embodiments, when the first target processing is to calculate the wavelet coefficients, the first processing parameters include the first wavelet coefficients, the second wavelet coefficients, the third wavelet coefficients and the fourth wavelet coefficients obtained after the wavelet decomposition of the difference image block; the first wavelet coefficients refer to the wavelet coefficients generated by processing the difference image block in both the row direction and the column direction using a high-pass wavelet filter, the second wavelet coefficients refer to the wavelet coefficients generated by processing the difference image block in the row direction using a low-pass wavelet filter and then in the column direction using a high-pass wavelet filter, the third wavelet coefficients refer to the wavelet coefficients generated by processing the difference image block in the row direction using a low-pass wavelet filter and then in the column direction using a high-pass wavelet filter, The coefficient refers to a wavelet coefficient generated by processing the difference image block in the row direction using a high-pass wavelet filter and then processing the difference image block in the column direction using a low-pass wavelet filter; the fourth wavelet coefficient refers to a wavelet coefficient generated by processing the difference image block in both the row direction and the column direction using a low-pass wavelet filter; the at least one first target parameter includes at least one first parameter and at least one second parameter, and the first target parameter corresponds to the first parameter and the second parameter one by one; the determining of at least one first target parameter less than a first threshold value from the at least one first processing parameter includes:

[0018] determining at least one second wavelet coefficient and at least one third wavelet coefficient from the at least one first processing parameter;

[0019] determining, from the at least one second wavelet coefficient, the at least one first parameter that is less than the first threshold;

[0020] From the at least one third wavelet coefficient, the at least one second parameter is determined which is smaller than the first threshold value.

[0021] In one of the embodiments, determining the difference feature value of the difference frame from the at least one first target parameter comprises:

[0022] Determine a first peak value and a first mean value of the at least one first target parameter, wherein the first peak value refers to the first target parameter with the highest occurrence frequency, and the first mean value refers to the average value of the at least one first target parameter;

[0023] The first peak value and the first mean value are used as the difference feature values ​​of the difference frame.

[0024] In one embodiment, before taking the first peak value and the first mean value as the difference feature value of the difference frame, the method further includes:

[0025] Performing eigenvalue decomposition on the covariance of the difference frame to obtain a target eigenvalue, wherein the target eigenvalue is used to represent the noise level of the difference frame;

[0026] The step of taking the first peak value and the first mean value as the difference feature value of the difference frame comprises:

[0027] The first peak value, the first mean value and the target feature value are used as the difference feature value of the difference frame.

[0028] In one embodiment, performing eigenvalue decomposition on the covariance of the difference frame to obtain a target eigenvalue includes:

[0029] determining the covariance of the difference frame;

[0030] Performing eigenvalue decomposition on the covariance to obtain at least one eigenvalue;

[0031] sorting the at least one eigenvalue from small to large;

[0032] In the at least one sorted eigenvalue, if the absolute value of the difference between the adjacent first eigenvalue and the second eigenvalue is greater than the difference threshold, determining the first eigenvalue as the target eigenvalue;

[0033] Among them, the first eigenvalue is smaller than the second eigenvalue, and the first eigenvalue and the second eigenvalue are the smallest set of target values ​​among the at least one eigenvalue after sorting, and the target value refers to the adjacent eigenvalues ​​whose absolute value of the difference is greater than the difference threshold.

[0034] In one embodiment, obtaining the noise level of the video based on the first spatial noise includes:

[0035] Determine an average frame of the compensation frame and the current frame, wherein brightness information of an average pixel of the average frame is a weighted sum of brightness information of a third pixel of the compensation frame and brightness information of a fourth pixel of the current frame, and a position of the average pixel in the average frame is the same as a position of the third pixel in the compensation frame and a position of the fourth pixel in the current frame;

[0036] Extracting a mean feature value of the mean frame, wherein the mean feature value is used to represent a noise level of the mean frame, and the noise level of the mean frame is used to represent a second spatial domain noise of the current frame;

[0037] A noise level of the video is obtained based on the first spatial domain noise and the second spatial domain noise.

[0038] In one embodiment, the mean frame includes at least one mean image block; and extracting the mean feature value of the mean frame includes:

[0039] Performing a second target processing on the at least one mean image block of the mean frame to obtain at least one second processing parameter of the at least one mean image block, wherein the mean image block corresponds to the second processing parameter in a one-to-one manner; the second processing parameter is used to indicate a smoothness degree of the mean image block;

[0040] From the at least one second processing parameter, determining at least one second target parameter that is less than a second threshold; the smoothness of the mean image block corresponding to the second target parameter matches the noise of the mean frame;

[0041] The mean feature value of the mean frame is determined from the at least one second target parameter.

[0042] In one embodiment, obtaining the noise level of the video based on the first spatial noise and the second spatial noise includes:

[0043] Based on the difference between the pixel points at the same position of the previous frame and the current frame, obtaining the temporal noise between the previous frame and the current frame;

[0044] A noise level of the video is obtained based on the first spatial noise, the second spatial noise and the temporal noise.

[0045] In one embodiment, obtaining the temporal noise between the previous frame and the current frame based on the difference between the pixels at the same position of the previous frame and the current frame includes:

[0046] Determine a mixed frame of the previous frame and the current frame, the mixed image block of the mixed frame is formed by splicing the first image block of the previous frame and the second image block of the current frame, the mixed image block includes pixel points of the mixed frame, the first image block includes pixel points of the previous frame, the second image block includes pixel points of the current frame, and the position of the first image block in the previous frame is the same as the position of the second image block in the current frame; the noise level of the mixed frame is used to represent the temporal noise between the previous frame and the current frame;

[0047] A mixed feature value of the mixed frame is extracted, where the mixed feature value is used to represent a noise level of the mixed frame.

[0048] In one of the embodiments, the mixed frame includes at least one mixed image block; and extracting the mixed feature value of the mixed frame includes:

[0049] Performing a third target processing on the at least one mixed image block of the mixed frame to obtain at least one third processing parameter of the at least one mixed image block, wherein the mixed image block corresponds to the third processing parameter in a one-to-one manner; the third processing parameter is used to indicate the smoothness of the mixed image block;

[0050] Determining at least one third target parameter less than a third threshold value from the at least one third processing parameter; the smoothness of the mixed image block corresponding to the third target parameter matches the noise of the mixed frame;

[0051] The mixed characteristic value of the mixed frame is determined from the at least one third target parameter.

[0052] In one embodiment, obtaining the noise level of the video based on the first spatial noise, the second spatial noise and the temporal noise includes:

[0053] The first spatial domain noise, the second spatial domain noise, and the time domain noise are linearly fitted to obtain a noise level of the video.

[0054] In one embodiment, predicting the position of the pixel points of the previous frame based on the position offset of the pixel points of the current frame compared with the pixel points of the previous frame to obtain the compensated frame of the previous frame includes:

[0055] Determine a position offset of a fifth pixel point of the current frame compared to a sixth pixel point of the previous frame, the fifth pixel point and the sixth pixel point having the same semantics;

[0056] Based on the fifth pixel of the current frame, predict the sixth pixel of the previous frame according to the position offset to obtain a seventh pixel;

[0057] Based on the seventh pixel point, a compensation frame of the previous frame is obtained.

[0058] In a second aspect, the present application further provides a noise estimation device, the device comprising:

[0059] A video frame acquisition module, used to acquire a previous frame in a video and a current frame in the video, wherein the previous frame and the current frame are adjacent decoded video frames, and the decoding order of the previous frame is before the current frame;

[0060] A video frame prediction module, configured to predict the positions of the pixels of the previous frame based on the position offsets of the pixels of the current frame compared to the pixels of the previous frame, and obtain a compensation frame of the previous frame;

[0061] A spatial noise determination module, configured to obtain a first spatial noise of the previous frame based on a difference between a pixel point of the compensation frame and a pixel point of the previous frame;

[0062] The video noise estimation module obtains the noise level of the video based on the first spatial noise.

[0063] In a third aspect, the present application further provides a computer device, comprising: a memory and a processor, wherein the memory stores program instructions; when the program instructions are executed by the processor, the processor executes the method shown in the first aspect or any embodiment of the first aspect.

[0064] In a fourth aspect, the present application further provides a computer-readable storage medium, wherein a computer program is stored in the computer-readable storage medium; when the computer program runs on one or more processors, the method shown in the first aspect or any embodiment of the first aspect is executed.

[0065] In a fifth aspect, the present application also provides a computer program product, which includes a computer program or instructions; when the computer program or instructions are run on a computer, the computer is enabled to execute the method shown in the first aspect or any embodiment of the first aspect.

[0066] It can be understood that the noise estimation device provided in the second aspect, the computer device provided in the third aspect, the computer-readable storage medium provided in the fourth aspect, and the computer program product provided in the fifth aspect can be used to execute the noise estimation method shown in the first aspect of the present application or any embodiment of the first aspect. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding method, which will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0067] In order to clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments are introduced as follows.

[0068] Figure 1 One of the flowcharts of the noise estimation method provided in the embodiment of the present application;

[0069] Figure 2 A schematic diagram of a process for obtaining the first spatial noise of the previous frame based on the difference between the pixel points of the compensation frame and the pixel points of the previous frame provided in an embodiment of the present application;

[0070] Figure 3A schematic diagram of a process for extracting difference feature values ​​of a difference frame provided in an embodiment of the present application;

[0071] Figure 4 A schematic diagram of a process for determining at least one first target parameter that is less than a first threshold value from at least one first processing parameter provided in an embodiment of the present application;

[0072] Figure 5 A schematic diagram of a process for determining a difference feature value of a difference frame from at least one first target parameter provided in an embodiment of the present application;

[0073] Figure 6 A second flowchart of the noise estimation method provided in an embodiment of the present application;

[0074] Figure 7 A schematic diagram of a process for performing eigenvalue decomposition on the covariance of a difference frame to obtain a target eigenvalue provided in an embodiment of the present application;

[0075] Figure 8 A schematic diagram of a process for obtaining a noise level of a video based on the first spatial domain noise provided in an embodiment of the present application;

[0076] Fig. 9 A schematic diagram of a process for extracting a mean feature value of a mean frame provided in an embodiment of the present application;

[0077] Fig.10 A schematic diagram of a process for obtaining a noise level of a video based on first spatial noise and second spatial noise provided in an embodiment of the present application;

[0078] Fig.11 A schematic diagram of a process for obtaining the temporal noise between a previous frame and a current frame based on the difference between pixels at the same position of the previous frame and the current frame provided in an embodiment of the present application;

[0079] Fig.12 A schematic diagram of a process for extracting mixed feature values ​​of mixed frames provided in an embodiment of the present application;

[0080] Fig.13 A third flowchart of the noise estimation method provided in an embodiment of the present application;

[0081] Fig.14 A schematic diagram of the structure of a noise estimation device provided in an embodiment of the present application;

[0082] Fig.15 A schematic diagram of the structure of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0083] In order to facilitate understanding of the embodiments of the present application, the embodiments of the present application will be described more fully below with reference to the relevant drawings. The preferred embodiments of the embodiments of the present application are given in the drawings. However, the embodiments of the present application can be implemented in many different forms and are not limited to the embodiments described herein. On the contrary, the purpose of providing these embodiments is to make the disclosure of the embodiments of the present application more thorough and comprehensive.

[0084] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art of the technical field of the embodiments of the present application. The terms used herein in the description of the embodiments of the present application are only for the purpose of describing specific embodiments and are not intended to limit the embodiments of the present application.

[0085] The terms "first", "second", etc. used in this article are used to distinguish different objects rather than to describe a specific order. For the purpose of distinguishing different objects, the "first airspace noise" can also be described as the "second airspace noise", and the "second airspace noise" can be described as the "first airspace noise". When used, the singular expressions "one", "a kind", "said", "above", "the" and "this" are intended to also include plural expressions, unless there is a clear indication to the contrary in the context. It should also be understood that the term "includes / comprising" specifies the presence of the stated features, wholes, steps, operations, parts or their fusions, but does not exclude the possibility of the presence or addition of one or more other features, wholes, steps, operations, parts or their fusions.

[0086] The present application embodiment provides a noise estimation method. Figure 1 As shown, the noise estimation method includes the following steps S101 to S104.

[0087] S101, obtaining a previous frame in a video and a current frame in the video. The previous frame and the current frame are adjacent decoded video frames, and the decoding order of the previous frame is before the current frame.

[0088] Specifically, the received video code stream is decoded to obtain two adjacent decoded video frames in the video: the previous frame and the current frame, and the decoding order of the previous frame is before the decoding order of the current frame. Among them, (H, W) can be used to represent the resolution of the previous frame and the current frame, H is the abbreviation of Height, which is used to represent the number of pixels in the vertical direction of the video frame, and W is the abbreviation of Width, which is used to represent the number of pixels in the horizontal direction of the video frame.

[0089] Exemplarily, before the previous frame and the current frame are processed, the previous frame and the current frame are respectively downsampled so that their resolutions are (H / n, W / n), where n is a positive integer. In order to prevent the amount of information of the video frame from being lost too much and affecting the quality of the video frame, n can be taken as 2, that is, the resolution of the previous frame and the current frame is adjusted to (H / 2, W / 2). The resolution of the previous frame and the current frame is reduced, so that the pixels of the previous frame and the current frame are reduced, which is conducive to reducing the amount of calculation based on the pixels when the previous frame and the current frame are subsequently processed, which is conducive to reducing the computational complexity and computing resource overhead, and is conducive to more efficiently estimating the noise level of the video. It should be understood that when the computational complexity is reduced, the demand for computing resources is smaller, and the efficiency of estimating the noise level of the video is improved, the noise estimation method provided in the embodiment of the present application can be applied to more models and more types of terminal devices in addition to being used for servers. That is, it is conducive to solving the problem that it is difficult for low-profile terminal devices to accurately estimate the noise level of the video.

[0090] It should be noted that, in order to more accurately estimate the noise level of the video, the resolution of the previous frame and the current frame in the embodiment of the present application may be (H, W). In order to more accurately estimate the noise level of the video and further improve the efficiency of estimating the noise level of the video, the embodiment of the present application and subsequent embodiments will be discussed using the resolution of the previous frame and the current frame as (H / 2, W / 2) as an example.

[0091] Exemplarily, before processing the previous frame and the current frame, the pixel values ​​of the pixels of the previous frame, the compensation frame and other video frames are converted into the YUV format, wherein "Y" represents brightness (Luminance or Luma), that is, brightness information; "U" and "V" represent chrominance (Chrominance or Chroma). After the pixel values ​​of the pixels are converted into the YUV format, the video frame can be regarded as consisting of a Y channel image, a U channel image and a V channel image. Since the amount of information contained in the Y channel image is much greater than the amount of information contained in the U channel image and the V channel image, the noise contained in the Y channel image accounts for a large proportion of the noise contained in the video frame, and from a visual perspective, the object is most sensitive to the Y channel image. Therefore, the Y channel image of the video frame can be regarded as a video frame, the brightness information of the pixel point replaces the pixel value of the pixel point, and the noise of the Y channel image is used as the noise of the video frame. This approach is helpful to reduce the amount of calculation based on the pixel value of each pixel point when performing subsequent processing on each video frame, which is helpful to reduce the computational complexity, reduce the computational resource overhead, and more efficiently estimate the noise level of the video. In addition, in actual application scenarios, the noise of U channel images and V channel images is less common than the noise of Y channel images. If the noise of U channel images and V channel images is calculated together, the noise of the obtained video frame will only change slightly, but the amount of calculation and computational complexity will be greatly increased, and the cost performance is low.

[0092] It should be understood that, similar to downsampling the previous frame and the current frame to reduce the resolution, when the brightness information of the pixel point is used instead of the pixel value of the pixel point for calculation, the calculation complexity is reduced and the efficiency of estimating the noise level of the video is improved. In addition to being applicable to servers, the noise estimation method provided in the embodiment of the present application can also be applied to more models and types of terminal devices. That is, it is also helpful to solve the problem that it is difficult for low-end terminal devices to accurately estimate the noise level of the video.

[0093] It should be noted that if there is no restriction or requirement to reduce the computational complexity and improve the efficiency of estimating the noise level, the pixel value or YUV value of the pixel point can be directly used for calculation when the video frame is subsequently processed. In the embodiment of the present application and subsequent embodiments, the brightness information of the pixel point is used instead of the pixel value of the pixel point for calculation, and only the brightness information of the pixel point is processed accordingly, so as to more accurately estimate the noise level of the video and further improve the efficiency of estimating the noise level of the video.

[0094] S102, predicting the positions of the pixels of the previous frame based on the position offsets of the pixels of the current frame compared with the pixels of the previous frame, to obtain a compensated frame of the previous frame.

[0095] The noise estimation method of this embodiment introduces video motion estimation and performs motion compensation based on the previous frame. The position of the pixel of the previous frame is predicted based on the position offset of the pixel of the current frame compared with the pixel of the previous frame. Based on the predicted position of each pixel, a compensated frame after motion compensation is obtained.

[0096] In one embodiment, step S102, based on the position offset of the pixel points of the current frame compared with the pixel points of the previous frame, predicting the position of the pixel points of the previous frame to obtain a compensated frame of the previous frame, includes the following steps S1021 to S1023.

[0097] S1021, determining a position offset of a fifth pixel point of the current frame compared to a sixth pixel point of the previous frame. The fifth pixel point and the sixth pixel point have the same semantics.

[0098] Among them, the fifth pixel point can be any pixel point in the current frame. For the purpose of being able to determine the position offset between two pixel points with the same semantics, this embodiment does not limit the method for determining the position offset. Exemplarily, an optical flow calculation method can be used, that is, calculating the optical flow information between two adjacent frames: the previous frame and the current frame, to obtain the horizontal and vertical position offsets of any two pixel points with the same semantics in the two adjacent frames.

[0099] S1022, based on the fifth pixel of the current frame, predict the sixth pixel of the previous frame according to the position offset to obtain the seventh pixel.

[0100] Specifically, if the position of the fifth pixel of the current frame is expressed as (0,2) in the form of coordinates, the position offset of the fifth pixel compared to the sixth pixel of the previous frame is 0 units horizontally to the right and 2 units vertically upward. Based on the fifth pixel of the current frame, the sixth pixel of the previous frame is predicted based on the position offset, and the position of the seventh pixel obtained is expressed in the form of coordinates, which is (0,0).

[0101] It should be noted that the position of the seventh pixel obtained by predicting the sixth pixel is not necessarily the same as the actual position of the sixth pixel. There are two reasons: First, when calculating the position offset, the absolute value of the position difference between the fifth pixel and the sixth pixel in each direction is taken. If the absolute value of the position difference in a certain direction is less than the preset threshold, the position offset of the two pixels in a certain direction is considered to be 0. If the absolute value of the position difference between the two pixels in the horizontal direction is 0.5 and the preset threshold is 1, the position offset of the two pixels in the horizontal direction is considered to be 0. Second, the above preset threshold is not constant, and there will be errors when determining the preset threshold. If the preset threshold determined according to the preset rule is 0.5, the actual preset threshold is 1 obtained by rounding off 0.5. At this time, there is an error in the preset threshold, which will lead to an error in the position offset determined based on the preset threshold.

[0102] S1023, obtaining a compensation frame of the previous frame based on the seventh pixel point.

[0103] By using the above method of predicting the seventh pixel point from the sixth pixel point of the previous frame, each pixel point of the previous frame is predicted, and then each pixel point of the compensated frame after motion compensation can be obtained, that is, the compensated frame of the previous frame can be obtained.

[0104] S103, obtaining first spatial noise of the previous frame based on the difference between the pixel points of the compensation frame and the pixel points of the previous frame.

[0105] The spatial noise of a video frame refers to the difference in pixel values ​​or brightness information of different pixels in the area where the pixel values ​​or brightness information of different pixels should be the same. From the perspective of visual experience, the spatial noise of a video frame can be understood as the granular texture that appears in the video frame. The first spatial noise refers to the spatial noise of the previous frame.

[0106] Specifically, if the aforementioned motion compensation is successful, the previous frame is highly similar to the compensated frame, and there is almost no difference between the pixels of the previous frame and the pixels of the compensated frame. If there is a difference between the pixels of the previous frame and the pixels of the compensated frame, it indicates that the motion compensation is not fully successful, and the incomplete compensation is affected by the first spatial noise contained in the previous frame. Therefore, the difference between the pixels of the compensated frame and the pixels of the previous frame, that is, the difference between the predicted pixels of the previous frame and the actual pixels of the previous frame, can be used to characterize the information changes caused by the first spatial noise contained in the previous frame. Based on the difference between the pixels of the compensated frame and the pixels of the previous frame, the first spatial noise of the previous frame can be accurately obtained.

[0107] It should be noted that in this embodiment, the first spatial noise of the previous frame is not determined based on the difference between the pixels of the current frame and the pixels of the previous frame. The reason is that the relative difference between the pixels of the current frame and the previous frame is large. The large difference is most likely due to the information change caused by the movement of the object in the video frame. Therefore, the accuracy of the first spatial noise of the previous frame determined in this way is still low.

[0108] S104: Obtain a noise level of the video based on the first spatial noise.

[0109] Since the noise level of the video can be obtained based on the first spatial noise of the previous frame decoded adjacently in the video, the noise level of the video can be obtained more accurately when the first spatial noise of the previous frame is accurately obtained.

[0110] By using the noise estimation method in the embodiment of the present application, motion estimation is performed on the previous frame and the current frame of the adjacent decoded video to obtain a compensation frame after motion compensation. Based on the difference between the pixel points of the compensation frame and the pixel points of the previous frame, the first spatial noise of the previous frame can be accurately obtained. Thus, the noise level of the video can be estimated more accurately.

[0111] By using the noise estimation method in the embodiment of the present application, after estimating a more accurate noise level of the video, it is possible to more accurately determine whether the video needs to enter the noise reduction module for processing. Specifically, in the case where the noise reduction module is a module inside the decoder, if the noise level of the video is lower than the preset noise threshold, the noise reduction function of the decoder is turned off, so that the video does not need to enter the noise reduction module for processing, so as to reduce the number of video noise reduction in the video link system, thereby reducing the computational complexity in the video link system and improving the image quality processing efficiency of the video; if the noise level of the video is higher than the preset noise threshold, the noise reduction function of the decoder is turned on, so that the video enters the noise reduction module for processing. It should be understood that if the noise reduction function of the decoder is turned off by default, there is no need to repeatedly turn off the noise reduction function when the noise level of the video is lower than the preset noise threshold. In the case where the noise reduction module is independent of the decoder, if the noise level of the video is lower than the preset noise threshold, the video does not need to enter the noise reduction module for processing; if the noise level of the video is higher than the preset noise threshold, the video enters the noise reduction module for processing.

[0112] In the case of estimating the noise level of a video based on the current frame and the previous frame of the video, if it is determined based on the noise level of the video that the video needs to enter the noise reduction module for processing, the video will be denoised starting from the next video frame of the current frame. On this basis, if the noise level of the video is estimated periodically, the video will be denoised starting from the next video frame of the current frame of the current cycle, and the noise reduction will end until the current frame of the next cycle. When it is determined based on the noise level of the video that the video does not need to enter the noise reduction module for processing, the starting video frame and the ending video frame for not performing noise reduction can be determined in the same way.

[0113] Exemplarily, by more accurately estimating the noise level of the video, it is not directly determined whether the video needs to enter the noise reduction module for processing, but the video is made to enter the noise reduction module for processing, and based on the noise level of the video, the noise reduction parameters are adjusted to achieve different intensities of noise reduction for the video. Specifically, the higher the noise level, the stronger the noise reduction intensity; the lower the noise level, the weaker the noise reduction intensity. In this example, the noise reduction module can also be a module belonging to the decoder or independent of the decoder.

[0114] In one embodiment, if Figure 2 As shown, the above step S103, based on the difference between the pixel points of the compensation frame and the pixel points of the previous frame, obtains the first spatial noise of the previous frame, including the following steps S201 to S202.

[0115] S201, determining a difference frame between a compensation frame and a previous frame. The brightness information of a difference pixel of the difference frame is the absolute value of the difference between the brightness information of a first pixel of the previous frame and the brightness information of a second pixel of the compensation frame, and the position of the difference pixel in the difference frame is the same as the position of the first pixel in the previous frame and the position of the second pixel in the compensation frame; the noise level of the difference frame is used to represent the first spatial noise of the previous frame.

[0116] The difference between the pixels of the compensation frame and the pixels of the previous frame is converted into a difference frame, and the difference frame includes the difference information between the pixels. According to the discussion in the above embodiment, since the difference between the pixels can be used to characterize the information change caused by the first spatial noise included in the previous frame, the difference frame includes the first spatial noise of the previous frame, and the noise level of the difference frame can be used to represent the first spatial noise of the previous frame.

[0117] Specifically, the brightness information of the first pixel of the previous frame is subtracted from the brightness information of the second pixel of the compensation frame and the absolute value is calculated to obtain the difference pixel of the difference frame. The first pixel is an arbitrary pixel of the previous frame, and the position of the difference pixel in the difference frame is the same as the position of the first pixel in the previous frame and the position of the second pixel in the compensation frame. It can be understood that by using the above method of obtaining the difference pixel of the difference frame based on the first pixel of the previous frame and the second pixel of the compensation frame, each pixel of the previous frame and each pixel of the compensation frame are processed accordingly, and each pixel of the difference frame can be obtained, that is, the difference frame is determined.

[0118] S202: extracting a difference feature value of the difference frame. The difference feature value is used to represent the noise level of the difference frame.

[0119] In this embodiment, the difference between the pixel points of the compensation frame and the pixel points of the previous frame is converted into the form of a difference frame. Since the difference frame contains the difference information between the pixel points, the difference frame contains the first spatial noise of the previous frame. Furthermore, by extracting the difference characteristic value used to represent the noise level of the difference frame, the first spatial noise of the previous frame can be obtained. In addition, in this embodiment, the brightness information of the pixel point is used instead of the pixel value of the pixel point for calculation, and only the brightness information of the pixel point is processed accordingly to obtain the brightness information of the difference pixel point of the difference frame, which is conducive to reducing the calculation complexity and improving the efficiency of estimating the first spatial noise of the previous frame and the noise level of the video.

[0120] In one embodiment, the difference frame includes at least one difference image block. Figure 3 As shown, the above step S202, extracting the difference feature value of the difference frame, includes the following steps S301 to S303.

[0121] S301, performing a first target processing on at least one difference image block of a difference frame to obtain at least one first processing parameter of the at least one difference image block. The difference image block corresponds to the first processing parameter one by one; the first processing parameter is used to indicate the smoothness of the difference image block.

[0122] If the difference frame with a resolution of (H / 2, W / 2) is divided into 8*8 difference image blocks, the number of difference image blocks is:

[0123]

[0124] The purpose is to obtain a first processing parameter that can be used to represent the smoothness of the corresponding difference image block. The first target processing includes but is not limited to calculating variance, calculating gradient information, and calculating wavelet coefficients. The first target processing can be performed based on the brightness information of the pixels in the difference image block. When the first target processing is to calculate the variance, the first processing parameter is the variance of the difference image block; when the first target processing is to calculate the gradient information, the first processing parameter is the gradient information of the pixels of the difference image block, where the Sobel operator can be used to calculate the gradient information of the pixels; when the first target processing is to calculate the wavelet coefficients, the first processing parameter is the wavelet coefficients of the difference image block.

[0125] S302: Determine at least one first target parameter that is less than a first threshold value from at least one first processing parameter, wherein the smoothness of the difference image block corresponding to the first target parameter matches the noise of the difference frame.

[0126] The difference frame contains the difference information between the pixels. The difference between the pixels of the previous frame and the pixels of the compensation frame may be the information change caused by the first spatial noise contained in the previous frame, or the information change caused by the movement of the object in the previous frame. Based on this, the smoothness of the difference image block of the difference frame may match the texture information caused by the noise of the difference frame, or may match the texture information caused by the movement of the object. In order to more accurately obtain the noise level of the difference frame and more accurately determine the first spatial noise of the previous frame, it is necessary to determine the first target parameter representing the target smoothness from the first processing parameter used to represent the smoothness of the difference image block of the difference frame. This target smoothness is the smoothness that matches the texture information caused by the noise of the difference frame.

[0127] Specifically, the texture information caused by noise is less than the texture information caused by motion, and the smoothness matching the texture information caused by noise is smaller, while the smoothness matching the texture information caused by motion is larger. Based on this, the first threshold is set, and the first processing parameter less than the first threshold can be determined as the first target parameter. The smoothness of the difference image block corresponding to the first target parameter matches the noise of the difference frame, and the first target parameter can be used to represent the noise level of the difference frame. The smoothness of the difference image block corresponding to the first processing parameter greater than or equal to the first threshold matches the motion.

[0128] It should be understood that the difference between the pixels of the previous frame and the pixels of the compensated frame can be used to characterize the information change caused by the first spatial noise contained in the previous frame. On this basis, the difference information caused by the actual movement of the object is further eliminated from the difference information between the pixels, and the remaining difference information can more accurately characterize the information change caused by the first spatial noise contained in the previous frame.

[0129] S303: Determine a difference feature value of the difference frame from at least one first target parameter.

[0130] In theory, at least one first target parameter used to represent the smoothness of at least one difference image block can be used to represent the noise level of the difference frame. Therefore, the difference feature value of the difference frame can be determined from the at least one first target parameter.

[0131] In this embodiment, at least one first processing parameter for representing the smoothness of the difference image block of the difference frame is first obtained, and then at least one first target parameter representing the smoothness of the difference image block and the noise of the difference frame is determined from the at least one first processing parameter, and the at least one first target parameter can be used to represent the noise level of the difference frame. Furthermore, the difference feature value for representing the noise level of the difference frame can be determined from the at least one first target parameter.

[0132] In one embodiment, when the first target processing is to calculate the wavelet coefficient, the first processing parameter includes the first wavelet coefficient, the second wavelet coefficient, the third wavelet coefficient and the fourth wavelet coefficient obtained after wavelet decomposition of the difference image block. The first wavelet coefficient refers to the wavelet coefficient generated by processing the difference image block in both the row direction and the column direction using a high-pass wavelet filter, the second wavelet coefficient refers to the wavelet coefficient generated by processing the difference image block in the row direction using a low-pass wavelet filter and then in the column direction using a high-pass wavelet filter, the third wavelet coefficient refers to the wavelet coefficient generated by processing the difference image block in the row direction using a high-pass wavelet filter and then in the column direction using a low-pass wavelet filter, and the fourth wavelet coefficient refers to the wavelet coefficient generated by processing the difference image block in both the row direction and the column direction using a low-pass wavelet filter. At least one first target parameter includes at least one first parameter and at least one second parameter, and the first target parameter corresponds to the first parameter and the second parameter one by one.

[0133] like Figure 4 As shown, the above step S302, determining at least one first target parameter less than a first threshold from at least one first processing parameter, includes the following steps S401 to S403.

[0134] S401, determining at least one second wavelet coefficient and at least one third wavelet coefficient from at least one first processing parameter.

[0135] Among them, respectively The 8*8 difference image blocks are subjected to wavelet decomposition to extract characteristic information for representing the smoothness of the difference image blocks, and the first wavelet coefficient, the second wavelet coefficient, the third wavelet coefficient and the fourth wavelet coefficient of each difference image block are obtained.

[0136] The wavelet coefficients can more delicately represent the pixel changes of the difference frame. The first wavelet coefficient refers to the wavelet coefficient generated by processing the difference image block in both the row and column directions using a high-pass wavelet filter, such as the wavelet coefficient generated by convolution using a high-pass wavelet filter. Based on the characteristics of wavelet decomposition, the first wavelet coefficient can be used to characterize the most intense texture information in the corresponding difference image block, and its value is relatively large. The fourth wavelet coefficient refers to the wavelet coefficient generated by processing the difference image block in both the row and column directions using a low-pass wavelet filter, such as the wavelet coefficient generated by convolution using a low-pass wavelet filter. The fourth wavelet coefficient can be used to characterize the smoothest texture information in the corresponding difference image block, and its value is relatively small. The second wavelet coefficient and the third wavelet coefficient can be used to characterize the more common and balanced texture information in the corresponding difference image block. In order to improve the accuracy of the smoothness of the determined difference image block, the second wavelet coefficient and the third wavelet coefficient that are representative and characterize the universality can be selected from the four types of wavelet coefficients to characterize the smoothness of the difference image block. At the same time, the second wavelet coefficient and the third wavelet coefficient are selected from the first processing parameters including four types of wavelet coefficients, and then the first target parameter only needs to be determined from the second wavelet coefficient and the third wavelet coefficient, which can reduce the amount of calculation and reduce the calculation complexity.

[0137] S402: Determine at least one first parameter that is smaller than a first threshold from at least one second wavelet coefficient.

[0138] S403: Determine at least one second parameter that is smaller than a first threshold from at least one third wavelet coefficient.

[0139] Furthermore, in order to more accurately obtain the noise level of the difference frame and more accurately determine the first spatial noise of the previous frame, at least one first parameter representing the target smoothness is determined from at least one second wavelet coefficient representing the smoothness of the difference image block of the difference frame; at least one second parameter representing the target smoothness is determined from at least one third wavelet coefficient representing the smoothness of the difference image block of the difference frame, and the target smoothness is the smoothness that matches the texture information caused by the noise of the difference frame. It can be understood that, in theory, at least one first parameter and at least one second parameter can be used to represent the noise level of the difference frame. Furthermore, the difference feature value of the difference frame can be determined from at least one first parameter and at least one second parameter.

[0140] Exemplarily, when the first parameter and the second parameter are determined from the second wavelet coefficient and the third wavelet coefficient, the first threshold may be set to 40 according to experimental feedback.

[0141] In this embodiment, when the first target processing is to calculate the wavelet coefficients, the second wavelet coefficients and the third wavelet coefficients that are representative and can be used to characterize the more common and more balanced texture information in the corresponding difference image block are selected from the first processing parameters containing four types of wavelet coefficients to characterize the smoothness of the corresponding difference image block, which can improve the accuracy of the smoothness of the determined difference image block. Furthermore, from the second wavelet coefficients and the third wavelet coefficients that characterize the smoothness of the corresponding difference image block, the first parameter and the second parameter that match the texture information caused by the noise of the difference frame are determined, which is conducive to more accurately determining the noise level of the difference frame, and then more accurately determining the first spatial noise of the previous frame. In addition, it is only necessary to determine the first parameter and the second parameter from the second wavelet coefficients and the third wavelet coefficients, which can reduce the amount of calculation and reduce the complexity of calculation.

[0142] In one embodiment, if Figure 5 As shown, the above step S303, determining the difference feature value of the difference frame from at least one first target parameter, includes the following steps S501 to S502.

[0143] S501, determining a first peak value and a first mean value of at least one first target parameter. The first peak value refers to the first target parameter with the highest occurrence frequency, and the first mean value refers to the average value of at least one first target parameter.

[0144] S502: Use the first peak value and the first mean value as difference feature values ​​of the difference frame.

[0145] As described in the above embodiments, in theory, at least one first target parameter can be used to characterize the noise level of the difference frame. Based on this, in order to avoid inaccurate results caused by possible data errors, at least one first target parameter can be subjected to histogram statistics, the first target parameter with the highest frequency of occurrence is determined as the first peak, and the average value of at least one first target parameter is determined as the first mean. Then, the representative first peak and first mean are used as the difference feature values ​​of the difference frame to characterize the noise level of the difference frame.

[0146] In addition, in actual situations, the noise distribution of video frames is relatively complex, and the noise level of each pixel of the video frame is not completely stable. Therefore, determining the first peak value and the first mean value to feedback the noise level of the difference frame through more quantity and more dimensional data information is conducive to more accurately determining the noise level of the difference frame.

[0147] Exemplarily, when the at least one first target parameter includes at least one first parameter and at least one second parameter, and the first target parameter corresponds one-to-one to the first parameter and the second parameter, the above step S303, determining the difference feature value of the difference frame from the at least one first target parameter, includes the following steps:

[0148] Determining a first peak value in at least one first parameter and a first mean value in at least one second parameter; or determining a first peak value in at least one second parameter and a first mean value in at least one first parameter;

[0149] The first peak value and the first mean value are taken as difference feature values ​​of the difference frame.

[0150] In this embodiment, the first peak value and the first mean value are determined from at least one first target parameter, and then the first peak value and the first mean value are used as difference characteristic values ​​characterizing the noise level of the difference frame, so as to avoid the inaccuracy of the determined noise level caused by the data error that may exist in at least one first target parameter. In addition, by introducing more quantity and more dimensional data information to feedback the noise level of the difference frame, it is also conducive to more accurately determining the noise level of the difference frame.

[0151] In one embodiment, if Figure 6 As shown, the noise estimation method includes the following steps S601 to S609. Among them, steps S601 to S606 and S609 correspond to steps S101 to S102, S201, S301 to S302, S501, and S104 in the aforementioned embodiment. The specific discussion of steps S601 to S606 and S609 in this embodiment can refer to the specific discussion of steps S101 to S102, S201, S301 to S302, S501, and S104 in the aforementioned embodiment, which will not be repeated in this embodiment.

[0152] S601, obtaining a previous frame in a video and a current frame in a video. The previous frame and the current frame are adjacent decoded video frames, and the decoding order of the previous frame is before the current frame.

[0153] S602, predicting the positions of the pixels of the previous frame based on the position offsets of the pixels of the current frame compared with the pixels of the previous frame, to obtain a compensated frame of the previous frame.

[0154] S603, determining a difference frame between the compensation frame and the previous frame. The brightness information of the difference pixel of the difference frame is the absolute value of the difference between the brightness information of the first pixel of the previous frame and the brightness information of the second pixel of the compensation frame, and the position of the difference pixel in the difference frame is the same as the position of the first pixel in the previous frame and the position of the second pixel in the compensation frame; the noise level of the difference frame is used to represent the first spatial domain noise of the previous frame.

[0155] S604, performing a first target processing on at least one difference image block of the difference frame to obtain at least one first processing parameter of the at least one difference image block. The difference image block corresponds to the first processing parameter one by one; the first processing parameter is used to indicate the smoothness of the difference image block.

[0156] S605: Determine at least one first target parameter that is less than a first threshold value from the at least one first processing parameter, wherein the smoothness of the difference image block corresponding to the first target parameter matches the noise of the difference frame.

[0157] S606, determining a first peak value and a first mean value of at least one first target parameter. The first peak value refers to the first target parameter with the highest occurrence frequency, and the first mean value refers to the average value of at least one first target parameter.

[0158] S607, perform eigenvalue decomposition on the covariance of the difference frame to obtain a target eigenvalue. The target eigenvalue is used to represent the noise level of the difference frame.

[0159] As described in the above embodiments, the noise distribution of video frames in actual situations is relatively complex. For example, the video obtained in the live video link system is a video after acquisition, compression, and transmission, and the noise of the video and video frame is not a single additive noise based on Gaussian distribution. Therefore, in addition to determining the first peak value and the first mean value from at least one first target parameter, in this embodiment, the noise level of the difference frame is fed back through more quantity and more dimensional data information to more accurately determine the noise level of the difference frame.

[0160] Since the brightness information of the pixels of the difference frame is obtained by subtracting the brightness information of the pixels of the previous frame and the brightness information of the pixels of the compensation frame, the brightness information of the pixels of the difference frame is generally small, the pixels are relatively smooth, and the difference between the pixels is mainly caused by the noise of the difference frame. Based on this, the covariance of the difference frame is calculated, and the eigenvalue decomposition of the calculation result is performed, and the target eigenvalue that can also represent the noise level of the difference frame can be obtained.

[0161] S608: Use the first peak value, the first mean value and the target characteristic value as the difference characteristic value of the difference frame. The difference characteristic value is used to represent the noise level of the difference frame.

[0162] Furthermore, by using the first peak value, the first mean value and the target characteristic value, which can all be used to represent the noise level of the difference frame, as the difference characteristic value of the difference frame, it is helpful to more accurately determine the noise level of the difference frame.

[0163] S609: Obtain a noise level of the video based on the first spatial noise.

[0164] In this embodiment, in addition to determining the first peak value and the first mean value from at least one first target parameter, the covariance of the difference frame is subjected to eigenvalue decomposition to determine a target eigenvalue that can also represent the noise level of the difference frame, so as to feedback the noise level of the difference frame through more quantity and more dimensional data information, which is conducive to more accurately determining the noise level of the difference frame. Furthermore, it is conducive to determining the first spatial noise of the previous frame and the noise level of the video.

[0165] In one embodiment, if Figure 7 As shown, the above step S607, performing eigenvalue decomposition on the covariance of the difference frame to obtain the target eigenvalue, includes the following steps S701 to S704.

[0166] S701, determining the covariance of the difference frame.

[0167] S702: Perform eigenvalue decomposition on the covariance to obtain at least one eigenvalue.

[0168] S703, sort at least one eigenvalue from small to large.

[0169] S704: In at least one sorted eigenvalue, if the absolute value of the difference between the adjacent first eigenvalue and the second eigenvalue is greater than a difference threshold, determine the first eigenvalue as a target eigenvalue.

[0170] Among them, the first eigenvalue is smaller than the second eigenvalue, and the first eigenvalue and the second eigenvalue are the smallest set of target values ​​among at least one eigenvalue after sorting, and the target value refers to the adjacent eigenvalues ​​whose absolute value of the difference is greater than the difference threshold.

[0171] Specifically, if the difference frame with a resolution of (H / 2, W / 2) is divided into 8*8 difference image blocks, the number of difference image blocks is Get Dimension Will Convert to Further converted to The matrix X.

[0172] Based on this, it can be determined that the covariance of the difference frame is the matrix X*X′. Then, perform eigenvalue decomposition on the matrix X*X′ to obtain at least one eigenvalue. Sort at least one eigenvalue from small to large. Traverse the at least one eigenvalue after sorting until the first particularly large eigenvalue appears and stop traversing. The first particularly large eigenvalue refers to the second eigenvalue when the absolute value of the difference between the adjacent first eigenvalue and the second eigenvalue is greater than the difference threshold. Among them, the second eigenvalue is greater than the first eigenvalue, and the first eigenvalue and the second eigenvalue are the adjacent eigenvalues ​​whose absolute value of the smallest set of differences is greater than the difference threshold in at least one eigenvalue after sorting. For example, at least one eigenvalue is 1, 2, 3, 4, 5, 20, 21..., and the difference threshold is 2. Among them, 5 and 20 are the adjacent eigenvalues ​​whose absolute value of the smallest set of differences is greater than the difference threshold in at least one eigenvalue after sorting; 5 is the first eigenvalue; 20 is the second eigenvalue, that is, the first particularly large eigenvalue. It should be noted that the difference threshold can be set according to the actual situation. For example, the difference threshold in the above example can also be 1, 3, 4, etc. Then, the eigenvalue before the first particularly large eigenvalue, that is, the first eigenvalue before the second eigenvalue, is used as the target eigenvalue. Based on the eigenvalue decomposition characteristics of the covariance, this target eigenvalue is an adaptive threshold that can be used to represent the noise level of the difference frame.

[0173] In this embodiment, by sorting at least one eigenvalue obtained by performing eigenvalue decomposition on the covariance from small to large, and traversing the sorted at least one eigenvalue, a target eigenvalue that can be used to represent the noise level of the difference frame can be determined.

[0174] In one embodiment, if Figure 8 As shown, step S104 in any of the aforementioned embodiments, obtaining the noise level of the video based on the first spatial noise, includes the following steps S801 to S803.

[0175] In addition to being based on the first spatial noise of the previous frame of the adjacent decoded image, the noise level of the video also needs to be based on the second spatial noise of the current frame of the adjacent decoded image. The second spatial noise is the spatial noise of the current frame. Therefore, more accurately determining the second spatial noise of the current frame is also conducive to more accurately determining the noise level of the video. Based on this, the second spatial noise of the current frame is first more accurately determined based on steps S801 to S802.

[0176] S801, determining an average frame of a compensation frame and a current frame. The brightness information of an average pixel of the average frame is a weighted sum of the brightness information of a third pixel of the compensation frame and the brightness information of a fourth pixel of the current frame, and the position of the average pixel in the average frame is the same as the position of the third pixel in the compensation frame and the position of the fourth pixel in the current frame.

[0177] The brightness information of the third pixel of the compensation frame is weighted and the brightness information of the fourth pixel of the current frame is weighted, and the brightness information of the mean pixel of the mean frame is obtained. Wherein, the weighted sum refers to the sum of the first multiple of the brightness information of the third pixel and the second multiple of the brightness information of the fourth pixel, and the sum of the first multiple and the second multiple is 1. Optionally, the first multiple and the second multiple are both 0.5. In this case, the brightness information of the mean pixel is the average of the brightness information of the third pixel and the brightness information of the fourth pixel. The third pixel is any pixel of the compensation frame, and the position of the mean pixel in the mean frame is the same as the position of the third pixel in the compensation frame and the position of the fourth pixel in the current frame. It can be understood that by using the above method of obtaining the mean pixel of the mean frame according to the third pixel of the compensation frame and the fourth pixel of the current frame, the respective pixels of the compensation frame and the respective pixels of the current frame are processed accordingly, and the respective pixels of the mean frame can be obtained, that is, the mean frame is determined.

[0178] Since the brightness information of the pixels of the mean frame is obtained by weighted summing the brightness information of the pixels of the compensation frame and the brightness information of the pixels of the current frame, the mean frame includes the weighted sum of the noise of the current frame and the noise of the compensation frame. The compensation frame is used to compensate the previous frame, and the compensation frame is similar to the previous frame and the current frame of the adjacent decoded frame. Therefore, in this embodiment, the compensation frame can be regarded as the current frame. On this basis, the mean frame includes the noise of the current frame. That is, the mean frame includes the second spatial domain noise of the current frame, and the noise level of the mean frame can be used to represent the second spatial domain noise of the current frame.

[0179] S802: extract the mean feature value of the mean frame. The mean feature value is used to represent the noise level of the mean frame, and the noise level of the mean frame is used to represent the second spatial domain noise of the current frame.

[0180] S803: Obtain a noise level of the video based on the first spatial domain noise and the second spatial domain noise.

[0181] In this embodiment, the brightness information of the pixels of the compensation frame and the brightness information of the pixels of the current frame are weighted and summed to obtain the pixels of the mean frame, and then the mean frame is obtained. Since the mean frame contains the weighted sum of the noise of the current frame and the noise of the compensation frame, and the compensation frame is similar to the current frame, it can be regarded as the mean frame containing the second spatial noise of the current frame. Furthermore, by extracting the mean characteristic value that can represent the noise level of the mean frame, the second spatial noise of the current frame can be accurately determined. Based on the accurate determination of the second spatial noise of the current frame and the first spatial noise of the previous frame, it is conducive to more accurately obtaining the noise level of the video. In addition, in this embodiment, the brightness information of the pixel point is used instead of the pixel value of the pixel point for calculation, and only the brightness information of the pixel point is processed accordingly to obtain the brightness information of the mean pixel point of the mean frame, which is conducive to reducing the computational complexity and improving the efficiency of estimating the second spatial noise of the current frame and the noise level of the video.

[0182] In one embodiment, the mean frame includes at least one mean image block. Fig. 9 As shown, the above step S802, extracting the mean feature value of the mean frame, includes the following steps S901 to S903.

[0183] S901, performing a second target processing on at least one mean image block of the mean frame to obtain at least one second processing parameter of the at least one mean image block. The mean image block corresponds to the second processing parameter one by one; the second processing parameter is used to indicate the smoothness of the mean image block.

[0184] Similar to extracting the difference feature value of the difference frame, in order to extract the mean feature value for representing the noise level of the mean frame, the smoothness of the mean image block may be determined first.

[0185] Specifically, if the mean frame with a resolution of (H / 2, W / 2) is divided into 8*8 mean image blocks, the number of mean image blocks is also indivual.

[0186] The second target processing is performed on each mean image block respectively, and the second target processing can be performed based on the brightness information of the pixels in the mean image block. The second target processing can be calculating the variance. Correspondingly, the second processing parameter is the variance of the mean image block.

[0187] S902: Determine at least one second target parameter that is less than a second threshold value from at least one second processing parameter, wherein the smoothness of the mean image block corresponding to the second target parameter matches the noise of the mean frame.

[0188] Since the brightness information of the pixels of the mean frame is obtained by weighted summing the brightness information of the pixels of the compensation frame and the brightness information of the pixels of the current frame, the mean frame contains not only the weighted sum of the noise of the current frame and the noise of the compensation frame, but also the weighted sum of the semantics of the current frame and the semantics of the compensation frame. That is, the smoothness of the mean image block of the mean frame may match the texture information caused by the noise of the mean frame, and may also match the texture information caused by the semantics of the video frame. In order to more accurately obtain the noise level of the mean frame and more accurately determine the second spatial domain noise of the current frame, it is necessary to determine the second target parameter representing the target smoothness from the second processing parameter used to represent the smoothness of the mean image block of the mean frame. This target smoothness is the smoothness that matches the texture information caused by the noise of the mean frame.

[0189] Specifically, the degree of smoothness that matches the texture information caused by noise is smaller, and the degree of smoothness that matches the texture information corresponding to the semantics is larger. Based on this, by setting the second threshold, the second processing parameter that is less than the second threshold can be determined as the second target parameter, and the degree of smoothness of the mean image block corresponding to the second target parameter matches the noise of the mean frame, and the second target parameter can be used to represent the noise level of the mean frame. The degree of smoothness of the mean image block corresponding to the second processing parameter that is greater than or equal to the second threshold matches the semantics. Optionally, when the second target processing is to calculate the variance and the second processing parameter is the variance of the mean image block, the second threshold can be taken as 10 according to experimental feedback.

[0190] S903: Determine a mean feature value of the mean frame from at least one second target parameter.

[0191] In theory, at least one second target parameter used to represent the smoothness of at least one mean image block can be used to represent the noise level of the mean frame. Therefore, the mean feature value of the mean frame can be determined from the at least one second target parameter.

[0192] In this embodiment, at least one second processing parameter for representing the smoothness of the mean image block of the mean frame is first obtained, and then at least one second target parameter representing the smoothness of the mean image block matches the noise of the mean frame is determined from the at least one second processing parameter, and the at least one second target parameter can be used to represent the noise level of the mean frame. Furthermore, the mean feature value for representing the noise level of the mean frame can be determined from the at least one second target parameter.

[0193] Exemplarily, the above step S903, determining the mean feature value of the mean frame from at least one second target parameter, includes the following steps:

[0194] Determine a second peak value and a second mean value of at least one second target parameter, wherein the second peak value refers to the second target parameter with the highest occurrence frequency, and the second mean value refers to the average value of at least one second target parameter;

[0195] The second peak value and the second mean value are taken as the mean feature value of the mean frame.

[0196] For the specific discussion of this example, please refer to the specific discussion on determining the first peak value and the first mean value, and using the first peak value and the first mean value as the difference characteristic values ​​of the difference frame in the aforementioned embodiment, which will not be repeated in this example. In this example, the second peak value and the second mean value are used as the mean characteristic values ​​characterizing the noise level of the mean frame, which can avoid the inaccuracy of the determined noise level caused by the data error that may exist in at least one second target parameter. In addition, by introducing more quantity and more dimensional data information to feedback the noise level of the mean frame, it is also beneficial to more accurately determine the noise level of the mean frame.

[0197] It should be noted that, unlike the first target processing of the difference image block of the difference frame, the second target processing of the mean image block of the mean frame usually does not include the calculation of wavelet coefficients. The reason is that the brightness information of the pixels of the mean frame is relatively large, and the texture information of the mean image block is relatively rich, and the color information is relatively rich. If the mean image block is subjected to wavelet decomposition, the value of the second processing parameter obtained, that is, the value of the wavelet coefficient is generally large. Based on this, the second target parameters that can be used to characterize the noise level of the mean frame screened out by the second threshold are relatively small. Taking into account the possible data errors and the randomness of the values, it is not conducive to improving the accuracy of the determined noise level based on less data.

[0198] In one embodiment, if Fig.10 As shown, step S803 in any of the aforementioned embodiments, obtaining the noise level of the video based on the first spatial noise and the second spatial noise, includes the following steps S1001 to S1002.

[0199] S1001, obtaining the temporal noise between the previous frame and the current frame based on the difference between the pixels at the same position between the previous frame and the current frame.

[0200] In addition to being based on the first spatial noise of the previous frame and the second spatial noise of the current frame, the noise level of the video also needs to be based on the temporal noise between the previous frame and the current frame. Therefore, accurately determining the temporal noise between the previous frame and the current frame is also conducive to more accurately determining the noise level of the video. Based on this, the temporal noise between the previous frame and the current frame can be accurately obtained by the difference in the pixel points at the same position between the previous frame and the current frame of the video.

[0201] S1002: Obtain a noise level of the video based on the first spatial domain noise, the second spatial domain noise, and the time domain noise.

[0202] In this embodiment, based on accurately determining the first spatial noise of the previous frame of the video, the second spatial noise of the current frame, and the temporal noise between the previous frame and the current frame, it is helpful to more accurately obtain the noise level of the video.

[0203] In one embodiment, if Fig.11 As shown, the above step S1001, based on the difference between the pixel points at the same position of the previous frame and the current frame, obtains the time domain noise between the previous frame and the current frame, including the following steps S1101 to S1102.

[0204] S1101, determine a mixed frame of a previous frame and a current frame. The mixed image block of the mixed frame is formed by splicing the first image block of the previous frame and the second image block of the current frame, the mixed image block includes pixel points of the mixed frame, the first image block includes pixel points of the previous frame, the second image block includes pixel points of the current frame, and the position of the first image block in the previous frame is the same as the position of the second image block in the current frame; the noise level of the mixed frame is used to represent the temporal noise between the previous frame and the current frame.

[0205] The image blocks at the same position of the previous frame and the current frame, that is, the first image block and the second image block, are spliced ​​to obtain a mixed image block of the mixed frame. Specifically, if the first image block and the second image block are both 8*8 image blocks, both have 64 pixels, then the first image block and the second image block are respectively represented by vectors containing 64 data. Furthermore, the two vectors containing 64 data are spliced ​​into a one-dimensional vector containing 128 data, and this vector containing 128 data represents the mixed image block. The first image block is any image block of the previous frame. By splicing all the image blocks at the same position of the previous frame and the current frame in the above manner, all the image blocks of the mixed frame can be obtained, that is, the mixed frame.

[0206] Since the position of the first image block in the previous frame is the same as the position of the second image block in the current frame, the first image block includes the pixels of the previous frame, the second image block includes the pixels of the current frame, and the mixed image block includes the pixels of the mixed frame. Therefore, the mixed image block formed by splicing the first image block and the second image block includes the pixels of the same position of the previous frame and the current frame, and the pixels of the mixed frame are composed of the pixels of the same position of the previous frame and the current frame. Still taking the 8*8 image block as an example, the 64 pixels of the first image block in the previous frame to the 64 pixels of the second image block in the current frame are changed, and the mixed image block containing the 64 pixels of the previous frame and the 64 pixels of the current frame contains the change information of the 64 pixels of the first image block to the 64 pixels of the second image block. It can be seen that the mixed frame contains the change information of the pixels of the same position of the previous frame and the current frame, that is, it contains the difference in the brightness information of the pixels of the same position of the previous frame and the current frame.

[0207] The difference in brightness information of the pixel points at the same position between the previous frame and the current frame may be caused by the temporal noise between the previous frame and the current frame, or may be caused by the movement of the object in the video frame. The information change caused by the temporal noise can be regarded as the information change caused by the noise level of the mixed frame for the mixed frame. Based on this, when the mixed frame contains the difference in brightness information of the pixel points at the same position between the previous frame and the current frame, the temporal noise between the previous frame and the current frame can be represented by the noise level of the mixed frame.

[0208] It should be noted that in this embodiment, a mixed image block of a mixed frame is obtained by splicing the first image block of the previous frame with the second image block of the current frame, thereby obtaining a mixed frame. The previous frame and the current frame are not directly spliced ​​to obtain a mixed frame. This can avoid two pixels at the same position being too far apart, which is conducive to more convenient comparison of brightness information of pixels at the same position. Optionally, the position of the mixed image block obtained by splicing the first image block and the second image block in the mixed frame is the same as the position of the first image block in the previous frame and the position of the second image block in the current frame.

[0209] S1102: extracting mixed feature values ​​of the mixed frame. The mixed feature values ​​are used to represent the noise level of the mixed frame.

[0210] Furthermore, by extracting the mixed feature value used to represent the noise level of the mixed frame, the time domain noise between the previous frame and the current frame can be obtained.

[0211] The traditional method of estimating the noise level of a video is to use a temporal filter to filter two adjacent video frames, calculate the gradient information of the two video frames using methods such as Spatio Temporal Gradients, and then obtain the covariance of the gradient information, and perform eigenvalue decomposition on the covariance to obtain the temporal noise. When filtering based on the temporal filter and calculating the gradient information of the video in the temporal domain, in scenes with large motion amplitudes, the pixel points of the two previous and subsequent video frames differ greatly, resulting in large pixel gradient information and large gradient information errors. When calculating the covariance based on the gradient information, the covariance will also be relatively large, and the covariance error will also be relatively large. Furthermore, the error of the temporal noise obtained based on the covariance will also be relatively large. In a video link environment, the actual noise level of the video may be very high, but the temporal noise and the fractional values ​​of the video noise level calculated based on this method may be very small.

[0212] In this embodiment, the difference between the pixels at the same position of the previous frame and the current frame is converted into a mixed frame. Since the mixed frame contains the difference in brightness information of the pixels at the same position of the previous frame and the current frame, and this difference may be the information change caused by the temporal noise between the previous frame and the current frame, and may also be the information change caused by the movement of the object in the video frame. Therefore, the temporal noise between the previous frame and the current frame can be obtained by determining the noise level of the mixed frame. Furthermore, by extracting the mixed characteristic value used to represent the noise level of the mixed frame, the temporal noise between the previous frame and the current frame can be accurately obtained. Compared with the traditional method for estimating the noise level of a video, the noise estimation method in this embodiment does not need to calculate the gradient information of two adjacent decoded video frames of the video, and can obtain more accurate temporal noise, which is conducive to more accurate estimation of the noise level of the video.

[0213] In one embodiment, the mixed frame includes at least one mixed image block. Fig.12 As shown, the above step S1102, extracting the mixed feature value of the mixed frame, includes the following steps S1201 to S1203.

[0214] S1201, performing a third target processing on at least one mixed image block of the mixed frame to obtain at least one third processing parameter of the at least one mixed image block. The mixed image block corresponds to the third processing parameter one by one; the third processing parameter is used to indicate the smoothness of the mixed image block.

[0215] Similar to extracting the difference feature value of the difference frame and extracting the mixed feature value of the mixed frame, in order to extract the mixed feature value for representing the noise level of the mixed frame, the smoothness of the mixed image block may be determined first.

[0216] Specifically, the size of the image blocks in the previous frame and the current frame are both 8*8 and the number is In the case of mixed frame, the size of mixed image block is 8*8*2, and the number is also

[0217] The third target processing is performed on each mixed image block respectively, and the third target processing can be performed based on the brightness information of the pixels in the mixed image block. The third target processing can be calculating the variance. Correspondingly, the third processing parameter is the variance of the mixed image block.

[0218] S1202: Determine at least one third target parameter that is less than a third threshold value from at least one third processing parameter, wherein the smoothness of the mixed image block corresponding to the third target parameter matches the noise of the mixed frame.

[0219] Since the mixed frame contains the difference in brightness information of the pixel points at the same position of the previous frame and the current frame, this difference may be the information change caused by the temporal noise between the previous frame and the current frame, and may also be the information change caused by the movement of the object in the video frame. Therefore, the smoothness of the mixed image block of the mixed frame may match the texture information caused by the noise of the mixed frame, and may also match the texture information caused by the movement of the object. In order to more accurately obtain the noise level of the mixed frame and more accurately determine the temporal noise between the previous frame and the current frame, it is necessary to determine a third target parameter representing the target smoothness from the third processing parameter used to represent the smoothness of the mixed image block of the mixed frame. This target smoothness is the smoothness that matches the texture information caused by the noise of the mixed frame.

[0220] Specifically, the degree of smoothness that matches the texture information caused by noise is smaller, and the degree of smoothness that matches the texture information caused by motion is larger. Based on this, the third threshold is set, and the third processing parameter that is less than the third threshold can be determined as the third target parameter. The degree of smoothness of the mixed image block corresponding to the third target parameter matches the noise of the mixed frame, and the third target parameter can be used to represent the noise level of the mixed frame. The degree of smoothness of the mixed image block corresponding to the third processing parameter that is greater than or equal to the third threshold matches the motion of the object. Optionally, when the third target processing is to calculate the variance and the third processing parameter is the variance of the mixed image block, the third threshold can be taken as 10 according to experimental feedback.

[0221] S1203: Determine a mixed feature value of the mixed frame from at least one third target parameter.

[0222] In theory, at least one third target parameter used to represent the smoothness of at least one mixed image block can be used to represent the noise level of the mixed frame. Therefore, the mixed characteristic value of the mixed frame can be determined from the at least one third target parameter.

[0223] In this embodiment, at least one third processing parameter for representing the smoothness of the mixed image block of the mixed frame is first obtained, and then at least one third target parameter representing the smoothness of the mixed image block and the noise of the mixed frame is determined from the at least one third processing parameter, and the at least one third target parameter can be used to represent the noise level of the mixed frame. Furthermore, a mixed feature value for representing the noise level of the mixed frame can be determined from the at least one third target parameter.

[0224] Exemplarily, the above step S1203, determining the mixed feature value of the mixed frame from at least one third target parameter, includes the following steps:

[0225] Determine a third peak value and a third mean value of at least one third target parameter, wherein the third peak value refers to the third target parameter with the highest occurrence frequency, and the third mean value refers to the average value of at least one third target parameter;

[0226] The third peak value and the third mean value are taken as mixed feature values ​​of the mixed frame.

[0227] For the specific discussion of this example, please refer to the specific discussion on determining the first peak value and the first mean value, and using the first peak value and the first mean value as the difference characteristic values ​​of the difference frame in the aforementioned embodiment, which will not be repeated in this example. In this example, the third peak value and the third mean value are used as mixed characteristic values ​​that characterize the noise level of the mixed frame, which can avoid the inaccuracy of the determined noise level caused by the data error that may exist in at least one third target parameter. In addition, by introducing more quantity and more dimensional data information to feedback the noise level of the mixed frame, it is also beneficial to more accurately determine the noise level of the mixed frame.

[0228] In one embodiment, step S1002 in any of the aforementioned embodiments, obtaining the noise level of the video based on the first spatial noise, the second spatial noise and the time domain noise, includes the following steps: linearly fitting the first spatial noise, the second spatial noise and the time domain noise to obtain the noise level of the video.

[0229] Specifically, the score representing the noise level of the video can be calculated by the following formula:

[0230] score=x0+x1*b+x2*c+x3*d

[0231] Among them, x {0,1,2,2} is the coefficient for fitting on the business data set, which can be adjusted according to the actual situation. b is the parameter value used to represent the first spatial domain noise, c is the parameter value used to represent the second spatial domain noise, and d is the parameter value used to represent the time domain noise. b, c and d can be determined by the corresponding methods in the above embodiments.

[0232] Exemplarily, the score representing the noise level of the video can be calculated by the following formula:

[0233] score=y0+y1*a+y2*b1+y3*b2+y4*c1+y5*c2+y6*d1+y7*d2

[0234] Among them, y {0,1,2,3,4,5,6,7} is the coefficient for fitting on the business data set, which can be adjusted according to the actual situation. a is the target characteristic value described in the aforementioned embodiment, b1 and b2 are respectively the first peak value and the first mean value described in the aforementioned embodiment, a, b1 and b2 are all difference characteristic values ​​that can be used to represent the noise level of the difference frame, that is, parameter values ​​that can be used to represent the first spatial domain noise. c1 and c2 are respectively the second peak value and the second mean value described in the aforementioned embodiment, c1 and c2 are both mean characteristic values ​​that can be used to represent the noise level of the mean frame, that is, parameter values ​​that can be used to represent the second spatial domain noise. d1 and d2 are respectively the third peak value and the third mean value described in the aforementioned embodiment, d1 and d2 are both mixed characteristic values ​​that can be used to represent the noise level of the mixed frame, that is, parameter values ​​that can be used to represent the time domain noise.

[0235] In this embodiment, based on the accurate first spatial noise of the previous frame, the second spatial noise of the current frame and the time noise between the previous frame and the current frame, by linearly fitting the three types of parameter values ​​of the first spatial noise, the second spatial noise and the time noise, and adjusting the fitting coefficient of each parameter value according to the actual situation, the noise level of the video can be estimated more accurately.

[0236] Fig.13 Schematic diagram of the flow of the noise estimation method provided in the embodiment of the present application. Fig.13As shown, the execution subject of the server or terminal executes the noise estimation method, and after acquiring the video stream, performs video decoding to obtain the adjacent decoded previous frame A and current frame B. The optical flow calculation method is used to calculate the optical flow information between the previous frame A and the current frame B, that is, the horizontal and vertical position offsets of any two pixels with the same semantics in the two adjacent frames are obtained. Then, the position of the pixel of the previous frame A is predicted based on the pixel of the current frame B and the optical flow information, and the motion-compensated compensated frame C is obtained based on the predicted positions of each pixel. The pixel of the previous frame A and the pixel of the compensated frame C are difference-calculated to obtain a difference frame D, wherein the difference calculation refers to finding the absolute value of the difference in brightness information of two pixels at the same position. The noise level of the difference frame D can be used to represent the first spatial domain noise of the previous frame A. The pixel of the compensated frame C and the pixel of the current frame B are average-calculated to obtain a mean frame M, wherein the average calculation refers to finding the weighted sum of the brightness information of two pixels at the same position. Optionally, the mean value calculation refers to finding the average value of the brightness information of two pixels at the same position. The noise level of the mean frame M can be used to represent the second spatial domain noise of the current frame B.

[0237] By finding the difference eigenvalue of the difference frame D that can be used to represent the noise level of the difference frame D, the noise level of the difference frame D can be accurately determined, and then the first spatial noise of the previous frame A can be accurately determined. Specifically, the covariance-eigenvalue calculation is performed on the difference frame D, that is, the covariance of the difference frame D is determined, and the covariance is decomposed by eigenvalue to obtain the target eigenvalue a; the wavelet calculation is performed on the difference frame D, that is, the wavelet decomposition is performed on each difference image block of the difference frame D respectively, and the first wavelet coefficient, the second wavelet coefficient, the third wavelet coefficient and the fourth wavelet coefficient corresponding to each difference image block are obtained. Then, the second wavelet coefficient and the third wavelet coefficient are selected from the four types of wavelet coefficients corresponding to each difference image block, and the mean peak statistics are performed on the second wavelet coefficient and the third wavelet coefficient corresponding to each difference image block to determine the first peak b1 and the first mean b2. The target characteristic value a, the first peak value b1 and the first mean value b2 are respectively the difference characteristic values. The first spatial noise of the previous frame A can be accurately determined by these three parameter values.

[0238] By calculating the mean characteristic value of the mean frame M that can be used to represent the noise level of the mean frame M, the noise level of the mean frame M can be accurately determined, and then the second spatial noise of the current frame B can be accurately determined. Specifically, the variance of the mean frame M is calculated, that is, the variance of each mean image block of the mean frame M is calculated respectively to obtain the mean variance of each mean image block. Then, the mean peak statistics are performed on the mean variance corresponding to each mean image block to determine the second peak c1 and the second mean c2. The second peak c1 and the second mean c2 are respectively the mean characteristic values, and the second spatial noise of the current frame B can be accurately determined by these two parameter values.

[0239] In addition, the previous frame A and the current frame B decoded adjacently are mixed and spliced ​​to obtain a mixed frame AB. Specifically, the image blocks at the same position of the previous frame A and the current frame B can be spliced ​​to obtain the image blocks of the mixed frame AB, and then the mixed frame AB is obtained. The mixed frame AB contains the difference in the pixel points at the same position of the previous frame A and the current frame B decoded adjacently in the video. This difference may be the information change caused by the time domain noise between the previous frame A and the current frame B, or it may be the information change caused by the object movement in the video frame. The information change caused by the time domain noise can be regarded as the information change caused by the noise level of the mixed frame AB. Based on this, the time domain noise between the previous frame A and the current frame B can be represented by the noise level of the mixed frame AB. Furthermore, by calculating the mixed characteristic value of the mixed frame AB that can be used to represent the noise level of the mixed frame AB, the noise level of the mixed frame AB can be accurately determined, and then the time domain noise between the previous frame A and the current frame B can be accurately determined. Specifically, the variance of the mixed frame AB is calculated, that is, the variance of each mixed image block of the mixed frame AB is calculated respectively to obtain the mixed variance of each mixed image block. Then, the mean peak value statistics are performed on the mixed variance corresponding to each mixed image block to determine the third peak value d1 and the third mean value d2. The third peak value d1 and the third mean value d2 are respectively the mixed characteristic values, and the time domain noise between the previous frame A and the current frame B can be accurately determined by these two parameter values.

[0240] Furthermore, by performing linear fitting on the target characteristic value a, the first peak value b1 and the first mean value b2 of the first spatial noise of the previous frame A, which can accurately determine the second peak value c1 and the second mean value c2 of the second spatial noise of the current frame B, and the third peak value d1 and the third mean value d2 of the temporal noise between the previous frame A and the current frame B, the noise level of the video can be accurately estimated and a score representing the noise level can be output.

[0241] about Fig.13 For the detailed discussion of the implementation logic of the noise estimation method shown, please refer to the detailed discussion of the corresponding content in the above embodiments, which will not be repeated here. Fig.13The technical effects that can be achieved by the noise estimation method shown can also be referred to the description of the technical effects that can be achieved by the corresponding contents in the aforementioned embodiments, which will not be repeated here.

[0242] The embodiment of the present application also provides a noise estimation device 1400. Fig.14 As shown, the noise estimation device 1400 includes a video frame acquisition module 1401, a video frame prediction module 1402, a spatial noise determination module 1403 and a video noise estimation module 1404. The video frame acquisition module 1401 is used to acquire the previous frame in the video and the current frame in the video, the previous frame and the current frame are adjacent decoded video frames, and the decoding order of the previous frame is before the current frame. The video frame prediction module 1402 is used to predict the position of the pixel points of the previous frame based on the position offset of the pixel points of the current frame compared with the pixel points of the previous frame, and obtain the compensation frame of the previous frame. The spatial noise determination module 1403 is used to obtain the first spatial noise of the previous frame based on the difference between the pixel points of the compensation frame and the pixel points of the previous frame. The video noise estimation module 1404 obtains the noise level of the video based on the first spatial noise.

[0243] In one embodiment, the spatial noise determination module 1403 includes a difference frame determination unit and a difference feature value extraction unit. The difference frame determination unit is used to determine the difference frame between the compensation frame and the previous frame, the brightness information of the difference pixel of the difference frame is the absolute value of the difference between the brightness information of the first pixel of the previous frame and the brightness information of the second pixel of the compensation frame, the position of the difference pixel in the difference frame is the same as the position of the first pixel in the previous frame and the position of the second pixel in the compensation frame; the noise level of the difference frame is used to represent the first spatial noise of the previous frame. The difference feature value extraction unit is used to extract the difference feature value of the difference frame, and the difference feature value is used to represent the noise level of the difference frame.

[0244] In one embodiment, the difference frame includes at least one difference image block. The difference feature value extraction unit is further used to perform a first target processing on at least one difference image block of the difference frame to obtain at least one first processing parameter of the at least one difference image block, and the difference image block corresponds to the first processing parameter one by one; the first processing parameter is used to indicate the smoothness of the difference image block; from the at least one first processing parameter, at least one first target parameter less than a first threshold is determined; the smoothness of the difference image block corresponding to the first target parameter matches the noise of the difference frame; and the difference feature value of the difference frame is determined from the at least one first target parameter.

[0245] In one of the embodiments, when the first target processing is to calculate the wavelet coefficient, the first processing parameter includes the first wavelet coefficient, the second wavelet coefficient, the third wavelet coefficient and the fourth wavelet coefficient obtained after wavelet decomposition of the difference image block; the first wavelet coefficient refers to the wavelet coefficient generated by processing the difference image block in both the row direction and the column direction using a high-pass wavelet filter, the second wavelet coefficient refers to the wavelet coefficient generated by processing the difference image block in the row direction using a low-pass wavelet filter and then processing the difference image block in the column direction using a high-pass wavelet filter, the third wavelet coefficient refers to the wavelet coefficient generated by processing the difference image block in the row direction using a high-pass wavelet filter and then processing the difference image block in the column direction using a low-pass wavelet filter, and the fourth wavelet coefficient refers to the wavelet coefficient generated by processing the difference image block in both the row direction and the column direction using a low-pass wavelet filter; at least one first target parameter includes at least one first parameter and at least one second parameter, and the first target parameter corresponds one-to-one to the first parameter and the second parameter. The above-mentioned difference feature value extraction unit is also used to determine at least one second wavelet coefficient and at least one third wavelet coefficient from at least one first processing parameter; determine at least one first parameter less than a first threshold from at least one second wavelet coefficient; and determine at least one second parameter less than the first threshold from at least one third wavelet coefficient.

[0246] In one embodiment, the above-mentioned difference feature value extraction unit is also used to determine a first peak value and a first mean value in at least one first target parameter, where the first peak value refers to the first target parameter with the highest frequency of occurrence, and the first mean value refers to the average value of at least one first target parameter; the first peak value and the first mean value are used as difference feature values ​​of the difference frame.

[0247] In one of the embodiments, the above-mentioned difference eigenvalue extraction unit is also used to perform eigenvalue decomposition on the covariance of the difference frame to obtain a target eigenvalue, and the target eigenvalue is used to represent the noise level of the difference frame; the first peak value, the first mean value and the target eigenvalue are used as the difference eigenvalues ​​of the difference frame.

[0248] In one of the embodiments, the above-mentioned difference eigenvalue extraction unit is also used to determine the covariance of the difference frame; perform eigenvalue decomposition on the covariance to obtain at least one eigenvalue; sort the at least one eigenvalue from small to large; in the at least one eigenvalue after sorting, if the absolute value of the difference between the adjacent first eigenvalue and the second eigenvalue is greater than the difference threshold, then determine the first eigenvalue as the target eigenvalue; wherein the first eigenvalue is smaller than the second eigenvalue, and the first eigenvalue and the second eigenvalue are the smallest set of target values ​​in the at least one eigenvalue after sorting, and the target value refers to the adjacent eigenvalue whose absolute value of the difference is greater than the difference threshold.

[0249] In one embodiment, the video noise estimation module 1404 includes a mean frame determination unit, a mean feature value extraction unit and a video noise estimation unit. The mean frame determination unit is used to determine the mean frame of the compensation frame and the current frame, the brightness information of the mean pixel of the mean frame is the weighted sum of the brightness information of the third pixel of the compensation frame and the brightness information of the fourth pixel of the current frame, and the position of the mean pixel in the mean frame is the same as the position of the third pixel in the compensation frame and the position of the fourth pixel in the current frame. The mean feature value extraction unit is used to extract the mean feature value of the mean frame, the mean feature value is used to represent the noise level of the mean frame, and the noise level of the mean frame is used to represent the second spatial domain noise of the current frame. The video noise estimation unit is used to obtain the noise level of the video based on the first spatial domain noise and the second spatial domain noise.

[0250] In one embodiment, the mean frame includes at least one mean image block. The mean feature value extraction unit is further used to perform second target processing on at least one mean image block of the mean frame to obtain at least one second processing parameter of the at least one mean image block, and the mean image block corresponds to the second processing parameter one by one; the second processing parameter is used to indicate the smoothness of the mean image block; from the at least one second processing parameter, at least one second target parameter less than a second threshold is determined; the smoothness of the mean image block corresponding to the second target parameter matches the noise of the mean frame; and the mean feature value of the mean frame is determined from the at least one second target parameter.

[0251] In one of the embodiments, the video noise estimation unit is further used to obtain the temporal noise between the previous frame and the current frame based on the difference between the pixels at the same position between the previous frame and the current frame; and to obtain the noise level of the video based on the first spatial noise, the second spatial noise and the temporal noise.

[0252] In one of the embodiments, the above-mentioned video noise estimation unit is also used to determine a mixed frame of a previous frame and a current frame, the mixed image block of the mixed frame is spliced ​​by the first image block of the previous frame and the second image block of the current frame, the mixed image block includes pixel points of the mixed frame, the first image block includes pixel points of the previous frame, the second image block includes pixel points of the current frame, the position of the first image block in the previous frame is the same as the position of the second image block in the current frame; the noise level of the mixed frame is used to represent the time domain noise between the previous frame and the current frame; the mixed feature value of the mixed frame is extracted, and the mixed feature value is used to represent the noise level of the mixed frame.

[0253] In one embodiment, the mixed frame includes at least one mixed image block. The above-mentioned video noise estimation unit is also used to perform third target processing on at least one mixed image block of the mixed frame to obtain at least one third processing parameter of the at least one mixed image block, and the mixed image block corresponds to the third processing parameter one by one; the third processing parameter is used to indicate the smoothness of the mixed image block; from at least one third processing parameter, at least one third target parameter less than a third threshold is determined; the smoothness of the mixed image block corresponding to the third target parameter matches the noise of the mixed frame; and the mixed characteristic value of the mixed frame is determined from at least one third target parameter.

[0254] In one of the embodiments, the video noise estimation unit is further used to perform linear fitting on the first spatial noise, the second spatial noise and the time noise to obtain the noise level of the video.

[0255] In one embodiment, the video frame prediction module 1402 is further used to determine the position offset of the fifth pixel of the current frame compared to the sixth pixel of the previous frame, and the fifth pixel and the sixth pixel have the same semantics; based on the fifth pixel of the current frame, the sixth pixel of the previous frame is predicted based on the position offset to obtain the seventh pixel; based on the seventh pixel, a compensation frame of the previous frame is obtained.

[0256] For explanations of various terms, please refer to the relevant descriptions in the aforementioned noise estimation method embodiment, which will not be elaborated here.

[0257] It should be noted that the specific execution process of the noise estimation device 1400 can be found in Figures 1 to 13 The specific description of the illustrated embodiment will not be repeated here.

[0258] Each module in the noise estimation device 1400 can be implemented in whole or in part by software, hardware or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute operations corresponding to each module.

[0259] like Fig.15 As shown, the embodiment of the present application further provides a computer device 1500. Exemplarily, the computer device 1500 may include a processor 1501, a communication interface 1502, a communication bus 1503, and a memory 1504. Specifically, the computer device 1500 may include:

[0260] At least one processor 1501, such as a CPU, at least one communication interface 1502, a memory 1504, and at least one communication bus 1503. The communication bus 1503 is used to realize the connection and communication between these components. The communication interface 1502 may optionally include a standard wired interface, a wireless interface (such as a WI-FI interface or a Bluetooth interface, etc.). The memory 1504 may be a high-speed RAM memory, or a non-volatile memory (non-volatile memory), such as at least one disk storage. The memory 1504 may optionally be at least one storage device located away from the aforementioned processor 1501. Fig.15 As shown, the memory 1504 as a computer storage medium may include an operating system and program instructions.

[0261] Exemplarily, the processor 1501 may be used to implement the above Fig.14 The steps or methods performed by the video frame acquisition module 1401, the video frame prediction module 1402, the spatial noise determination module 1403 and the video noise estimation module 1404 in the embodiment.

[0262] It is understandable that the above method is only an example, and the processor 1501 and other modules in the above computer device 1500 may also cooperate to perform the above method. Fig.14 The steps or methods performed by the video frame acquisition module 1401, the video frame prediction module 1402, the spatial noise determination module 1403 and the video noise estimation module 1404 are not limited in this document.

[0263] exist Fig.15 In the computer device 1500 shown, the processor 1501 may be used to load program instructions stored in the memory 1504 and specifically perform the following operations:

[0264] Get the previous frame in the video and the current frame in the video, where the previous frame and the current frame are adjacent decoded video frames, and the decoding order of the previous frame is before the current frame;

[0265] Based on the position offset of the pixel points of the current frame compared with the pixel points of the previous frame, the position of the pixel points of the previous frame is predicted to obtain a compensation frame of the previous frame;

[0266] Based on the difference between the pixel points of the compensation frame and the pixel points of the previous frame, the first spatial noise of the previous frame is obtained;

[0267] Based on the first spatial noise, a noise level of the video is obtained.

[0268] For explanations of various terms, please refer to the relevant descriptions in the aforementioned noise estimation method embodiment, which will not be elaborated here.

[0269] It should be noted that the specific implementation process can be found in Figures 1 to 13 The specific description of the illustrated embodiment will not be repeated here.

[0270] The present application also provides a computer-readable storage medium, which can store a plurality of instructions, and the instructions are suitable for being loaded and executed by a processor as described above. Figures 1 to 13 The method steps of the embodiment shown in the figure can be found in the specific execution process. Figures 1 to 13 The specific description of the illustrated embodiment will not be repeated here.

[0271] As used in the above embodiments, the term "when..." may be interpreted to mean "if..." or "after..." or "in response to determining..." or "in response to detecting...", depending on the context. Similarly, the phrases "upon determining..." or "if (the stated condition or event) is detected" may be interpreted to mean "if determining..." or "in response to determining..." or "upon detecting (the stated condition or event)" or "in response to detecting (the stated condition or event)", depending on the context.

[0272] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any fusion thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function according to the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integration. The available medium can be a magnetic medium, (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state hard disk), etc.

[0273] Those skilled in the art can understand that to implement all or part of the processes in the above-mentioned embodiments, the processes can be completed by computer programs to instruct related hardware, and the programs can be stored in computer-readable storage media. When the programs are executed, they can include the processes of the above-mentioned method embodiments. The aforementioned storage media include: ROM or random access memory RAM, magnetic disk or optical disk and other media that can store program codes.

[0274] The technical features of the above embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0275] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A noise estimation method, characterized in that: The method comprises: Obtaining a previous frame in a video and a current frame in the video, wherein the previous frame and the current frame are adjacent decoded video frames, and a decoding order of the previous frame is before that of the current frame; Predicting the positions of the pixels of the previous frame based on the position offsets of the pixels of the current frame compared to the pixels of the previous frame to obtain a compensated frame of the previous frame; Based on the difference between the pixel points of the compensation frame and the pixel points of the previous frame, obtaining the first spatial noise of the previous frame; A noise level of the video is obtained based on the first spatial noise.

2. The method according to claim 1, characterized in that: The obtaining the first spatial noise of the previous frame based on the difference between the pixel points of the compensation frame and the pixel points of the previous frame comprises: Determine the difference frame between the compensation frame and the previous frame, the brightness information of the difference pixel of the difference frame is the absolute value of the difference between the brightness information of the first pixel of the previous frame and the brightness information of the second pixel of the compensation frame, the position of the difference pixel in the difference frame is the same as the position of the first pixel in the previous frame and the position of the second pixel in the compensation frame; the noise level of the difference frame is used to represent the first spatial domain noise of the previous frame; A difference feature value of the difference frame is extracted, where the difference feature value is used to represent a noise level of the difference frame.

3. The method according to claim 2, characterized in that The difference frame includes at least one difference image block; and extracting the difference feature value of the difference frame includes: Performing a first target processing on the at least one difference image block of the difference frame to obtain at least one first processing parameter of the at least one difference image block, wherein the difference image block corresponds to the first processing parameter in a one-to-one manner; the first processing parameter is used to indicate a smoothness degree of the difference image block; Determining at least one first target parameter less than a first threshold value from the at least one first processing parameter; the smoothness of the difference image block corresponding to the first target parameter matches the noise of the difference frame; The difference feature value of the difference frame is determined from the at least one first target parameter.

4. The method according to claim 3, characterized in that In the case where the first target processing is to calculate the wavelet coefficients, the first processing parameters include the first wavelet coefficients, the second wavelet coefficients, the third wavelet coefficients and the fourth wavelet coefficients obtained after wavelet decomposition of the difference image block; the first wavelet coefficients refer to the wavelet coefficients generated by processing the difference image block in both the row direction and the column direction using a high-pass wavelet filter, the second wavelet coefficients refer to the wavelet coefficients generated by processing the difference image block in the row direction using a low-pass wavelet filter and then in the column direction using a high-pass wavelet filter, the third wavelet coefficients refer to the wavelet coefficients generated by processing the difference image block in the row direction using a high-pass wavelet filter and then in the column direction using a low-pass wavelet filter, and the fourth wavelet coefficients refer to the wavelet coefficients generated by processing the difference image block in both the row direction and the column direction using a low-pass wavelet filter; the at least one first target parameter includes at least one first parameter and at least one second parameter, and the first target parameter corresponds one-to-one to the first parameter and the second parameter; Determining at least one first target parameter less than a first threshold value from the at least one first processing parameter comprises: determining at least one second wavelet coefficient and at least one third wavelet coefficient from the at least one first processing parameter; determining, from the at least one second wavelet coefficient, the at least one first parameter that is less than the first threshold; From the at least one third wavelet coefficient, the at least one second parameter is determined which is smaller than the first threshold value.

5. The method according to claim 3, characterized in that: The determining the difference feature value of the difference frame from the at least one first target parameter comprises: Determine a first peak value and a first mean value of the at least one first target parameter, wherein the first peak value refers to the first target parameter with the highest occurrence frequency, and the first mean value refers to the average value of the at least one first target parameter; The first peak value and the first mean value are used as the difference feature values ​​of the difference frame.

6. The method according to claim 5, characterized in that Before taking the first peak value and the first mean value as the difference feature value of the difference frame, the method further includes: Performing eigenvalue decomposition on the covariance of the difference frame to obtain a target eigenvalue, wherein the target eigenvalue is used to represent the noise level of the difference frame; The step of taking the first peak value and the first mean value as the difference feature value of the difference frame comprises: The first peak value, the first mean value and the target feature value are used as the difference feature value of the difference frame.

7. The method according to claim 6, characterized in that The step of performing eigenvalue decomposition on the covariance of the difference frame to obtain a target eigenvalue includes: determining the covariance of the difference frame; Performing eigenvalue decomposition on the covariance to obtain at least one eigenvalue; sorting the at least one eigenvalue from small to large; In the at least one sorted eigenvalue, if the absolute value of the difference between the adjacent first eigenvalue and the second eigenvalue is greater than the difference threshold, determining the first eigenvalue as the target eigenvalue; Among them, the first eigenvalue is smaller than the second eigenvalue, and the first eigenvalue and the second eigenvalue are the smallest set of target values ​​among the at least one eigenvalue after sorting, and the target value refers to the adjacent eigenvalues ​​whose absolute value of the difference is greater than the difference threshold.

8. The method according to any one of claims 1 to 7, characterized in that: The obtaining the noise level of the video based on the first spatial noise includes: Determine an average frame of the compensation frame and the current frame, wherein brightness information of an average pixel of the average frame is a weighted sum of brightness information of a third pixel of the compensation frame and brightness information of a fourth pixel of the current frame, and a position of the average pixel in the average frame is the same as a position of the third pixel in the compensation frame and a position of the fourth pixel in the current frame; Extracting a mean feature value of the mean frame, wherein the mean feature value is used to represent a noise level of the mean frame, and the noise level of the mean frame is used to represent a second spatial domain noise of the current frame; A noise level of the video is obtained based on the first spatial domain noise and the second spatial domain noise.

9. The method according to claim 8, characterized in that The mean frame includes at least one mean image block; and extracting the mean feature value of the mean frame includes: Performing a second target processing on the at least one mean image block of the mean frame to obtain at least one second processing parameter of the at least one mean image block, wherein the mean image block corresponds to the second processing parameter in a one-to-one manner; the second processing parameter is used to indicate a smoothness degree of the mean image block; From the at least one second processing parameter, determining at least one second target parameter that is less than a second threshold; the smoothness of the mean image block corresponding to the second target parameter matches the noise of the mean frame; The mean feature value of the mean frame is determined from the at least one second target parameter.

10. The method according to claim 8, characterized in that The obtaining the noise level of the video based on the first spatial domain noise and the second spatial domain noise includes: Based on the difference between the pixel points at the same position of the previous frame and the current frame, obtaining the temporal noise between the previous frame and the current frame; A noise level of the video is obtained based on the first spatial noise, the second spatial noise and the temporal noise.

11. The method according to claim 10, characterized in that The obtaining the temporal noise between the previous frame and the current frame based on the difference between the pixels at the same position of the previous frame and the current frame comprises: Determine a mixed frame of the previous frame and the current frame, the mixed image block of the mixed frame is formed by splicing the first image block of the previous frame and the second image block of the current frame, the mixed image block includes pixel points of the mixed frame, the first image block includes pixel points of the previous frame, the second image block includes pixel points of the current frame, and the position of the first image block in the previous frame is the same as the position of the second image block in the current frame; the noise level of the mixed frame is used to represent the temporal noise between the previous frame and the current frame; A mixed feature value of the mixed frame is extracted, where the mixed feature value is used to represent a noise level of the mixed frame.

12. The method according to claim 11, characterized in that The mixed frame includes at least one mixed image block; and extracting the mixed feature value of the mixed frame includes: Performing a third target processing on the at least one mixed image block of the mixed frame to obtain at least one third processing parameter of the at least one mixed image block, wherein the mixed image block corresponds to the third processing parameter in a one-to-one manner; the third processing parameter is used to indicate the smoothness of the mixed image block; Determining at least one third target parameter less than a third threshold value from the at least one third processing parameter; the smoothness of the mixed image block corresponding to the third target parameter matches the noise of the mixed frame; The mixed characteristic value of the mixed frame is determined from the at least one third target parameter.

13. The method according to any one of claims 10 to 12, characterized in that: The obtaining the noise level of the video based on the first spatial noise, the second spatial noise and the time domain noise includes: The first spatial domain noise, the second spatial domain noise, and the time domain noise are linearly fitted to obtain a noise level of the video.

14. The method according to any one of claims 1 to 7, characterized in that: The predicting the position of the pixel point of the previous frame based on the position offset of the pixel point of the current frame compared with the pixel point of the previous frame to obtain the compensated frame of the previous frame includes: Determine a position offset of a fifth pixel point of the current frame compared to a sixth pixel point of the previous frame, the fifth pixel point and the sixth pixel point having the same semantics; Based on the fifth pixel of the current frame, predict the sixth pixel of the previous frame according to the position offset to obtain a seventh pixel; Based on the seventh pixel point, a compensation frame of the previous frame is obtained.

15. A noise estimation device, characterized in that: The device comprises: A video frame acquisition module, used to acquire a previous frame in a video and a current frame in the video, wherein the previous frame and the current frame are adjacent decoded video frames, and the decoding order of the previous frame is before the current frame; A video frame prediction module, configured to predict the positions of the pixels of the previous frame based on the position offsets of the pixels of the current frame compared to the pixels of the previous frame, and obtain a compensation frame of the previous frame; A spatial noise determination module, configured to obtain a first spatial noise of the previous frame based on a difference between a pixel point of the compensation frame and a pixel point of the previous frame; The video noise estimation module obtains the noise level of the video based on the first spatial noise.

16. A computer device, characterized in that: include: A memory and a processor, wherein the memory stores program instructions; when the program instructions are executed by the processor, the processor executes the method as claimed in any one of claims 1 to 14.

17. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program; when the computer program is run on one or more processors, the method according to any one of claims 1 to 14 is executed.

18. A computer program product, characterized in that The computer program product comprises a computer program or instructions; when the computer program or instructions are run on a computer, the computer is caused to perform the method according to any one of claims 1 to 14.