Video noise detection method, apparatus, device, and medium

Abstract

The present disclosure relates to a video noise detection method, device, equipment and medium. The video noise detection method comprises extracting a first video frame and a second video frame in a target video, the first video frame and the second video frame being adjacent video frames; performing difference processing on the first video frame and the second video frame to obtain an inter-frame difference image between the first video frame and the second video frame; performing flat region detection on the first video frame and the second video frame to obtain an intersection of flat regions in the first video frame and the second video frame; and calculating a time domain noise value corresponding to the first video frame by using pixel information of the intersection of flat regions in the inter-frame difference image. The method can accurately evaluate video frame noise in the time domain and can meet the demand for real-time processing of a shot video.

Description

Technical Field

[0001] This disclosure relates to the field of video processing technology, and in particular to a video noise detection method, apparatus, device, and medium. Background Technology

[0002] Currently, users are accustomed to shooting videos using their smartphones' built-in cameras. However, due to the performance limitations of smartphone built-in cameras, certain types of videos shot with these cameras may have significant noise, which severely impacts the viewing experience.

[0003] To reduce the viewing experience caused by video noise, noise detection algorithms are needed to identify noise in the captured video and then remove it. However, current noise detection algorithms in the industry cannot balance accuracy and real-time performance, thus failing to meet the needs for real-time video noise processing. Summary of the Invention

[0004] To solve the above-mentioned technical problems, or at least partially solve them, this disclosure provides a video noise detection method, apparatus, device, and medium.

[0005] In a first aspect, embodiments of this disclosure provide a video noise detection method, including:

[0006] Extract a first video frame and a second video frame from the target video. The first video frame and the second video frame are adjacent video frames.

[0007] The first video frame and the second video frame are differentially processed to obtain an inter-frame difference image between the first video frame and the second video frame.

[0008] Perform flat region detection on the first video frame and the second video frame to obtain the intersection of the flat regions in the first video frame and the second video frame;

[0009] The temporal noise value corresponding to the first video frame is calculated using the pixel information of the intersection of the flat regions in the inter-frame difference image.

[0010] Optionally, the pixel information includes the pixel values ​​of each pixel in the inter-frame difference image where the intersection of the flat regions is located;

[0011] The step of calculating the temporal noise value corresponding to the target timestamp using the pixel information of the intersection of the flat regions in the inter-frame difference image includes:

[0012] Calculate the weighted average of the pixel values ​​of each pixel;

[0013] The weighted average value is used as the temporal noise value corresponding to the target timestamp.

[0014] Optionally, before calculating the weighted average of the pixel values ​​of each pixel, the method further includes:

[0015] Based on the preset correspondence between pixel values ​​and weight values, the weight value corresponding to each pixel value is determined.

[0016] The calculation of the weighted average of the pixel values ​​of each pixel includes:

[0017] For each pixel value, calculate the product of the pixel value and the weight value corresponding to the pixel value to obtain the weighted pixel value corresponding to the pixel value;

[0018] The weighted average value is obtained by performing a weighted average of the weighted pixel values ​​corresponding to each pixel value.

[0019] Optionally, the method further includes: evaluating the noise perception impact on the first video frame or the second video frame to obtain a noise perception impact coefficient in at least one dimension;

[0020] The method further includes, after calculating the temporal noise value corresponding to the first video frame using the pixel information in the inter-frame difference image obtained from the intersection of the flat regions, the method further includes:

[0021] Based on the temporal noise value and the noise perception influence coefficient of at least one dimension, a noise perception score for the first video frame is obtained. The noise perception score is used to evaluate the visual perception quality of noise in the first video frame and / or whether to perform noise reduction processing on the first video frame.

[0022] Optionally, the noise perceptibility impact coefficient of at least one dimension includes a detail richness impact coefficient. The step of evaluating noise perceptibility on the first video frame or the second video frame to obtain noise perceptibility evaluation coefficients of at least one dimension includes: performing detail intensity detection on the first video frame or the second video frame to obtain the detail richness impact coefficient; and / or,

[0023] The evaluation coefficient for at least one dimension includes a displacement rate influence coefficient. The step of evaluating noise perception on the first video frame or the second video frame to obtain noise perception evaluation coefficients for at least one dimension includes: performing image displacement detection based on the first video frame and the second video frame to obtain the displacement rate influence coefficient; and / or,

[0024] The evaluation coefficient of at least one dimension includes a brightness influence coefficient. The step of evaluating the noise perception of the first video frame or the second video frame to obtain the noise perception evaluation coefficient of at least one dimension includes: performing highlight area detection on the first video frame or the second video frame to obtain the brightness influence coefficient.

[0025] Optionally, the step of performing flat region detection on the first video frame and the second video frame to obtain the intersection of the flat regions in the first video frame and the second video frame includes:

[0026] For any one of the first video frame and the second video frame, a flat region is extracted from the video frame to obtain the flat region of the video frame;

[0027] Perform a bitwise AND operation on the flat regions of the first video frame and the flat regions of the second video frame to obtain the intersection of the flat regions.

[0028] Optionally, the step of extracting flat regions from the video frame to obtain flat regions of the video frame includes:

[0029] The video frame is segmented to obtain multiple image regions of the video frame;

[0030] Determine the texture parameters for each of the image regions;

[0031] Image regions with texture parameters less than a preset threshold are designated as flat regions of the video frame.

[0032] Optionally, before performing differential processing on the first video frame and the second video frame to obtain an inter-frame difference image between the first video frame and the second video frame, the method further includes:

[0033] The first video frame and the second video frame are globally aligned to obtain the aligned first video frame and second video frame.

[0034] The step of performing differential processing on the first video frame and the second video frame to obtain an inter-frame difference image between the first video frame and the second video frame includes:

[0035] The first and second video frames after alignment are subjected to a difference operation to obtain the inter-frame difference image.

[0036] Optionally, the step of globally aligning the first video frame and the second video frame to obtain aligned first and second video frames includes:

[0037] The first video frame and the second video frame are subjected to brightness alignment processing to obtain the brightness-aligned first video frame and second video frame.

[0038] A phase alignment operation is performed on the first and second video frames after brightness alignment to obtain the coordinate transformation relationship between the first and second video frames;

[0039] The first video frame after brightness alignment is subjected to an affine transformation using the coordinate transformation relationship to obtain the first video frame after affine transformation.

[0040] The first video frame after the affine transformation is used as the first aligned video frame, and the second video frame after the brightness alignment is used as the second aligned video frame.

[0041] Optionally, performing phase alignment operation on the luminance-aligned first and second video frames to obtain the coordinate transformation relationship between the first and second video frames includes:

[0042] The first and second video frames after brightness alignment are downsampled by a preset multiple to obtain the downsampled first and second video frames.

[0043] A phase alignment operation is performed on the downsampled first video frame and the downsampled second video frame to obtain a rotation matrix and a downsampled translation vector;

[0044] Multiply the downsampled translation vector by the preset factor to obtain the original offset;

[0045] The coordinate transformation relationship is determined based on the rotation matrix and the original offset.

[0046] Optionally, after calculating the temporal noise value corresponding to the first video frame, the method further includes:

[0047] Based on the temporal noise value, determine whether to perform noise reduction processing on the first video frame.

[0048] Secondly, embodiments of this disclosure provide a video noise detection device, comprising:

[0049] An extraction unit is used to extract a first video frame and a second video frame from a target video, wherein the first video frame and the second video frame are adjacent video frames.

[0050] The processing unit is configured to perform differential processing on the first video frame and the second video frame to obtain an inter-frame difference image between the first video frame and the second video frame.

[0051] The intersection determination unit is used to perform flat region detection on the first video frame and the second video frame to obtain the intersection of the flat regions in the first video frame and the second video frame.

[0052] The calculation unit is used to calculate the temporal noise value corresponding to the first video frame by utilizing the pixel information of the intersection of the flat regions in the inter-frame difference image.

[0053] Thirdly, embodiments of this disclosure provide a computing device, including: a processor; and a memory for storing executable instructions; wherein the processor is configured to read the executable instructions from the memory and execute the executable instructions to implement the video noise detection method as described above.

[0054] Fourthly, embodiments of this disclosure provide a computer-readable storage medium storing a computer program that, when executed by a processor, causes the processor to implement the video noise detection method as described above.

[0055] The technical solution provided in this disclosure has the following advantages compared with the prior art:

[0056] The video noise detection method, apparatus, device, and medium of this disclosure determine the intersection of an inter-frame difference image and a flat region based on adjacent first and second video frames in a target video. Then, based on the intersection of the flat regions and pixel information in the inter-frame difference image, the temporal noise value corresponding to the first video frame is calculated. Because the inter-frame difference image can evaluate the noise fluctuation characteristics of a video frame in the temporal domain, and the intersection of flat regions represents an image region with relatively obvious noise characteristics within the video frame, the noise characteristics of the intersection of flat regions in the inter-frame difference image are significant. Therefore, the temporal noise value calculated using the pixel information of the intersection of flat regions in the inter-frame difference image can more accurately evaluate video frame noise in the temporal domain, thereby improving the accuracy of noise evaluation. Furthermore, using the method provided in this disclosure, the video noise detection method only requires a reduced processing load to identify the temporal noise value of the video frame, which can meet the needs of real-time processing of captured videos. Attached Figure Description

[0057] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. Throughout the drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic, and the originals and elements are not necessarily drawn to scale.

[0058] Figure 1 This is a flowchart of a video noise detection method provided in some embodiments of this disclosure;

[0059] Figure 2This is a flowchart illustrating the process of obtaining inter-frame difference images provided in some embodiments of this disclosure;

[0060] Figure 3 This is a flowchart of determining a common minimum flat region provided in some embodiments of this disclosure;

[0061] Figure 4 This is a flowchart illustrating the determination of time-domain noise values ​​provided in some embodiments of this disclosure;

[0062] Figure 5 This is a flowchart of a video noise detection method provided in some other examples of this disclosure;

[0063] Figure 6 This is a schematic diagram of the video noise detection device provided in an embodiment of this disclosure;

[0064] Figure 7 A schematic diagram of the structure of a computing device provided in an embodiment of this disclosure is shown. Detailed Implementation

[0065] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0066] It should be understood that the steps described in the method embodiments of this disclosure may be performed in different orders and / or in parallel. Furthermore, the method embodiments may include additional steps and / or omit the steps shown. The scope of this disclosure is not limited in this respect.

[0067] The term "comprising" and its variations as used herein are open-ended inclusions, meaning "including but not limited to". The term "based on" means "at least partially based on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". Definitions of other terms will be given in the description below.

[0068] It should be noted that the concepts of "first" and "second" mentioned in this disclosure are used only to distinguish different devices, modules or units, and are not used to limit the order of functions performed by these devices, modules or units or their interdependencies.

[0069] It should be noted that the terms "a" and "a plurality of" used in this disclosure are illustrative rather than restrictive, and those skilled in the art should understand that, unless otherwise expressly indicated in the context, they should be understood as "one or more".

[0070] The names of messages or information exchanged between multiple devices in the embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of such messages or information.

[0071] Figure 1 This is a flowchart of a video noise detection method provided in some embodiments of this disclosure. For example... Figure 1 As shown, the video noise detection method provided in this embodiment includes steps S101-S104.

[0072] It should be noted that the video noise detection method provided in this disclosure can be executed by a computing device. The computing device includes, but is not limited to, electronic devices such as smartphones, laptops, personal digital assistants (PDAs), tablet computers (PADs), portable multimedia players (PMPs), in-vehicle terminals (e.g., in-vehicle navigation terminals), wearable devices, etc., and may also include servers.

[0073] Step S101: Extract the first video frame and the second video frame from the target video. The first video frame and the second video frame are adjacent video frames.

[0074] In this embodiment of the disclosure, the computing device can acquire the target video. For example, when the computing device is an electronic device such as a smartphone, it can capture images using its built-in camera to form the target video to be processed. As another example, when the computing device is a server, it can receive the target video to be processed sent by an electronic device via a network. Of course, the computing device can also read the target video stored locally and process it.

[0075] The target video can be in its original format or it can be an encoded video; this disclosure does not impose any particular limitation. For example, when the computing device is an electronic device and the target video is a video to be processed captured by the electronic device, the target video is preferably in its original format. When the computing device is a server and the target video is a video to be processed transmitted over a network, the target video is preferably an encoded video to ensure real-time video transmission.

[0076] After acquiring the target video, the computing device extracts video frames from it. Specifically, it extracts the first and second video frames. These are adjacent video frames. In practical applications, the first video frame can be the later of two adjacent video frames, and the second video frame can be the earlier of two adjacent video frames. It should be noted that the first and second video frames have the same resolution in both the horizontal and vertical directions.

[0077] Step S102: Perform differential processing on the first video frame and the second video frame to obtain the inter-frame difference image between the first video frame and the second video frame.

[0078] In some embodiments of this disclosure, differential processing of the first video frame and the second video frame involves subtracting the grayscale values ​​of pixels at corresponding positions in the first and second video frames to obtain a pixel grayscale difference. In other words, the inter-frame difference image of the first and second video frames is an image composed of the aforementioned pixel grayscale differences arranged in the sorting order of corresponding pixels in the first and second video frames.

[0079] In some specific applications, when capturing video of a target, the shooting device (such as a smartphone with a built-in camera) moves rapidly relative to the subject, causing the image content of the first and second video frames to differ. If the first and second video frames are directly differentially processed, the resulting frame difference image may not show the pixel grayscale differences of the subject (i.e., the fast-moving object) in the two video frames.

[0080] Figure 2 This is a flowchart illustrating the process of obtaining inter-frame difference images according to some embodiments of this disclosure. For example... Figure 2 As shown, in order to solve the aforementioned problems, in some embodiments of this disclosure, performing the aforementioned step S102 on a computing device may include steps S1021-S1022.

[0081] Step S1021: Perform global alignment on the first video frame and the second video frame to obtain the aligned first video frame and the second video frame.

[0082] Global alignment of the first video frame and the second video frame involves using one of the two video frames as a reference and matching the image pixel content representing a certain object in the other video frame with the image pixel content representing the same object in the aforementioned reference video frame.

[0083] In this embodiment of the disclosure, since the first video frame is the video frame to be processed and the second video frame is the reference video frame, the second video frame is used as the reference, and the first video frame is used as the video frame to be aligned. The first video frame is processed to obtain the aligned first video frame and the second video frame.

[0084] Specifically, global alignment of the first and second video frames can be achieved by selecting feature regions in both frames and matching them. After matching the feature regions in the first and second video frames, the coordinate transformation relationship of the target object represented by the feature regions in the two video frames can be determined based on the pixel coordinates of the feature regions in the two video frames.

[0085] After determining the coordinate transformation relationship of the target object in the first image frame and the second image frame, an affine transformation is performed on the first image frame according to the aforementioned coordinate transformation relationship, which can achieve global alignment of the first image frame with respect to the second image frame.

[0086] In some embodiments of this disclosure, step S1021 performs global alignment of the first video frame and the second video frame to obtain the aligned first video frame and the second video frame, which may specifically include steps S1021A-S1021B.

[0087] Step S1021A: Perform brightness alignment processing on the first video frame and the second video frame to obtain the brightness-aligned first video frame and the second video frame.

[0088] In practical applications, due to the real-time changes in ambient light intensity, the light intensity illuminating the target object may not be the same, resulting in different pixel brightness of the target object represented in the first video frame and the second video frame. This leads to the introduction of system errors into the inter-frame difference image of the first video frame and the second video frame.

[0089] To avoid the aforementioned problems, in some embodiments of this disclosure, before performing global phase alignment on the first video frame and the second video frame, luminance alignment can be performed first. Luminance alignment of the first video frame and the second video frame involves adjusting the maximum brightness of the first video frame and the second video frame to be consistent, or adjusting the black and white points in the first video frame and the second video frame to be consistent.

[0090] In some embodiments of this disclosure, brightness alignment processing is performed on the first video frame and the second video frame. First, grayscale histograms of the pixels in the first and second video frames are obtained. Then, based on the grayscale histograms of the first and second video frames, the grayscale values ​​of the pixels in the first video frame are adjusted according to a pre-set unidirectional adjustment rule, so that the brightness of each pixel in the first video frame is adjusted according to the adjustment rule, thereby aligning the brightness of the first and second video frames. In specific embodiments, the brightness alignment of the first and second video frames can be manifested by the fact that the distribution patterns of their grayscale histograms are substantially the same.

[0091] Step S1021B: Perform phase alignment operation on the first and second video frames after brightness alignment to obtain the coordinate transformation relationship between the first and second video frames.

[0092] The computing device can select feature regions in the first and second video frames and match these feature regions. After matching the feature regions in the first and second video frames, the computing device determines the coordinate transformation relationship of the target object represented by the feature regions in the two video frames based on the pixel coordinates of the feature regions in the two video frames.

[0093] Optionally, in some embodiments of this disclosure, in order to improve processing speed, the computing device may include steps B1-B4 when performing phase alignment operations on the first video frame and the second video frame.

[0094] B1: Downsample the first and second video frames after brightness alignment by a preset factor to obtain the downsampled first and second video frames.

[0095] B2: Perform phase alignment operation on the first and second downsampled video frames to obtain the rotation matrix and the downsampled translation vector.

[0096] B3: Multiply the downsampled translation vector by a preset factor to obtain the original offset.

[0097] After determining the coordinate transformation relationship between the first video frame and the second video frame using the aforementioned method, step S1021C can then be executed.

[0098] Step S1021C: Perform an affine transformation on the first video frame after brightness alignment using coordinate transformation relationships to obtain the first video frame after affine transformation.

[0099] After obtaining the coordinate transformation relationship, the first video frame after brightness alignment is subjected to an affine transformation, that is, a rotation and / or translation transformation, using the aforementioned coordinate transformation relationship, to obtain the first video frame after the affine transformation.

[0100] Step S1022: Perform a difference operation on the aligned first and second video frames to obtain an inter-frame difference image.

[0101] After obtaining the aligned first and second video frames, the pixels representing the same object in the aligned first and second video frames are used to perform a difference operation, which yields the inter-frame difference image.

[0102] It should be noted that when performing step S1022, some pixels in the first video frame and the second video frame may not have corresponding pixels for the difference operation, and only some pixels in the first video frame and the second video frame are subjected to the difference operation to obtain the inter-frame difference image.

[0103] In the foregoing embodiments of this disclosure, the alignment of the first video frame and the second video frame is a global alignment. Global alignment is particularly suitable for processing target videos formed by a moving shooting device capturing a stationary target object.

[0104] In other embodiments of this disclosure, the alignment of the first video frame and the second video frame can also be partial alignment. That is, when aligning the first video frame and the second video frame, only a portion of the pixel regions in the first video frame are aligned. After performing partial alignment and obtaining the aligned first video frame and the second video frame, a differential calculation can be performed on the first video frame and the second video frame to obtain an inter-frame difference image.

[0105] Step S103: Perform flat region detection on the first video frame and the second video frame to obtain the intersection of the flat regions in the first video frame and the second video frame.

[0106] In this embodiment, the flat region is a region in the first and second video frames where the changes in image pixels are relatively gradual. For example, the flat region may be a region in the first and second video frames that has a specific color and brightness.

[0107] Figure 3 This is a flowchart illustrating the determination of a common minimum flat region, provided in some embodiments of this disclosure. For example... Figure 3 As shown, in some embodiments of this disclosure, step S103 performs flat region processing on the first video frame and the second video frame to obtain the intersection of the flat regions in the first video frame and the second video frame, which may include steps S1031-S1032.

[0108] Step S1031: For any video frame in the first video frame and the second video frame, extract the flat region of the video frame to obtain the flat region of the video frame.

[0109] In this embodiment of the disclosure, flattening the region for either the first video frame or the second video frame is performed by performing flattening the region for both the first video frame and the second video frame respectively.

[0110] In some embodiments, performing flat region processing on the first video frame and the second video frame to obtain a flat region of the video frame may include steps S1031A-S1031C.

[0111] Step S1031A: Perform image segmentation on the video frame to obtain multiple image regions of the video frame.

[0112] Optionally, in this embodiment of the disclosure, image segmentation of the video frame can be performed by calculating the gradient image of the video frame, which is an image constructed using gradients. Subsequently, the video frame image is segmented based on the gradient image to obtain multiple image regions within the video frame. The aforementioned gradient image is determined based on the pixel grayscale difference between adjacent pixels in the video frame. Segmenting the video frame image based on the gradient image involves selecting pixels in the gradient image that are greater than a preset value as the edges of the segmented images, and determining multiple image regions of the video frame based on the edges of the segmented images.

[0113] Step S1031B: Determine the texture parameters for each image region.

[0114] After obtaining multiple image regions, each region can be processed separately to obtain its texture parameters. These texture parameters characterize the texture features of the image region.

[0115] In some embodiments of this disclosure, the texture feature parameters of an image region can be represented by the trace of the matrix covariance matrix within the image region. Specifically, firstly, based on the intra-block gradient covariance matrix of each image region, the eigenvalues ​​of the covariance matrix are obtained; then, the eigenvalues ​​of the covariance matrix are accumulated to obtain the trace of the matrix, which is used as the texture parameter of the image region.

[0116] Step S1031C: Image regions with texture parameters less than a preset parameter threshold are used as flat regions of the video frame.

[0117] A larger texture parameter indicates richer texture information in the image region. Flat regions are image regions with less texture features, so image regions with texture parameters less than a preset threshold can be considered as flat regions of a video frame.

[0118] Step S1032: Perform a bitwise AND operation on the flat regions of the first video frame and the flat regions of the second video frame to obtain the intersection of the flat regions.

[0119] After obtaining the flat regions of the first video frame and the second video frame respectively, performing a bitwise AND operation on the flat regions of the two video frames yields the intersection of the flat regions. In other words, if a pixel region at a specific location is a flat region in both the first and second video frames, then the pixel region at that specific location can be considered as the intersection of the flat regions.

[0120] After obtaining the common minimum flat region, the computing device can execute step S104.

[0121] Step S104: Calculate the temporal noise value corresponding to the first video frame by using the pixel information of the intersection of flat regions in the inter-frame difference image.

[0122] In this embodiment of the disclosure, after obtaining the intersection of the flat regions, the computing device can use the intersection of the flat regions to filter the pixel information in the inter-frame difference image to determine the sub-frame difference image with the view noise value. Subsequently, the temporal noise value of the first video frame can be calculated using the pixel information of the sub-frame difference image.

[0123] Figure 4 This is a flowchart illustrating the determination of time-domain noise values ​​provided in some embodiments of this disclosure. For example... Figure 4 As shown, in some embodiments of this disclosure, the pixel information in step S104 includes the pixel values ​​of each pixel in the inter-frame difference image where the intersection of flat regions is located. In this case, step S104 includes steps S1041-S1042.

[0124] Step S1041: Calculate the weighted average of the pixel values ​​of each pixel.

[0125] Step S1042: Use the weighted average value as the temporal noise value corresponding to the target timestamp.

[0126] In some embodiments of this disclosure, different pixel values ​​(i.e., the grayscale values ​​of pixels) correspond to different weighting weights. In this case, in order to obtain the temporal noise value corresponding to the target timestamp, it is first necessary to calculate the weighted average of the pixel values ​​of each pixel, and then use the weighted average as the temporal noise value corresponding to the target timestamp.

[0127] In a specific implementation, the electronic device may also include step S1043 before performing step S1041.

[0128] Step S1043: Based on the preset correspondence between pixel values ​​and weight values, determine the weight value corresponding to each pixel value.

[0129] In this embodiment of the disclosure, the electronic device stores the correspondence between each pixel value and its weight. After obtaining the pixel information of the intersection of flat regions in the inter-frame difference image, the weight value corresponding to each pixel value can be determined based on the aforementioned correspondence.

[0130] After determining the weight values, step S1041 can be executed. In specific implementations, step S1041 may include steps S1041A-S1042B.

[0131] Step S1041A: For each pixel value, calculate the product of the pixel value and the corresponding weight value to obtain the weighted pixel value.

[0132] Step S1041B: Perform a weighted average based on the weighted pixel values ​​corresponding to each pixel value to obtain the weighted average value.

[0133] For example, the pixel information of the intersection of flat regions in the inter-frame difference image are x1, x2, L, x n The corresponding weight values ​​are ω1, ω2, L, ω n Then, steps S1041A–S1041B can be performed using the formula. The weighted average value is calculated.

[0134] The video noise detection method provided in this disclosure determines the intersection of an inter-frame difference image and a flat region based on adjacent first and second video frames in the target video. Then, based on the intersection of the flat regions and pixel information in the inter-frame difference image, it calculates the temporal noise value corresponding to the first video frame. Because the inter-frame difference image can evaluate the noise fluctuation characteristics of a video frame in the temporal domain, and the intersection of flat regions represents an image region with relatively obvious noise characteristics within the video frame, the noise characteristics of the intersection of flat regions in the inter-frame difference image are significant. Therefore, the temporal noise value calculated using the pixel information of the intersection of flat regions in the inter-frame difference image can more accurately evaluate video frame noise in the temporal domain, thereby improving the accuracy of noise evaluation. Furthermore, using the method provided in this disclosure, the video noise detection method only requires a reduced processing load to identify the temporal noise value of the video frame, which can meet the needs of real-time processing of captured videos.

[0135] In some embodiments of this disclosure, after calculating the temporal noise value corresponding to the first video frame using step S104, the video noise detection method may further include step S105.

[0136] Step S105: Determine whether to perform noise reduction processing on the first video frame based on the temporal noise value.

[0137] By determining whether to perform noise reduction on the first video frame based on the temporal noise value, noise reduction can be performed on the first video frame if the temporal noise value exceeds a set value, and not if the temporal noise value does not exceed the set value.

[0138] Figure 5 This is a flowchart illustrating video noise detection methods provided in other examples of this disclosure. For example... Figure 5 As shown, in some embodiments of this disclosure, the video noise detection method includes steps S501-S506 as described above.

[0139] Step S501: Extract the first video frame and the second video frame from the target video. The first video frame and the second video frame are adjacent video frames.

[0140] Step S502: Perform differential processing on the first video frame and the second video frame to obtain the inter-frame difference image between the first video frame and the second video frame.

[0141] Step S503: Perform flat region detection on the first video frame and the second video frame to obtain the intersection of the flat regions in the first video frame and the second video frame.

[0142] Step S504: Calculate the temporal noise value corresponding to the first video frame using the pixel information of the intersection of flat regions in the inter-frame difference image.

[0143] The aforementioned steps S501-S504 are the same as steps S101-S104 in the previous embodiment, and can be found in the previous description, which will not be repeated here.

[0144] Step S505: Evaluate the impact of noise perception on the first video frame or the second video frame to obtain the noise perception impact coefficient of at least one dimension.

[0145] In this embodiment of the disclosure, noise perception impact evaluation is performed on the first video frame or the second video frame to obtain a noise perception impact coefficient in at least one dimension. This is achieved by processing the first video frame or the second video frame using a pre-selected multi-dimensional image evaluation method to obtain a noise perception evaluation coefficient in at least one dimension. In this embodiment of the disclosure, since the second video frame is the reference video frame, step S505 preferably performs noise perception impact evaluation on the second video frame to obtain a noise perception impact coefficient.

[0146] In some embodiments of this disclosure, the noise perception influence coefficient may include at least one of the detail richness influence coefficient, displacement rate influence coefficient, and brightness influence coefficient.

[0147] The detail richness influence coefficient is an evaluation coefficient used to characterize the impact of the detail richness of an image frame on noise perception. The detail richness influence coefficient can be obtained by performing detail intensity detection on the image frame. In some embodiments of this disclosure, the Laplace transform can be used to process the image frame to obtain its detail information, and the detail richness influence coefficient can be obtained using this detail information.

[0148] The displacement rate influence coefficient is an evaluation coefficient used to assess the impact of an object's displacement rate on noise perception in two video frames. In some embodiments of this disclosure, image displacement detection can be performed on a first video frame and a second video frame to obtain the displacement amount. Image displacement detection on the first and second video frames involves feature extraction analysis of the first and second video frames to determine the displacement amount of the object in the first and second video frames. The displacement amount can include displacement in the length direction and displacement in the width direction. Subsequently, the displacement rate influence coefficient can be obtained based on the displacement amount and either the first or second video frame. Taking the second video frame as an example, the length displacement amount can be divided by the image length of the second video frame to obtain the length direction rate influence coefficient, and the width displacement amount can be divided by the image width of the second video frame to obtain the width direction rate influence coefficient. Finally, the displacement rate influence coefficient can be obtained based on the length direction rate influence coefficient and the width direction rate influence coefficient. Specifically, the root mean square of the length direction rate influence coefficient and the width direction rate influence coefficient can be used to obtain the displacement rate influence coefficient.

[0149] The brightness impact coefficient is an evaluation coefficient used to assess the influence of video frame brightness on noise perception. In some embodiments of this disclosure, the number of pixels in the video frame whose grayscale exceeds a set value can be counted, and the counted number of pixels can be divided by the total number of pixels in the video frame to obtain the area ratio of the bright area as the brightness impact coefficient.

[0150] Step S506: Obtain the noise perception score of the first video frame based on the temporal noise value and the noise perception influence coefficient of at least one dimension.

[0151] After obtaining the temporal noise value and at least one dimension of noise perception influence coefficient, the noise perception score of the first video frame can be obtained according to the pre-specified scoring rules.

[0152] In some embodiments of this disclosure, a video frame score is obtained based on the temporal noise value and at least one noise perception influence coefficient. This can be achieved by multiplying the temporal noise value and each noise perception influence coefficient in sequence to obtain a product, and then using the product as the noise perception score of the first video frame.

[0153] In other embodiments of this disclosure, the noise perception score of the first video frame is obtained based on the temporal noise value and the noise perception influence coefficient of at least one dimension. This can be achieved by multiplying the temporal noise value by each noise perception influence coefficient to obtain a product, and then summing the products to obtain the noise perception score of the first video frame.

[0154] In other embodiments of this disclosure, the temporal noise value and the noise perception influence coefficients of each dimension can be input into a pre-trained deep learning scoring model. The deep learning scoring model is then used to comprehensively process the temporal noise value and the noise perception influence coefficients of each dimension to obtain the noise perception score of the first video frame. The deep learning scoring model is trained using sample images, the evaluation coefficients of each dimension corresponding to the sample images, and manually labeled scores.

[0155] By using temporal noise values ​​and noise perception scores of at least one dimension to obtain a noise perception score for the first video, the visual perception of noise in the first video frame can be evaluated, or it can be determined whether the first video should be denoised.

[0156] This disclosure also provides a video noise detection device. The video noise detection device can be installed in the aforementioned computing device to detect noise in a target video.

[0157] Figure 6 This is a schematic diagram of the video noise detection device 600 provided in an embodiment of this disclosure, as shown below. Figure 6 As shown, the video noise detection device 600 provided in this embodiment may include an extraction unit 601, a processing unit 602, an intersection determination unit 603, and a calculation unit 604.

[0158] The extraction unit 601 is used to extract a first video frame and a second video frame from the target video, wherein the first video frame and the second video frame are adjacent video frames.

[0159] The processing unit 602 is used to perform differential processing on the first video frame and the second video frame to obtain an inter-frame difference image between the first video frame and the second video frame.

[0160] The intersection determination unit 603 is used to perform flat region detection on the first video frame and the second video frame to obtain the intersection of the flat regions in the first video frame and the second video frame.

[0161] The calculation unit 604 is used to calculate the temporal noise value corresponding to the first video frame by utilizing the pixel information of the intersection of flat regions in the inter-frame difference image.

[0162] In some embodiments of this disclosure, pixel information includes the pixel values ​​of each pixel in the inter-frame difference image, representing the intersection of flat regions. Correspondingly, the calculation unit 604 includes a weighted average calculation subunit and a noise value calculation subunit. The weighted average calculation subunit is used to calculate a weighted average of the pixel values ​​of each pixel, and the noise value calculation subunit is used to use the weighted average as the temporal noise value corresponding to the target timestamp.

[0163] In some embodiments of this disclosure, the calculation unit 604 further includes a weight value acquisition subunit, which is used to determine the weight value corresponding to each pixel value based on a preset correspondence between pixel values ​​and weight values. Correspondingly, the weighted average calculation subunit first calculates the product of the pixel value and the corresponding weight value for each pixel value to obtain the weighted pixel value corresponding to the pixel value; then, it performs a weighted average based on the weighted pixel values ​​corresponding to each pixel value to obtain the weighted average value.

[0164] In some embodiments of this disclosure, the video noise detection device 600 further includes a noise perception influence coefficient acquisition unit and a noise perception score calculation unit. The noise perception influence coefficient acquisition unit is used to evaluate the noise perception influence on a first video frame and / or a second video frame, obtaining a noise perception influence coefficient in at least one dimension. The video frame scoring unit is used to obtain a noise perception score for the first video frame based on the temporal noise value and the noise perception influence coefficient in at least one dimension. The noise perception score is used to evaluate the visual perception of noise in the first video frame, and / or, whether noise reduction processing is performed on the first video frame.

[0165] In some embodiments of this disclosure, the noise perception impact coefficient of at least one dimension includes the detail richness impact coefficient. To evaluate the noise perception of a first video frame or a second video frame and obtain the noise perception evaluation coefficient of at least one dimension, the method includes: performing detail intensity detection on the first video frame or the second video frame to obtain the detail richness impact coefficient.

[0166] In some embodiments of this disclosure, the evaluation coefficient of at least one dimension includes a displacement rate influence coefficient. The noise perception evaluation of the first video frame or the second video frame is performed to obtain the noise perception evaluation coefficient of at least one dimension, including: performing image displacement detection based on the first video frame and the second video frame to obtain the displacement rate influence coefficient.

[0167] In some embodiments of this disclosure, the evaluation coefficient for at least one dimension includes a brightness influence coefficient. To obtain a noise perception evaluation coefficient for at least one dimension by performing noise perception evaluation on a first video frame or a second video frame, the evaluation includes: performing highlight region detection on the first video frame or the second video frame to obtain a brightness influence coefficient.

[0168] In some embodiments of this disclosure, the detection unit 603 includes a flat region extraction subunit and an intersection determination subunit. The flat region extraction subunit is used to extract flat regions from either the first video frame or the second video frame to obtain the flat regions of the video frame. The intersection determination subunit is used to perform a bitwise AND operation on the flat regions of the first video frame and the flat regions of the second video frame to obtain the intersection of the flat regions.

[0169] In some embodiments of this disclosure, the flat region extraction subunit includes an image region segmentation module, a texture parameter determination module, and a flat region selection module. The image region segmentation module performs image segmentation on the video frame to obtain multiple image regions of the video frame. The texture parameter determination module determines the texture parameters of each image region. The flat region selection module selects image regions with texture parameters less than a preset threshold as flat regions of the video frame.

[0170] In some embodiments of this disclosure, the video noise detection device 600 further includes an alignment unit. The alignment unit performs global alignment on the first video frame and the second video frame to obtain aligned first and second video frames. Correspondingly, the processing unit 602 performs a difference operation on the aligned first and second video frames to obtain an inter-frame difference image.

[0171] In some embodiments of this disclosure, the alignment unit includes a luminance alignment subunit, a coordinate transformation relationship determination subunit, an affine transformation subunit, and an aligned video frame determination subunit. The alignment subunit performs luminance alignment processing on a first video frame and a second video frame to obtain luminance-aligned first and second video frames. The coordinate transformation relationship determination subunit performs phase alignment operations on the luminance-aligned first and second video frames to obtain the coordinate transformation relationship between the first and second video frames. The affine transformation subunit performs an affine transformation on the luminance-aligned first video frame using the coordinate transformation relationship to obtain an affine-transformed first video frame. The aligned video frame determination subunit uses the affine-transformed first video frame as the aligned first video frame and the luminance-aligned second video frame as the aligned second video frame.

[0172] In some embodiments of this disclosure, the coordinate transformation relationship determination subunit includes a downsampling module, a phase alignment calculation module, an original offset calculation module, and a coordinate transformation relationship determination module. The downsampling module is used to downsample the first and second video frames after brightness alignment by a preset factor, obtaining the downsampled first and second video frames. The phase alignment calculation module is used to perform phase alignment operations on the downsampled first and second video frames, obtaining a rotation matrix and a downsampled translation vector. The original offset calculation module is used to multiply the downsampled translation vector by a preset factor to obtain the original offset. The coordinate transformation relationship determination module is used to determine the coordinate transformation relationship based on the rotation matrix and the original offset.

[0173] In some embodiments of this disclosure, the video noise detection apparatus further includes a noise reduction processing unit. The noise reduction processing unit is used to determine whether to perform noise reduction processing on the first video frame based on the temporal noise value.

[0174] It should be noted that, Figure 6 The video noise detection device 600 shown can perform... Figures 1 to 5 The various steps in the method embodiment shown are implemented. Figures 1 to 5 The processes and effects in the method embodiments shown are not described in detail here.

[0175] This disclosure also provides a computing device, which may include a processor and a memory, the memory being used to store executable instructions. The processor can be used to read the executable instructions from the memory and execute the executable instructions to implement the information display method described in the above embodiments.

[0176] Figure 7 A schematic diagram of a computing device provided in an embodiment of this disclosure is shown. See below for details. Figure 7 It shows a schematic diagram of a structure suitable for implementing the computing device 700 in the embodiments of this disclosure.

[0177] The computing device 700 in this embodiment can be an electronic device or a server. The electronic device may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), in-vehicle terminals (e.g., in-vehicle navigation terminals), wearable devices, etc., as well as fixed terminals such as digital TVs, desktop computers, smart home devices, etc.

[0178] It should be noted that, Figure 7 The computing device 700 shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments disclosed herein.

[0179] like Figure 7 As shown, the computing device 700 may include a processing unit (e.g., a central processing unit, a graphics processing unit, etc.) 701, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 702 or a program loaded from a storage device 708 into a random access memory (RAM) 703. The RAM 703 also stores various programs and data required for the operation of the information processing device 700. The processing unit 701, ROM 702, and RAM 703 are interconnected via a bus 704. An input / output (I / O) interface 705 is also connected to the bus 704.

[0180] Typically, the following devices can be connected to I / O interface 705: input devices 706 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 707 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 708 including, for example, magnetic tapes, hard disks, etc.; and communication devices 709. Communication device 709 allows computing device 700 to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 7 A computing device 700 with various devices is shown; however, it should be understood that it is not required to implement or have all of the devices shown. More or fewer devices may be implemented or have alternatively.

[0181] This disclosure also provides a computer-readable storage medium storing a computer program that, when executed by a processor, causes the processor to implement the information display method described above.

[0182] In particular, according to embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication device 709, or installed from storage device 708, or installed from ROM 702. When the computer program is executed by processing device 701, it performs the functions defined in the information display method of embodiments of this disclosure.

[0183] It should be noted that the computer-readable medium described in this disclosure can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this disclosure, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in connection with an instruction execution system, apparatus, or device. In this disclosure, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.

[0184] In some implementations, clients and servers can communicate using any currently known or future-developed network protocol such as HTTP, and can interconnect with digital data communication (e.g., communication networks) of any form or medium. Examples of communication networks include local area networks (“LANs”), wide area networks (“WANs”), the Internet (e.g., the Internet of Things), and end-to-end networks (e.g., ad hoc end-to-end networks), as well as any currently known or future-developed networks.

[0185] The aforementioned computer-readable medium may be included in the aforementioned computing device; or it may exist independently and not assembled into the computing device.

[0186] The aforementioned computer-readable medium carries one or more programs that, when executed by the computing device, cause the computing device to perform the following: extracting a first video frame and a second video frame from a target video, wherein the first video frame and the second video frame are adjacent video frames; performing differential processing on the first video frame and the second video frame to obtain an inter-frame difference image between the first video frame and the second video frame; performing flat region detection on the first video frame and the second video frame to obtain the intersection of flat regions in the first video frame and the second video frame; and using the pixel information of the intersection of flat regions in the inter-frame difference image to calculate the temporal noise value corresponding to the first video frame.

[0187] In embodiments of this disclosure, computer program code for performing the operations of this disclosure can be written in one or more programming languages ​​or a combination thereof. These programming languages ​​include, but are not limited to, object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0188] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0189] The units described in the embodiments of this disclosure can be implemented in software or hardware. The names of the units are not, in some cases, intended to limit the specific unit.

[0190] The functions described above in this document can be performed, at least in part, by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: Field Programmable Gate Arrays (FPGAs), Application-Specific Integrated Circuits (ASICs), Application Standard Products (ASSPs), System-on-Chip (SoCs), Complex Programmable Logic Devices (CPLDs), and so on.

[0191] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0192] The above description is merely a preferred embodiment of this disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of this disclosure is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features disclosed in this disclosure that have similar functions.

[0193] Furthermore, while the operations are described in a specific order, this should not be construed as requiring these operations to be performed in the specific order shown or in a sequential order. In certain environments, multitasking and parallel processing may be advantageous. Similarly, while several specific implementation details are included in the above discussion, these should not be construed as limiting the scope of this disclosure. Certain features described in the context of individual embodiments may also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented individually or in any suitable sub-combination in multiple embodiments.

[0194] Although the subject matter has been described using language specific to structural features and / or methodological logic, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. Rather, the specific features and actions described above are merely illustrative examples of implementing the claims.

Claims

1. A method of video noise detection, characterized by, include: Extract a first video frame and a second video frame from the target video. The first video frame and the second video frame are adjacent video frames. The first video frame and the second video frame are differentially processed to obtain an inter-frame difference image between the first video frame and the second video frame. The first video frame and the second video frame are globally aligned before the differential processing. The global alignment between the first video frame and the second video frame is achieved by sequentially performing brightness alignment processing, phase alignment operation and affine transformation on the first video frame and the second video frame. Flat region detection is performed on the first video frame and the second video frame to obtain the intersection of flat regions in the first video frame and the second video frame, wherein the intersection of flat regions is a flat region in both the first video frame and the second video frame; The temporal noise value corresponding to the first video frame is calculated using the pixel information of the intersection of the flat regions in the inter-frame difference image.

2. The method of claim 1, wherein, The pixel information includes the pixel values ​​of each pixel in the inter-frame difference image where the intersection of the flat regions is located; The step of calculating the temporal noise value corresponding to the target timestamp using the pixel information of the intersection of the flat regions in the inter-frame difference image includes: Calculate the weighted average of the pixel values ​​of each pixel; The weighted average value is used as the temporal noise value corresponding to the target timestamp.

3. The method of claim 2, wherein, Before calculating the weighted average of the pixel values ​​of the individual pixels, the method further includes: Based on the preset correspondence between pixel values ​​and weight values, the weight value corresponding to each pixel value is determined. The calculation of the weighted average of the pixel values ​​of each pixel includes: For each pixel value, the product of the pixel value and the weight value corresponding to the pixel value is calculated to obtain the weighted pixel value corresponding to the pixel value; The weighted average value is obtained by performing a weighted average of the weighted pixel values ​​corresponding to each pixel value.

4. The method of claim 1, wherein, The method further includes: The noise perception impact is evaluated on the first video frame and / or the second video frame to obtain a noise perception impact coefficient in at least one dimension. The method further includes, after calculating the temporal noise value corresponding to the first video frame using the pixel information in the inter-frame difference image obtained from the intersection of the flat regions, the method further includes: Based on the temporal noise value and the noise perception influence coefficient of at least one dimension, a noise perception score for the first video frame is obtained. The noise perception score is used to evaluate the visual perception of noise in the first video frame and / or whether to perform noise reduction processing on the first video frame.

5. The method according to claim 4, characterized in that, The noise perceptibility impact coefficient of at least one dimension includes a detail richness impact coefficient. The step of evaluating noise perceptibility on the first video frame or the second video frame to obtain noise perceptibility evaluation coefficients of at least one dimension includes: performing detail intensity detection on the first video frame or the second video frame to obtain the detail richness impact coefficient; and / or, The evaluation coefficient for at least one dimension includes a displacement rate influence coefficient. The step of evaluating noise perception on the first video frame or the second video frame to obtain noise perception evaluation coefficients for at least one dimension includes: performing image displacement detection based on the first video frame and the second video frame to obtain the displacement rate influence coefficient; and / or, The evaluation coefficient of at least one dimension includes a brightness influence coefficient. The step of evaluating the noise perception of the first video frame or the second video frame to obtain the noise perception evaluation coefficient of at least one dimension includes: performing highlight area detection on the first video frame or the second video frame to obtain the brightness influence coefficient.

6. The method of claim 1, wherein, The step of performing flat region detection on the first video frame and the second video frame to obtain the intersection of the flat regions in the first video frame and the second video frame includes: For any one of the first video frame and the second video frame, a flat region is extracted from the video frame to obtain the flat region of the video frame; Perform a bitwise AND operation on the flat regions of the first video frame and the flat regions of the second video frame to obtain the intersection of the flat regions.

7. The method of claim 6, wherein, The step of extracting flat regions from the video frame to obtain flat regions of the video frame includes: The video frame is segmented to obtain multiple image regions of the video frame; Determine the texture parameters for each of the image regions; Image regions with texture parameters less than a preset threshold are designated as flat regions of the video frame.

8. The method of claim 1, wherein, Before performing differential processing on the first video frame and the second video frame to obtain an inter-frame difference image between the first video frame and the second video frame, the method further includes: The first video frame and the second video frame are globally aligned to obtain the aligned first video frame and second video frame. The step of performing differential processing on the first video frame and the second video frame to obtain an inter-frame difference image between the first video frame and the second video frame includes: The first and second video frames after alignment are subjected to a difference operation to obtain the inter-frame difference image.

9. The method of claim 8, wherein, The step of globally aligning the first video frame and the second video frame to obtain aligned first and second video frames includes: The first video frame and the second video frame are subjected to brightness alignment processing to obtain the brightness-aligned first video frame and second video frame. A phase alignment operation is performed on the first and second video frames after brightness alignment to obtain the coordinate transformation relationship between the first and second video frames; The first video frame after brightness alignment is subjected to an affine transformation using the coordinate transformation relationship to obtain the first video frame after affine transformation. The first video frame after the affine transformation is used as the first aligned video frame, and the second video frame after the brightness alignment is used as the second aligned video frame.

10. The method of claim 9, wherein, The step of performing phase alignment operation on the first and second video frames after brightness alignment to obtain the coordinate transformation relationship between the first and second video frames includes: The first and second video frames after brightness alignment are downsampled by a preset multiple to obtain the downsampled first and second video frames. Perform phase alignment operation on the downsampled first video frame and the downsampled second video frame to obtain a rotation matrix and a downsampled translation vector; Multiply the downsampled translation vector by the preset factor to obtain the original offset; The coordinate transformation relationship is determined based on the rotation matrix and the original offset.

11. The method of claim 1, wherein, After calculating the temporal noise value corresponding to the first video frame, the method further includes: Based on the temporal noise value, determine whether to perform noise reduction processing on the first video frame.

12. A video noise detection apparatus characterized by comprising: include: An extraction unit is used to extract a first video frame and a second video frame from a target video, wherein the first video frame and the second video frame are adjacent video frames. The processing unit is used to perform differential processing on the first video frame and the second video frame to obtain an inter-frame differential image between the first video frame and the second video frame. The first video frame and the second video frame have been globally aligned before the differential processing. The global alignment between the first video frame and the second video frame is achieved by sequentially performing brightness alignment processing, phase alignment operation and affine transformation on the first video frame and the second video frame. The intersection determination unit is used to perform flat region detection on the first video frame and the second video frame to obtain the intersection of the flat regions in the first video frame and the second video frame, wherein the intersection of the flat regions is a flat region in both the first video frame and the second video frame. The calculation unit is used to calculate the temporal noise value corresponding to the first video frame by utilizing the pixel information of the intersection of the flat regions in the inter-frame difference image.

13. A computing device, comprising: include: processor; Memory, used to store executable instructions; The processor is configured to read the executable instructions from the memory and execute the executable instructions to implement the video noise detection method according to any one of claims 1-11.

14. A computer-readable storage medium, characterized in that, The storage medium stores a computer program, which, when executed by a processor, causes the processor to implement the video noise detection method according to any one of claims 1-11.

Images