Video image noise reduction method and device based on time domain information and weight factor

Abstract

The invention relates to the technical field of video image noise reduction, and discloses a video image noise reduction method and device based on time domain information and a weight factor, and the method comprises the steps: determining an optimal reference image block of a to-be-processed image block according to the weight factor, and determining the optimal reference image block of the to-be-processed image block according to the time domain information of the to-be-processed image block and the optimal reference image block; determining the motion intensity of the to-be-processed image block, determining a reference block filter coefficient of the optimal reference image block according to the motion intensity, and calculating a reference pixel point filter coefficient of the reference pixel point and a to-be-processed pixel point filter coefficient of the to-be-processed pixel point according to the reference block filter coefficient, and carrying out filtering and noise reduction on the to-be-processed pixel point according to the to-be-processed pixel point filtering coefficient and the reference pixel point filtering coefficient. Therefore, the optimal reference image block can be determined based on the strategy of the local weight factor, the filtering and noise reduction efficiency and accuracy of the image video can be improved, and the probability of inter-frame flicker and motion smear problems after noise reduction is reduced.

Description

Technical Field

[0001] The present invention relates to the technical field of video image noise reduction, and in particular to a video image noise reduction method and device based on time domain information and weight factors. Background Art

[0002] With the rapid development of Internet technology, the demand for ultra-high-definition video is becoming increasingly strong. Noise will inevitably be introduced during the video acquisition process. Noise reduction technology has always been one of the key technologies in the fields of video encoding and image processing.

[0003] At present, the noise reduction processing methods are mainly divided into two major directions: traditional noise reduction methods and learning-based noise reduction methods. Among them, the commonly used method in traditional video coding noise reduction technology is spatiotemporal motion adaptive noise reduction (abbreviated as: 3D noise reduction), which uses spatial domain similarity and temporal continuity of previous and next frames and current frames to remove noise in the video. Compared with spatial domain noise reduction (abbreviated as: 2D noise reduction), 3D noise reduction technology can reduce the flickering of background areas, and motion adaptation can reduce the "smear" problem in temporal noise reduction. However, when continuous background areas appear in video sequences, flickering and motion smearing problems are likely to occur between frames in continuous background areas after noise reduction; while the learning-based noise reduction method uses a network model for motion estimation, noise level estimation, and judgment of noise reduction intensity. The parameters that need to be trained for the network model are huge, and it is difficult to achieve real-time noise reduction, which is difficult to implement in real-time video coding applications. Therefore, it is particularly important to propose a technical solution that can reduce the probability of inter-frame flickering and motion smearing problems after video noise reduction. Summary of the invention

[0004] The present invention provides a method and device for video image denoising based on time domain information and weight factors, which can help reduce the probability of inter-frame flickering and motion smearing problems after video denoising.

[0005] In order to solve the above technical problems, the first aspect of the present invention discloses a video image denoising method based on time domain information and weight factors, the method comprising:

[0006] Acquire at least one image block to be processed in the image frame to be processed, and for each of the image blocks to be processed, determine, in a reference image frame corresponding to the image frame to be processed, an optimal reference image block corresponding to the image block to be processed according to a preset weight factor;

[0007] For each of the image blocks to be processed, calculating the time domain information between the image block to be processed and the corresponding best reference image block, wherein the time domain information includes similarity and motion vector;

[0008] For each of the image blocks to be processed, determining the motion intensity of the image block to be processed according to the time domain information corresponding to the image block to be processed, and determining the reference block filter coefficient of the best reference image block corresponding to the image block to be processed according to the motion intensity;

[0009] For each of the image blocks to be processed, a reference pixel point filter coefficient of each reference pixel point in the best reference image block corresponding to the image block to be processed is calculated, and a pixel point filter coefficient of each pixel point to be processed in the image block to be processed is calculated according to the reference pixel point filter coefficient of each reference pixel point;

[0010] For each of the image blocks to be processed and each of the pixel points to be processed in the image blocks to be processed, filtering and denoising is performed on the pixel point to be processed according to the pixel point filtering coefficient to be processed and the reference pixel point filtering coefficient of the reference pixel point corresponding to the pixel point to be processed, so as to obtain a denoised output result corresponding to the pixel point to be processed.

[0011] The second aspect of the present invention discloses a video image denoising device based on time domain information and weight factors, the device comprising:

[0012] An acquisition module is used to acquire at least one image block to be processed in the image frame to be processed, and for each of the image blocks to be processed, determine the best reference image block corresponding to the image block to be processed in the reference image frame corresponding to the image frame to be processed according to a preset weight factor;

[0013] A calculation module, configured to calculate, for each of the image blocks to be processed, time domain information between the image block to be processed and the corresponding optimal reference image block, wherein the time domain information includes similarity and motion vector;

[0014] A determination module, configured to determine, for each of the image blocks to be processed, a motion intensity of the image block to be processed according to the time domain information corresponding to the image block to be processed, and determine, according to the motion intensity, a reference block filter coefficient of the best reference image block corresponding to the image block to be processed;

[0015] The calculation module is further configured to calculate, for each of the image blocks to be processed, a reference pixel point filter coefficient of each reference pixel point in the best reference image block corresponding to the image block to be processed according to the reference block point filter coefficient of the best reference image block to be processed, and calculate a to-be-processed pixel point filter coefficient of each to-be-processed pixel point in the image block to be processed according to the reference pixel point filter coefficient of each of the reference pixels;

[0016] The denoising module is used to filter and denoise each of the image blocks to be processed and each of the pixel points to be processed in the image blocks to be processed according to the pixel point filtering coefficient of the pixel point to be processed and the reference pixel point filtering coefficient of the reference pixel point corresponding to the pixel point to be processed, so as to obtain the denoising output result corresponding to the pixel point to be processed.

[0017] The third aspect of the present invention discloses another video image noise reduction device based on time domain information and weight factors, the device comprising:

[0018] A memory storing executable program code;

[0019] a processor coupled to the memory;

[0020] The processor calls the executable program code stored in the memory to execute the video image denoising method based on time domain information and weight factors disclosed in the first aspect of the present invention.

[0021] The fourth aspect of the present invention discloses a computer storage medium, which stores computer instructions. When the computer instructions are called, they are used to execute the video image denoising method based on time domain information and weight factors disclosed in the first aspect of the present invention.

[0022] Compared with the prior art, the embodiments of the present invention have the following beneficial effects:

[0023] In the embodiment of the present invention, the best reference image block corresponding to the image block to be processed in the image frame to be processed is determined according to the preset weight factor, and the motion intensity of the image block to be processed is determined according to the time domain information between the image block to be processed and the corresponding best reference image block, and the reference block filter coefficient of the best reference image block corresponding to the image block to be processed is determined according to the motion intensity, and the reference pixel filter coefficient of each reference pixel in the best reference image block and the processed pixel filter coefficient of each pixel to be processed in the image block to be processed are calculated according to the reference block filter coefficient, and the pixel to be processed is filtered and denoised according to the pixel filter coefficient to be processed and the reference pixel filter coefficient to obtain the denoised output result corresponding to the pixel to be processed. It can be seen that the implementation of the present invention can determine the best reference image block corresponding to the image block to be processed based on the strategy of the local weight factor, improve the accuracy of determining the best reference image block, calculate the time domain information in blocks as units, and fewer reference frames can reduce the occupancy of additional cache data, which can improve the filtering and denoising efficiency and accuracy of image video, and reduce the probability of inter-frame flickering and motion smearing problems after video denoising. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0025] Figure 1 It is a flow chart of a method for video image denoising based on time domain information and weight factors disclosed in an embodiment of the present invention;

[0026] Figure 2 It is a flowchart of another method for video image denoising based on time domain information and weight factors disclosed in an embodiment of the present invention;

[0027] Figure 3 It is a structural schematic diagram of a video image denoising device based on time domain information and weight factors disclosed in an embodiment of the present invention;

[0028] Figure 4 It is a structural schematic diagram of another video image denoising device based on time domain information and weight factors disclosed in an embodiment of the present invention. DETAILED DESCRIPTION

[0029] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0030] The terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish different objects rather than to describe a specific order. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, device, product or end including a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units that are not listed, or may optionally include other steps or units inherent to these processes, methods, products or ends.

[0031] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present invention. The appearance of the phrase in various places in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0032] The present invention discloses a method and device for video image denoising based on time domain information and weight factors, which can determine the best reference image block corresponding to the image block to be processed based on the strategy of local weight factors, improve the accuracy of determining the best reference image block, calculate the time domain information in blocks, and reduce the occupancy of additional cache data with fewer reference frames, thereby improving the efficiency and accuracy of filtering and denoising of image videos, and reducing the probability of inter-frame flickering and motion smearing problems after video denoising. The following are detailed descriptions.

[0033] Embodiment 1

[0034] See also Figure 1 , Figure 1 is a flow chart of a method for video image denoising based on time domain information and weight factors disclosed in an embodiment of the present invention. Figure 1 The described video image denoising method based on time domain information and weight factors can be applied to a video image denoising device based on time domain information and weight factors, wherein the video image denoising device based on time domain information and weight factors may include an intelligent server or intelligent platform for denoising video images, and the intelligent server includes a local server or a cloud server, which is not limited in the embodiments of the present invention. Figure 1 As shown, the video image denoising method based on time domain information and weight factors may include the following operations:

[0035] 101. Obtain at least one image block to be processed in an image frame to be processed, and for each image block to be processed, determine, in a reference image frame corresponding to the image frame to be processed, an optimal reference image block corresponding to the image block to be processed according to a preset weight factor.

[0036] In an embodiment of the present invention, optionally, the image frame to be processed can be divided into at least one macroblock, i.e., the image block to be processed, each image block to be processed is of equal size, each image frame to be processed corresponds to a reference image frame, each image block to be processed in each image frame to be processed corresponds to an optimal reference image block in the reference image frame, and the optimal reference image block corresponding to the image block to be processed can be determined according to a preset weight factor. Specifically, the optimal reference image block corresponding to the image block to be processed can be determined based on a strategy based on a local weight factor.

[0037] 102. For each image block to be processed, calculate the time domain information between the image block to be processed and the corresponding best reference image block, determine the motion intensity of the image block to be processed based on the time domain information corresponding to the image block to be processed, and determine the reference block filter coefficient of the best reference image block corresponding to the image block to be processed based on the motion intensity.

[0038] In the embodiment of the present invention, optionally, the time domain information between the image block to be processed and the corresponding best reference image block may include the similarity and motion vector between the image block to be processed and the corresponding best reference image block, and the motion intensity of the image block to be processed may be determined according to the similarity and the size of the motion vector between the image block to be processed and the corresponding best reference image block, and according to the motion intensity, the reference block filter coefficient of the best reference image block corresponding to the image block to be processed may be selected from a preset filter coefficient table.

[0039] 103. For each image block to be processed, the reference pixel point filter coefficient of each reference pixel in the best reference image block corresponding to the image block to be processed is calculated, and the processed pixel point filter coefficient of each reference pixel in the image block to be processed is calculated based on the reference pixel point filter coefficient of each reference pixel.

[0040] 104. For each image block to be processed and for each pixel to be processed in the image block to be processed, filtering and denoising is performed on the pixel to be processed according to a filtering coefficient of the pixel to be processed and a reference pixel filtering coefficient of a reference pixel corresponding to the pixel to be processed to obtain a denoised output result corresponding to the pixel to be processed.

[0041] In an embodiment of the present invention, optionally, for each image block to be processed, filtering and denoising can be performed on the pixel to be processed according to the filtering coefficient of the pixel to be processed and the reference pixel filtering coefficient of the reference pixel to obtain a denoising output result corresponding to the pixel to be processed, and then the denoising output results of all pixels of the image block to be processed are synthesized into a new image block to replace the image block to be processed, thereby completing filtering and denoising of the image block to be processed, which is not limited by the present invention.

[0042] It can be seen that the implementation Figure 1The described video image denoising method based on time domain information and weight factors can determine the best reference image block corresponding to the image block to be processed in the image frame to be processed according to the preset weight factor, and determine the motion intensity of the image block to be processed according to the time domain information between the image block to be processed and the corresponding best reference image block, and determine the reference block filter coefficient of the best reference image block corresponding to the image block to be processed according to the motion intensity, calculate the reference pixel point filter coefficient of each reference pixel point in the best reference image block and the to-be-processed pixel point filter coefficient of each to-be-processed pixel point in the image block to be processed according to the reference block filter coefficient, filter and denoise the pixel point to be processed according to the to-be-processed pixel point filter coefficient and the reference pixel point filter coefficient, and obtain the denoising output result corresponding to the pixel point to be processed, and can determine the best reference image block corresponding to the image block to be processed based on the strategy of local weight factors, improve the accuracy of determining the best reference image block, calculate the time domain information in blocks, and fewer reference frames can reduce the occupation of additional cache data, which can improve the filtering and denoising efficiency and accuracy of image video, and reduce the probability of inter-frame flickering and motion smear problems after video denoising.

[0043] In an optional embodiment, for each image block to be processed, calculating the reference pixel point filtering coefficient of each reference pixel point in the best reference image block according to the reference block filtering coefficient of the best reference image block corresponding to the image block to be processed may include the following operations:

[0044] For each image block to be processed, for each pixel to be processed in the image block to be processed, calculating the absolute value of the difference between the pixel to be processed and each corresponding reference pixel in the best reference image block;

[0045] For each image block to be processed, determine the difference absolute value extreme value between the image block to be processed and the corresponding best reference image block in each difference absolute value, and calculate the difference absolute value extreme value difference according to the difference absolute value extreme value, wherein the difference absolute value extreme value includes the difference absolute value maximum value and the difference absolute value minimum value;

[0046] For each image block to be processed, for each pixel to be processed in the image block to be processed, according to the absolute value of the difference between the pixel to be processed and the corresponding reference pixel, the minimum absolute value of the difference, and the extreme absolute value of the difference, a normalized result of the absolute value of the difference between the pixel to be processed and the corresponding reference pixel within a preset target range is calculated;

[0047] For each image block to be processed and for each pixel to be processed in the image block to be processed, a scaling coefficient corresponding to the pixel to be processed is calculated based on a normalization result between the pixel to be processed and the corresponding reference pixel, and a reference pixel point filtering coefficient for each reference pixel in the best reference image block corresponding to the image block to be processed is calculated based on the scaling coefficient and the reference block filtering coefficient of the best reference image block corresponding to the image block to be processed.

[0048] In this optional embodiment, optionally, the calculation formula for the absolute value of the difference between the pixel to be processed and each corresponding reference pixel in the best reference image block is:

[0049] pixeldiffval k (i,j)=abs(curpixelval k (i,j)-refpixelval k (i,j))

[0050] Among them, pixeldiffval k (i, j) represents the absolute value of the difference between the pixel to be processed with coordinates (i, j) in the kth image block to be processed and the corresponding reference pixel;

[0051] Optionally, an extreme absolute value of the difference between the image block to be processed and the corresponding best reference image block is determined in each absolute value of the difference, and the extreme absolute value of the difference may include a maximum absolute value of the difference and a minimum absolute value of the difference. The specific formula includes:

[0052] maxpixeldiff k =max(pixeldiffval k (i,j)), minpixeldiff k =min(pixeldiffval k (i,j))

[0053] Among them, maxpixeldiff k Indicates the maximum absolute value of the difference corresponding to the kth image block to be processed, minpixeldiff k Indicates the minimum absolute value of the difference corresponding to the kth image block to be processed;

[0054] Optionally, the absolute value extreme value difference of the difference may be calculated according to the absolute value extreme value of the difference. The specific calculation formula includes:

[0055] gap k =maxpixeldiff k -minpixeldiff k

[0056] Among them, gap k Represents the absolute value and extreme value of the difference corresponding to the kth image block to be processed.

[0057] In this optional embodiment, optionally, for each image block to be processed, for each pixel to be processed in the image block to be processed, according to the absolute value of the difference between the pixel to be processed and the corresponding reference pixel, the minimum absolute value of the difference and the extreme absolute value of the difference, a normalized result of the absolute value of the difference between the pixel to be processed and the corresponding reference pixel within a preset target range is calculated, and the preset target range can be set by the user. According to the normalized result between the pixel to be processed and the corresponding reference pixel, a scaling coefficient corresponding to the pixel to be processed is calculated, and according to the scaling coefficient and the reference block filtering coefficient of the best reference image block corresponding to the image block to be processed, a reference pixel point filtering coefficient of each reference pixel in the best reference image block corresponding to the image block to be processed is calculated, which is not limited in this embodiment.

[0058] It can be seen that the implementation of this optional embodiment can calculate the absolute value of the difference between the pixel to be processed and each corresponding reference pixel in the best reference image block, and determine the absolute value extreme value of the difference and the absolute value extreme value difference of the difference. According to the absolute value of the difference, the minimum absolute value of the difference and the absolute value extreme value difference of the difference, the normalization result of the absolute value of the difference within the preset target range is calculated. According to the normalization result between the pixel to be processed and the corresponding reference pixel, the scaling coefficient corresponding to the pixel to be processed is calculated, and according to the scaling coefficient and the reference block filtering coefficient of the best reference image block corresponding to the image block to be processed, the reference pixel point filtering coefficient of each reference pixel in the best reference image block corresponding to the image block to be processed is calculated. The scaling coefficient of the exponential model can be calculated according to the absolute value of the difference of the image block to be processed, and the reference pixel point filtering coefficient of each reference pixel is calculated according to the scaling coefficient, so as to improve the accuracy of the calculated reference pixel point filtering coefficient, thereby improving the accuracy and efficiency of filtering and denoising the image block to be processed, and reducing the probability of inter-frame flickering and motion smearing problems after video denoising.

[0059] In another optional embodiment, the calculation formula of the normalized result includes:

[0060]

[0061] Among them, normalpixeldiffval k (i, j) represents the normalized result of the absolute value of the difference between the pixel to be processed with coordinates (i, j) and the corresponding reference pixel in the kth image block to be processed in the image frame to be processed, in the range [-a, a], pixeldiffval k(i, j) represents the absolute value of the difference between the pixel to be processed with coordinates (i, j) in the kth image block to be processed and the corresponding reference pixel. minpixeldiff k Indicates the minimum absolute value of the difference corresponding to the kth image block to be processed, gap k represents the absolute value extreme value difference of the difference corresponding to the kth image block to be processed;

[0062] And, the calculation formula of the reference pixel point filter coefficient of each reference pixel point in the best reference image block corresponding to each image block to be processed includes:

[0063] pixelRefCoef k (i,j)=refblkCoef k *exp(-normalpixeldiffval k (i,j))

[0064] Among them, pixelRefCoef k (i, j) represents the reference pixel filter coefficient of the reference pixel corresponding to the pixel to be processed with coordinates (i, j) in the kth image block to be processed, refblkCoef k The reference block filter coefficient of the best reference image block corresponding to the kth image block to be processed, exp(-normalpixeldiffval k (i, j)) represents the scaling factor corresponding to the pixel to be processed with coordinates (i, j) in the kth image block to be processed;

[0065] And, the calculation formula of the filter coefficient of each pixel to be processed in each image block to be processed includes:

[0066] pixelCurCoef k (i,j)=2 W -pixelRefCoef k (i,j)

[0067] Among them, pixelCurCoef k (i, j) represents the filtering coefficient of the pixel to be processed with coordinates (i, j) in the kth image block to be processed, and W represents the preset weight value accuracy.

[0068] In this optional embodiment, optionally, normalpixeldiffval k(i, j) represents the normalized result of the absolute value of the difference between the pixel to be processed with coordinates (i, j) and the corresponding reference pixel in the kth image block to be processed in the image frame to be processed in the range [-a, a]. The preset target range [-a, a] can be set by the user. For example, the preset target range [-a, a] can be [-0.5, 0.5]. pixelRefCoef k (i, j) represents the reference pixel filter coefficient of the reference pixel corresponding to the pixel to be processed with coordinates (i, j) in the kth image block to be processed, refblkCoef k The reference block filter coefficient of the best reference image block corresponding to the kth image block to be processed, exp(-normalpixeldiffval k (i, j)) represents the scaling factor corresponding to the pixel to be processed with coordinates (i, j) in the kth image block to be processed, pixelCurCoef k (i, j) represents the filtering coefficient of the pixel to be processed with coordinates (i, j) in the kth image block to be processed, and W represents the preset weight value accuracy.

[0069] It can be seen that the implementation of this optional embodiment can calculate the normalized result of the absolute value of the difference within the preset target range according to the absolute value of the difference, the minimum absolute value of the difference and the extreme absolute value difference of the difference, and then calculate the scaling factor of the exponential model according to the normalized result, and calculate the reference pixel point filtering coefficient and the pixel point filtering coefficient to be processed according to the scaling factor, thereby improving the calculation accuracy and efficiency of the pixel point filtering coefficient, thereby improving the accuracy and efficiency of filtering and denoising the image block to be processed, and reducing the probability of frame flickering and motion ghosting problems after video denoising.

[0070] In yet another optional embodiment, for each image block to be processed, for each pixel to be processed in the image block to be processed, filtering and denoising the pixel to be processed according to a pixel filter coefficient to be processed of the pixel to be processed and a reference pixel filter coefficient of a reference pixel corresponding to the pixel to be processed, and obtaining a denoised output result corresponding to the pixel to be processed may include the following operations:

[0071] For each image block to be processed, for each pixel to be processed in the image block to be processed, filtering and denoising is performed on the pixel to be processed according to the pixel filter coefficient to be processed, the reference pixel filter coefficient of the reference pixel corresponding to the pixel to be processed, and a preset filtering and denoising formula, to obtain a denoising output result corresponding to the pixel to be processed;

[0072] The filtering noise reduction formula includes:

[0073]

[0074] denoiseOut k (i,j)=clip(0,255,denoiseOut k (i,j))

[0075] Among them, denoiseOut k (i, j) represents the denoising output result of the pixel to be processed with coordinates (i, j) in the kth image block to be processed in the image frame to be processed, curpixelval k (i, j) represents the pixel value of the pixel to be processed with coordinates (i, j) in the kth image block to be processed, refpixelval k (i, j) represents the pixel value of the reference pixel corresponding to the pixel to be processed with coordinates (i, j) in the kth image block to be processed, clip(0,255,denoiseOut k (i, j)) means that the denoising output result of the pixel to be processed with coordinates (i, j) in the kth image block to be processed in the image frame to be processed is clamped within [0, 255].

[0076] In this optional embodiment, optionally, denoiseOut k (i, j) represents the denoising output result of the pixel to be processed with coordinates (i, j) in the kth image block to be processed in the image frame to be processed, W represents the preset weight value accuracy, the larger the W, the higher the accuracy. For example, W can be 4 or 6, which is not limited in this embodiment. clip(0,255,denoiseOut k (i, j)) means that the denoising output result of the pixel to be processed with coordinates (i, j) in the kth image block to be processed in the image frame to be processed is clamped within [0, 255]. For each image block to be processed, after obtaining the denoising output result corresponding to each pixel to be processed in the image block to be processed, the denoising output results of all the pixels of the image block to be processed are synthesized into a new image block to replace the image block to be processed, thereby completing the filtering and denoising of the image block to be processed.

[0077] It can be seen that the implementation of this optional embodiment can filter and denoise the pixel to be processed according to the preset filtering and denoising formula, obtain the denoising output result corresponding to the pixel to be processed, and clamp the denoising output result within [0,255]. The denoising output results of all pixels of the image block to be processed are synthesized into a new image block to replace the image block to be processed, thereby completing the filtering and denoising of the image block to be processed, which can improve the efficiency and accuracy of filtering and denoising of images and videos, and reduce the probability of frame flickering and motion ghosting problems after video denoising.

[0078] Embodiment 2

[0079] See also Figure 2 , Figure 2 is a flow chart of a method for video image denoising based on time domain information and weight factors disclosed in an embodiment of the present invention. Figure 2 The described video image denoising method based on time domain information and weight factors can be applied to a video image denoising device based on time domain information and weight factors, wherein the video image denoising device based on time domain information and weight factors may include an intelligent server or intelligent platform for denoising video images, and the intelligent server includes a local server or a cloud server, which is not limited in the embodiments of the present invention. Figure 2 As shown, the video image denoising method based on time domain information and weight factors may include the following operations:

[0080] 201. Obtain at least one to-be-processed image block in an to-be-processed image frame, divide the to-be-processed image frame and a reference image frame corresponding to the to-be-processed image frame into regions, and obtain at least one to-be-processed region corresponding to the to-be-processed image frame and at least one reference region corresponding to the reference image frame.

[0081] In an embodiment of the present invention, optionally, the reference image frame corresponding to the image frame to be processed may include at least the previous frame image of the image frame to be processed, that is, the reference image frame may include the previous frame image of the image frame to be processed, and may also include the previous multiple frames of images of the image frame to be processed. After the image frame to be processed and the reference image frame are divided into regions, at least one to-be-processed region corresponding to the image frame to be processed and at least one reference region corresponding to the reference image frame are the same in size and number, that is, the to-be-processed region corresponds to the reference region one-to-one, and the region size of the to-be-processed region is larger than the image block size of the image block to be processed, that is, one to-be-processed region may include at least one to-be-processed image block, which is not limited by the present invention.

[0082] 202. Count the motion vector histogram of each reference area in the reference image frame, and for each area to be processed, predict the motion or still type of the area to be processed based on the motion vector histogram of the reference area corresponding to the area to be processed, and determine the weight factor of the area to be processed based on the motion or still type of the area to be processed.

[0083] In an embodiment of the present invention, optionally, a motion vector histogram of each reference area in a reference image frame may be counted, and for each area to be processed, the motion or static type of the area to be processed may be predicted based on the motion vector histogram of the reference area corresponding to the area to be processed. Further optionally, the pixel difference between the current block at the same position in the previous frame and the reference block at the same position may be counted as a metric, which is not limited by the present invention.

[0084] In an embodiment of the present invention, optionally, the static and dynamic type of the area to be processed may include a motion type or a static type. Affected by noise, the similarity between the reference block and the current block is relatively large. If the original Lambda is used, the motion vector of the background area will fluctuate slightly. In order to reduce the influence of noise on the motion estimation of the background area, the present invention proposes a local Lambda strategy, which divides the previous frame into multiple areas, counts the motion vector histogram of each area, and adjusts the Lambda according to the motion characteristics of each area. If the noise intensity of the current frame is relatively large and the area where the current block is located belongs to the static area, the Lambda of the current area is relatively large. If the area where the current block is located belongs to the motion area, the Lambda of the current area remains unchanged. Specifically, the best reference image block corresponding to the image block to be processed can be determined based on the strategy of the local weight factor, that is, different weight factors of the area to be processed are determined according to the static and dynamic type of the area to be processed. Specifically, optionally, the weight factor of the area to be processed determined according to the static and dynamic type of the area to be processed can specifically include:

[0085] When the static and dynamic type of the area to be processed includes a motion type, the weight factor of the area to be processed is calculated according to a first formula, wherein the first formula includes:

[0086]

[0087] When the static type of the area to be processed includes a motion type, the weight factor of the area to be processed is calculated according to the first formula, wherein the second formula includes:

[0088]

[0089] Among them, LambdaOri represents the weight factor corresponding to the motion type of the area to be processed, deltaOri and alphaOri represent the auxiliary weight factors corresponding to the motion type of the area to be processed, LambdaOpt represents the weight factor corresponding to the static type of the area to be processed, deltaOpt and alphaOpt represent the auxiliary weight factors corresponding to the static type of the area to be processed, and qp represents the preset quantization coefficient.

[0090] In an embodiment of the present invention, optionally, for example, when the dynamic and static types of the area to be processed include a motion type, deltaOri=13.7122, alphaOri=4.2005; when the static type of the area to be processed includes a motion type, deltaOpt=13.5, alphaOpt=4.2; the present invention does not limit this.

[0091] 203. For each image block to be processed, calculate the cost between the image block to be processed and each reference image block in the corresponding reference image frame according to the weight factor of the area to be processed corresponding to the image block to be processed, and determine the reference image block corresponding to the cost that meets the preset selection condition as the best reference image block corresponding to the image block to be processed.

[0092] In an embodiment of the present invention, optionally, the cost that satisfies the preset selection condition may include the minimum value of the cost between the image block to be processed and each reference image block in the corresponding reference image frame, that is, the reference image block corresponding to the minimum cost is determined as the best reference image block corresponding to the image block to be processed, and the present invention is not limited thereto.

[0093] 204. For each image block to be processed, calculate the time domain information between the image block to be processed and the corresponding best reference image block, determine the motion intensity of the image block to be processed based on the time domain information corresponding to the image block to be processed, and determine the reference block filter coefficient of the best reference image block corresponding to the image block to be processed based on the motion intensity.

[0094] 205. For each image block to be processed, the reference pixel point filter coefficient of each reference pixel in the best reference image block corresponding to the image block to be processed is calculated, and the processed pixel point filter coefficient of each reference pixel in the image block to be processed is calculated based on the reference pixel point filter coefficient of each reference pixel.

[0095] 206. For each image block to be processed and for each pixel to be processed in the image block to be processed, filtering and denoising is performed on the pixel to be processed according to a filtering coefficient of the pixel to be processed and a reference pixel filtering coefficient of a reference pixel corresponding to the pixel to be processed to obtain a denoised output result corresponding to the pixel to be processed.

[0096] In the embodiment of the present invention, for other descriptions of step 204 to step 206, please refer to the detailed description of step 102 to step 104 in the first embodiment, and the embodiment of the present invention will not be repeated here.

[0097] It can be seen that the implementation Figure 1The described video image denoising method based on time domain information and weight factors can divide the image frame to be processed and the reference image frame into regions, calculate the motion vector histogram of each reference region in the reference image frame, and predict the motion and still type of the area to be processed based on the motion vector histogram, thereby improving the efficiency of determining the motion and still type of the area to be processed, determining the weight factor of the area to be processed based on the motion and still type of the area to be processed, and calculating the cost between the image block to be processed and each reference image block in the corresponding reference image frame based on the weight factor, determining the reference image block corresponding to the cost that meets the preset selection condition as the best reference image block corresponding to the image block to be processed, and being able to determine the best reference image block corresponding to the image block to be processed through the strategy of local weight factors, thereby improving the accuracy and efficiency of determining the best reference image block and improving the denoising of the image block to be processed. Accuracy and efficiency, determining the best reference image block corresponding to the image block to be processed in the image frame to be processed according to a preset weight factor, determining the motion intensity of the image block to be processed according to the time domain information between the image block to be processed and the corresponding best reference image block, and determining the reference block filter coefficient of the best reference image block corresponding to the image block to be processed according to the motion intensity, calculating the reference pixel point filter coefficient of each reference pixel point in the best reference image block and the to-be-processed pixel point filter coefficient of each to-be-processed pixel point in the image block to be processed according to the reference block filter coefficient, filtering and denoising the to-be-processed pixel point according to the to-be-processed pixel point filter coefficient and the reference pixel point filter coefficient, obtaining the denoising output result corresponding to the to-be-processed pixel point, which can improve the filtering and denoising efficiency and accuracy of image and video, and reduce the probability of frame flickering and motion smearing problems after video denoising.

[0098] In an optional embodiment, for each image block to be processed, determining the motion intensity of the image block to be processed according to the time domain information corresponding to the image block to be processed may include the following operations:

[0099] Obtaining a level threshold corresponding to the time domain information, where the level threshold includes a similarity level threshold and a motion vector level threshold;

[0100] For each image block to be processed, determining the motion intensity of the image block to be processed according to a target relationship between the time domain information corresponding to the image block to be processed and the level threshold, the target relationship including a first relationship between the similarity of the image block to be processed and the similarity level threshold and a second relationship between the motion vector of the image block to be processed and the motion vector level threshold;

[0101] The first relationship includes:

[0102]

[0103] The second relationship includes:

[0104]

[0105] The motion intensity of the image block to be processed includes:

[0106]

[0107] Among them, sadStatus k represents the first relationship corresponding to the kth image block to be processed in the image frame to be processed, sad k Indicates the similarity between the kth image block to be processed in the image frame to be processed and the best reference image block, mvStatus k represents the second relationship corresponding to the kth image block to be processed in the image frame to be processed, mv k represents the motion vector between the kth image block to be processed and the best reference image block in the image frame to be processed, moveStatus k represents the motion intensity of the kth image block to be processed in the image frame to be processed, sadTh0-sadTh3 represent the similarity level threshold, and mvTh0-mvTh3 represent the motion vector level threshold.

[0108] In this optional embodiment, optionally, the level threshold corresponding to the time domain information may be pre-set by the user according to the actual video image noise reduction requirements, or may be automatically set by the system according to the historical video image noise reduction records. The level threshold may include a similarity level threshold and a motion vector level threshold. The target relationship between the time domain information corresponding to the image block to be processed and the level threshold may include a first relationship and a second relationship. The first relationship may represent the size relationship between the similarity of the image block to be processed and the similarity level threshold, and the second relationship may include the size relationship between the motion vector of the image block to be processed and the motion vector level threshold. Specifically, the first relationship may include:

[0109]

[0110] Secondary relationships may include:

[0111]

[0112] The motion intensity of the image block to be processed may include:

[0113]

[0114] Among them, sadStatus k represents the first relationship corresponding to the kth image block to be processed in the image frame to be processed, sad k Indicates the similarity between the kth image block to be processed in the image frame to be processed and the best reference image block, mvStatus krepresents the second relationship corresponding to the kth image block to be processed in the image frame to be processed, mv k represents the motion vector between the kth image block to be processed and the best reference image block in the image frame to be processed, moveStatus k represents the motion intensity of the kth image block to be processed in the image frame to be processed, sadTh0-sadTh3 represent the similarity level threshold, and mvTh0-mvTh3 represent the motion vector level threshold.

[0115] In this optional embodiment, optionally, the calculation formula for the similarity between the kth image block to be processed in the image frame to be processed and the best reference image block is:

[0116]

[0117] Among them, curpixelval k (i, j) represents the pixel value of the pixel to be processed with coordinates (i, j) in the kth image block to be processed, refpixelval k (i, j) represents the pixel value of the reference pixel corresponding to the pixel to be processed with coordinates (i, j) in the kth image block to be processed, and L and K represent the number of pixels in the horizontal and vertical directions in the kth image block to be processed, respectively;

[0118] The calculation formula of the motion vector between the kth image block to be processed and the best reference image block in the image frame to be processed is:

[0119] mv k =abs(mvx)+abs(mvy)

[0120] Wherein, mvx and mvy represent the motion vector components of the kth image block to be processed in the x direction and the y direction respectively.

[0121] In this optional embodiment, optionally, for each image block to be processed, a reference block filter coefficient of an optimal reference image block corresponding to the image block to be processed may be selected from a preset filter coefficient table according to the motion intensity of the image block to be processed, wherein the filter coefficient table may include:

[0122] Filter coefficient table

[0123] <![CDATA[mvStatus k ]]> <![CDATA[refblkCoef k ]]> 0 14 1 10 2 8 3 6 4 0

[0124] Among them, refblkCoef kRepresents the reference block filter coefficient of the best reference image block corresponding to the k-th image block to be processed. Optionally, refblkCoef decreases as the strength of moveStatus increases. The data in the filter coefficient table is only for exemplifying the corresponding relationship between refblkCoef and moveStatus. This embodiment does not limit the specific values ​​of refblkCoef and moveStatus.

[0125] In this optional embodiment, optionally, a reference block filter coefficient refblkCoef of the best reference image block corresponding to the kth image block to be processed is determined. k After that, the filter coefficient of the image block to be processed corresponding to the kth image block to be processed is (N-refblkCoef k ), where N = 2 M , M represents the bit width.

[0126] It can be seen that implementing this optional embodiment can obtain the level threshold corresponding to the time domain information, the level threshold includes the similarity level threshold and the motion vector level threshold, and determine the motion intensity of the image block to be processed according to the target relationship between the time domain information corresponding to the image block to be processed and the level threshold. The motion intensity of the image block to be processed can be determined according to the relationship between the similarity and motion vector corresponding to the image block to be processed and the corresponding threshold level, which can improve the accuracy and efficiency of determining the motion intensity of the image block to be processed, thereby improving the denoising efficiency of the video image.

[0127] In another optional embodiment, for each image block to be processed, according to the weight factor of the area to be processed corresponding to the image block to be processed, calculating the cost between the image block to be processed and each reference image block in the reference image frame may include the following operations:

[0128] For each image block to be processed, calculating reference time domain information between the image block to be processed and each reference image block in the reference image frame;

[0129] For each image block to be processed, according to the weight factor of the area to be processed corresponding to the image block to be processed and the reference time domain information, the cost between the image block to be processed and each reference image block in the reference image frame is calculated according to a preset cost calculation formula;

[0130] The cost calculation formula includes:

[0131] cost k，p =sad k，p +mvbits k，p Lambda k

[0132] Among them, cost k，prepresents the cost between the kth image block to be processed in the image frame to be processed and the pth reference image block in the reference image frame, sad k，p Indicates the similarity between the kth image block to be processed in the image frame to be processed and the pth reference image block in the reference image frame, mvbits k，p represents the number of bits of the motion vector between the kth image block to be processed in the image frame to be processed and the pth reference image block in the reference image frame, Lambda k represents the weight factor of the to-be-processed area corresponding to the kth to-be-processed image block in the to-be-processed image frame;

[0133] And, the calculation formula of the weight factor includes:

[0134] lambda k =exp((qp-delta k ) / alpha k )

[0135] Among them, qp represents the preset quantization coefficient, delta k and alpha k They respectively represent the auxiliary weight factors of the area to be processed corresponding to the kth image block to be processed in the image frame to be processed.

[0136] In this optional embodiment, optionally, the reference time domain information between the image block to be processed and each reference image block in the reference image frame may include the similarity and motion vector between the image block to be processed and each reference image block in the reference image frame, cost k，p represents the cost between the kth image block to be processed in the image frame to be processed and the pth reference image block in the reference image frame, sad k，p Indicates the similarity between the kth image block to be processed in the image frame to be processed and the pth reference image block in the reference image frame, mvbits k，p Indicates the number of bits of the motion vector between the kth image block to be processed in the image frame to be processed and the pth reference image block in the reference image frame.

[0137] In this optional embodiment, optionally, the best reference image block corresponding to the image block to be processed can be determined based on the strategy of the local weight factor, that is, different weight factors of the area to be processed are determined according to the static and dynamic type of the area to be processed. When the static and dynamic type of the area to be processed is a motion type, the calculation formula of the weight factor is:

[0138]

[0139] When the static type of the area to be processed is static, the calculation formula of the weight factor is:

[0140]

[0141] Among them, Lambda k Lambda represents the weight factor of the area to be processed corresponding to the kth image block to be processed in the image frame to be processed. k Include LambdaOri or LambdaOpt, delta k and alpha k They represent the auxiliary weight factors of the processing area corresponding to the kth image block to be processed in the image frame to be processed, delta k Includes deltaOri or deltaOpt, alpha k Includes alphaOri or alphaOpt, and qp represents the preset quantization coefficient.

[0142] In an embodiment of the present invention, optionally, for example, when the dynamic and static types of the area to be processed include a motion type, deltaOri=13.7122, alphaOri=4.2005; when the static type of the area to be processed includes a motion type, deltaOpt=13.5, alphaOpt=4.2; the present invention does not limit this.

[0143] It can be seen that implementing this optional embodiment can calculate the reference time domain information between the image block to be processed and each reference image block in the reference image frame. For each image block to be processed, the cost between the image block to be processed and each reference image block in the reference image frame is calculated according to the weight factor of the area to be processed corresponding to the image block to be processed and the reference time domain information, according to a preset cost calculation formula. The weight factor can be adjusted according to the dynamic and static states of the area to be processed based on a strategy of local weight factors, and the cost can be calculated based on the weight factor, similarity and number of motion vector bits. This can improve the accuracy and efficiency of determining the best reference image block for the image block to be processed, thereby improving the accuracy and efficiency of subsequent denoising of the image block to be processed.

[0144] Embodiment 3

[0145] See also Figure 3 , Figure 3 : is a schematic diagram of the structure of a video image noise reduction device based on time domain information and weight factors disclosed in an embodiment of the present invention. Figure 3 The described video image denoising device based on time domain information and weight factors may include an intelligent server or intelligent platform for denoising video images, and the intelligent server includes a local server or a cloud server, which is not limited in the embodiment of the present invention. Figure 3 As shown, the video image noise reduction device based on time domain information and weight factors may include:

[0146] The acquisition module 301 is used to acquire at least one image block to be processed in the image frame to be processed, and for each image block to be processed, determine the best reference image block corresponding to the image block to be processed in the reference image frame corresponding to the image frame to be processed according to a preset weight factor;

[0147] A calculation module 302 is used to calculate, for each image block to be processed, time domain information between the image block to be processed and a corresponding best reference image block, where the time domain information includes similarity and motion vector;

[0148] The determination module 303 is used to determine the motion intensity of each image block to be processed according to the time domain information corresponding to the image block to be processed, and determine the reference block filter coefficient of the best reference image block corresponding to the image block to be processed according to the motion intensity;

[0149] The calculation module 302 is further configured to calculate, for each image block to be processed, a reference pixel point filter coefficient of each reference pixel point in the best reference image block corresponding to the image block to be processed according to the reference block point filter coefficient of the best reference image block to be processed, and calculate a to-be-processed pixel point filter coefficient of each to-be-processed pixel point in the image block to be processed according to the reference pixel point filter coefficient of each reference pixel point;

[0150] The denoising module 304 is used to filter and denoise each pixel to be processed in each image block to be processed, according to a filter coefficient of the pixel to be processed and a reference pixel filter coefficient of a reference pixel corresponding to the pixel to be processed, to obtain a denoised output result corresponding to the pixel to be processed.

[0151] It can be seen that the implementation Figure 3 The described video image denoising device based on time domain information and weight factors can determine the best reference image block corresponding to the image block to be processed in the image frame to be processed according to the preset weight factor, and determine the motion intensity of the image block to be processed according to the time domain information between the image block to be processed and the corresponding best reference image block, and determine the reference block filter coefficient of the best reference image block corresponding to the image block to be processed according to the motion intensity, calculate the reference pixel point filter coefficient of each reference pixel point in the best reference image block and the to-be-processed pixel point filter coefficient of each to-be-processed pixel point in the image block to be processed according to the reference block filter coefficient, filter and denoise the to-be-processed pixel point according to the to-be-processed pixel point filter coefficient and the reference pixel point filter coefficient, and obtain the denoising output result corresponding to the to-be-processed pixel point, calculate the time domain information in blocks as units, and fewer reference frames can reduce the occupancy of additional cache data, improve the filtering and denoising efficiency and accuracy of image and video, and reduce the probability of inter-frame flickering and motion smearing problems after video denoising.

[0152] In an optional embodiment, if Figure 3 As shown, the specific manner in which the acquisition module 301 determines the best reference image block corresponding to the image block to be processed in the reference image frame corresponding to the image frame to be processed according to the preset weight factor for each image block to be processed includes:

[0153] Performing region division on the image frame to be processed and the reference image frame corresponding to the image frame to be processed to obtain at least one region to be processed corresponding to the image frame to be processed and at least one reference region corresponding to the reference image frame, wherein the reference image frame at least includes a previous frame image corresponding to the image frame to be processed, and the region size of the region to be processed is larger than the image block size of the image block to be processed;

[0154] Counting the motion vector histogram of each reference area in the reference image frame, for each area to be processed, predicting the motion and stillness type of the area to be processed according to the motion vector histogram of the reference area corresponding to the area to be processed, and determining the weight factor of the area to be processed according to the motion and stillness type of the area to be processed, where the motion and stillness type includes a motion type or a stillness type;

[0155] For each image block to be processed, the cost between the image block to be processed and each reference image block in the corresponding reference image frame is calculated according to the weight factor of the area to be processed corresponding to the image block to be processed, and the reference image block corresponding to the cost that meets the preset selection condition is determined as the best reference image block corresponding to the image block to be processed.

[0156] It can be seen that the implementation Figure 3The described video image denoising device based on time domain information and weight factors can divide the image frame to be processed and the reference image frame into regions, calculate the motion vector histogram of each reference region in the reference image frame, and predict the motion and still type of the area to be processed based on the motion vector histogram, thereby improving the efficiency of determining the motion and still type of the area to be processed, determining the weight factor of the area to be processed based on the motion and still type of the area to be processed, and calculating the cost between the image block to be processed and each reference image block in the corresponding reference image frame based on the weight factor, determining the reference image block corresponding to the cost that meets the preset selection condition as the best reference image block corresponding to the image block to be processed, and being able to determine the best reference image block corresponding to the image block to be processed through the strategy of local weight factors, thereby improving the accuracy and efficiency of determining the best reference image block and improving the denoising of the image block to be processed. Accuracy and efficiency, determining the best reference image block corresponding to the image block to be processed in the image frame to be processed according to a preset weight factor, determining the motion intensity of the image block to be processed according to the time domain information between the image block to be processed and the corresponding best reference image block, and determining the reference block filter coefficient of the best reference image block corresponding to the image block to be processed according to the motion intensity, calculating the reference pixel point filter coefficient of each reference pixel point in the best reference image block and the to-be-processed pixel point filter coefficient of each to-be-processed pixel point in the image block to be processed according to the reference block filter coefficient, filtering and denoising the to-be-processed pixel point according to the to-be-processed pixel point filter coefficient and the reference pixel point filter coefficient, obtaining the denoising output result corresponding to the to-be-processed pixel point, which can improve the filtering and denoising efficiency and accuracy of image and video, and reduce the probability of frame flickering and motion smearing problems after video denoising.

[0157] In another optional embodiment, Figure 3 As shown, the specific manner in which the determination module 303 determines the motion intensity of each image block to be processed according to the time domain information corresponding to the image block to be processed includes:

[0158] Obtaining a level threshold corresponding to the time domain information, where the level threshold includes a similarity level threshold and a motion vector level threshold;

[0159] For each image block to be processed, determining the motion intensity of the image block to be processed according to a target relationship between the time domain information corresponding to the image block to be processed and the level threshold, the target relationship including a first relationship between the similarity of the image block to be processed and the similarity level threshold and a second relationship between the motion vector of the image block to be processed and the motion vector level threshold;

[0160] The first relationship includes:

[0161]

[0162] The second relationship includes:

[0163]

[0164] The motion intensity of the image block to be processed includes:

[0165]

[0166] Among them, sadStatus k represents the first relationship corresponding to the kth image block to be processed in the image frame to be processed, sad k Indicates the similarity between the kth image block to be processed in the image frame to be processed and the best reference image block, mvStatus k represents the second relationship corresponding to the kth image block to be processed in the image frame to be processed, mv k represents the motion vector between the kth image block to be processed and the best reference image block in the image frame to be processed, moveStatus k represents the motion intensity of the kth image block to be processed in the image frame to be processed, sadTh0-sadTh3 represent the similarity level threshold, and mvTh0-mvTh3 represent the motion vector level threshold.

[0167] It can be seen that the implementation Figure 3 The described video image denoising device based on time domain information and weight factors can obtain the level threshold corresponding to the time domain information, the level threshold includes the similarity level threshold and the motion vector level threshold, and determine the motion intensity of the image block to be processed according to the target relationship between the time domain information corresponding to the image block to be processed and the level threshold. It can determine the motion intensity of the image block to be processed according to the relationship between the similarity and motion vector corresponding to the image block to be processed and the corresponding threshold level, which can improve the accuracy and efficiency of determining the motion intensity of the image block to be processed, thereby improving the denoising efficiency of the video image.

[0168] In yet another optional embodiment, Figure 3 As shown, the specific manner in which the calculation module 302 calculates the reference pixel point filter coefficient of each reference pixel point in the best reference image block for each image block to be processed according to the reference block filter coefficient of the best reference image block corresponding to the image block to be processed includes:

[0169] For each image block to be processed, for each pixel to be processed in the image block to be processed, calculating the absolute value of the difference between the pixel to be processed and each corresponding reference pixel in the best reference image block;

[0170] For each image block to be processed, determine the difference absolute value extreme value between the image block to be processed and the corresponding best reference image block in each difference absolute value, and calculate the difference absolute value extreme value difference according to the difference absolute value extreme value, wherein the difference absolute value extreme value includes the difference absolute value maximum value and the difference absolute value minimum value;

[0171] For each image block to be processed, for each pixel to be processed in the image block to be processed, according to the absolute value of the difference between the pixel to be processed and the corresponding reference pixel, the minimum absolute value of the difference, and the extreme absolute value of the difference, a normalized result of the absolute value of the difference between the pixel to be processed and the corresponding reference pixel within a preset target range is calculated;

[0172] For each image block to be processed and for each pixel to be processed in the image block to be processed, a scaling coefficient corresponding to the pixel to be processed is calculated based on a normalization result between the pixel to be processed and the corresponding reference pixel, and a reference pixel point filtering coefficient for each reference pixel in the best reference image block corresponding to the image block to be processed is calculated based on the scaling coefficient and the reference block filtering coefficient of the best reference image block corresponding to the image block to be processed.

[0173] It can be seen that the implementation Figure 3 The described video image denoising device based on time domain information and weight factors can calculate the absolute value of the difference between the pixel to be processed and each corresponding reference pixel in the best reference image block, and determine the absolute value extreme value of the difference and the absolute value extreme value difference of the difference. According to the absolute value of the difference, the minimum absolute value of the difference and the absolute value extreme value difference of the difference, the normalized result of the absolute value of the difference within the preset target range is calculated. According to the normalized result between the pixel to be processed and the corresponding reference pixel, the scaling coefficient corresponding to the pixel to be processed is calculated, and according to the scaling coefficient and the reference block filtering coefficient of the best reference image block corresponding to the image block to be processed, the reference pixel point filtering coefficient of each reference pixel in the best reference image block corresponding to the image block to be processed is calculated. The scaling coefficient of the exponential model can be calculated according to the absolute value of the difference of the image block to be processed, and the reference pixel point filtering coefficient of each reference pixel is calculated according to the scaling coefficient, so as to improve the accuracy of the calculated reference pixel point filtering coefficient, thereby improving the accuracy and efficiency of filtering and denoising the image block to be processed, and reducing the probability of inter-frame flickering and motion smearing problems after video denoising.

[0174] In yet another optional embodiment, Figure 3 As shown, the calculation formula for the normalized result includes:

[0175]

[0176] Among them, normalpixeldiffvalk (i, j) represents the normalized result of the absolute value of the difference between the pixel to be processed with coordinates (i, j) and the corresponding reference pixel in the kth image block to be processed in the image frame to be processed, in the range [-a, a], pixeldiffval k (i, j) represents the absolute value of the difference between the pixel to be processed with coordinates (i, j) in the kth image block to be processed and the corresponding reference pixel. minpixeldiff k Indicates the minimum absolute value of the difference corresponding to the kth image block to be processed, gap k represents the absolute value extreme value difference of the difference corresponding to the kth image block to be processed;

[0177] And, the calculation formula of the reference pixel point filter coefficient of each reference pixel point in the best reference image block corresponding to each image block to be processed includes:

[0178] pixelRefCoef k (i,j)=refblkCoef k *exp(-normalpixeldiffval k (i,j))

[0179] Among them, pixelRefCoef k (i, j) represents the reference pixel filter coefficient of the reference pixel corresponding to the pixel to be processed with coordinates (i, j) in the kth image block to be processed, refblkCoef k The reference block filter coefficient of the best reference image block corresponding to the kth image block to be processed, exp(-normalpixeldiffval k (i, j)) represents the scaling factor corresponding to the pixel to be processed with coordinates (i, j) in the kth image block to be processed;

[0180] And, the calculation formula of the filter coefficient of each pixel to be processed in each image block to be processed includes:

[0181] pixelCurCoef k (i,j)=2 W -pixelRefCoef k (i,j)

[0182] Among them, pixelCurCoef k (i, j) represents the filtering coefficient of the pixel to be processed with coordinates (i, j) in the kth image block to be processed, and W represents the preset weight value accuracy.

[0183] It can be seen that the implementation Figure 3The described video image denoising device based on time domain information and weight factors can calculate the normalized result of the absolute value of the difference within a preset target range according to the absolute value of the difference, the minimum absolute value of the difference and the extreme absolute value difference of the difference, and then calculate the scaling factor of the exponential model according to the normalized result, and calculate the reference pixel point filtering coefficient and the pixel point filtering coefficient to be processed according to the scaling factor, thereby improving the calculation accuracy and efficiency of the pixel point filtering coefficient, thereby improving the accuracy and efficiency of filtering and denoising the image block to be processed, and reducing the probability of frame flickering and motion ghosting problems after video denoising.

[0184] In yet another optional embodiment, Figure 3 As shown, the noise reduction module 304 performs filtering and noise reduction on each to-be-processed pixel in each to-be-processed image block according to the to-be-processed pixel filter coefficient of the to-be-processed pixel and the reference pixel filter coefficient of the reference pixel corresponding to the to-be-processed pixel, and obtains the noise reduction output result corresponding to the to-be-processed pixel in a specific manner including:

[0185] For each image block to be processed, for each pixel to be processed in the image block to be processed, filtering and denoising is performed on the pixel to be processed according to the pixel filter coefficient to be processed, the reference pixel filter coefficient of the reference pixel corresponding to the pixel to be processed, and a preset filtering and denoising formula, to obtain a denoising output result corresponding to the pixel to be processed;

[0186] The filtering noise reduction formula includes:

[0187]

[0188] denoiseOut k (i,j)=clip(0,255,denoiseOut k (i,j))

[0189] Among them, denoiseOut k (i, j) represents the denoising output result of the pixel to be processed with coordinates (i, j) in the kth image block to be processed in the image frame to be processed, curpixelval k (i, j) represents the pixel value of the pixel to be processed with coordinates (i, j) in the kth image block to be processed, refpixelval k (i, j) represents the pixel value of the reference pixel corresponding to the pixel to be processed with coordinates (i, j) in the kth image block to be processed, clip(0,255,denoiseOut k(i, j)) means that the denoising output result of the pixel to be processed with coordinates (i, j) in the kth image block to be processed in the image frame to be processed is clamped within [0, 255].

[0190] It can be seen that the implementation Figure 3 The described video image denoising device based on time domain information and weight factors can perform filtering and denoising on the pixel to be processed according to a preset filtering and denoising formula, obtain the denoising output result corresponding to the pixel to be processed, and clamp the denoising output result within [0,255]. The denoising output results of all pixels of the image block to be processed are synthesized into a new image block to replace the image block to be processed, thereby completing the filtering and denoising of the image block to be processed. The filtering and denoising efficiency and accuracy of the image video can be improved, and the probability of inter-frame flickering and motion ghosting problems after video denoising is reduced.

[0191] In yet another optional embodiment, Figure 3 As shown, the specific manner in which the acquisition module 301 calculates the cost between each image block to be processed and each reference image block in the reference image frame according to the weight factor of the area to be processed corresponding to the image block to be processed includes:

[0192] For each image block to be processed, calculating reference time domain information between the image block to be processed and each reference image block in the reference image frame;

[0193] For each image block to be processed, according to the weight factor of the area to be processed corresponding to the image block to be processed and the reference time domain information, the cost between the image block to be processed and each reference image block in the reference image frame is calculated according to a preset cost calculation formula;

[0194] The cost calculation formula includes:

[0195] cost k，p =sad k，p +mvbits k，p Lambda k

[0196] Among them, cost k，p represents the cost between the kth image block to be processed in the image frame to be processed and the pth reference image block in the reference image frame, sad k，p Indicates the similarity between the kth image block to be processed in the image frame to be processed and the pth reference image block in the reference image frame, mvbits k，p represents the number of bits of the motion vector between the kth image block to be processed in the image frame to be processed and the pth reference image block in the reference image frame, Lambda krepresents the weight factor of the to-be-processed area corresponding to the kth to-be-processed image block in the to-be-processed image frame;

[0197] And, the calculation formula of the weight factor includes:

[0198] lambda k =exp((qp-delta k ) / alpha k )

[0199] Among them, qp represents the preset quantization coefficient, delta k and alpha k They respectively represent the auxiliary weight factors of the area to be processed corresponding to the kth image block to be processed in the image frame to be processed.

[0200] It can be seen that the implementation Figure 3 The described video image denoising device based on time domain information and weight factors can calculate the reference time domain information between the image block to be processed and each reference image block in the reference image frame. For each image block to be processed, the cost between the image block to be processed and each reference image block in the reference image frame is calculated according to the weight factor of the area to be processed corresponding to the image block to be processed and the reference time domain information, according to a preset cost calculation formula. The weight factor can be adjusted according to the dynamic and static states of the area to be processed based on a strategy of local weight factors, and the cost can be calculated based on the weight factor, similarity and number of motion vector bits. The accuracy and efficiency of determining the best reference image block for the image block to be processed can be improved, thereby improving the accuracy and efficiency of subsequent denoising of the image block to be processed.

[0201] Embodiment 4

[0202] See also Figure 4 , Figure 4 FIG. 1 is a schematic diagram of the structure of another video image noise reduction device based on time domain information and weight factors disclosed in an embodiment of the present invention. Figure 4 As shown, the video image noise reduction device based on time domain information and weight factors may include:

[0203] A memory 401 storing executable program codes;

[0204] a processor 402 coupled to the memory 401;

[0205] The processor 402 calls the executable program code stored in the memory 401 to execute the steps in the video image denoising method based on time domain information and weight factors described in the first embodiment of the present invention or the second embodiment of the present invention.

[0206] Embodiment 5

[0207] An embodiment of the present invention discloses a computer storage medium, which stores computer instructions. When the computer instructions are called, they are used to execute the steps in the video image denoising method based on time domain information and weight factors described in Embodiment 1 or Embodiment 2 of the present invention.

[0208] Embodiment 6

[0209] An embodiment of the present invention discloses a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program, and the computer program is operable to enable a computer to execute the steps in the video image denoising method based on time domain information and weight factors described in Embodiment 1 or Embodiment 2.

[0210] The device embodiments described above are only illustrative, wherein the modules described as separate components may or may not be physically separated, and the components displayed as modules may or may not be physical modules, i.e., they may be located in one place, or they may be distributed on multiple network modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Those of ordinary skill in the art may understand and implement it without creative work.

[0211] Through the specific description of the above embodiments, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on such an understanding, the above technical solution can be essentially or partly contributed to the prior art in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, and the storage medium includes a read-only memory (ROM), a random access memory (RAM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), a one-time programmable read-only memory (OTPROM), an electronically erasable rewritable read-only memory (EEPROM), a compact disc (CD-ROM) or other optical disc storage, magnetic disk storage, magnetic tape storage, or any other computer-readable medium that can be used to carry or store data.

[0212] Finally, it should be noted that the method and device for video image denoising based on time domain information and weight factors disclosed in the embodiments of the present invention are only preferred embodiments of the present invention, and are only used to illustrate the technical solution of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A video image denoising method based on time domain information and weight factors, characterized in that: The method comprises: Acquire at least one image block to be processed in the image frame to be processed, and for each of the image blocks to be processed, determine, in a reference image frame corresponding to the image frame to be processed, an optimal reference image block corresponding to the image block to be processed according to a preset weight factor; For each of the image blocks to be processed, calculating the time domain information between the image block to be processed and the corresponding best reference image block, wherein the time domain information includes similarity and motion vector; For each of the image blocks to be processed, determining the motion intensity of the image block to be processed according to the time domain information corresponding to the image block to be processed, and determining the reference block filter coefficient of the best reference image block corresponding to the image block to be processed according to the motion intensity; For each of the image blocks to be processed, a reference pixel point filter coefficient of each reference pixel point in the best reference image block corresponding to the image block to be processed is calculated, and a pixel point filter coefficient of each pixel point to be processed in the image block to be processed is calculated according to the reference pixel point filter coefficient of each reference pixel point; For each of the image blocks to be processed and each of the pixel points to be processed in the image blocks to be processed, filtering and denoising is performed on the pixel point to be processed according to the pixel point filtering coefficient to be processed and the reference pixel point filtering coefficient of the reference pixel point corresponding to the pixel point to be processed, so as to obtain a denoised output result corresponding to the pixel point to be processed.

2. The video image denoising method based on time domain information and weight factors according to claim 1, characterized in that: For each of the image blocks to be processed, determining the best reference image block corresponding to the image block to be processed in the reference image frame corresponding to the image frame to be processed according to a preset weight factor, comprising: Performing region division on the image frame to be processed and a reference image frame corresponding to the image frame to be processed to obtain at least one region to be processed corresponding to the image frame to be processed and at least one reference region corresponding to the reference image frame, wherein the reference image frame at least includes a previous frame image corresponding to the image frame to be processed, and the region size of the region to be processed is larger than the image block size of the image block to be processed; Counting the motion vector histogram of each reference area in the reference image frame, for each area to be processed, predicting the motion or still type of the area to be processed according to the motion vector histogram of the reference area corresponding to the area to be processed, and determining the weight factor of the area to be processed according to the motion or still type of the area to be processed, wherein the motion or still type includes a motion type or a still type; For each of the image blocks to be processed, the cost between the image block to be processed and each reference image block in the corresponding reference image frame is calculated according to the weight factor of the area to be processed corresponding to the image block to be processed, and the reference image block corresponding to the cost that meets the preset selection condition is determined as the best reference image block corresponding to the image block to be processed.

3. The video image denoising method based on time domain information and weight factors according to claim 2, characterized in that: The step of determining, for each of the image blocks to be processed, the motion intensity of the image block to be processed according to the time domain information corresponding to the image block to be processed, includes: Acquire a level threshold corresponding to the time domain information, wherein the level threshold includes a similarity level threshold and a motion vector level threshold; For each of the image blocks to be processed, determining the motion intensity of the image block to be processed according to a target relationship between the time domain information corresponding to the image block to be processed and the level threshold, wherein the target relationship includes a first relationship between the similarity of the image block to be processed and the similarity level threshold and a second relationship between the motion vector of the image block to be processed and the motion vector level threshold; The first relationship includes: The second relationship includes: The motion intensity of the image block to be processed includes: Among them, sadStatus k represents the first relationship corresponding to the kth image block to be processed in the image frame to be processed, sad k Indicates the similarity between the kth image block to be processed in the image frame to be processed and the best reference image block, mvStatus k represents the second relationship corresponding to the kth image block to be processed in the image frame to be processed, mv k represents the motion vector between the kth image block to be processed in the image frame to be processed and the best reference image block, moveStatus k represents the motion intensity of the kth image block to be processed in the image frame to be processed, sadTh0-sadTh3 represent the similarity level threshold, and mvTh0-mvTh3 represent the motion vector level threshold.

4. The video image denoising method based on time domain information and weight factors according to any one of claims 1 to 3, characterized in that: The step of calculating, for each of the image blocks to be processed, a reference pixel point filtering coefficient of each reference pixel point in the best reference image block according to the reference block filtering coefficient of the best reference image block corresponding to the image block to be processed, comprises: For each of the to-be-processed image blocks, for each to-be-processed pixel point in the to-be-processed image block, calculating the absolute value of the difference between the to-be-processed pixel point and each corresponding reference pixel point in the best reference image block; For each of the image blocks to be processed, determining the difference absolute value extreme value between the image block to be processed and the corresponding optimal reference image block in each of the difference absolute values, and calculating the difference absolute value extreme value difference according to the difference absolute value extreme value, wherein the difference absolute value extreme value includes the difference absolute value maximum value and the difference absolute value minimum value; For each of the image blocks to be processed, for each of the pixel points to be processed in the image blocks to be processed, according to the absolute value of the difference between the pixel point to be processed and the corresponding reference pixel point, the minimum absolute value of the difference, and the extreme absolute value difference of the difference, a normalized result of the absolute value of the difference between the pixel point to be processed and the corresponding reference pixel point within a preset target range is calculated; For each of the image blocks to be processed and for each of the pixel points to be processed in the image block to be processed, a scaling coefficient corresponding to the pixel point to be processed is calculated based on the normalization result between the pixel point to be processed and the corresponding reference pixel point, and a reference pixel point filtering coefficient for each of the reference pixel points in the best reference image block corresponding to the image block to be processed is calculated based on the scaling coefficient and the reference block filtering coefficient of the best reference image block corresponding to the image block to be processed.

5. The video image denoising method based on time domain information and weight factors according to claim 4, characterized in that: The calculation formula of the normalized result includes: Among them, normalpixeldiffval k (i, j) represents the normalized result of the absolute value of the difference between the pixel to be processed with coordinates (i, j) in the kth image block to be processed in the image frame to be processed and the corresponding reference pixel in the range [-a, a], pixeldiffval k (i, j) represents the absolute value of the difference between the pixel to be processed with coordinates (i, j) in the kth image block to be processed and the corresponding reference pixel, minpixeldiff k represents the minimum absolute value of the difference corresponding to the kth image block to be processed, gap k represents the absolute value extreme value difference of the difference corresponding to the kth image block to be processed; And, the calculation formula of the reference pixel point filtering coefficient of each reference pixel point in the optimal reference image block corresponding to each image block to be processed includes: pixelRefCoef k (i,j)=refblkCoef k *exp(-normalpixeldiffval k (i,j)) Among them, pixelRefCoef k (i, j) represents the reference pixel filter coefficient of the reference pixel corresponding to the pixel to be processed with coordinates (i, j) in the kth image block to be processed, refblkCoef k represents the reference block filter coefficient of the best reference image block corresponding to the kth image block to be processed, exp(-normalpixeldiffval k (i, j)) represents the scaling factor corresponding to the pixel to be processed with coordinates (i, j) in the kth image block to be processed; And, the calculation formula of the filter coefficient of each pixel to be processed in each image block to be processed includes: pixelCurCoef k (i,j)=2 W -pixelRefCoef k (i,j) Among them, pixelCurCoef k (i, j) represents the filtering coefficient of the pixel to be processed with coordinates (i, j) in the kth image block to be processed, and W represents the preset weight value accuracy.

6. The video image denoising method based on time domain information and weight factors according to claim 5, characterized in that: For each of the image blocks to be processed, for each of the pixels to be processed in the image blocks to be processed, filtering and denoising the pixels to be processed according to the filtering coefficient of the pixels to be processed and the filtering coefficient of the reference pixels corresponding to the pixels to be processed, to obtain the denoised output result corresponding to the pixels to be processed, including: For each of the image blocks to be processed, for each of the pixels to be processed in the image blocks to be processed, filtering and denoising the pixels to be processed according to the filtering coefficient of the pixels to be processed, the reference pixel filtering coefficient of the reference pixel corresponding to the pixels to be processed, and a preset filtering and denoising formula, to obtain a denoising output result corresponding to the pixels to be processed; The filtering and noise reduction formula includes: denoiseOut k (i,j)=clip(0,255,denoiseOut k (i,j)) Among them, denoiseOut k (i, j) represents the denoising output result of the pixel to be processed with coordinates (i, j) in the kth image block to be processed in the image frame to be processed, curpixelval k (i, j) represents the pixel value of the pixel to be processed with coordinates (i, j) in the kth image block to be processed, refpixelval k (i, j) represents the pixel value of the reference pixel corresponding to the pixel to be processed with coordinates (i, j) in the kth image block to be processed, clip(0,255,denoiseOut k (i, j)) means clamping the denoising output result of the pixel to be processed with coordinates (i, j) in the kth image block to be processed in the image frame to be processed within [0, 255].

7. The video image denoising method based on time domain information and weight factors according to claim 2 or 3, characterized in that: The step of calculating, for each of the image blocks to be processed, the cost between the image block to be processed and each reference image block in the reference image frame according to the weight factor of the area to be processed corresponding to the image block to be processed, comprises: For each of the image blocks to be processed, calculating reference time domain information between the image block to be processed and each reference image block in the reference image frame; For each of the image blocks to be processed, according to the weight factor of the area to be processed corresponding to the image block to be processed and the reference time domain information, the cost between the image block to be processed and each of the reference image blocks in the reference image frame is calculated according to a preset cost calculation formula; The cost calculation formula includes: cost k，p =sad k，p +mvbits k，p *Lambda k Among them, cost k，p represents the cost between the kth image block to be processed in the image frame to be processed and the pth reference image block in the reference image frame, sad k，p represents the similarity between the kth image block to be processed in the image frame to be processed and the pth reference image block in the reference image frame, mvbits k，p represents the number of bits of the motion vector between the kth image block to be processed in the image frame to be processed and the pth reference image block in the reference image frame, Lambda k The weight factor representing the area to be processed corresponding to the kth image block to be processed in the image frame to be processed; And, the calculation formula of the weight factor includes: lambda k =exp((qp-delta k ) / alpha k ) Among them, qp represents the preset quantization coefficient, delta k and alpha k They respectively represent the auxiliary weight factors of the area to be processed corresponding to the kth image block to be processed in the image frame to be processed.

8. A video image denoising device based on time domain information and weight factors, characterized in that: The device comprises: An acquisition module is used to acquire at least one image block to be processed in the image frame to be processed, and for each of the image blocks to be processed, determine the best reference image block corresponding to the image block to be processed in the reference image frame corresponding to the image frame to be processed according to a preset weight factor; A calculation module, configured to calculate, for each of the image blocks to be processed, time domain information between the image block to be processed and the corresponding optimal reference image block, wherein the time domain information includes similarity and motion vector; A determination module, configured to determine, for each of the image blocks to be processed, a motion intensity of the image block to be processed according to the time domain information corresponding to the image block to be processed, and determine, according to the motion intensity, a reference block filter coefficient of the best reference image block corresponding to the image block to be processed; The calculation module is further configured to calculate, for each of the image blocks to be processed, a reference pixel point filter coefficient of each reference pixel point in the best reference image block corresponding to the image block to be processed according to the reference block point filter coefficient of the best reference image block to be processed, and calculate a to-be-processed pixel point filter coefficient of each to-be-processed pixel point in the image block to be processed according to the reference pixel point filter coefficient of each of the reference pixels; The denoising module is used to filter and denoise each of the image blocks to be processed and each of the pixel points to be processed in the image blocks to be processed according to the pixel point filtering coefficient of the pixel point to be processed and the reference pixel point filtering coefficient of the reference pixel point corresponding to the pixel point to be processed, so as to obtain the denoising output result corresponding to the pixel point to be processed.

9. A video image denoising device based on time domain information and weight factors, characterized in that: The device comprises: A memory storing executable program code; a processor coupled to the memory; The processor calls the executable program code stored in the memory to execute the video image denoising method based on time domain information and weight factors as described in any one of claims 1 to 7.

10. A computer storage medium, characterized in that: The computer storage medium stores computer instructions, and when the computer instructions are called, they are used to execute the video image denoising method based on time domain information and weight factors as described in any one of claims 1 to 7.

Images