Video noise reduction method and device, computer equipment and medium

Abstract

The embodiment of the invention discloses a video noise reduction method and device, computer equipment and a medium, and the method comprises the steps: obtaining a target video, and traversing image frames in the target video; when the current image frame of the target video is traversed, acquiring motion information of pixel movement between the current image frame and an adjacent image frame of the current image frame, and determining a pixel-level weight map for the current image frame based on the motion information; performing noise reduction processing on the current image frame by using the pixel-level weight map of the current image frame to obtain a noise-reduced image frame of the current image frame; obtaining a noise reduction image frame of each image frame in the target video, and generating a noise reduction video of the target video based on the noise reduction image frame of each image frame in the target video; the video quality can be improved.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a video noise reduction method, device, computer equipment and medium. Background Art

[0002] With the rapid development of computer vision technology and multimedia technology, a large amount of video data has emerged, and users can usually watch videos in various video players. However, the video quality of the videos watched by users is usually uneven. For example, some videos contain noise, resulting in poor video quality and clarity, which affects the overall quality of the video and the viewing experience of users. Based on this, how to improve the video quality of the video has become a current research hotspot. Summary of the invention

[0003] The embodiments of the present application provide a video noise reduction method, apparatus, computer equipment and medium, which can improve the video quality of a video.

[0004] The first aspect of the embodiment of the present application discloses a video noise reduction method, the method comprising:

[0005] Acquire a target video, and traverse image frames in the target video;

[0006] When traversing to the current image frame of the target video, obtaining motion information of pixel movement between the current image frame and an adjacent image frame of the current image frame, and determining a pixel-level weight map for the current image frame based on the motion information;

[0007] Performing noise reduction processing on the current image frame by using the pixel-level weight map of the current image frame to obtain a noise-reduced image frame of the current image frame;

[0008] A denoised image frame of each image frame in the target video is acquired, and a denoised video of the target video is generated based on the denoised image frame of each image frame in the target video.

[0009] A second aspect of an embodiment of the present application discloses a video noise reduction device, the device comprising:

[0010] An acquisition unit, used for acquiring a target video and traversing image frames in the target video;

[0011] a determining unit, configured to, when traversing to a current image frame of the target video, obtain motion information of pixel movement between the current image frame and an adjacent image frame of the current image frame, and determine a pixel-level weight map for the current image frame based on the motion information;

[0012] A denoising unit, configured to perform denoising processing on the current image frame by using a pixel-level weight map of the current image frame to obtain a denoised image frame of the current image frame;

[0013] The determining unit is further configured to obtain a denoised image frame of each image frame in the target video, and generate a denoised video of the target video based on the denoised image frame of each image frame in the target video.

[0014] A third aspect of an embodiment of the present application discloses a computer device, including a processor and a memory, wherein the memory is used to store a computer program, the computer program includes program instructions, and the processor is configured to call the program instructions to execute the method of the first aspect above.

[0015] A fourth aspect of an embodiment of the present application discloses a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, wherein the computer program includes program instructions, and when the program instructions are executed by a processor, the processor executes the method of the first aspect.

[0016] The fifth aspect of the embodiments of the present application discloses a computer program product or a computer program, wherein the computer program product or the computer program comprises program instructions, and when the program instructions are executed by a processor, the method of the first aspect is implemented.

[0017] In an embodiment of the present application, a target video can be acquired and image frames in the target video can be traversed; when the current image frame of the target video is traversed, motion information of pixel movement between the current image frame and adjacent image frames of the current image frame can be acquired, and a pixel-level weight map for the current image frame can be determined based on the motion information; further, the current image frame can be denoised using the pixel-level weight map of the current image frame to obtain a denoised image frame of the current image frame; finally, the denoised image frame of each image frame in the target video can be acquired, and a denoised video of the target video can be generated based on the denoised image frame of each image frame in the target video. By implementing the above method, when performing noise reduction on a video, the current image frame can be denoised in combination with adjacent image frames in the video, so as to achieve noise reduction by combining multiple frames and improve the reliability of noise reduction. Specifically, the pixel-level weights can be adaptively calculated in combination with the running information between the two image frames, and finally the current image frame can be denoised by the pixel-level weights, so as to effectively filter out noise and motion artifacts in the current image frame by weighting different pixels in the current image frame with different weights, thereby improving the noise reduction effect, thereby improving the image frame quality, and further improving the video quality of the video where the image is located. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0019] Figure 1 is a schematic diagram of the architecture of a noise reduction system provided in an embodiment of the present application;

[0020] Figure 2 It is a flow chart of a video noise reduction method provided in an embodiment of the present application;

[0021] Figure 3 is a schematic diagram of determining a deviation graph provided in an embodiment of the present application;

[0022] Figure 4 It is a structural diagram of determining whether there is a noise reduction requirement for a target video provided by an embodiment of the present application;

[0023] Figure 5 It is a schematic diagram of changes of various graphs involved in a video noise reduction method provided in an embodiment of the present application;

[0024] Figure 6 It is a flowchart of another video noise reduction method provided in an embodiment of the present application;

[0025] Figure 7 is a structural schematic diagram of a video noise reduction device provided in an embodiment of the present application;

[0026] Figure 8 It is a structural diagram of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0027] The technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application.

[0028] The embodiment of the present application proposes a video noise reduction scheme, and the principle of the video noise reduction scheme is as follows: the motion information of the pixel movement between the video frames in the target video can be used to perform noise reduction processing on the target video to obtain a noise-reduced video of the target video. In one embodiment, for the target video, the image frames in the target video can be traversed, and when the current image frame is traversed, the motion information between the current image frame and the adjacent image frames can be used to perform noise reduction processing on the current image frame to obtain a noise-reduced image frame of the current image frame. When the noise-reduced image frame of each image frame in the target video is determined by the above method, the noise-reduced video of the target video can be generated by these noise-reduced image frames. In a possible implementation, the specific implementation of performing noise reduction processing on the current image frame using the motion information between the current image frame and the adjacent image frames to obtain the noise-reduced image frame of the current image frame can be: first determine a pixel-level weight map for the current image frame based on the motion information, and then perform noise reduction processing on the current image frame using the pixel-level weight map of the current image frame to obtain a noise-reduced image frame of the current image frame.

[0029] Through the above method, when denoising a video, the current image frame can be denoised in combination with adjacent image frames in the video, so as to achieve noise reduction by combining multiple frames and improve the reliability of noise reduction. Specifically, the pixel-level weights can be adaptively calculated in combination with the running information between the two image frames, and finally the current image frame can be denoised by the pixel-level weights, so as to effectively filter out the noise and picture motion artifacts in the current image frame by weighting different pixels in the current image frame with different weights, thereby improving the noise reduction effect, thereby improving the image frame quality, and then improving the video quality of the video where the image is located.

[0030] Computer equipment, which can be a terminal or a server. The terminal mentioned here can be a smart phone, tablet computer, laptop computer, desktop computer and other devices; the server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDN), and big data and artificial intelligence platforms, etc.

[0031] For example, when the computer device is a server, the embodiment of the present application provides a noise reduction system, such as Figure 1As shown, the noise reduction system may include at least one terminal and at least one server; the terminal may obtain a target video to be denoised, and upload the obtained target video to a server (i.e., a computer device) so that the server may obtain the target video, and use a video noise reduction scheme to perform noise reduction processing on each image frame in the target video to obtain a noise-reduced image frame of each image frame, and then the noise-reduced image frame of each image frame may constitute a noise-reduced video of the target video.

[0032] Based on the video noise reduction solution provided above, the embodiment of the present application provides a video noise reduction method, which can be executed by the computer device mentioned above. Figure 2 The video noise reduction method includes but is not limited to the following steps:

[0033] S201, obtaining a target video, and traversing image frames in the target video.

[0034] The target video may be any video, for example, the video may be a video of any length, for example, the video may be a long video or a short video; for example, the video may be any type of video, such as a film and television type, or a science and education type video.

[0035] It is understandable that during the video acquisition process, noise is usually introduced due to equipment limitations or environmental factors (such as shooting videos at night or in low-light environments), affecting the image quality; or, during the video compression and transmission stage, the encoding and decoding process to reduce the file size will introduce additional noise and distortion, resulting in reduced image clarity; or, noise may also be introduced during video editing, special effects synthesis and other operations. Based on this, it can be seen that the target video may affect the quality of the video due to the presence of noise. In order to improve the video quality, the target video can be denoised to ensure the quality of the target video. For example, in scenes such as advertising and high-definition television, denoising the video can make the visual effect of the video better when it is played. For another example, in the scene of enhancing old video materials, noise reduction can be used as part of the restoration process of historical videos or old videos to improve and preserve these precious audio-visual materials. For example, in a live broadcast scenario, the video may refer to a real-time live video, and noise reduction of the video may make the live broadcast effect better during the live broadcast process. When watching the live broadcast, live broadcast users may see clearer pictures and thus obtain a better live viewing experience.

[0036] Based on the above description, it can be known that in order to improve the quality of the target video, the target video can be subjected to noise reduction processing to obtain a video with higher quality than the target video. It can be understood that when the target video is subjected to noise reduction processing, each image frame in the target video is subjected to noise reduction processing in turn. After the noise reduction processing of each image frame is completed, a noise-reduced image frame of each image frame can be obtained, and the noise-reduced image frame of each image frame can be combined into a noise-reduced video for the target video. Based on this, it can be known that the image frames in the target video can be traversed here, and when an image frame is traversed, the image frame can be subjected to noise reduction processing to obtain the corresponding noise-reduced image frame.

[0037] In one implementation, when there is a noise reduction requirement for the target video, the step of acquiring the target video may be triggered.

[0038] In one embodiment, a noise reduction request for a target video can be obtained on a computer device, and it is determined that there is a need for noise reduction for the target video. The noise reduction request can be triggered by any user performing a related operation on a user operation interface. For example, when a user needs to perform noise reduction on a target video, a related operation can be performed on a user operation interface output by a terminal used to send a noise reduction request for the target video to the computer device. For example, see Figure 3 As shown, a user operation interface may be displayed on the terminal screen of the terminal used by the user, and the user operation interface may at least include a video input area 301 and a confirmation control 302. If the user wants to perform noise reduction processing on the target video to obtain a video with better video quality than the target video, the user may first input relevant information of the video in the video input area 301 (such as the video directly, or the link address corresponding to the video); then, a trigger operation (such as clicking, pressing, etc.) may be performed on the confirmation control 302; after the terminal detects that the confirmation control 302 is triggered, the video may be obtained based on the information in the input area 301, and after the terminal obtains the video, a noise reduction request carrying the target video may be sent to the computer device, and the noise reduction request may carry the target video, so that the target video may be subsequently subjected to noise reduction processing, thereby obtaining a noise-reduced video.

[0039] In one embodiment, when a noise reduction timing task is triggered, it can be determined that the noise reduction requirement for the target video is obtained. For example, a noise reduction timing task can be set, and when the trigger condition in the noise reduction timing task is triggered, it can be determined that the noise reduction requirement is obtained. In one embodiment, a large number of videos can be stored in a designated storage area, and the trigger condition can be that the current time reaches the preset processing time, or the remaining storage space of the designated storage area exceeds the preset remaining storage space, and so on. Among them, the video that needs to be denoised (i.e., the target video) can be a video in the designated area.

[0040] In one embodiment, when a video streaming task is triggered, it can be determined that a noise reduction requirement for a target video is obtained, and the target video is also the video being streamed. For example, the video streaming task can refer to a task in a live video broadcast scenario, and specifically can refer to a task for improving the video quality in a live video broadcast scenario, such as the video noise reduction processing here is a prerequisite for beauty processing; exemplarily, in a live broadcast scenario, when the live streaming side is streaming live to the live audience side, the live video stream (which can be understood as the target video) can be first subjected to noise reduction processing, so that the live video stream pushed to the live audience side is a noise-reduced video stream, thereby improving the live audience experience of the audience. For another example, the video streaming task can be a task in a video recommendation scenario; exemplarily, in a video recommendation scenario, for a recommended video recommended to a user, the recommended video (which can be understood as the target video) can be first subjected to noise reduction processing, so as to ensure that the video recommended to the user is a high-quality video.

[0041] S202, when traversing to the current image frame of the target video, obtaining motion information of pixel movement between the current image frame and adjacent image frames of the current image frame, and determining a pixel-level weight map for the current image frame based on the motion information.

[0042] Among them, the current image frame can be any image frame in the target video, and the adjacent image frame can refer to the previous image frame relative to the current image frame in the target video, that is, the embodiment of the present application can use the adjacent image frame of the current image frame as a reference, so that the noise reduction processing of the current image frame can be implemented based on the noise reduction method of the difference between frames.

[0043] It should be understood that, for the first image frame in the target video, since there is no previous image frame as a reference, it is also impossible to use the video noise reduction method proposed in the embodiment of the present application to implement noise reduction processing on the first image frame. Based on this, it can be known that a noise reduction processing method for the first image frame can be pre-set to obtain a noise reduction image frame of the first image frame, so that when the second image frame is subsequently processed, the first image frame and the noise reduction image frame corresponding to the first image frame can be used to perform noise reduction processing on the second image frame.

[0044] Among them, the noise reduction processing method is not limited. For example, the noise reduction processing method can be a single-frame noise reduction method, a spatial domain noise reduction method, etc. The single-frame noise reduction method refers to not relying on the information of the previous and next frame images in the video, but processing a frame of image separately; the single-frame noise reduction method can be implemented by using bilateral filters, wavelet transforms and other technologies. The principle of the spatial domain noise reduction method is to assume that adjacent pixels in the image are similar, and to reduce random noise by smoothing these pixels; in the spatial domain noise reduction method, arithmetic mean filtering, median filtering, Gaussian filtering, etc. can be used to reduce the noise in the image. The following is a detailed description of these noise reduction processing methods.

[0045] Among them, the bilateral filter in the single-frame denoising method is a nonlinear filtering method that can combine the spatial proximity and pixel value similarity of the image, that is, it can consider the spatial difference and intensity difference of pixels at the same time. The characteristic of this filter is that it reduces noise while keeping the edge clear, because it considers not only the distance between pixels but also the difference between pixel values ​​when calculating the weighted average.

[0046] Among them, the specific implementation of the denoising process of the first image frame by the wavelet transform in the single-frame denoising method can be: the first image frame can be firstly subjected to wavelet transform to decompose it into wavelet coefficients of different scales and directions; in this process, the useful information (such as edges and textures) of the first image frame is usually concentrated in the larger wavelet coefficients, while the noise is distributed in the smaller wavelet coefficients, that is, the image can be decomposed into different frequency bands, while retaining certain spatial information, which is helpful to better separate the noise and image signals. Then, a preset reference threshold is obtained, and all wavelet coefficients less than the reference threshold are set to zero or shrunk to near zero; through this operation, the noise component can be eliminated or reduced, while retaining the large coefficients representing the main features of the image. Finally, the processed wavelet coefficients can be subjected to inverse wavelet transform to reconstruct the denoised image, which is the denoised image frame for the first image frame. From the above description, it can be seen that in the process of denoising the image by using wavelet transform, since the wavelet coefficients of the noise are discarded or greatly reduced, the noise level in the reconstructed image will be significantly reduced, and the noise can be removed while maintaining important features such as the edge and texture of the image.

[0047] Among them, under the spatial denoising method, the following is an explanation of the denoising process of any pixel in the first image frame. The arithmetic mean filtering in the spatial denoising method refers to: using the average value of the pixel values ​​of all pixels in the neighborhood of any pixel in the first image frame to replace the original pixel value of any pixel; performing the above operation on each pixel in the first image frame, the resulting image is the denoised image frame. The median filtering in the spatial denoising method refers to: using the median value of the pixel values ​​of all pixels in the neighborhood of any pixel in the first image frame to replace the original pixel value of any pixel; performing the above operation on each pixel in the first image frame, the resulting image is the denoised image frame.

[0048] Based on the above description, it can be known that for the first image frame in the target video, the above-mentioned method can be used to obtain the corresponding denoised image frame, and the other image frames in the corresponding target video can be implemented by the video denoising method proposed in the embodiment of the present application, that is, the other image frames in the target video except the first image frame can execute the determination operation and the determination operation in step 202.

[0049] In one implementation, an acquisition operation may be performed first, that is, motion information of pixel movement between the current image frame and the adjacent image frame of the current image frame may be acquired first. It is understandable that since objects (such as people and objects) in the video may move, or the camera may shake, there may be some displacement between different frames in the video, that is, there may be running information of pixel movement between the two frames. Therefore, in order to use the previous image frame as a reference for noise reduction processing during the image noise reduction process, the two frames may be adjusted to compensate for this movement through image alignment, so that the position of the same object on different frames remains consistent, thereby improving the noise reduction effect.

[0050] Based on this, it can be known that the specific implementation of the acquisition operation in step S201 can be: the current image frame can be aligned with the adjacent image frame to determine the coordinate mapping diagram of the current image frame relative to the corresponding pixel points of the adjacent image frame; in one embodiment, the computer device can calculate the pixel movement between the adjacent image frame and the current image frame based on the optical flow alignment algorithm to obtain a coordinate mapping diagram, which is the coordinate mapping diagram of the current image frame relative to the corresponding pixel points of the adjacent image frame. The coordinate mapping diagram contains the displacement vector of each pixel point from the adjacent image frame to the current image frame, and the coordinate mapping diagram is the motion information of the pixel movement between the current image frame and the adjacent image frame of the current image frame.

[0051] Among them, the optical flow alignment algorithm is a method for estimating the movement of pixels in an image, which can determine the movement of pixels by analyzing the changes in the time domain of pixels in the image sequence and the correlation between adjacent frames. In one embodiment, the specific implementation of obtaining a coordinate mapping diagram using the optical flow alignment algorithm can be: feature points or regions can be first extracted from the current image frame and the adjacent image frame, and then the displacement vector (optical flow vector) of these feature points between the two image frames can be calculated using the optical flow method. Finally, the current image frame can be remapped according to the calculated displacement vector to align it with the adjacent image frame at the pixel level. Specifically, the displacement vector can be added to the pixel coordinates of the current image frame to obtain the aligned pixel coordinates, so that the coordinate mapping diagram can be constructed based on the aligned pixel coordinates. In general, this coordinate mapping diagram contains pixel-level displacement vectors, indicating how to deform the adjacent image frame to the shape and position of the current image frame.

[0052] Here, the optical flow alignment operation can ensure that the object motion in the image sequence is accurately matched, which is beneficial to the subsequent noise reduction process. Because the noise reduction algorithm usually assumes that the image content is consistent in time, if there is motion, it will cause the denoised image to be blurred or ghosted. Optical flow alignment can also help determine the temporal correspondence of each pixel in the image, so that the pixel values ​​at different time points can be better integrated when synthesizing the image. Then, in high dynamic range imaging (HDR), optical flow alignment can also be used to reduce ghosting caused by object movement. After optical flow alignment processing, the noise reduction algorithm can be more focused on eliminating static background noise, rather than mistakenly treating the edges of moving objects as noise, which helps to maintain the details and texture of the image and enhance the image quality. In general, through the processing of optical flow alignment, the noise reduction effect can be effectively improved, the ghosting phenomenon can be reduced, and the image quality can be enhanced.

[0053] After obtaining the coordinate mapping map, the computer device can determine the pixel-level weight map for the current image frame based on the coordinate mapping map, that is, the determination operation in step S202 can be performed. In a specific implementation, an adjacent denoised image frame of an adjacent image frame can be obtained to determine the pixel-level weight map of the current image frame based on the coordinate mapping map and the adjacent denoised image frame. The adjacent denoised image frame is a denoised image frame of an adjacent image frame.

[0054] In one embodiment, a specific implementation of determining a pixel-level weight map of a current image frame based on a coordinate mapping map and adjacent denoised image frames may be as follows: the adjacent denoised image frames may be backward mapped using the coordinate mapping map to obtain adjacent aligned denoised image frames aligned with the current image frame; then the pixel value deviations of corresponding pixel points between the current image frame and the adjacent aligned denoised image frames may be determined to obtain a deviation map; and further, the deviation map may be used to determine a pixel-level weight map for the current image frame.

[0055] In one embodiment, the specific implementation of the above-mentioned determination of adjacent aligned denoised image frames may be: the computer device uses the coordinate mapping map to perform a backward mapping process on the adjacent denoised image frames corresponding to the adjacent image frames to obtain adjacent aligned denoised image frames aligned with the current image frame. The backward mapping process may refer to mapping the coordinate mapping map to the adjacent denoised image frames, that is, finding the corresponding position of each pixel point in the adjacent denoised image frame in the current image frame according to the displacement vector in the coordinate mapping map.

[0056] Among them, the use of a coordinate mapping diagram to perform backward mapping processing on adjacent denoised image frames can make the adjacent denoised image frames aligned with the current image frame at the pixel level, that is, the adjacent aligned denoised image frame is an image that is pixel-aligned with the current image frame. The adjacent aligned denoised image frame retains the denoising results of the adjacent image frames and is aligned with the current image frame at the pixel level. Through this operation, it can be ensured as much as possible that the pixels between consecutive frames can correspond accurately. When using multi-frame denoising technology to perform denoising processing, denoising artifacts caused by inter-frame motion can be avoided as much as possible, and the details in the video content can be better preserved.

[0057] In one embodiment, the specific implementation of determining the deviation map may be: after obtaining the adjacent aligned denoised image frames, the deviation map may be determined based on the adjacent aligned denoised image frames and the current image frame. In a possible implementation, the pixel value deviation of the corresponding pixel points between the current image frame and the adjacent aligned denoised image frames may be calculated to obtain the deviation map.

[0058] Among them, the deviation map has the same size as the current image frame, and the deviation map contains the pixel value deviation of the pixel value between each pixel point in the current image frame and the corresponding pixel point in the adjacent aligned denoised image frame. The pixel value deviation here can refer to the absolute difference in the pixel value of a pixel point in two image frames (the current image frame and the adjacent aligned denoised image frame), that is, the deviation map contains the absolute difference between the pixel values ​​of the corresponding pixel points of the current image frame and the adjacent aligned denoised image frame, such as the deviation map can also be called a difference map. Among them, the absolute difference refers to the absolute value of the difference. It should be noted that, for the convenience of description, each position in the deviation map is understood here as a pixel point corresponding to the current image frame, that is, the numerical value corresponding to a pixel point in the deviation map represents the absolute difference between the pixel values ​​of the current image frame and the adjacent aligned denoised image frame at the pixel point.

[0059] For example, see Figure 4 As shown, the image marked by 41 is the current image frame, the image marked by 42 is the adjacent aligned denoised image frame, and the image marked by 43 is the deviation map; the markers S, P, and Q respectively represent the pixel points on the current image frame 41, the pixel points on the adjacent aligned denoised image frame 42, and the pixel points on the deviation map 43. The value corresponding to the pixel point Q on the deviation map 43 is: the absolute difference between the pixel value of the pixel point S on the current image frame 41 and the pixel value of the pixel point P on the adjacent aligned denoised image frame 42.

[0060] In one embodiment, the above description on determining the deviation map is expressed in a mathematical formula, which can be referred to as the following formula (1):

[0061] DiffMap=fabs(WarpPreOut-CurIn)(1)

[0062] Among them, DiffMap represents the deviation map, WarpPreOut represents the adjacent aligned denoised image frames, CurIn represents the current image frame, and fabs(A) represents the absolute value of A.

[0063] In one embodiment, the specific implementation of determining the pixel-level weight map for the current image frame using the deviation map may be: first obtaining at least one segmentation threshold, and using at least one segmentation threshold to segment the deviation values ​​in the deviation map to obtain a segmentation map. In one embodiment, firstly using at least one segmentation threshold to segment the deviation values ​​in the deviation map to obtain an initial segmentation map; then, filtering the initial segmentation map to obtain a segmentation map.

[0064] Among them, at least one segmentation threshold can be pre-set, such as the segmentation threshold can be set by a technician based on actual experience. For example, at least one segmentation threshold includes a first segmentation threshold and a second segmentation threshold; the first segmentation threshold is less than the second segmentation threshold. In one embodiment, the specific implementation of segmenting the deviation value in the deviation map using at least one segmentation threshold to obtain the initial segmentation map can be described as follows. Considering that the processing principle of any pixel point in the deviation map is similar, the following description is made using any pixel point in the deviation map as an example.

[0065] For any pixel in the deviation map, if the deviation value of any pixel in the deviation map is less than the first segmentation threshold, the deviation value of the pixel is updated to the first value; if the deviation value of a pixel in the deviation map is greater than the second segmentation threshold, the deviation value of the pixel is updated to the second value; if the deviation value of a pixel in the deviation map is within the range of the first segmentation threshold and the second segmentation threshold, the deviation value of the pixel is updated to the third value. The first value is greater than the third value, and the third value is greater than the second value. The above-mentioned update operation is performed on each pixel in the deviation map to obtain the updated value of each pixel, and the updated value of each pixel can be constructed as an initial segmentation map.

[0066] It can be seen from the above segmentation processing that the deviation map is binarized. After the deviation map is binarized, the deviation value greater than or less than a certain segmentation threshold can be updated to a specific value (such as the first value, the second value, and the third value mentioned above), thereby simplifying the complexity of the image and simplifying subsequent image processing operations.

[0067] In an embodiment of the present application, different segmentation thresholds can be set to distinguish between noise and actual signals in an image. For example, referring to the above description, by comparing the deviation value in the deviation map with the segmentation threshold, the deviation value (i.e., possible noise) beyond the normal range can be identified and processed accordingly. At the same time, the edge and detail information in the image can also be protected. Considering that in the segmentation process, the high-frequency part (such as edge and texture) and the low-frequency part (such as smooth area) of the image can be distinguished by the segmentation threshold, so that while filtering out noise, its original quality and information content (such as main structure and detail information) can be maintained. In general, using segmentation thresholds to process the deviation map can better strike a balance between denoising and retaining important features in the image, ensuring that the image can maintain its original quality and information content while removing noise.

[0068] Wherein, if the above description on determining the initial segmentation graph is expressed in a mathematical formula, it can be referred to the following formula (2):

[0069]

[0070] Wherein, ThreshMap represents the initial segmentation map, DiffMap represents the deviation map, K1 and K2 represent the first segmentation threshold and the second segmentation threshold, respectively, and H1, H2, and H3 represent the first value, the second value, and the third value, respectively. Formula (2) can also be simply understood as a linear truncation function.

[0071] For example, assuming K1=5, K2=10; H1=40, H2=5, H3=20; then the above formula (2) can be transformed into the following formula (3):

[0072]

[0073] For example, see Figure 5 As shown, Figure 5 Various figures involved in the video noise reduction method proposed in the embodiment of the present application are shown. Figure 5 As shown, the image marked by 51 is the deviation map, and the image marked by 52 is the initial segmented map, which is obtained by segmenting the deviation map 51 using a linear truncation function (such as the above formula (3)). It can be seen from the initial segmented map 52 that after the deviation map is segmented, the noise and the actual signal in the image can be clearly distinguished.

[0074] Among them, the above-mentioned initial segmentation map and segmentation map have the same size as the current image frame. For the convenience of description, each position in the initial segmentation map can be understood as a pixel point corresponding to the current image frame, and the numerical value corresponding to each position is called the initial segmentation value (or it can also be called other terms, without limitation); similarly, each position in the segmentation map is understood as a pixel point corresponding to the current image frame, and the numerical value corresponding to each position is called the segmentation value.

[0075] The implementation of filtering the initial segmentation can be achieved by using a box filter. The processing principle of the box filter is to replace the pixel value of each pixel in the image with the average value of the pixel values ​​in its neighborhood. The neighborhood can be a square area, and its size is determined by the radius. For example, if the radius is set to 1, the neighborhood of a pixel is an area of ​​3×3 size, that is, the pixel value of a pixel can be: the average value of the pixel values ​​of all pixels in the 3×3 area.

[0076] Based on this, it can be known that the specific implementation of filtering the initial segmented image to obtain the segmented image can be described as follows. Considering that the principle of filtering any pixel point in the initial segmented image is consistent, the following is explained by taking any pixel point in the initial segmented image as an example:

[0077] The neighborhood of any pixel in the initial segmentation map can be determined according to a preset radius; then, the initial segmentation values ​​corresponding to all the pixels in the neighborhood can be obtained, and the obtained initial segmentation values ​​can be averaged to obtain an average value; finally, the initial segmentation value of any pixel can be replaced with the average value. Through the above operations, the initial segmentation value of each pixel in the initial segmentation map can be replaced with the corresponding average value, and the initial segmentation map obtained after completing all replacement operations is the segmentation map, that is, the segmentation value of each pixel in the segmentation map is the above average value.

[0078] It should be noted that when calculating the edge pixels in the initial segmentation image, since such pixels usually do not have a sufficiently complete neighborhood, you can In addition, when processing boundary pixels, since they may not have enough neighboring pixels, special processing is required, such as repeating boundary pixels or using a smaller neighborhood.

[0079] As mentioned above, the filtering processing in the embodiment of the present application can locally average the initial segmentation values ​​of each pixel point in the initial segmentation map, so as to reduce the high-frequency noise and details in the initial segmentation map, thereby achieving the purpose of smoothing the initial segmentation map, and further improving the quality of the segmentation map obtained using the initial segmentation map. Then, when the segmentation map is used for subsequent noise reduction processing, the reliability of noise reduction can also be improved.

[0080] After obtaining the segmentation map, a pixel-level weight map for the current image frame can be obtained based on the deviation map and the segmentation map. In one embodiment, the computer device can use the segmentation map to normalize the deviation map to obtain an initial weight map; then, the initial weight map can be truncated in a target truncation manner to obtain a pixel-level weight map for the current image frame.

[0081] The specific implementation of the normalization process may be to divide the deviation map by the segmentation map, and use the result of the operation as the initial weight map, that is, to obtain the initial weight map by dividing the deviation map by the segmentation map. The division operation between the two maps here may refer to the division operation of the values ​​at the corresponding positions between the two maps.

[0082] Among them, the target truncation method can be a 0-1 truncation method. In this way, the weight values ​​in the initial weight map can be limited to the range of 0-1, so that the weight values ​​in the final pixel-level weight map are limited to the range of 0-1. Based on this, the initial weight map is truncated in a 0-1 truncation method to obtain a specific implementation of the pixel-level weight map for the current image frame, which can be described as follows:

[0083] Considering that the principle of truncating the weight value of each pixel in the initial weight map is consistent, the following is an example of the weight value of any pixel in the initial weight map. For the weight value of any pixel in the initial weight map, if the weight value is in the range of 0 to 1, the weight value is kept unchanged; if the weight value is less than 0, the weight value is updated to 0, and if the weight value is greater than 1, the weight value is updated to 1.

[0084] Through the 0-1 truncation method, it can be ensured that the weights used in the subsequent fusion process of the current image frame and the adjacent aligned denoised image frames will not exceed the range of 0-1, thereby avoiding overexposure or underexposure of the denoised image frame finally fused, making the transition of the denoised image frame between different exposure areas more natural, ensuring the overall brightness balance of the denoised image frame, and also maintaining the details and texture features, thereby improving the image quality of the denoised image frame.

[0085] In one embodiment, the above description of determining the pixel-level weight map of the current image frame is expressed in a mathematical formula, which can be referred to as the following formula (4):

[0086] WeightMap=clamp(DiffMap / BoxBlur(ThreshMap),0,1)(4)

[0087] Among them, WeightMap represents the pixel-level weight map, DiffMap represents the deviation map, ThreshMap represents the initial segmentation map, BoxBlur(A) represents the filtering effect on A, and the filtering effect is achieved by using a box filter; DiffMap / BoxBlur(ThreshMap) represents the initial weight map; the function of the clamp function is to limit a value within a specified range. For example, the meaning of formula (4) here is to limit DiffMap / BoxBlur(ThreshMap) within the range of 0 to 1. In short, the clamp function here represents a 0 to 1 truncation method.

[0088] Among them, the pixel-level weight map of the current image frame has the same size as the current image frame, and the weight map may include the weight value corresponding to each pixel in the current image frame. For ease of understanding, each position in the weight image corresponds to a pixel in the current image frame, that is, the weight map represents the proportion of image information of the current image frame used when performing noise reduction processing on the current image frame. Among them, the size of the weight value in the weight map is positively correlated with the amount of image information of the current image frame used; that is, when the weight value of a certain position in the weight map is larger, it means that more image information of the current image frame is used; when the weight value of a certain position in the weight map is smaller, it means that less image information of the current image frame is used. For example, see Figure 5 As shown, Figure 5 The image marked by 53 in FIG. 1 may refer to a pixel-level weight map.

[0089] S203, performing noise reduction processing on the current image frame by using the pixel-level weight map of the current image frame to obtain a noise-reduced image frame of the current image frame.

[0090] In one implementation, a computer device may first determine a weight map for adjacent aligned denoised image frames based on a specified weight map and a pixel-level weight map. The specified weight map has the same size as the pixel-level weight map. As described above, the pixel-level weight map includes the weight value of each pixel in the current image frame, and the specified weight map also includes each pixel in the current image frame. For example, the specified weight map and the pixel-level weight map may be subtracted to obtain a weight map for adjacent aligned denoised image frames. Exemplarily, taking the weight value of any pixel in the weight map of adjacent aligned denoised image frames as an example, the weight value of any pixel is the difference between the numerical value corresponding to the pixel in the specified weight map and the weight value of the pixel in the pixel-level weight map. That is, the difference between the numerical values ​​at corresponding positions in the two weight maps can be constructed as a weight map for adjacent aligned denoised image frames.

[0091] After obtaining the weight map for adjacent aligned denoised image frames, the denoised image frame of the current image frame can be determined using the two weight maps, the current image frame and the adjacent aligned denoised image frames. In one embodiment, the weight map of adjacent aligned denoised image frames can be used to perform pixel-level weighted processing on adjacent aligned denoised image frames to obtain a first weighted map, and the pixel-level weight map can be used to perform pixel-level weighted processing on the current image frame to obtain a second weighted map. The denoised image frame of the current image frame can then be determined based on the first weighted map and the second weighted map. For example, the weighted values ​​of the corresponding pixels in the first weighted map and the second weighted map can be summed, and the calculation result corresponding to each pixel is the pixel value of the corresponding pixel subjected to denoising, that is, the calculation result corresponding to each pixel in the current image frame can be constructed as the denoised image frame of the current image frame.

[0092] Among them, the weight map of adjacent aligned denoised image frames is used to perform pixel-level weighted processing on adjacent aligned denoised image frames to obtain the first weighted map. The specific implementation can be: use the weight value of each pixel point in the weight map to perform weighted processing on the pixel values ​​of the corresponding pixel points in the adjacent aligned denoised image frames to obtain the weighted value for each pixel point in the previous aligned noise image, and the weighted value for each pixel point in the previous aligned noise image can be constructed as the first weight map.

[0093] Among them, the specific implementation of performing pixel-level weighted processing on the current image frame using the pixel-level weight map to obtain the second weighted map can be: using the weight value of each pixel point in the weight map to perform weighted processing on the pixel value of the corresponding pixel point in the current image frame, to obtain the weighted value for each pixel point in the current image frame, and the weighted value for each pixel point in the current image frame can be constructed as the second weight map.

[0094] In one embodiment, the above-mentioned description of determining the noise reduction image frame is expressed by a mathematical formula, which can be referred to as the following formula (5):

[0095] CurOut=WarpPreOut*(1-WeightMap)+CuIn*WeightMap(5)

[0096] Among them, CurOut represents the denoised image frame of the current image frame, WarpPreOut represents the adjacent aligned denoised image frame, CuIn represents the current image frame, WeightMap represents the pixel-level weight map for the current image frame, and (1-WeightMap) represents the weight map for the adjacent aligned denoised image frame.

[0097] In summary, the embodiment of the present application can be combined with multiple frames of time domain images for noise reduction. On this basis, the inter-frame pixel-level fusion weights can be adaptively calculated based on the inter-frame difference map after optical flow alignment. Finally, through the inter-frame recursive fusion method, strong noise can be effectively filtered out and the picture motion artifacts generated during the fusion process can be eliminated to obtain a highly guaranteed video noise reduction result. In general, the embodiment of the present application proposes a pixel-level multi-frame adaptive fusion noise reduction method, which is a highly robust video noise reduction method with strong noise reduction effect and high definition fidelity. Under this method, the video quality clarity can be effectively maintained and motion artifacts can be eliminated while effectively filtering out video noise.

[0098] S204, obtaining a denoised image frame of each image frame in the target video, and generating a denoised video of the target video based on the denoised image frame of each image frame in the target video.

[0099] In one implementation, each image frame in the target video can be denoised by the above-mentioned method of determining the denoised image frame of the current image frame to obtain a denoised image frame for each image frame. After obtaining these denoised image frames, a denoised video of the target video can be generated based on the denoised image frame of each image frame in the target video.

[0100] In one implementation, the noise-reduced video can be applied in a variety of scenarios, such as in analysis and recognition tasks; for example, the noise-reduced video corresponding to the target video can be used for tasks such as facial recognition and object detection to effectively improve the accuracy and robustness of the analysis and recognition tasks. For another example, the noise-reduced video can be applied in video playback scenarios (such as advertising playback, short video playback, TV playback, etc.); for example, when there is a demand for video playback, the target video with the video playback demand can first be subjected to the video noise reduction method proposed in the embodiment of the present application to obtain a noise-reduced video, and then the noise-reduced video can be played to effectively enhance the visual effect of the video playback.

[0101] For example, the following specific explanation is given by taking the video live broadcast scenario as an example. In the specific application scenario, the video noise reduction method proposed in the embodiment of the present application can be applied to the noise reduction image quality processing task in the video live broadcast streaming. Usually, the quality of the live broadcast image on the live audience side mainly focuses on two aspects: one is the facial clarity of the anchor. Currently, good portrait beautification and clarity effects can be obtained through algorithms such as portrait skin refining, portrait clarity, and portrait whitening; the second is the overall picture quality clarity of the live broadcast room. Among them, the salt and pepper noise and flicker noise caused by the differences in hardware such as mobile phones and cameras and the live broadcast environment (such as point light sources, dark light, and night scenes) are the main reasons that affect the picture quality experience of the live broadcast room. Therefore, the noise reduction processing of the live video stream is a more important module on the live broadcast streaming end, and it is usually also the pre-processing module of algorithms such as portrait skin refining. Based on the above considerations, the video noise reduction method proposed in the embodiment of the present application can be applied to the noise reduction processing of the live video stream to reduce the noise of the live stream, thereby realizing a video live noise reduction method based on multi-frame adaptive fusion, so that a live video stream with better quality can be pushed to the live audience side, thereby improving the overall picture quality experience of the live audience side.

[0102] In an embodiment of the present application, when performing noise reduction on a video, the current image frame can be denoised in combination with adjacent image frames in the video, so as to achieve noise reduction by combining multiple frames and improve the reliability of noise reduction. Specifically, the pixel-level weights can be adaptively calculated in combination with the running information between the two image frames, and finally the current image frame is denoised by the pixel-level weights, so as to effectively filter out noise and motion artifacts in the current image frame by weighting different pixels in the current image frame with different weights, thereby improving the noise reduction effect, thereby improving the image frame quality, and further improving the video quality of the video in which the image is located.

[0103] To better understand the video noise reduction method proposed in this application, the following combination Figure 6 Further description. The following describes the video noise reduction by taking the current image frame in the target video as an example, wherein the noise reduction process may involve the following steps:

[0104] S1: caching adjacent image frames for the current image frame and adjacent denoised image frames corresponding to the adjacent image frames; the adjacent image frames are recorded as PreIn, and the adjacent denoised image frames are recorded as PreOut.

[0105] S2: Capture the current image frame and record it as CurIn; calculate the coordinate mapping of the corresponding pixel points of the current image frame CurIn relative to the adjacent image frame PreIn through the optical flow alignment algorithm.

[0106] S3: using the coordinate mapping diagram obtained in step S2, performing backward mapping processing on the adjacent denoised image frames to obtain adjacent aligned denoised image frames aligned with the pixels of the current image frame, and recording the adjacent aligned denoised image frames as WarpPreOut.

[0107] S4: Calculate the absolute difference between the corresponding pixels of the current image frame CurIn and the adjacent aligned denoised image frame WarpPreOut to obtain a deviation map (or a difference map), and record the deviation map as DiffMap. The specific implementation of determining the deviation map DiffMap here can refer to the above formula (1). After obtaining the deviation map, adaptive fusion weight calculation can be performed to obtain a pixel-level weight map, which is specifically implemented in the following steps S5 to S7.

[0108] S5: According to the set segmentation threshold, the deviation map is segmented (such as threshold segmentation) and filtered (such as box filtering) to obtain a truncated threshold map, which is also the above-mentioned segmentation map; the truncated threshold map is denoted as TreshMap here. The specific implementation of threshold segmentation of the deviation map here can refer to the above formula (2).

[0109] S6: performing a division operation on the deviation map and the truncation threshold map, that is, dividing the deviation map by the truncation threshold map, to obtain an initial fusion weight map, which is also the initial weight map mentioned above.

[0110] S7: Cut the initial fusion weight map from 0 to 1 to obtain the final pixel-level fusion weight map, which is the pixel-level weight map for the current image frame mentioned above; the pixel-level fusion weight map is referred to as WeightMap. The specific implementation of determining the pixel-level fusion weight map WeightMap here can refer to the above formula (4).

[0111] Among them, the pixel-level fusion weight map can represent: when performing denoising on the current image frame, the proportion of image information in the current image frame is used. The larger the weight value of the pixel point in the pixel-level fusion weight map, the more image information of the current image frame is used, and the smaller the weight value, the more image information of the adjacent aligned denoised image frame is used.

[0112] S8: According to the pixel-level fusion weight map, the current image frame CurIn and the adjacent aligned denoised image frame WarpPreOut are weighted to obtain a denoised image frame of the current image frame CurIn, which can be recorded as CurOut. The specific implementation of determining the denoised image frame CurOut here can refer to the above formula (5).

[0113] Combining the pixel-level fusion weight map of step S7 and the weighting method of step S8, it can be seen that: in the static area, the fusion weight of the adjacent aligned denoised image frame WarpPreOut is relatively large, which can effectively eliminate large noise; in the moving area, the fusion weight of the current image frame is relatively large, which can effectively eliminate motion artifacts while ensuring that the picture clarity is not lost.

[0114] From the above description, it can be seen that, based on the multi-frame denoising process, the present application designs a method for calculating the pixel-level fusion weights between frames. In the specific implementation, the current image frame and the adjacent images can be aligned to adaptively calculate the pixel-level weights between frames according to the aligned inter-frame deviation map. Finally, the current image frame is denoised through recursive inter-frame fusion. Not only the strong noise in the image frame is effectively filtered out, but also the motion artifacts caused by the fusion of image frames are eliminated, thereby improving the image quality and obtaining a high-fidelity denoised video.

[0115] See also Figure 7 , Figure 7 : is a schematic diagram of the structure of a video noise reduction device provided in an embodiment of the present application. The video noise reduction device described in this embodiment includes:

[0116] An acquisition unit 701 is used to acquire a target video and traverse image frames in the target video;

[0117] A determination unit 702 is configured to, when traversing to a current image frame of the target video, obtain motion information of pixel movement between the current image frame and an adjacent image frame of the current image frame, and determine a pixel-level weight map for the current image frame based on the motion information;

[0118] A denoising unit 703 is configured to perform denoising processing on the current image frame by using a pixel-level weight map of the current image frame to obtain a denoised image frame of the current image frame;

[0119] The determining unit 702 is further configured to obtain a denoised image frame of each image frame in the target video, and generate a denoised video of the target video based on the denoised image frame of each image frame in the target video.

[0120] In one implementation, the determining unit 702 is specifically configured to:

[0121] Calculating pixel movement between the adjacent image frames and the current image frame based on an optical flow alignment algorithm to obtain a coordinate map, and using the coordinate map as the motion information; the coordinate map includes a displacement vector of each pixel from the adjacent image frame to the current image frame;

[0122] Acquire an adjacent noise reduction image frame of the adjacent image frame; the adjacent noise reduction image frame is a noise reduction image frame of the adjacent image frame;

[0123] Based on the coordinate mapping diagram and the adjacent denoised image frames, a pixel-level weight map of the current image frame is determined.

[0124] In one implementation, the determining unit 702 is specifically configured to:

[0125] Performing a backward mapping process on the adjacent denoised image frames using the coordinate mapping diagram to obtain adjacent aligned denoised image frames aligned with the current image frame;

[0126] Determining pixel value deviations of corresponding pixel points between the current image frame and the adjacent aligned denoised image frames to obtain a deviation map;

[0127] A pixel-level weight map for the current image frame is determined using the deviation map.

[0128] In one implementation, the determining unit 702 is specifically configured to:

[0129] Obtaining at least one segmentation threshold, and using the at least one segmentation threshold to segment the pixel value deviation in the deviation map to obtain a segmentation map;

[0130] Based on the deviation map and the segmentation map, a pixel-level weight map for the current image frame is obtained.

[0131] In one implementation, the determining unit 702 is specifically configured to:

[0132] Segmenting the pixel value deviations in the deviation map using the at least one segmentation threshold to obtain an initial segmentation map;

[0133] The initial segmented graph is subjected to filtering processing to obtain a segmented graph.

[0134] In one implementation, the determining unit 702 is specifically configured to:

[0135] Using the segmented graph to normalize the deviation graph to obtain an initial weight graph;

[0136] The initial weight map is truncated in a target truncation manner to obtain a pixel-level weight map for the current image frame.

[0137] In one implementation, the noise reduction unit 703 is specifically configured to:

[0138] Determining a weight map for the adjacent aligned denoised image frames based on a specified weight map and the pixel-level weight map;

[0139] Performing pixel-level weighted processing on the adjacent aligned denoised image frames using the weight map of the adjacent aligned denoised image frames to obtain a first weighted map;

[0140] Performing pixel-level weighted processing on the current image frame using the pixel-level weight map to obtain a second weighted map;

[0141] A denoised image frame of the current image frame is determined based on the first weighted map and the second weighted map.

[0142] It is understood that the division of units in the embodiments of the present application is schematic and is only a logical function division. There may be other division methods in actual implementation. The functional units in the embodiments of the present application may be integrated into a processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated units may be implemented in the form of hardware or in the form of software functional units.

[0143] See also Figure 8 , Figure 8 801 and 802. The computer device includes a processor 801 and a memory 802. Optionally, the computer device may also include a network interface 803. The processor 801, the memory 802 and the network interface 803 may exchange data.

[0144] The processor 801 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0145] The memory 802 may include a read-only memory and a random access memory, and provide program instructions and data to the processor 801. A portion of the memory 802 may also include a non-volatile random access memory. When the processor 801 calls the program instructions, it is used to execute:

[0146] Acquire a target video, and traverse image frames in the target video;

[0147] When traversing to the current image frame of the target video, obtaining motion information of pixel movement between the current image frame and an adjacent image frame of the current image frame, and determining a pixel-level weight map for the current image frame based on the motion information;

[0148] Performing noise reduction processing on the current image frame by using the pixel-level weight map of the current image frame to obtain a noise-reduced image frame of the current image frame;

[0149] A denoised image frame of each image frame in the target video is acquired, and a denoised video of the target video is generated based on the denoised image frame of each image frame in the target video.

[0150] In one implementation, the processor 801 is specifically configured to:

[0151] Calculating pixel movement between the adjacent image frames and the current image frame based on an optical flow alignment algorithm to obtain a coordinate map, and using the coordinate map as the motion information; the coordinate map includes a displacement vector of each pixel from the adjacent image frame to the current image frame;

[0152] Acquire an adjacent noise reduction image frame of the adjacent image frame; the adjacent noise reduction image frame is a noise reduction image frame of the adjacent image frame;

[0153] Based on the coordinate mapping diagram and the adjacent denoised image frames, a pixel-level weight map of the current image frame is determined.

[0154] In one implementation, the processor 801 is specifically configured to:

[0155] Performing a backward mapping process on the adjacent denoised image frames using the coordinate mapping diagram to obtain adjacent aligned denoised image frames aligned with the current image frame;

[0156] Determining pixel value deviations of corresponding pixel points between the current image frame and the adjacent aligned denoised image frames to obtain a deviation map;

[0157] A pixel-level weight map for the current image frame is determined using the deviation map.

[0158] In one implementation, the processor 801 is specifically configured to:

[0159] Obtaining at least one segmentation threshold, and using the at least one segmentation threshold to segment the pixel value deviation in the deviation map to obtain a segmentation map;

[0160] Based on the deviation map and the segmentation map, a pixel-level weight map for the current image frame is obtained.

[0161] In one implementation, the processor 801 is specifically configured to:

[0162] Segmenting the pixel value deviations in the deviation map using the at least one segmentation threshold to obtain an initial segmentation map;

[0163] The initial segmented graph is subjected to filtering processing to obtain a segmented graph.

[0164] In one implementation, the processor 801 is specifically configured to:

[0165] Using the segmented graph to normalize the deviation graph to obtain an initial weight graph;

[0166] The initial weight map is truncated in a target truncation manner to obtain a pixel-level weight map for the current image frame.

[0167] In one implementation, the processor 801 is specifically configured to:

[0168] Determining a weight map for the adjacent aligned denoised image frames based on a specified weight map and the pixel-level weight map;

[0169] Performing pixel-level weighted processing on the adjacent aligned denoised image frames using the weight map of the adjacent aligned denoised image frames to obtain a first weighted map;

[0170] Performing pixel-level weighted processing on the current image frame using the pixel-level weight map to obtain a second weighted map;

[0171] A denoised image frame of the current image frame is determined based on the first weighted map and the second weighted map.

[0172] The present application also provides a computer storage medium in which program instructions are stored. When the program is executed, it may include: Figure 2 Part or all of the steps of the video noise reduction method in the corresponding embodiment.

[0173] It should be noted that, for the above-mentioned various method embodiments, for the sake of simplicity of description, they are all expressed as a series of action combinations, but those skilled in the art should be aware that this application is not limited by the order of the actions described, because according to this application, some steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by this application.

[0174] A person skilled in the art may understand that all or part of the steps in the various methods of the above embodiments may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, and the storage medium may include: a flash drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0175] The embodiment of the present application also provides a computer program product or computer program, which includes program instructions, and when the program instructions are executed by a processor, some or all of the steps in the above method can be implemented. For example, the program instructions are stored in a computer-readable storage medium. The processor of the computer device reads the program instructions from the computer-readable storage medium, and the processor executes the program instructions, so that the computer device executes the steps performed in the embodiments of the above methods.

[0176] The above is a detailed introduction to a video noise reduction method, device, computer equipment and medium provided in the embodiments of the present application. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea; at the same time, for general technical personnel in this field, according to the ideas of the present application, there will be changes in the specific implementation methods and application scopes. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A video noise reduction method, characterized in that: The method comprises: Acquire a target video, and traverse image frames in the target video; When traversing to the current image frame of the target video, obtaining motion information of pixel movement between the current image frame and an adjacent image frame of the current image frame, and determining a pixel-level weight map for the current image frame based on the motion information; Performing noise reduction processing on the current image frame by using the pixel-level weight map of the current image frame to obtain a noise-reduced image frame of the current image frame; A denoised image frame of each image frame in the target video is acquired, and a denoised video of the target video is generated based on the denoised image frame of each image frame in the target video.

2. The method according to claim 1, characterized in that The acquiring motion information of pixel movement between the current image frame and an image frame adjacent to the current image frame includes: Calculating pixel movement between the adjacent image frames and the current image frame based on an optical flow alignment algorithm to obtain a coordinate map, and using the coordinate map as the motion information; the coordinate map includes a displacement vector of each pixel from the adjacent image frame to the current image frame; The determining of a pixel-level weight map for the current image frame by using motion information of pixel movement between the current image frame and an adjacent image frame of the current image frame comprises: Acquire an adjacent noise-reduced image frame of the adjacent image frame; the adjacent noise-reduced image frame is a noise-reduced image frame of the adjacent image frame; Based on the coordinate mapping diagram and the adjacent denoised image frames, a pixel-level weight map of the current image frame is determined.

3. The method according to claim 2, characterized in that The step of determining a pixel-level weight map of the current image frame based on the coordinate mapping map and the adjacent denoised image frames includes: Performing a backward mapping process on the adjacent denoised image frames using the coordinate mapping diagram to obtain adjacent aligned denoised image frames aligned with the current image frame; Determining pixel value deviations of corresponding pixel points between the current image frame and the adjacent aligned denoised image frames to obtain a deviation map; The deviation map is used to determine a pixel-level weight map for the current image frame.

4. The method according to claim 3, characterized in that The step of using the deviation map to determine a pixel-level weight map for the current image frame includes: Obtaining at least one segmentation threshold, and using the at least one segmentation threshold to segment the pixel value deviation in the deviation map to obtain a segmentation map; Based on the deviation map and the segmentation map, a pixel-level weight map for the current image frame is obtained.

5. The method according to claim 4, characterized in that The step of segmenting the pixel value deviation in the deviation map by using the at least one segmentation threshold to obtain a segmentation map comprises: Segmenting the pixel value deviations in the deviation map using the at least one segmentation threshold to obtain an initial segmentation map; The initial segmented graph is subjected to filtering processing to obtain a segmented graph.

6. The method according to claim 4 or 5, characterized in that: The step of obtaining a pixel-level weight map for the current image frame based on the deviation map and the segmentation map includes: Using the segmented graph to normalize the deviation graph to obtain an initial weight graph; The initial weight map is truncated in a target truncation manner to obtain a pixel-level weight map for the current image frame.

7. The method according to claim 3, characterized in that The step of performing noise reduction processing on the current image frame by using the pixel-level weight map of the current image frame to obtain a noise-reduced image frame of the current image frame includes: Determining a weight map for the adjacent aligned denoised image frames based on a specified weight map and the pixel-level weight map; Performing pixel-level weighted processing on the adjacent aligned denoised image frames using the weight map of the adjacent aligned denoised image frames to obtain a first weighted map; Performing pixel-level weighted processing on the current image frame using the pixel-level weight map to obtain a second weighted map; A denoised image frame of the current image frame is determined based on the first weighted map and the second weighted map.

8. A video noise reduction device, characterized in that: include: An acquisition unit, used for acquiring a target video and traversing image frames in the target video; a determining unit, configured to, when traversing to a current image frame of the target video, obtain motion information of pixel movement between the current image frame and an adjacent image frame of the current image frame, and determine a pixel-level weight map for the current image frame based on the motion information; A denoising unit, configured to perform denoising processing on the current image frame by using a pixel-level weight map of the current image frame to obtain a denoised image frame of the current image frame; The determining unit is further configured to obtain a denoised image frame of each image frame in the target video, and generate a denoised video of the target video based on the denoised image frame of each image frame in the target video.

9. A computer device, characterized in that: The method comprises a processor and a memory, wherein the memory is used to store a computer program, the computer program comprises program instructions, and the processor is configured to call the program instructions to execute the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, wherein the computer program includes program instructions, and when the program instructions are executed by a processor, the processor is caused to perform the method according to any one of claims 1 to 7.

Images