Video stream noise reduction method, system, device and storage medium

Abstract

A video stream noise reduction method includes the following steps: identifying multiple dynamic actions in a video stream, analyzing the speed change pattern of each dynamic action, using one or more video frames with abnormal speed change as reference video frames, and using one or more video frames with normal speed change as control video frames; capturing a first dynamic object of the dynamic action in the reference video frame, capturing a second dynamic object of the dynamic action in a control video frame adjacent to the reference video frame, blurring the other unfiltered one or more control video frames and marking them as blurred video frames; redrawing based on the reference video frame or the filtered blurred video frame to obtain a redrawn video frame; and sequentially outputting the control video frame, the redrawn video frame, and the reference video frame according to a time queue. The present invention can improve the noise reduction effect and efficiency, and is suitable for video stream noise reduction processing in different encoding modes, thereby expanding the application scope of video stream noise reduction processing.

Description

Technical Field

[0001] The present invention relates to the technical field of video stream noise reduction, and in particular to a video stream noise reduction method, system, device and storage medium. Background Art

[0002] In daily life, due to the limitations of shooting conditions and the influence of sending, transmitting, and receiving devices, videos are often interfered with by noise, which degrades the video quality, affects the visual effect, and hinders further processing of the video. Therefore, video noise reduction is necessary to improve video quality.

[0003] At present, in order to improve the quality of video noise reduction, a commonly used method is to perform noise reduction on each image frame in the video to obtain a noise-reduced video. This noise reduction process requires processing each frame, which consumes a lot of computing time and computing resources, reducing the efficiency of video noise reduction. Summary of the Invention

[0004] In view of the above-mentioned drawbacks, the present invention aims to provide a method, system, device and storage medium for video stream noise reduction, thereby improving the noise reduction effect and efficiency and expanding the application scope of video stream noise reduction processing.

[0005] To achieve this purpose, the present invention adopts the following technical solution: a method for reducing noise in a video stream, comprising the following steps:

[0006] Step S101: identifying multiple dynamic actions in a video stream, analyzing the speed change pattern of each dynamic action, using one or more video frames with abnormal speed changes as reference video frames, and using one or more video frames with normal speed changes as control video frames;

[0007] Step S102: capturing a first dynamic object of the dynamic action in the reference video frame, and capturing a second dynamic object of the dynamic action in a comparison video frame adjacent to the reference video frame;

[0008] Comparing the first dynamic object with the second dynamic object to determine whether the difference in size and shape between the first dynamic object and the second dynamic object is greater than a threshold, if so, deleting the corresponding reference video frame; if not, retaining the corresponding reference video frame;

[0009] Step S103: identifying the attributes of the first dynamic object in the retained reference video frame, and sequentially comparing the attributes of the first dynamic object with preset motion trajectories to obtain the motion trajectory of the first dynamic object;

[0010] screening a control video frame that matches the activity trajectory of the first dynamic object;

[0011] Step S104: performing blurring processing on one or more unscreened control video frames and marking them as blurred video frames;

[0012] Obtaining a reference video frame or a filtered control video frame adjacent to the blurred video frame, and redrawing the video frame according to the reference video frame or the filtered blurred video frame to obtain a redrawn video frame;

[0013] The control video frame, the redrawn video frame and the reference video frame are output in sequence according to the time queue.

[0014] Preferably, the blurring process in step S104 specifically includes the following steps: offsetting the edge position of the dynamic action according to the activity trajectory of the blurred video frame and blurring the edge position;

[0015] Each pixel point in the blurred video frame is converted into a grayscale value with reference to the reference video frame.

[0016] Preferably, the specific steps of redrawing the blurred video frame are as follows:

[0017] Step A: obtaining coordinate points of a first region to be blurred in a blurred video frame, and obtaining a color block of a reference video frame at the coordinate points according to the first region coordinate points, and performing color block filling on the blurred video frame;

[0018] Step B: Obtain the second area coordinate point of the blank position in the blurred video frame, obtain the color block of the reference video frame at the coordinate according to the second area coordinate point, and fill the blank position in the blurred video frame with the color block.

[0019] Preferably, the reference video frame used in step B is: a reference video frame adjacent to the blurred video frame.

[0020] Preferably, in step B, if there are multiple control video frames adjacent to the blurred video frame, the multiple control video frames are sequentially filled with color blocks together with the blurred video frame to obtain multiple filled images;

[0021] The filling image is processed by an image smoothing algorithm, and the blurred video frame is used to perform image comparison with the filling image respectively, and the filling image with the smallest difference value is used as the redrawing video frame.

[0022] Preferably, the image smoothing algorithm uses adaptive median filtering to process the image.

[0023] Preferably, in step B, the following operation is also required: calculate the average grayscale value of each pixel point of the filled color block in the fill image and multiple pixel points around the color block. If the average grayscale value is greater than the grayscale threshold, the grayscale value of the color block is manually processed.

[0024] A video stream noise reduction system, using the video stream noise reduction method, is characterized by comprising a video frame recognition module, a video frame screening module, a video frame analysis module and a video frame processing module;

[0025] The video frame recognition module is used to recognize multiple dynamic actions in the video stream, analyze the speed change pattern of each dynamic action, use one or more video frames with abnormal speed changes as reference video frames, and use one or more video frames with normal speed changes as control video frames;

[0026] The video frame screening module is used to capture the first dynamic object of the dynamic action in the reference video frame, and capture the second dynamic object of the dynamic action in the comparison video frame adjacent to the reference video frame;

[0027] Comparing the first dynamic object with the second dynamic object to determine whether the difference in size and shape between the first dynamic object and the second dynamic object is greater than a threshold, if so, deleting the corresponding reference video frame; if not, retaining the corresponding reference video frame;

[0028] The video frame analysis module is used to identify the attributes of the first dynamic object in the retained reference video frame, and compare the attributes of the first dynamic object with the preset movement trajectory in sequence to obtain the movement trajectory of the first dynamic object;

[0029] screening a control video frame that matches the activity trajectory of the first dynamic object;

[0030] The video frame processing module is used to perform blurring processing on one or more unscreened control video frames and mark them as blurred video frames;

[0031] Obtaining a reference video frame or a filtered control video frame adjacent to the blurred video frame, and redrawing the video frame according to the reference video frame or the filtered blurred video frame to obtain a redrawn video frame;

[0032] The control video frame, the redrawn video frame and the reference video frame are output in sequence according to the time queue.

[0033] A device includes a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor implements a video stream noise reduction method.

[0034] A storage medium stores a computer program, which implements the video stream noise reduction method when executed by a processor.

[0035] One of the above technical solutions has the following advantages or beneficial effects: the present invention can improve the noise reduction effect and efficiency, and is adaptable to video stream noise reduction processing in different encoding modes, thereby expanding the application scope of video stream noise reduction processing. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 The figure is a flow chart of a method for reducing noise in a video stream according to an embodiment of the present invention.

[0037] Figure 2 A redrawn schematic diagram of a flow chart in one embodiment of the present invention;

[0038] Figure 3 FIG. 4 is a structural diagram of a video stream noise reduction system according to an embodiment of the present invention. DETAILED DESCRIPTION

[0039] The embodiments of the present invention are described in detail below. Examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals represent the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention and are not to be construed as limiting the present invention.

[0040] In the description of the embodiments of the present invention, the terms "first" and "second" are used for descriptive purposes only and should not be understood to indicate or imply relative importance or implicitly specify the number of the technical features indicated. Therefore, a feature specified as "first" or "second" may explicitly or implicitly include one or more of the described features. In the description of the embodiments of the present invention, "plurality" means two or more, unless otherwise specifically specified.

[0041] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of the technical features being referred to. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of such features. In the description of the present invention, unless otherwise specified, "plurality" means two or more. Those skilled in the art will understand the specific meanings of the above terms in the present invention based on the specific circumstances.

[0042] like Figures 1 to 3 As shown, a video stream noise reduction method includes the following steps:

[0043] Step S101: identifying multiple dynamic actions in a video stream, analyzing the speed change pattern of each dynamic action, using one or more video frames with abnormal speed changes as reference video frames, and using one or more video frames with normal speed changes as control video frames;

[0044] The same set of video frames contains at least one dynamic action, which is then identified and processed. The speed changes and magnitude of the dynamic action, such as acceleration, deceleration, and constant speed, are used to determine whether there is an abnormality, thereby eliminating noise caused by repeated frames and frame skipping.

[0045] Step S102: capturing a first dynamic object of the dynamic action in the reference video frame, and capturing a second dynamic object of the dynamic action in a comparison video frame adjacent to the reference video frame;

[0046] Comparing the first dynamic object with the second dynamic object to determine whether the difference in size and shape between the first dynamic object and the second dynamic object is greater than a threshold, if so, deleting the corresponding reference video frame; if not, retaining the corresponding reference video frame;

[0047] The dynamic objects of the dynamic actions may include people, objects or animals. The sizes and shapes of the first dynamic object and the second action object are compared, so as to delete the reference video frames with large differences. When applied to the step video frame processing, the noise of the reference video frame is avoided when it is used as the basis for the control video frame processing.

[0048] Step S103: identifying the attributes of the first dynamic object in the retained reference video frame, and sequentially comparing the attributes of the first dynamic object with preset motion trajectories to obtain the motion trajectory of the first dynamic object;

[0049] screening a control video frame that matches the activity trajectory of the first dynamic object;

[0050] Specifically, the motion trajectory of the action is analyzed based on the properties of the dynamic object. Object properties can include various motion templates, such as free falling, walking, running, and squatting. The reference video frames are filtered using the preset motion trajectory of the object properties. The reference video frames with matching motion trajectories are selected and retained, and the remaining video frames are processed based on the selected reference video frames and the reference video frames.

[0051] Step S104: performing blurring processing on one or more unscreened control video frames and marking them as blurred video frames;

[0052] Obtaining a reference video frame or a filtered control video frame adjacent to the blurred video frame, and redrawing the video frame according to the reference video frame or the filtered blurred video frame to obtain a redrawn video frame;

[0053] The control video frame, the redrawn video frame and the reference video frame are output in sequence according to the time queue.

[0054] Detailed, unfiltered control video frames are blurred to capture details such as the trajectory, size, shape, edges, and brightness of dynamic movements, thereby reshaping the video stream.

[0055] The blurred video frame can be redrawn based on the adjacent comparison video frame or reference video frame. This process is similar to restoration, so that the blurred part of the blurred video frame can be restored to a clear image. The comparison video frame, the redrawn video frame, and the reference video frame are input into the array in reverse order according to the playback time, so as to form the video stream after noise reduction processing.

[0056] The present invention identifies multiple dynamic actions in a video stream, thereby analyzing the speed change rules of each dynamic action, and then filtering out video frames with abnormal speed changes. The frames are used as a reference for filtering video frames with size and shape differences, and multiple video frames with high dynamic action, shape and size restoration can be retained for further identification and analysis. The present invention identifies the attributes of the dynamic action object, thereby analyzing the trajectory changes of the dynamic action using the attributes, and filtering the control video frames using the activity trajectory to retain the matching control video frames, and blurring the filtered control video frames to redraw the blurred video frames based on the selected control video frames and reference video frames, thereby eliminating the noise of the blurred video frames. The video frames, the redrawn video frames and the reference video frames are sequentially input into an array in reverse order according to the playback time sequence, so that people can view the video stream after noise reduction processing.

[0057] Therefore, the present invention can improve the noise reduction effect and efficiency, and is adaptable to video stream noise reduction processing in different coding modes, thereby expanding the application scope of video stream noise reduction processing.

[0058] Preferably, the blurring process in step S104 specifically includes the following steps: offsetting the edge position of the dynamic action according to the activity trajectory of the blurred video frame and blurring the edge position;

[0059] Each pixel point in the blurred video frame is converted into a grayscale value with reference to the reference video frame.

[0060] The present invention uses the moving track to offset the edge position and blur it, making it easier to re-edit the position and shape of the dynamic object. And after processing, it is adjusted to grayscale value, making it easier to restore the image color through color, brightness and grayscale, further improving the image processing effect.

[0061] Preferably, the specific steps of redrawing the blurred video frame are as follows:

[0062] Step A: obtaining coordinate points of a first region to be blurred in a blurred video frame, and obtaining a color block of a reference video frame at the coordinate points according to the first region coordinate points, and performing color block filling on the blurred video frame;

[0063] Step B: Obtain the second area coordinate point of the blank position in the blurred video frame, obtain the color block of the reference video frame at the coordinate according to the second area coordinate point, and fill the blank position in the blurred video frame with the color block.

[0064] During the redrawing process, the above step A is the first color block infilling, which is to automatically identify the pixel coordinates of the reference video frame and then fill the blurred video frame with color blocks, thereby preliminarily restoring the image color of the blurred video frame.

[0065] Furthermore, the blank locations in the blurred video frame after color filling are similarly obtained from the reference video frame using the same method described above. The white locations in the blurred video frame are then color-filled, thereby further color-filling the image using the differences caused by the motion trajectory. The coordinates of the pixels in these blank locations can be referenced to the corresponding selected reference video frame, and after the second infill in step B, color correction can be performed based on the color transitions between adjacent pixels.

[0066] Preferably, the reference video frame used in step B is: a reference video frame adjacent to the blurred video frame.

[0067] By comparing the video frame and the blurred video frame using the adjacent frame numbers, the color editing of the blurred video frame can be performed more comprehensively, and the efficiency of color editing can be improved.

[0068] Preferably, in step B, if there are multiple control video frames adjacent to the blurred video frame, the multiple control video frames are sequentially filled with color blocks together with the blurred video frame to obtain multiple filled images;

[0069] The filling image is processed by an image smoothing algorithm, and the blurred video frame is used to perform image comparison with the filling image respectively, and the filling image with the smallest difference value is used as the redrawing video frame.

[0070] This approach can reduce the image disharmony between the blurred video frame and adjacent frames, thereby further reducing the image noise generated between adjacent frames of the video stream.

[0071] Preferably, the image smoothing algorithm uses an adaptive median filter to process the image. The use of the adaptive median filter can effectively preserve the edges of dynamic movements and reduce detail noise.

[0072] Preferably, in step B, the following operation is also required: calculate the average grayscale value of each pixel point of the filled color block in the fill image and multiple pixel points around the color block. If the average grayscale value is greater than the grayscale threshold, the grayscale value of the color block is manually processed.

[0073] By comparing adjacent pixels and the grayscale value of the middle pixel with the grayscale values ​​of multiple surrounding pixels, we can further find the suspected noise part in the video stream. When the average grayscale value is greater than the grayscale threshold, the color block may be noise, so it needs to be sent to manual judgment and selection, using the human eye to check and eliminate noise problems.

[0074] A video stream noise reduction system, using the video stream noise reduction method, is characterized by comprising a video frame recognition module, a video frame screening module, a video frame analysis module and a video frame processing module;

[0075] The video frame recognition module is used to recognize multiple dynamic actions in the video stream, analyze the speed change pattern of each dynamic action, use one or more video frames with abnormal speed changes as reference video frames, and use one or more video frames with normal speed changes as control video frames;

[0076] The video frame screening module is used to capture the first dynamic object of the dynamic action in the reference video frame, and capture the second dynamic object of the dynamic action in the comparison video frame adjacent to the reference video frame;

[0077] Comparing the first dynamic object with the second dynamic object to determine whether the difference in size and shape between the first dynamic object and the second dynamic object is greater than a threshold, if so, deleting the corresponding reference video frame; if not, retaining the corresponding reference video frame;

[0078] The video frame analysis module is used to identify the attributes of the first dynamic object in the retained reference video frame, and compare the attributes of the first dynamic object with the preset movement trajectory in sequence to obtain the movement trajectory of the first dynamic object;

[0079] screening a control video frame that matches the activity trajectory of the first dynamic object;

[0080] The video frame processing module is used to perform blurring processing on one or more unscreened control video frames and mark them as blurred video frames;

[0081] Obtaining a reference video frame or a filtered control video frame adjacent to the blurred video frame, and redrawing the video frame according to the reference video frame or the filtered blurred video frame to obtain a redrawn video frame;

[0082] The control video frame, the redrawn video frame and the reference video frame are output in sequence according to the time queue.

[0083] A device includes a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor implements a video stream noise reduction method.

[0084] A storage medium stores a computer program, which implements the video stream noise reduction method when executed by a processor.

[0085] Throughout this specification, reference to terms such as "one embodiment," "some embodiments," "illustrative embodiments," "examples," "specific examples," or "some examples" means that a specific feature, structure, material, or characteristic described in conjunction with that embodiment or example is included in at least one embodiment or example of the present invention. In this specification, illustrative uses of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.

[0086] Although the embodiments of the present invention have been shown and described above, it will be understood that the above embodiments are exemplary and are not to be construed as limitations on the present invention. A person skilled in the art may change, modify, replace and modify the above embodiments within the scope of the present invention.

Claims

1. A video stream noise reduction method, characterized in that: The following steps are involved: Step S101: identifying multiple dynamic actions in a video stream, analyzing the speed change pattern of each dynamic action, using one or more video frames with abnormal speed changes as reference video frames, and using one or more video frames with normal speed changes as control video frames; Step S102: capturing a first dynamic object of the dynamic action in the reference video frame, and capturing a second dynamic object of the dynamic action in a comparison video frame adjacent to the reference video frame; Comparing the first dynamic object with the second dynamic object to determine whether the difference in size and shape between the first dynamic object and the second dynamic object is greater than a threshold, if so, deleting the corresponding reference video frame; if not, retaining the corresponding reference video frame; Step S103: identifying the attributes of the first dynamic object in the retained reference video frame, and sequentially comparing the attributes of the first dynamic object with preset motion trajectories to obtain the motion trajectory of the first dynamic object; screening a control video frame that matches the activity trajectory of the first dynamic object; Step S104: performing blurring processing on one or more unscreened control video frames and marking them as blurred video frames; Obtaining a reference video frame or a filtered control video frame adjacent to the blurred video frame, and redrawing the video frame according to the reference video frame or the filtered blurred video frame to obtain a redrawn video frame; The control video frame, the redrawn video frame and the reference video frame are output in sequence according to the time queue.

2. The method for video stream noise reduction according to claim 1, wherein: The blurring process in step S104 specifically includes the following steps: offsetting the edge position of the dynamic action according to the moving trajectory of the blurred video frame and blurring it; Each pixel point in the blurred video frame is converted into a grayscale value with reference to the reference video frame.

3. The method for video stream noise reduction according to claim 2, wherein: The specific steps of redrawing the blurred video frame are as follows: Step A: obtaining coordinate points of a first region to be blurred in a blurred video frame, and obtaining a color block of a reference video frame at the coordinate points according to the first region coordinate points, and performing color block filling on the blurred video frame; Step B: Obtain the second area coordinate point of the blank position in the blurred video frame, obtain the color block of the reference video frame at the coordinate according to the second area coordinate point, and fill the blank position in the blurred video frame with the color block.

4. The method for video stream noise reduction according to claim 3, wherein: The reference video frame used in step B is: a reference video frame adjacent to the blurred video frame.

5. The method for video stream noise reduction according to claim 4, wherein: In step B, if there are multiple reference video frames adjacent to the blurred video frame, the multiple reference video frames are sequentially filled with color blocks together with the blurred video frame to obtain multiple filled images; The filling image is processed by an image smoothing algorithm, and the blurred video frame is used to perform image comparison with the filling image respectively, and the filling image with the smallest difference value is used as the redrawing video frame.

6. A video stream noise reduction method according to claim 5, characterized in that: The image smoothing algorithm uses adaptive median filtering to process the image.

7. The method for video stream noise reduction according to claim 5, wherein: In step B, the following operation also needs to be performed: calculate the average grayscale value of each pixel point of the filled color block in the fill image and multiple pixels around the color block. If the average grayscale value is greater than the grayscale threshold, the grayscale value of the color block is manually processed.

8. A video stream noise reduction system, using a video stream noise reduction method according to any one of claims 1 to 7, characterized in that: It includes a video frame recognition module, a video frame screening module, a video frame analysis module and a video frame processing module; The video frame recognition module is used to recognize multiple dynamic actions in the video stream, analyze the speed change pattern of each dynamic action, use one or more video frames with abnormal speed changes as reference video frames, and use one or more video frames with normal speed changes as control video frames; The video frame screening module is used to capture the first dynamic object of the dynamic action in the reference video frame, and capture the second dynamic object of the dynamic action in the comparison video frame adjacent to the reference video frame; Comparing the first dynamic object with the second dynamic object to determine whether the difference in size and shape between the first dynamic object and the second dynamic object is greater than a threshold, if so, deleting the corresponding reference video frame; if not, retaining the corresponding reference video frame; The video frame analysis module is used to identify the attributes of the first dynamic object in the retained reference video frame, and compare the attributes of the first dynamic object with the preset movement trajectory in sequence to obtain the movement trajectory of the first dynamic object; screening a control video frame that matches the activity trajectory of the first dynamic object; The video frame processing module is used to perform blurring processing on one or more unscreened control video frames and mark them as blurred video frames; Obtaining a reference video frame or a filtered control video frame adjacent to the blurred video frame, and redrawing the video frame according to the reference video frame or the filtered blurred video frame to obtain a redrawn video frame; The control video frame, the redrawn video frame and the reference video frame are output in sequence according to the time queue.

9. A device, characterized in that The method comprises a memory and a processor, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the processor implements a video stream noise reduction method according to any one of claims 1 to 7.

10. A storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the video stream noise reduction method according to any one of claims 1 to 7 is implemented.

Images