Video noise reduction method, device and equipment

Abstract

The present invention provides a method, device and equipment for video noise reduction, the method comprising: performing target recognition on each frame image of a video to obtain the coordinates and type of at least one target in each frame image; for the at least one target in each frame image, determining the movement speed of the target according to the coordinates of the target in the current frame image and the previous frame image and the time interval between adjacent frames; determining the spatial noise reduction parameters and time domain noise reduction parameters corresponding to the target according to the type and movement speed of the target; performing image noise reduction on the target according to the spatial noise reduction parameters and time domain noise reduction parameters corresponding to the target. By using the method provided by the present invention, the corresponding proportions of spatial noise reduction and time domain noise reduction are determined according to the movement speed and type of each target, and noise reduction processing is performed on each target respectively, which can improve the video noise reduction effect and reduce the encoding bit rate.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a video noise reduction method, device and equipment. Background Art

[0002] Since the digital images obtained by terminal shooting are often affected by the noise interference of imaging equipment and external environment during the process of digitization and data transmission, they all contain noise. Noise will affect the image effect and increase the encoding bit rate of the image. Therefore, in order to obtain better image effects and reduce the encoding bit rate, it is necessary to perform noise reduction processing on the above-mentioned noisy images.

[0003] Related technologies generally use time-domain noise reduction, spatial-domain noise reduction, and a combination of the two to perform image noise reduction: Among them, spatial-domain noise reduction, also known as intra-frame noise reduction or 2D noise reduction, only requires a single image to be input, and various noise reduction filters are used to reduce the noise of the input image. The noise reduction effect depends on the design of the filter and the filter parameters; however, the stronger the spatial-domain noise reduction intensity, the more serious the loss of details of the image after noise reduction, and the blurrier the picture. Time-domain noise reduction, also known as 3D noise reduction, uses motion estimation technology to perform time-domain inter-frame filtering along the motion trajectory of the object in the image layer constructed by the current frame and multiple frames before and after, to remove noise in the video sequence while maintaining the details of the video image. Although time-domain noise reduction does not lose picture details like spatial-domain noise reduction, due to the limitations of motion estimation, when the time-domain noise reduction intensity is too strong, obvious ghosting will occur on the trajectory of the moving object, affecting the video picture effect. The most common method for denoising video images in the prior art is to perform noise reduction on each frame by combining spatial noise reduction with temporal noise reduction. However, how to set the parameters and intensity of spatial noise reduction, and temporal noise reduction and intensity to obtain higher image effects and reduce the encoding bit rate is a problem to be solved. Summary of the invention

[0004] The present invention provides a video noise reduction method, a device and a equipment, which can obtain a better video noise reduction effect and reduce the encoding bit rate of the image.

[0005] In a first aspect, the present invention provides a video noise reduction method, the method comprising:

[0006] Performing target recognition on each frame image of the video to obtain the coordinates and type of at least one target in each frame image;

[0007] For the at least one target in each frame image, determine the moving speed of the target according to the coordinates of the target in the current frame image and the previous frame image and the time interval between adjacent frames;

[0008] According to the type and movement speed of the target, determine the spatial noise reduction parameters and the temporal noise reduction parameters corresponding to the target;

[0009] Image denoising is performed on the target according to the spatial denoising parameters and temporal denoising parameters corresponding to the target.

[0010] The video denoising method provided by the present invention extracts targets in an image through target recognition, distinguishes the types and coordinates of the targets, calculates the movement speed of the targets according to the coordinates, matches spatial denoising parameters and temporal denoising parameters based on the types and movement speeds, and can adopt different spatial denoising parameters and temporal denoising parameters for different types of targets in the image and their movement intensity and static state to perform differentiated denoising; removes image noise, improves image quality, and reduces the encoding bit rate of the image while ensuring the details of the target image; can solve the problem of motion ghosting, and ensure the denoising effect of targets with different motion intensities.

[0011] Optionally, performing image denoising on the target according to a spatial denoising parameter and a temporal denoising parameter corresponding to the target includes:

[0012] Determine at least one image area corresponding to the target in N image areas; wherein the N image areas are obtained by dividing the current frame image according to a preset size;

[0013] Taking the image area as a unit, noise reduction is performed on at least one image area corresponding to the target according to the spatial noise reduction parameters and the temporal noise reduction parameters corresponding to the target.

[0014] The video denoising method provided by the present invention divides the current frame image into N image regions of the same size, and performs partition denoising on at least one image region corresponding to the target, thereby improving the effect of video denoising and reducing the encoding bit rate of the image.

[0015] Optionally, after determining the spatial domain noise reduction parameters and the temporal domain noise reduction parameters corresponding to the target, the method further includes:

[0016] For each frame image, according to preset spatial domain noise reduction parameters and temporal domain noise reduction parameters, noise reduction is performed on the area other than the at least one target in the current frame image.

[0017] The video denoising method provided by the present invention can strengthen the spatial domain denoising parameters and the temporal domain denoising parameters for non-target areas, and can effectively reduce the encoding bit rate of the image.

[0018] Optionally, performing noise reduction on an area outside the at least one target in the current frame image includes:

[0019] Taking image regions as units, performing noise reduction on image regions other than at least one image region corresponding to the at least one target in the N image regions;

[0020] The N image regions are obtained by dividing the current frame image according to a preset size.

[0021] The video noise reduction method provided by the present invention adopts a partitioning method to perform noise reduction on each image area, provides a specific noise reduction processing method, and improves the feasibility of the embodiment of the present invention.

[0022] Optionally, determining at least one image area corresponding to the target includes:

[0023] Determine at least one first image region intersecting with the target according to the coordinates of the target;

[0024] Calculate the moving direction and moving speed of the target according to the coordinates of the target in the current frame image and the previous frame image and the time interval between adjacent frames;

[0025] According to the moving direction and moving speed of the target, predicting, in the N image areas, a second image area that will intersect with the target after a preset time;

[0026] The first image area and the second image area are determined to be at least one image area corresponding to the target.

[0027] The video denoising method provided by the present invention divides the current frame image into N image areas of the same size, and predicts the position of the target after a preset time according to the coordinates and movement speed of the target, and uses the image area where the current position and the predicted position are located as the denoising area of ​​the target, thereby reserving processing time for time domain denoising, improving the effect of video denoising and reducing the encoding bit rate of the image.

[0028] Optionally, determining the moving speed of the target according to the coordinates of the target in the current frame image and the previous frame image and the time interval between adjacent frames includes:

[0029] Calculating the displacement between the coordinates of the pixel corresponding to the preset point of the target in the current frame image and the coordinates of the pixel corresponding to the preset point in the previous frame image as the pixel displacement of the target;

[0030] Compare the pixel displacement with the time interval between adjacent frames to obtain the pixel movement speed of the target;

[0031] The pixel moving speed is compared with the pixel width of the current frame image to obtain the moving speed of the target.

[0032] The video denoising method provided by the present invention calculates the moving speed of the target through the coordinates of the preset point of the same target in two adjacent frames and the time interval between adjacent frames, and provides a specific implementation method for calculating the moving speed of the target, which provides the necessary technical basis for subsequently determining the spatial domain denoising parameters and the temporal domain denoising parameters, helps to improve the effect of video denoising and reduce the encoding bit rate.

[0033] Optionally, determining a spatial noise reduction parameter and a temporal noise reduction parameter corresponding to the target according to the type and movement speed of the target includes:

[0034] Determine the exercise intensity level of the target according to the preset correspondence between the exercise speed and the exercise intensity level and the exercise speed of the target;

[0035] Determine the spatial noise reduction parameters and the temporal noise reduction parameters corresponding to the target according to the correspondence between the preset motion parameters and the noise reduction parameters and the motion parameters of the target; wherein the motion parameters include the type and the motion intensity level, and the noise reduction parameters include the spatial noise reduction parameters and the temporal noise reduction parameters;

[0036] The spatial noise reduction parameters include at least one of different types of parameters that affect the strength of spatial noise reduction, and the temporal noise reduction parameters include at least one of different types of parameters that affect the strength of temporal noise reduction.

[0037] The video denoising method provided by the present invention matches a corresponding motion intensity level for each target according to the motion speed of each target, distinguishes the types and motion intensity levels of different targets, matches corresponding spatial noise reduction parameters and temporal noise reduction parameters for targets of different types and / or motion intensity levels, thereby ensuring a higher noise reduction effect and reducing the encoding bit rate.

[0038] Optionally, determining the exercise intensity level of the target according to a preset correspondence between the exercise speed and the exercise intensity level and the exercise speed of the target includes:

[0039] When the movement speed of the target is less than a preset threshold, determining the movement intensity level of the target to be a preset value;

[0040] When the target's movement speed is not less than a preset threshold, the target's movement intensity level is determined according to the preset correspondence between the movement speed and the movement intensity level and the target's movement speed; wherein, the greater the movement speed, the higher the corresponding movement intensity level.

[0041] The video noise reduction method provided by the present invention can reduce the impact of target recognition errors on video noise reduction and improve the video noise reduction effect by setting a preset threshold for the target's movement speed and setting the movement intensity level of the target whose movement speed is less than the preset threshold to a preset value.

[0042] Optionally, the correspondence between the motion parameters and the noise reduction parameters has the following principle:

[0043] For the same type of target, the strength of the corresponding spatial noise reduction is positively correlated with the motion intensity level, and the strength of the corresponding temporal noise reduction is negatively correlated with the motion intensity level; and / or

[0044] For targets with the same motion intensity level, the corresponding spatial noise reduction parameters and temporal noise reduction parameters are adjusted according to the image clarity requirements and image effect requirements corresponding to different types of targets.

[0045] The video denoising method provided by the present invention adjusts the corresponding spatial denoising parameters and temporal denoising parameters according to different motion intensity levels and different types, and determines the corresponding relationship between the above motion parameters and the denoising parameters, which can improve the effect of video denoising and reduce the encoding bit rate.

[0046] In a second aspect, the present invention provides a video noise reduction device, including a memory and a processor, wherein:

[0047] The memory is used to store computer programs;

[0048] The processor is used to read the program in the memory and execute the following steps:

[0049] Performing target recognition on each frame image of the video to obtain the coordinates and type of at least one target in each frame image;

[0050] For the at least one target in each frame image, determine the moving speed of the target according to the coordinates of the target in the current frame image and the previous frame image and the time interval between adjacent frames;

[0051] According to the type and movement speed of the target, determine the spatial noise reduction parameters and the temporal noise reduction parameters corresponding to the target;

[0052] Image denoising is performed on the target according to the spatial denoising parameters and temporal denoising parameters corresponding to the target.

[0053] Optionally, the processor performs image denoising on the target according to a spatial denoising parameter and a temporal denoising parameter corresponding to the target, including:

[0054] Determine at least one image area corresponding to the target in N image areas; wherein the N image areas are obtained by dividing the current frame image according to a preset size;

[0055] Taking the image area as a unit, noise reduction is performed on at least one image area corresponding to the target according to the spatial noise reduction parameters and the temporal noise reduction parameters corresponding to the target.

[0056] Optionally, after determining the spatial domain noise reduction parameter and the temporal domain noise reduction parameter corresponding to the target, the processor is further configured to:

[0057] For each frame image, according to preset spatial domain noise reduction parameters and temporal domain noise reduction parameters, noise reduction is performed on the area other than the at least one target in the current frame image.

[0058] Optionally, the processor performs noise reduction on an area outside the at least one target in the current frame image, comprising:

[0059] Taking image regions as units, performing noise reduction on image regions other than at least one image region corresponding to the at least one target in the N image regions;

[0060] The N image regions are obtained by dividing the current frame image according to a preset size.

[0061] Optionally, the processor determines at least one image area corresponding to the target, including:

[0062] Determine at least one first image region intersecting with the target according to the coordinates of the target;

[0063] Calculate the moving direction and moving speed of the target according to the coordinates of the target in the current frame image and the previous frame image and the time interval between adjacent frames;

[0064] According to the moving direction and moving speed of the target, predicting, in the N image areas, a second image area that will intersect with the target after a preset time;

[0065] The first image area and the second image area are determined to be at least one image area corresponding to the target.

[0066] Optionally, the processor determines the moving speed of the target according to the coordinates of the target in the current frame image and the previous frame image and the time interval between adjacent frames, including:

[0067] Calculating the displacement between the coordinates of the pixel corresponding to the preset point of the target in the current frame image and the coordinates of the pixel corresponding to the preset point in the previous frame image as the pixel displacement of the target;

[0068] Compare the pixel displacement with the time interval between adjacent frames to obtain the pixel movement speed of the target;

[0069] The pixel moving speed is compared with the pixel width of the current frame image to obtain the moving speed of the target.

[0070] Optionally, the processor determines, according to the type and movement speed of the target, a spatial noise reduction parameter and a temporal noise reduction parameter corresponding to the target, including:

[0071] Determine the exercise intensity level of the target according to the preset correspondence between the exercise speed and the exercise intensity level and the exercise speed of the target;

[0072] Determine the spatial noise reduction parameters and the temporal noise reduction parameters corresponding to the target according to the correspondence between the preset motion parameters and the noise reduction parameters and the motion parameters of the target; wherein the motion parameters include the type and the motion intensity level, and the noise reduction parameters include the spatial noise reduction parameters and the temporal noise reduction parameters;

[0073] The spatial noise reduction parameters include at least one of different types of parameters that affect the strength of spatial noise reduction, and the temporal noise reduction parameters include at least one of different types of parameters that affect the strength of temporal noise reduction.

[0074] Optionally, the processor determines the exercise intensity level of the target according to a preset correspondence between the exercise speed and the exercise intensity level and the exercise speed of the target, including:

[0075] When the movement speed of the target is less than a preset threshold, determining the movement intensity level of the target to be a preset value;

[0076] When the target's movement speed is not less than a preset threshold, the target's movement intensity level is determined according to the preset correspondence between the movement speed and the movement intensity level and the target's movement speed; wherein, the greater the movement speed, the higher the corresponding movement intensity level.

[0077] Optionally, the correspondence between the motion parameters and the noise reduction parameters has the following principle:

[0078] For the same type of target, the strength of the corresponding spatial noise reduction is positively correlated with the motion intensity level, and the strength of the corresponding temporal noise reduction is negatively correlated with the motion intensity level; and / or

[0079] For targets with the same motion intensity level, the corresponding spatial noise reduction parameters and temporal noise reduction parameters are adjusted according to the image clarity requirements and image effect requirements corresponding to different types of targets.

[0080] In a third aspect, the present invention provides a video noise reduction device, comprising:

[0081] The target recognition unit is used to perform target recognition on each frame image of the video to obtain the coordinates and type of each target in each frame image;

[0082] A speed determination unit, for determining, for each target in each frame image, a moving speed of the target according to the coordinates of the target in the current frame image and the previous frame image and the time interval between the current frame image and the previous frame image;

[0083] A parameter determination unit, for determining, for each target in each frame image, a spatial domain noise reduction parameter and a temporal domain noise reduction parameter corresponding to the target according to the type and movement speed of the target;

[0084] The image denoising unit is used to perform image denoising on each target in each frame image according to the spatial denoising parameters and temporal denoising parameters corresponding to the target.

[0085] Optionally, the image denoising unit performs image denoising on the target according to a spatial denoising parameter and a temporal denoising parameter corresponding to the target, including:

[0086] Determine at least one image area corresponding to the target in N image areas; wherein the N image areas are obtained by dividing the current frame image according to a preset size;

[0087] Taking the image area as a unit, noise reduction is performed on at least one image area corresponding to the target according to the spatial noise reduction parameters and the temporal noise reduction parameters corresponding to the target.

[0088] Optionally, after determining the spatial domain noise reduction parameter and the temporal domain noise reduction parameter corresponding to the target, the image noise reduction unit is further used to:

[0089] For each frame image, according to preset spatial domain noise reduction parameters and temporal domain noise reduction parameters, noise reduction is performed on the area other than the at least one target in the current frame image.

[0090] Optionally, the image noise reduction unit performs noise reduction on a region other than the at least one target in the current frame image, including:

[0091] Taking image regions as units, performing noise reduction on image regions other than at least one image region corresponding to the at least one target in the N image regions;

[0092] The N image regions are obtained by dividing the current frame image according to a preset size.

[0093] Optionally, the image denoising unit determines at least one image area corresponding to the target, including:

[0094] Determine at least one first image region intersecting with the target according to the coordinates of the target;

[0095] Calculate the moving direction and moving speed of the target according to the coordinates of the target in the current frame image and the previous frame image and the time interval between adjacent frames;

[0096] According to the moving direction and moving speed of the target, predicting, in the N image areas, a second image area that will intersect with the target after a preset time;

[0097] The first image area and the second image area are determined to be at least one image area corresponding to the target.

[0098] Optionally, the speed determination unit determines the moving speed of the target according to the coordinates of the target in the current frame image and the previous frame image and the time interval between adjacent frames, including:

[0099] Calculating the displacement between the coordinates of the pixel corresponding to the preset point of the target in the current frame image and the coordinates of the pixel corresponding to the preset point in the previous frame image as the pixel displacement of the target;

[0100] Compare the pixel displacement with the time interval between adjacent frames to obtain the pixel movement speed of the target;

[0101] The pixel moving speed is compared with the pixel width of the current frame image to obtain the moving speed of the target.

[0102] Optionally, the parameter determination unit determines the spatial noise reduction parameter and the temporal noise reduction parameter corresponding to the target according to the type and movement speed of the target, including:

[0103] Determine the exercise intensity level of the target according to the preset correspondence between the exercise speed and the exercise intensity level and the exercise speed of the target;

[0104] Determine the spatial noise reduction parameters and the temporal noise reduction parameters corresponding to the target according to the correspondence between the preset motion parameters and the noise reduction parameters and the motion parameters of the target; wherein the motion parameters include the type and the motion intensity level, and the noise reduction parameters include the spatial noise reduction parameters and the temporal noise reduction parameters;

[0105] The spatial noise reduction parameters include at least one of different types of parameters that affect the strength of spatial noise reduction, and the temporal noise reduction parameters include at least one of different types of parameters that affect the strength of temporal noise reduction.

[0106] Optionally, the parameter determination unit determines the exercise intensity level of the target according to a preset correspondence between the exercise speed and the exercise intensity level and the exercise speed of the target, including:

[0107] When the movement speed of the target is less than a preset threshold, determining the movement intensity level of the target to be a preset value;

[0108] When the target's movement speed is not less than a preset threshold, the target's movement intensity level is determined according to the preset correspondence between the movement speed and the movement intensity level and the target's movement speed; wherein, the greater the movement speed, the higher the corresponding movement intensity level.

[0109] Optionally, the correspondence between the motion parameters and the noise reduction parameters has the following principle:

[0110] For the same type of target, the strength of the corresponding spatial noise reduction is positively correlated with the motion intensity level, and the strength of the corresponding temporal noise reduction is negatively correlated with the motion intensity level; and / or

[0111] For targets with the same motion intensity level, the corresponding spatial noise reduction parameters and temporal noise reduction parameters are adjusted according to the image clarity requirements and image effect requirements corresponding to different types of targets.

[0112] In a fourth aspect, the present invention provides a computer program medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the video noise reduction method provided in the first aspect above.

[0113] In a fifth aspect, the present invention provides a chip, which is coupled to a memory in a device so that the chip calls program instructions stored in the memory during operation to implement the above-mentioned various aspects of the embodiments of the present application and any possible video noise reduction method involved in each aspect.

[0114] In a sixth aspect, the present invention provides a computer program product. When the computer program product runs on an electronic device, the electronic device executes the video noise reduction method that implements the above-mentioned various aspects of the embodiments of the present application and any possible video noise reduction method involved in each aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0115] Figure 1 A schematic diagram of an application scenario of a video noise reduction method provided by an embodiment of the present invention;

[0116] Figure 2 A schematic diagram of an application scenario of another video noise reduction method provided by an embodiment of the present invention;

[0117] Figure 3 A flowchart of a video noise reduction method provided by an embodiment of the present invention;

[0118] Figure 4 A schematic diagram of a target recognition result provided by an embodiment of the present invention;

[0119] Figure 5 A schematic diagram of an image area provided by an embodiment of the present invention;

[0120] Figure 6 A flowchart of an implementation of a video noise reduction method provided by an embodiment of the present invention;

[0121] Figure 7 A schematic diagram of a video noise reduction device provided by an embodiment of the present invention;

[0122] Figure 8 A schematic diagram of a video noise reduction device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0123] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0124] It should be noted that the terms "first", "second", etc. in the specification and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the numbers used in this way can be interchanged where appropriate, so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein.

[0125] The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the attached claims. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0126] In the description of the embodiments of the present application, unless otherwise specified, " / " means or, for example, A / B can mean A or B; "and / or" in the text is merely a description of the association relationship of associated objects, indicating that three relationships may exist, for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, in the description of the embodiments of the present application, "multiple" refers to two or more than two, and other quantifiers are similar. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present application, and are not used to limit the present application, and the embodiments of the present application and the features in the embodiments may be combined with each other without conflict.

[0127] Hereinafter, some terms used in the embodiments of the present invention are explained to facilitate understanding by those skilled in the art.

[0128] (1) The term "Digital Video Recorder (DVR)" in the embodiments of the present invention refers to a digital video recorder. Compared with a traditional analog video recorder, a digital video recorder uses a hard disk for recording, and is therefore often referred to as a hard disk recorder. It is a computer system for image storage and processing, and has the functions of long-term video recording, audio recording, remote monitoring and control of images and / or voices.

[0129] (2) The term “Image Signal Processor (ISP)” in the embodiments of the present invention refers to a pipelined image processing engine that can process image signals at high speed. It occupies a core and dominant position in the camera system and is an important device that constitutes the camera.

[0130] (3) The term "central processing unit (CPU)" in the embodiments of the present invention refers to one of the main devices of an electronic computer and a core component in a computer. Its function is to interpret computer instructions and process data in computer software.

[0131] (4) The term "frames per second (FPS)" in the embodiments of the present invention is defined in the field of images and refers to the number of frames of an animation or video.

[0132] (5) The term "pixel width" in the embodiments of the present invention, also called resolution, refers to the array of horizontal and vertical pixels that can be displayed, and is the number of pixels into which a picture or screen is divided horizontally. For example, when the resolution is 1240×768, it means that the picture or screen is divided horizontally into 1240 pixels and vertically into 768 pixels, where 1240 is the pixel width.

[0133] In view of the above-mentioned problems existing in the video noise reduction method of the related art, the present application proposes a video noise reduction method, device and equipment.

[0134] A video noise reduction method, device and equipment in an embodiment of the present application are described in detail below with reference to the accompanying drawings.

[0135] Example 1

[0136] like Figure 1 As shown, an embodiment of the present invention provides a schematic diagram of an application scenario of a video noise reduction method, including:

[0137] The camera device 101 is used to capture video images and transmit the captured video images to a hard disk recorder.

[0138] The hard disk video recorder 102 is used to perform target recognition on each frame image of the video to obtain the coordinates and type of at least one target in each frame image; for the above-mentioned at least one target in each frame image, determine the movement speed of the target according to the coordinates of the target in the current frame image and the previous frame image and the time interval between adjacent frames; determine the spatial domain noise reduction parameters and time domain noise reduction parameters corresponding to the target according to the type and movement speed of the target; and perform image noise reduction on the target according to the spatial domain noise reduction parameters and time domain noise reduction parameters corresponding to the target.

[0139] The hard disk video recorder includes a central processing unit, the camera device transmits the collected video signal to the hard disk video recorder, and the hard disk video recorder uses the central processing unit to perform noise reduction processing on the video signal.

[0140] It should be noted that any device that can realize the functions of the above-mentioned camera device and the above-mentioned hard disk recorder can be applied to the embodiments of the present invention, and the type, model, specifications, etc. of the above-mentioned camera device and the above-mentioned hard disk recorder can be specifically set according to the specific implementation situation. For example, the type of the above-mentioned camera device is set to be a camera, and the embodiments of the present invention do not impose any limitation on this.

[0141] It should be noted that the above application scenario is only an illustration of a possible implementation method in the embodiment of the present invention, and does not limit the embodiment of the present invention. On the contrary, the implementation method of the present application can be applied to any applicable scenario, and the above application scenario can be physically added, deleted, or modified according to the specific implementation situation, for example, using multiple cameras to capture video images.

[0142] like Figure 2 As shown, an embodiment of the present invention provides a schematic diagram of another application scenario of a video noise reduction method, including:

[0143] The image sensor 201 is used to capture video images and transmit the captured video images to the image signal processor.

[0144] An image signal processor 202 is used to process the image signal output by the image sensor and transmit the processed image signal to the central processing unit;

[0145] The central processing unit 203 is used to perform target recognition on each frame image of the video to obtain the coordinates and type of at least one target in each frame image; for the above-mentioned at least one target in each frame image, determine the movement speed of the target according to the coordinates of the target in the current frame image and the previous frame image and the time interval between adjacent frames; determine the spatial noise reduction parameters and time domain noise reduction parameters corresponding to the target according to the type and movement speed of the target; and perform image noise reduction on the target according to the spatial noise reduction parameters and time domain noise reduction parameters corresponding to the target.

[0146] Above Figure 2 A camera architecture is provided, wherein the image signal processor outputs a video image to the central processor, and the central processor performs noise reduction processing on the input video image.

[0147] It should be noted that any device that can realize the functions of the above-mentioned image sensor, image signal processor and central processing unit can be applied to the embodiments of the present invention. The types, models, specifications, etc. of the above-mentioned image sensors, image signal processors and central processing units can be specifically set according to the specific implementation situation. The embodiments of the present invention do not impose any limitations on this.

[0148] It should be noted that the above application scenario is only an illustration of a possible implementation method in the embodiment of the present invention, and does not limit the embodiment of the present invention. On the contrary, the implementation method of the present application can be applied to any applicable scenario, and the above application scenario can be physically added, deleted, or modified according to the specific implementation situation. For example, the above central processing unit can be changed to a graphics processor, etc.

[0149] The embodiment of the present invention provides a flowchart of a video noise reduction method, which is applied to the hard disk video recorder 102 and / or the central processor 203. Figure 3 As shown, including:

[0150] Step S301, performing target recognition on each frame image of the video to obtain the coordinates and type of at least one target in each frame image;

[0151] For each frame of the input video, target recognition is performed using a target recognition algorithm in combination with a preset model to obtain the coordinates and type of at least one target in the frame.

[0152] The above-mentioned target recognition algorithm can be any algorithm that can realize the target recognition function, including but not limited to: (1) Region-Convolutional Neural Networks (R-CNN) algorithms based on candidate regions (Region Proposal): for example, Region-Convolutional Neural Networks (R-CNN) algorithm, Fast Region-Convolutional Neural Networks (Fast R-CNN) algorithm, Faster Region-Convolutional Neural Networks (Faste R-CNN) algorithm, etc.); (2) You Only Look Once (YOLO) algorithm, Single Shot MultiBox Detector (SSD) algorithm, etc.

[0153] The above preset model is a set of neural network parameters obtained through pre-training of the neural network based on sample data, and is an important component of the target recognition algorithm.

[0154] By combining the above target recognition algorithm with the above preset model, the function of identifying the type of target and its coordinate position in the input image can be realized.

[0155] It should be noted that any target recognition method that can achieve target recognition effect can be applied to the embodiments of the present invention, and is not limited to the implementation method of combining the target recognition algorithm with a preset model. The embodiments of the present invention do not impose any limitations on this.

[0156] In specific implementation, it can be set to identify all the targets in the above image, or it can be set to identify some targets in the above image, that is, targets of interest.

[0157] For example, in a security monitoring scenario where the input video is a traffic video, the types of targets to be identified can be set to include any one or more of the following: people, animals, plants, motor vehicles, non-motor vehicles, and signs. The above types can also be expanded.

[0158] It should be noted that the types of the above-mentioned targets of interest can be specifically set according to the specific implementation scenario. For example, in the security monitoring scenario of an automated teller machine (ATM), the types of the above-mentioned targets of interest can be set to wallets, bank cards, cash, etc. The embodiments of the present invention do not impose any limitations on this.

[0159] The coordinates of the above-mentioned targets can be expressed in the form of numerical values, such as (left, top, right, bottom), where left represents the position of the leftmost side of the target in the above-mentioned image, right represents the position of the rightmost side of the target in the above-mentioned image, top represents the position of the uppermost end of the target in the above-mentioned image, and bottom represents the position of the lowermost end of the target in the above-mentioned image; the coordinates of the above-mentioned targets can also be expressed in the form of area markings, for example, using rectangular areas to mark each target in the above-mentioned image, where (left, top) represents the coordinates of the upper left corner of the rectangular area, and (right, bottom) represents the coordinates of the lower right corner of the rectangular area.

[0160] like Figure 4 As shown, an embodiment of the present invention provides a schematic diagram of a target recognition result.

[0161] In the above image, the first target is identified, the type of the above first target is output as human, and the above first target is marked with a rectangular area, wherein (left1, top1) is used to represent the upper left corner coordinate of the rectangular area of ​​the above first target, and (right1, bottom1) is used to represent the lower right corner coordinate of the rectangular area of ​​the above first target.

[0162] The second target is identified, the type of the second target is output as human, and the second target is marked with a rectangular area, wherein (left2, top2) is used to represent the upper left corner coordinate of the rectangular area of ​​the second target, and (right2, bottom2) is used to represent the lower right corner coordinate of the rectangular area of ​​the second target.

[0163] The third target is identified, the type of the third target is output as a motor vehicle, and the third target is marked with a rectangular area, wherein (left3, top3) is used to represent the upper left corner coordinate of the rectangular area of ​​the third target, and (right3, bottom3) is used to represent the lower right corner coordinate of the rectangular area of ​​the third target.

[0164] Step S302, for the at least one target in each frame image, determine the moving speed of the target according to the coordinates of the target in the current frame image and the previous frame image and the time interval between adjacent frames;

[0165] For each target in each frame image, the pixel displacement of the preset point of the target between adjacent frames is calculated based on the coordinates of the target in the current frame image and the previous frame image and the time interval between adjacent frames, and the relative pixel speed obtained by dividing the above pixel displacement by the image width is determined as the movement speed of the target.

[0166] Step S303, determining the spatial domain noise reduction parameters and the temporal domain noise reduction parameters corresponding to the target according to the type and movement speed of the target;

[0167] The above-mentioned spatial noise reduction parameters include at least one of the different types of parameters that affect the strength of spatial noise reduction, and the above-mentioned time domain noise reduction parameters include at least one of the different types of parameters that affect the strength of time domain noise reduction.

[0168] The above-mentioned spatial denoising parameters and the above-mentioned temporal denoising parameters can be specifically set according to the specific implementation scenario. For example, the above-mentioned spatial denoising parameters are set to include the spatial luminance denoising intensity, which has a value range of 0 to 2047. The larger the value, the greater the intensity. The default value is 128; the spatial chrominance denoising intensity has a value range of 0 to 255. The larger the value, the greater the intensity. The default value is 12. The above-mentioned temporal denoising parameters are set to include the temporal luminance denoising intensity, which has a value range of 0 to 63. The larger the value, the greater the intensity. The default value is 12; the temporal chrominance denoising intensity has a value range of 0 to 32. The larger the value, the greater the intensity. The default value is 16.

[0169] The embodiment of the present invention comprehensively considers the type and movement speed of the target, and determines corresponding spatial noise reduction parameters and temporal noise reduction parameters for different targets, which is beneficial to improving the effect of video noise reduction.

[0170] It should be noted that the above-mentioned spatial domain noise reduction parameters and time domain noise reduction parameters include not only the types of parameters, but also the specific values ​​of the parameters.

[0171] Step S304: performing image denoising on the target according to the spatial denoising parameters and the temporal denoising parameters corresponding to the target.

[0172] According to the spatial domain denoising parameters and temporal domain denoising parameters corresponding to each target, image denoising is performed on each target respectively.

[0173] The embodiment of the present invention adopts the following implementation method to determine the moving speed of the target according to the coordinates of the target in the current frame image and the previous frame image and the time interval between adjacent frames:

[0174] Calculate the displacement between the coordinates of the pixel corresponding to the preset point of the target in the current frame image and the coordinates of the pixel corresponding to the preset point in the previous frame image as the pixel displacement of the target;

[0175] Compare the above pixel displacement with the time interval between adjacent frames to obtain the pixel movement speed of the target;

[0176] The pixel moving speed is compared with the pixel width of the current frame image to obtain the moving speed of the target.

[0177] The motion speed obtained by comparing the above pixel movement speed with the pixel width of the above current frame image is the relative pixel movement speed.

[0178] When the above preset point is the center point of the target, the formula is used: Calculate the target's moving speed;

[0179] in, is the horizontal coordinate of the center point of the target in the current frame image, is the vertical coordinate of the center point of the target in the current frame image, is the horizontal coordinate of the center point of the target in the previous frame image, is the vertical coordinate of the center point of the target in the previous frame image, is the time interval between adjacent frames, and W is the pixel width of the current frame image.

[0180] Among them, (left, top, right, bottom) is the coordinate of the target in the current frame image, (left', top', right', bottom') is the coordinate of the target in the previous frame image, and Fps is the number of frames transmitted per second of the video.

[0181] It should be noted that the above-mentioned preset point can be any point in the above-mentioned target, and the calculation method is basically the same as the above-mentioned calculation method, and it is only necessary to adaptably modify the above-mentioned calculation formulas of X, Y, X', and Y'.

[0182] When determining the spatial noise reduction parameters and the time domain noise reduction parameters, the embodiment of the present invention takes into account the movement speed of the target and provides a specific implementation method for calculating the movement speed of the target: the movement speed of the target is calculated through the coordinates of a preset point of the same target in two adjacent frames and the time interval between adjacent frames, which provides the necessary technical basis for subsequently determining the spatial noise reduction parameters and the time domain noise reduction parameters, and helps to improve the effect of video noise reduction and reduce the encoding bit rate.

[0183] Based on the type of the target determined in step S301 and the moving speed of the target determined in step S302, the embodiment of the present invention adopts the following implementation to determine the spatial noise reduction parameters and the temporal noise reduction parameters corresponding to the target:

[0184] Determine the exercise intensity level of the target according to the preset correspondence between the exercise speed and the exercise intensity level and the exercise speed of the target;

[0185] According to the correspondence between the preset motion parameters and the noise reduction parameters and the motion parameters of the target, the spatial noise reduction parameters and the time domain noise reduction parameters corresponding to the target are determined; wherein the above-mentioned motion parameters include the type and the motion intensity level, and the above-mentioned noise reduction parameters include the spatial noise reduction parameters and the time domain noise reduction parameters.

[0186] As an optional implementation, the correspondence between the preset exercise speed and exercise intensity level may be in the form of the following first index table:

[0187]

[0188] It should be noted that the specific values ​​of the first index table and the number of level divisions can be specifically set according to specific implementation conditions, and the embodiment of the present invention does not impose any limitation on this.

[0189] The above-mentioned A is a preset threshold value, which can be specifically set according to the specific implementation situation, and the embodiment of the present invention does not impose any limitation on this.

[0190] As an optional implementation, the correspondence between the preset motion parameters and the noise reduction parameters may be in the form of the following second index table:

[0191]

[0192]

[0193] It should be noted that the spatial noise reduction parameters and temporal noise reduction parameters in the above-mentioned second index table are general terms, referring to one or more parameters that affect the intensity or effect of spatial noise reduction and temporal noise reduction.

[0194] In the second index table, different motion intensity levels and different types correspond to different spatial noise reduction parameters and temporal noise reduction parameters.

[0195] It should be noted that the spatial noise reduction parameters and the temporal noise reduction parameters in the above-mentioned second index table are pre-adjusted preset values, and the specific numerical values ​​of the preset values ​​can be specifically set according to the specific implementation situation, and the embodiment of the present invention does not impose any limitation on this.

[0196] The embodiments of the present invention match the corresponding motion intensity level for each target through the relative pixel speed of each target; distinguish the types and motion intensity levels of different targets, and match the corresponding spatial noise reduction parameters and temporal noise reduction parameters for targets of different types and / or motion intensity levels, so as to ensure a higher noise reduction effect and reduce the encoding bit rate.

[0197] The embodiment of the present invention adopts the following implementation method to determine the exercise intensity level of the target according to the corresponding relationship between the preset exercise speed and the exercise intensity level and the exercise speed of the target:

[0198] When the movement speed of the target is less than a preset threshold, determining the movement intensity level of the target to be a preset value;

[0199] When the target's movement speed is not less than a preset threshold, the target's movement intensity level is determined according to the preset correspondence between the movement speed and the movement intensity level and the target's movement speed; wherein, the greater the movement speed, the higher the corresponding movement intensity level.

[0200] As an optional implementation, the above preset value is 0.

[0201] Taking into account the errors of the target recognition algorithm itself, for example, for a stationary target, there are still differences in the coordinates of the two frames before and after. In order to eliminate the jitter of the target recognition algorithm itself, A in the above-mentioned first index table is used to represent the above-mentioned preset threshold. When the target's movement speed is less than the above-mentioned preset threshold, the target is considered to be stationary, and the corresponding motion intensity level is 0; when the target's movement speed is not less than the above-mentioned preset threshold and falls within any interval in the table, a corresponding motion intensity level will be obtained, indicating the intensity of the target's movement. The larger the motion intensity level, the more intense the movement of the target.

[0202] The embodiment of the present invention can reduce the error of target recognition and improve the video noise reduction effect by setting a preset threshold for the target's motion speed and setting the motion intensity level of the target whose motion speed is less than the preset threshold to a preset value.

[0203] The corresponding relationship between the above motion parameters and noise reduction parameters has the following principles:

[0204] For the same type of target, the strength of the corresponding spatial noise reduction is positively correlated with the motion intensity level, and the strength of the corresponding temporal noise reduction is negatively correlated with the motion intensity level; and / or

[0205] For targets with the same motion intensity level, the corresponding spatial noise reduction parameters and temporal noise reduction parameters are adjusted according to the image clarity requirements and image effect requirements corresponding to different types of targets.

[0206] For the same type of target, the lower the motion intensity level, the stronger the temporal noise reduction is, and the intensity of spatial noise reduction is appropriately reduced, so as to give full play to the advantages of temporal noise reduction and remove the noise in the video sequence while maintaining the details of the video image; the higher the motion intensity level, the weaker the temporal noise reduction is, and the intensity of spatial noise reduction is appropriately increased to avoid the phenomenon of ghosting, and try to use the remaining noise reduction to reduce the noise and ensure the image effect.

[0207] For targets with the same motion intensity level, in actual application scenarios, different types of targets will have different requirements for picture effects. For example, we hope to see the facial features of a person and the license plate number of a car clearly, but the clarity requirements for animals are not high. In addition, the textures and color ranges of faces and license plates vary greatly. We can fine-tune different noise reduction parameters to optimize different types of targets in a targeted manner and determine the corresponding spatial noise reduction parameters and time domain noise reduction parameters.

[0208] The embodiment of the present invention adopts the above principles to determine the correspondence between the above motion parameters and noise reduction parameters, and adjusts the corresponding spatial noise reduction parameters and temporal noise reduction parameters for different motion intensity levels and different types of targets, which can improve the effect of video noise reduction and reduce the encoding bit rate.

[0209] The embodiment of the present invention adopts the following implementation method to perform image denoising on the target according to the spatial denoising parameters and the temporal denoising parameters corresponding to the target:

[0210] Determine at least one image area corresponding to the target in N image areas; wherein the N image areas are obtained by dividing the current frame image according to a preset size;

[0211] Taking the image area as a unit, noise reduction is performed on at least one image area corresponding to the target according to the spatial noise reduction parameters and the temporal noise reduction parameters corresponding to the target.

[0212] The specific value of N can be specifically set according to specific implementation conditions, and the division method of the image area can also be specifically set according to specific implementation conditions, which is not limited in any way in the embodiment of the present invention.

[0213] Considering that the image of the input video is generally rectangular, the current frame image can be divided into N rectangular areas of equal size.

[0214] As an optional implementation manner, determining at least one image area corresponding to the target includes:

[0215] Determine at least one first image region intersecting with the target according to the coordinates of the target;

[0216] Calculate the moving direction and moving speed of the target according to the coordinates of the target in the current frame image and the previous frame image and the time interval between adjacent frames;

[0217] According to the moving direction and moving speed of the target, predicting, in the N image areas, a second image area that will intersect with the target after a preset time;

[0218] The first image region and the second image region are determined to be at least one image region corresponding to the target.

[0219] When the target and a certain image region have an intersection, the image region is determined to be the first image region.

[0220] The specific value of the preset time can be set according to the specific implementation situation, for example, it can be set to 2 to 3 frames, and the embodiment of the present invention does not impose any limitation on this.

[0221] The embodiment of the present invention divides the current frame image into N image areas of the same size, predicts the position of the target after a preset time according to the coordinates and movement speed of the target, and uses the image area where the current position and the predicted position are located as the noise reduction area of ​​the target, taking into account the processing time of time domain noise reduction, improving the video noise reduction effect and reducing the image encoding bit rate.

[0222] like Figure 5 As shown, an embodiment of the present invention provides a schematic diagram of an image area.

[0223] In the above Figure 5 In the example, N=63, that is, the image is divided into 63 equal parts, each of which has a number (1 to 63). Figure 5 Taking the car in the figure as an example, the process of determining at least one image area corresponding to the target is explained:

[0224] According to the coordinates of the target, at least one first image region A={40, 41, 42, 49, 50, 51} intersecting with the target is determined.

[0225] The moving direction and moving speed of the target are calculated according to the coordinates of the target in the current frame image and the previous frame image and the time interval between adjacent frames.

[0226] The above movement direction and movement speed can be expressed as the movement speed in vector form: Indicates, where a = (X′-X) / T, represents the moving speed of the target in the horizontal coordinate direction, b = (Y′-Y) / T, represents the moving speed of the target in the vertical coordinate direction, X′ is the horizontal coordinate of the preset point of the target in the previous frame image, Y′ is the vertical coordinate of the preset point of the target in the previous frame image, X is the horizontal coordinate of the preset point of the target in the current frame image, and Y is the horizontal coordinate of the preset point of the target in the current frame image. is the time interval between adjacent frames, and Fps is the number of frames transmitted per second in the video.

[0227] The preset point may be any point in the target. As an optional implementation, the preset point is the center point of the target.

[0228] According to the moving direction and moving speed of the target, a second image area intersecting with the target after a preset time is predicted in the N image areas.

[0229] Determine the speed of the above movement After that, predict the new position of the preset point of the target after t1 time:

[0230] X″=X+a*t1;Y″=X+b*t1;wherein, X″ is the abscissa of the new position of the preset point of the target, and Y″ is the ordinate of the new position of the preset point of the target.

[0231] As mentioned above Figure 5 In the above second target car, the moving direction of the car is rightward. After the preset time t, it moves a short distance to the right and moves to the dotted area. Then the second image area B = {41, 42, 43, 50, 51, 52}.

[0232] The first image region and the second image region are determined to be at least one image region corresponding to the target.

[0233] Finally, at least one image area corresponding to the target is determined, that is, the denoising area set C=A∪B of the target, which is {40, 41, 42, 43, 49, 50, 51, 52}. The image area will use the same set of spatial denoising parameters and temporal denoising parameters.

[0234] As an optional implementation manner, after determining the spatial domain noise reduction parameter and the temporal domain noise reduction parameter corresponding to the target, the method further includes:

[0235] For each frame image, according to preset spatial domain noise reduction parameters and temporal domain noise reduction parameters, noise reduction is performed on the area other than the at least one target in the current frame image.

[0236] The embodiment of the present invention adopts preset spatial noise reduction parameters and time domain noise reduction parameters for non-target areas. As an optional implementation mode, the strength of the above-mentioned preset spatial noise reduction parameters and time domain noise reduction parameters is greater than the strength of the spatial noise reduction parameters and time domain noise reduction parameters corresponding to the target, which can reduce the picture details of non-interest areas to reduce the subsequent encoding bit rate.

[0237] The embodiment of the present invention adopts the following implementation to reduce noise in the area other than the at least one target in the current frame image:

[0238] Taking image regions as units, performing noise reduction on image regions other than at least one image region corresponding to the at least one target in the N image regions;

[0239] The N image regions are obtained by dividing the current frame image according to a preset size.

[0240] The embodiment of the present invention adopts a partitioning method to perform noise reduction on each image area, provides a specific noise reduction processing method, and improves the feasibility of the embodiment of the present invention.

[0241] After determining the noise reduction area of ​​each target in the current frame, the noise reduction parameters corresponding to the target are matched, and all image areas in the noise reduction area use the noise reduction parameters; a set of default noise reduction parameters are used for each image area in the remaining non-target noise reduction area, and the noise reduction parameters can be appropriately stronger to reduce the bit rate; after the noise reduction parameters are matched in all image areas, noise reduction processing is performed on each image area as a unit.

[0242] In the related art, the spatial noise reduction parameters and strength, temporal noise reduction parameters and strength of each frame of the image are static and unchanged, and the image content is not treated differently, which cannot guarantee the noise reduction effect of the image in different scenes, and is not conducive to reducing the encoding bit rate in a scene-based manner. However, this application comprehensively considers the type and movement speed of the target, matches the corresponding spatial noise reduction parameters and temporal noise reduction parameters for targets of different types and / or movement speeds, ensures the noise reduction effect of the image in different scenes, and reduces the encoding bit rate in a scene-based manner.

[0243] like Figure 6 As shown, an embodiment of the present invention provides a flowchart of an implementation of a video noise reduction method, including:

[0244] Step S601, performing target recognition on each frame image of the video to obtain the coordinates and type of at least one target in each frame image;

[0245] Step S602, for the at least one target in each frame image, determine the moving speed of the target according to the coordinates of the target in the current frame image and the previous frame image and the time interval between adjacent frames;

[0246] Step S603, determining the exercise intensity level of the target according to the preset correspondence between the exercise speed and the exercise intensity level and the exercise speed of the target;

[0247] Step S604, determining the spatial noise reduction parameters and the temporal noise reduction parameters corresponding to the target according to the correspondence between the preset motion parameters and the noise reduction parameters and the motion parameters of the target;

[0248] The above-mentioned motion parameters include the type and motion intensity level, and the above-mentioned noise reduction parameters include the spatial noise reduction parameters and the temporal noise reduction parameters;

[0249] The above-mentioned spatial noise reduction parameters include at least one of the different types of parameters that affect the strength of spatial noise reduction, and the above-mentioned time domain noise reduction parameters include at least one of the different types of parameters that affect the strength of time domain noise reduction.

[0250] Step S605, determining at least one image area corresponding to the target in the N image areas;

[0251] The N image regions are obtained by dividing the current frame image according to a preset size;

[0252] Step S606, taking the image area as a unit, and performing noise reduction on at least one image area corresponding to the target according to the spatial noise reduction parameter and the temporal noise reduction parameter corresponding to the target;

[0253] Step S607, for each frame of image, according to preset spatial noise reduction parameters and temporal noise reduction parameters, taking image regions as units, denoising is performed on image regions other than at least one image region corresponding to the at least one target in the N image regions.

[0254] Example 2

[0255] The embodiment of the present invention provides a schematic diagram of a video noise reduction device 700, including a memory 701 and a processor 702. Figure 7 As shown, where:

[0256] The memory is used to store computer programs;

[0257] The processor is used to read the program in the memory and execute the following steps:

[0258] Performing target recognition on each frame image of the video to obtain the coordinates and type of at least one target in each frame image;

[0259] For the at least one target in each frame image, determine the moving speed of the target according to the coordinates of the target in the current frame image and the previous frame image and the time interval between adjacent frames;

[0260] According to the type and movement speed of the target, determine the spatial noise reduction parameters and the temporal noise reduction parameters corresponding to the target;

[0261] Image denoising is performed on the target according to the spatial denoising parameters and temporal denoising parameters corresponding to the target.

[0262] Optionally, the processor performs image noise reduction on the target according to a spatial noise reduction parameter and a temporal noise reduction parameter corresponding to the target, including:

[0263] Determine at least one image area corresponding to the target in N image areas; wherein the N image areas are obtained by dividing the current frame image according to a preset size;

[0264] Taking the image area as a unit, noise reduction is performed on at least one image area corresponding to the target according to the spatial noise reduction parameters and the temporal noise reduction parameters corresponding to the target.

[0265] Optionally, after determining the spatial domain noise reduction parameter and the temporal domain noise reduction parameter corresponding to the target, the processor is further configured to:

[0266] For each frame image, according to preset spatial domain noise reduction parameters and temporal domain noise reduction parameters, noise reduction is performed on the area other than the at least one target in the current frame image.

[0267] Optionally, the processor performs noise reduction on an area outside the at least one target in the current frame image, comprising:

[0268] Taking image regions as units, performing noise reduction on image regions other than at least one image region corresponding to the at least one target in the N image regions;

[0269] The N image regions are obtained by dividing the current frame image according to a preset size.

[0270] Optionally, the processor determines at least one image area corresponding to the target, including:

[0271] Determine at least one first image region intersecting with the target according to the coordinates of the target;

[0272] Calculate the moving direction and moving speed of the target according to the coordinates of the target in the current frame image and the previous frame image and the time interval between adjacent frames;

[0273] According to the moving direction and moving speed of the target, predicting, in the N image areas, a second image area that will intersect with the target after a preset time;

[0274] The first image area and the second image area are determined to be at least one image area corresponding to the target.

[0275] Optionally, the processor determines the moving speed of the target according to the coordinates of the target in the current frame image and the previous frame image and the time interval between adjacent frames, including:

[0276] Calculating the displacement between the coordinates of the pixel corresponding to the preset point of the target in the current frame image and the coordinates of the pixel corresponding to the preset point in the previous frame image as the pixel displacement of the target;

[0277] Compare the pixel displacement with the time interval between adjacent frames to obtain the pixel movement speed of the target;

[0278] The pixel moving speed is compared with the pixel width of the current frame image to obtain the moving speed of the target.

[0279] Optionally, the processor determines, according to the type and movement speed of the target, a spatial noise reduction parameter and a temporal noise reduction parameter corresponding to the target, including:

[0280] Determine the exercise intensity level of the target according to the preset correspondence between the exercise speed and the exercise intensity level and the exercise speed of the target;

[0281] Determine the spatial noise reduction parameters and the temporal noise reduction parameters corresponding to the target according to the correspondence between the preset motion parameters and the noise reduction parameters and the motion parameters of the target; wherein the motion parameters include the type and the motion intensity level, and the noise reduction parameters include the spatial noise reduction parameters and the temporal noise reduction parameters;

[0282] The spatial noise reduction parameters include at least one of different types of parameters that affect the strength of spatial noise reduction, and the temporal noise reduction parameters include at least one of different types of parameters that affect the strength of temporal noise reduction.

[0283] Optionally, the processor determines the exercise intensity level of the target according to a preset correspondence between the exercise speed and the exercise intensity level and the exercise speed of the target, including:

[0284] When the movement speed of the target is less than a preset threshold, determining the movement intensity level of the target to be a preset value;

[0285] When the target's movement speed is not less than a preset threshold, the target's movement intensity level is determined according to the preset correspondence between the movement speed and the movement intensity level and the target's movement speed; wherein, the greater the movement speed, the higher the corresponding movement intensity level.

[0286] Optionally, the correspondence between the motion parameters and the noise reduction parameters has the following principle:

[0287] For the same type of target, the strength of the corresponding spatial noise reduction is positively correlated with the motion intensity level, and the strength of the corresponding temporal noise reduction is negatively correlated with the motion intensity level; and / or

[0288] For targets with the same motion intensity level, the corresponding spatial noise reduction parameters and temporal noise reduction parameters are adjusted according to the image clarity requirements and image effect requirements corresponding to different types of targets.

[0289] An embodiment of the present invention provides a schematic diagram of a video noise reduction device, such as Figure 8 As shown, including:

[0290] The target recognition unit 801 is used to perform target recognition on each frame image of the video to obtain the coordinates and type of each target in each frame image;

[0291] The speed determination unit 802 is used to determine the moving speed of each target in each frame image according to the coordinates of the target in the current frame image and the previous frame image and the time interval between the current frame image and the previous frame image;

[0292] A parameter determination unit 803 is used to determine, for each target in each frame image, a spatial noise reduction parameter and a temporal noise reduction parameter corresponding to the target according to the type and movement speed of the target;

[0293] The image denoising unit 804 is used to perform image denoising on each target in each frame image according to the spatial denoising parameter and the temporal denoising parameter corresponding to the target.

[0294] Optionally, the image denoising unit performs image denoising on the target according to a spatial denoising parameter and a temporal denoising parameter corresponding to the target, including:

[0295] Determine at least one image area corresponding to the target in N image areas; wherein the N image areas are obtained by dividing the current frame image according to a preset size;

[0296] Taking the image area as a unit, noise reduction is performed on at least one image area corresponding to the target according to the spatial noise reduction parameters and the temporal noise reduction parameters corresponding to the target.

[0297] Optionally, after determining the spatial domain noise reduction parameter and the temporal domain noise reduction parameter corresponding to the target, the image noise reduction unit is further used to:

[0298] For each frame image, according to preset spatial domain noise reduction parameters and temporal domain noise reduction parameters, noise reduction is performed on the area other than the at least one target in the current frame image.

[0299] Optionally, the image noise reduction unit performs noise reduction on a region other than the at least one target in the current frame image, including:

[0300] Taking image regions as units, performing noise reduction on image regions other than at least one image region corresponding to the at least one target in the N image regions;

[0301] The N image regions are obtained by dividing the current frame image according to a preset size.

[0302] Optionally, the image denoising unit determines at least one image area corresponding to the target, including:

[0303] Determine at least one first image region intersecting with the target according to the coordinates of the target;

[0304] Calculate the moving direction and moving speed of the target according to the coordinates of the target in the current frame image and the previous frame image and the time interval between adjacent frames;

[0305] According to the moving direction and moving speed of the target, predicting, in the N image areas, a second image area that will intersect with the target after a preset time;

[0306] The first image area and the second image area are determined to be at least one image area corresponding to the target.

[0307] Optionally, the speed determination unit determines the moving speed of the target according to the coordinates of the target in the current frame image and the previous frame image and the time interval between adjacent frames, including:

[0308] Calculating the displacement between the coordinates of the pixel corresponding to the preset point of the target in the current frame image and the coordinates of the pixel corresponding to the preset point in the previous frame image as the pixel displacement of the target;

[0309] Compare the pixel displacement with the time interval between adjacent frames to obtain the pixel movement speed of the target;

[0310] The pixel moving speed is compared with the pixel width of the current frame image to obtain the moving speed of the target.

[0311] Optionally, the parameter determination unit determines the spatial noise reduction parameter and the temporal noise reduction parameter corresponding to the target according to the type and movement speed of the target, including:

[0312] Determine the exercise intensity level of the target according to the preset correspondence between the exercise speed and the exercise intensity level and the exercise speed of the target;

[0313] Determine the spatial noise reduction parameters and the temporal noise reduction parameters corresponding to the target according to the correspondence between the preset motion parameters and the noise reduction parameters and the motion parameters of the target; wherein the motion parameters include the type and the motion intensity level, and the noise reduction parameters include the spatial noise reduction parameters and the temporal noise reduction parameters;

[0314] The spatial noise reduction parameters include at least one of different types of parameters that affect the strength of spatial noise reduction, and the temporal noise reduction parameters include at least one of different types of parameters that affect the strength of temporal noise reduction.

[0315] Optionally, the parameter determination unit determines the exercise intensity level of the target according to a preset correspondence between the exercise speed and the exercise intensity level and the exercise speed of the target, including:

[0316] When the movement speed of the target is less than a preset threshold, determining the movement intensity level of the target to be a preset value;

[0317] When the target's movement speed is not less than a preset threshold, the target's movement intensity level is determined according to the preset correspondence between the movement speed and the movement intensity level and the target's movement speed; wherein, the greater the movement speed, the higher the corresponding movement intensity level.

[0318] Optionally, the correspondence between the motion parameters and the noise reduction parameters has the following principle:

[0319] For the same type of target, the strength of the corresponding spatial noise reduction is positively correlated with the motion intensity level, and the strength of the corresponding temporal noise reduction is negatively correlated with the motion intensity level; and / or

[0320] For targets with the same motion intensity level, the corresponding spatial noise reduction parameters and temporal noise reduction parameters are adjusted according to the image clarity requirements and image effect requirements corresponding to different types of targets.

[0321] The present invention also provides a computer program medium on which a computer program is stored. When the program is executed by a processor, the steps of the video noise reduction method provided in the above embodiment 1 are implemented.

[0322] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the modules is only a logical function division. There may be other division methods in actual implementation, such as multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or modules, which can be electrical, mechanical or other forms.

[0323] The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical modules, that is, they may be located in one place or distributed on multiple network modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0324] In addition, each functional module in each embodiment of the present application can be integrated into a processing module, or each module can exist physically separately, or two or more modules can be integrated into one module. The above-mentioned integrated module can be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it can be stored in a computer-readable storage medium.

[0325] In the above embodiments, all or part of the embodiments may be implemented by software, hardware, firmware or any combination thereof. When implemented by software, all or part of the embodiments may be implemented in the form of a computer program product.

[0326] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be transmitted from a website site, a computer, a server or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a server or a data center that includes one or more available media integrations. The available medium may be a magnetic medium, (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid state drive (SSD)), etc.

[0327] The technical solution provided by the present application is introduced in detail above. The principles and implementation methods of the present application are explained by using specific examples in the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea. At the same time, for those skilled in the art, according to the idea of ​​the present application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

[0328] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0329] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0330] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0331] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0332] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A video noise reduction method, characterized in that: include: Performing target recognition on each frame image of the video to obtain the coordinates and type of at least one target in each frame image; For the at least one target in each frame image, determine the moving speed of the target according to the coordinates of the target in the current frame image and the previous frame image and the time interval between adjacent frames; Determine the motion intensity level of the target according to the preset correspondence between the motion speed and the motion intensity level and the motion speed of the target; determine the spatial noise reduction parameters and the temporal noise reduction parameters corresponding to the target according to the preset correspondence between the motion parameters and the noise reduction parameters and the motion parameters of the target, wherein the motion parameters include the type and the motion intensity level, the noise reduction parameters include the spatial noise reduction parameters and the temporal noise reduction parameters, the spatial noise reduction parameters include at least one of the different types of parameters that affect the spatial noise reduction intensity, and the temporal noise reduction parameters include at least one of the different types of parameters that affect the temporal noise reduction intensity; Image denoising is performed on the target according to the spatial denoising parameters and temporal denoising parameters corresponding to the target.

2. The method according to claim 1, characterized in that According to the spatial domain noise reduction parameters and the temporal domain noise reduction parameters corresponding to the target, the image noise reduction is performed on the target, including: Determine at least one image area corresponding to the target in N image areas; wherein the N image areas are obtained by dividing the current frame image according to a preset size; Taking the image area as a unit, noise reduction is performed on at least one image area corresponding to the target according to the spatial noise reduction parameters and the temporal noise reduction parameters corresponding to the target.

3. The method according to claim 1, characterized in that After determining the spatial domain noise reduction parameters and the temporal domain noise reduction parameters corresponding to the target, it also includes: For each frame image, according to preset spatial domain noise reduction parameters and temporal domain noise reduction parameters, noise reduction is performed on a region other than at least one target in the current frame image.

4. The method according to claim 3, characterized in that The step of reducing noise in an area other than the at least one target in the current frame image includes: Taking image regions as units, performing noise reduction on image regions other than at least one image region corresponding to the at least one target in the N image regions; The N image regions are obtained by dividing the current frame image according to a preset size.

5. The method according to claim 2, characterized in that: Determining at least one image region corresponding to the target includes: Determine at least one first image region intersecting with the target according to the coordinates of the target; Calculate the moving direction and moving speed of the target according to the coordinates of the target in the current frame image and the previous frame image and the time interval between adjacent frames; According to the moving direction and moving speed of the target, predicting, in the N image areas, a second image area that will intersect with the target after a preset time; The first image area and the second image area are determined to be at least one image area corresponding to the target.

6. The method according to claim 1, characterized in that Determine the moving speed of the target according to the coordinates of the target in the current frame image and the previous frame image and the time interval between adjacent frames, including: Calculating the displacement between the coordinates of the pixel corresponding to the preset point of the target in the current frame image and the coordinates of the pixel corresponding to the preset point in the previous frame image as the pixel displacement of the target; Compare the pixel displacement with the time interval between adjacent frames to obtain the pixel movement speed of the target; The pixel moving speed is compared with the pixel width of the current frame image to obtain the moving speed of the target.

7. The method according to claim 6, characterized in that Determining the exercise intensity level of the target according to the preset correspondence between the exercise speed and the exercise intensity level and the exercise speed of the target includes: When the movement speed of the target is less than a preset threshold, determining the movement intensity level of the target to be a preset value; When the target's movement speed is not less than a preset threshold, the target's movement intensity level is determined according to the preset correspondence between the movement speed and the movement intensity level and the target's movement speed; wherein, the greater the movement speed, the higher the corresponding movement intensity level.

8. The method according to claim 6, characterized in that The corresponding relationship between the motion parameters and the noise reduction parameters has the following principles: For the same type of target, the strength of the corresponding spatial noise reduction is positively correlated with the motion intensity level, and the strength of the corresponding temporal noise reduction is negatively correlated with the motion intensity level; and / or For targets with the same motion intensity level, the corresponding spatial noise reduction parameters and temporal noise reduction parameters are adjusted according to the image clarity requirements and image effect requirements corresponding to different types of targets.

9. A video noise reduction device, characterized in that: comprising a memory and a processor, wherein: The memory is used to store computer programs; The processor is used to read the program in the memory and execute the steps of the video noise reduction method according to any one of claims 1 to 8.

10. A video noise reduction device, characterized in that: include: The target recognition unit is used to perform target recognition on each frame image of the video to obtain the coordinates and type of each target in each frame image; A speed determination unit, for determining, for each target in each frame image, a moving speed of the target according to the coordinates of the target in the current frame image and the previous frame image and the time interval between the current frame image and the previous frame image; a parameter determination unit, configured to determine the motion intensity level of the target according to a preset correspondence between the motion speed and the motion intensity level and the motion speed of the target, and to determine the spatial noise reduction parameters and the temporal noise reduction parameters corresponding to the target according to a preset correspondence between the motion parameters and the noise reduction parameters and the motion parameters of the target, wherein the motion parameters include a type and a motion intensity level, the noise reduction parameters include spatial noise reduction parameters and temporal noise reduction parameters, the spatial noise reduction parameters include at least one of the different types of parameters affecting the spatial noise reduction intensity, and the temporal noise reduction parameters include at least one of the different types of parameters affecting the temporal noise reduction intensity; The image denoising unit is used to perform image denoising on each target in each frame image according to the spatial denoising parameters and temporal denoising parameters corresponding to the target.

11. The device according to claim 10, characterized in that The image denoising unit performs image denoising on the target according to the spatial denoising parameter and the temporal denoising parameter corresponding to the target, including: Determine at least one image area corresponding to the target in N image areas; wherein the N image areas are obtained by dividing the current frame image according to a preset size; Taking the image area as a unit, noise reduction is performed on at least one image area corresponding to the target according to the spatial noise reduction parameters and the temporal noise reduction parameters corresponding to the target.

12. The device according to claim 10, characterized in that After determining the spatial domain noise reduction parameters and the temporal domain noise reduction parameters corresponding to the target, the image noise reduction unit is further used to: For each frame image, according to preset spatial domain noise reduction parameters and temporal domain noise reduction parameters, noise reduction is performed on a region other than at least one target in the current frame image.

13. A computer program medium, characterized in that A computer program is stored thereon, and when the program is executed by a processor, the steps of the video noise reduction method as claimed in any one of claims 1 to 8 are implemented.

Images