Video noise reduction method, device, equipment and storage medium

Abstract

The application provides a video noise reduction method, device and equipment and a storage medium, and belongs to the field of image processing. The method comprises the following steps: acquiring a to-be-reduced-noise video, wherein the to-be-reduced-noise video comprises a plurality of to-be-reduced-noise image frames; determining a motion probability map of each to-be-reduced-noise image frame based on a preset motion probability identification model, wherein the motion probability map comprises the motion probability of each image region, and the motion probability identification model is obtained by training a neural network model based on a plurality of sample data in advance, and the sample data comprises an image frame and a labeled motion probability map; determining the noise reduction strategy of each image region in each to-be-reduced-noise image frame according to the motion probability map of each to-be-reduced-noise image frame; and performing noise reduction processing on the image frame according to the noise reduction strategy of each image region in each to-be-reduced-noise image frame. The motion probability identification by the preset motion probability identification model can effectively improve the accuracy of video noise reduction.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to a video noise reduction method, apparatus, device and storage medium. Background Technology

[0002] As video coding technology becomes increasingly widely used, people's demands for video clarity are also rising. However, during the acquisition, transmission, processing, and reception of videos, noise can cause blurring due to the influence of equipment and the external environment. Currently, traditional video denoising algorithms often suffer from motion detection limitations, resulting in problems such as motion blur and blurring in moving areas, as well as noise fluctuations in stationary areas.

[0003] Therefore, how to accurately reduce noise in videos to improve video quality is an urgent problem to be solved. Summary of the Invention

[0004] The main objective of this application is to provide a video noise reduction method, apparatus, device, and storage medium, which aims to improve the efficiency and accuracy of video noise reduction.

[0005] In a first aspect, this application provides a video noise reduction method, which includes the following steps: Acquire a video to be denoised, the video to be denoised including multiple image frames to be denoised; The motion probability map of each image frame to be denoised is determined based on a preset motion probability recognition model. The motion probability map includes the motion probability of each image region. The motion probability recognition model is obtained by training a neural network model in advance based on multiple sample data. The sample data includes image frames and labeled motion probability maps. Based on the motion probability map of each of the image frames to be denoised, a denoising strategy for each image region in each of the image frames to be denoised is determined; The image frame is denoised according to the denoising strategy of each image region of the image frame to be denoised.

[0006] Secondly, this application also provides a video noise reduction device, which includes an acquisition module, a determination module, and a noise reduction module, wherein: The acquisition module is used to acquire the video to be denoised, which includes multiple image frames to be denoised. The determining module is used to determine the motion probability map of each of the image frames to be denoised based on a preset motion probability recognition model. The motion probability map includes the motion probability of each image region. The motion probability recognition model is obtained by training a neural network model in advance based on multiple sample data. The sample data includes image frames and labeled motion probability maps. The determining module is further configured to determine the denoising strategy for each image region in each image frame to be denoised based on the motion probability map of each image frame to be denoised. The noise reduction module is used to perform noise reduction processing on the image frame according to the noise reduction strategy of each image region of the image frame to be denoised.

[0007] Thirdly, this application also provides a computer device, the computer device including a processor, a memory, and a computer program stored in the memory and executable by the processor, wherein when the computer program is executed by the processor, it implements the steps of the video noise reduction method described above.

[0008] Fourthly, this application also provides a computer-readable storage medium storing a computer program, wherein when the computer program is executed by a processor, it implements the steps of the video noise reduction method described above.

[0009] This application provides a video denoising method, apparatus, device, and storage medium. The method involves acquiring a video to be denoised, comprising multiple image frames; determining a motion probability map for each image frame based on a preset motion probability recognition model, where the motion probability map includes the motion probability of each image region; the motion probability recognition model being pre-trained on a neural network model based on multiple sample data, including image frames and labeled motion probability maps; determining a denoising strategy for each image region within each image frame based on its motion probability map; and performing denoising processing on the image frames according to the denoising strategies for each image region. This application accurately outputs the motion probability map for each image frame using the preset motion probability recognition model, accurately determines the denoising strategy for each image region within each image frame based on the motion probability map, and then performs denoising processing on the image frames according to the denoising strategies for each image region. This significantly improves the efficiency and accuracy of video denoising. Attached Figure Description

[0010] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0011] Figure 1 A flowchart illustrating a video noise reduction method provided in an embodiment of this application; Figure 2A flowchart illustrating another video noise reduction method provided in an embodiment of this application; Figure 3 This is a schematic diagram of the structure of a motion probability recognition model provided in an embodiment of this application; Figure 4 for Figure 1 A flowchart illustrating the sub-steps of the video noise reduction method in the image; Figure 5 A schematic block diagram of a video noise reduction device provided in an embodiment of this application; Figure 6 for Figure 5 A schematic block diagram of a submodule of the video noise reduction device in the image; Figure 7 This is a schematic block diagram of the structure of a computer device provided in an embodiment of this application.

[0012] The realization of the purpose, functional features and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0013] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0014] The flowchart shown in the attached diagram is for illustrative purposes only and does not necessarily include all content and operations / steps, nor does it necessarily have to be performed in the order described. For example, some operations / steps can be broken down, combined, or partially merged, so the actual execution order may change depending on the actual situation.

[0015] As video coding technology becomes increasingly widely used, people's demands for video clarity are also rising. However, during the acquisition, transmission, processing, and reception of videos, noise can cause blurring due to the influence of equipment and the external environment. Currently, traditional video denoising algorithms often suffer from motion detection limitations, resulting in problems such as motion blur and blurring in moving areas, as well as noise fluctuations in stationary areas.

[0016] To address the aforementioned problems, embodiments of this application provide a video denoising method, apparatus, device, and storage medium. The video denoising method includes: acquiring a video to be denoised, the video including multiple image frames; determining a motion probability map for each image frame based on a preset motion probability recognition model, the motion probability map including the motion probability of each image region, the motion probability recognition model being pre-trained on a neural network model based on multiple sample data, the sample data including image frames and labeled motion probability maps; determining a denoising strategy for each image region in each image frame based on the motion probability map of each image frame; and performing denoising processing on the image frames according to the denoising strategies for each image region of each image frame.

[0017] This video noise reduction method can be applied to computer devices, such as mobile phones, cameras, camcorders, tablets, laptops, desktop computers, personal digital assistants, and wearable devices.

[0018] The following detailed description of some embodiments of this application is provided in conjunction with the accompanying drawings. Unless otherwise specified, the following embodiments and features can be combined with each other.

[0019] Please refer to Figure 1 , Figure 1 This is a flowchart illustrating a video noise reduction method provided in an embodiment of this application.

[0020] like Figure 1 As shown, the video noise reduction method includes steps S101 to S104.

[0021] Step S101: Obtain the video to be denoised, which includes multiple image frames to be denoised.

[0022] The video to be denoised includes multiple image frames to be denoised, each image frame having the same resolution. This resolution can be determined according to actual conditions, and this embodiment does not impose a specific limitation on it. For example, the resolution can be 1920*1080 or 2560*1440. It should be noted that the image frame to be denoised can be a noisy RAW image frame.

[0023] In some embodiments, a video to be denoised is acquired, and the video is then segmented into frames to obtain multiple image frames to be denoised. By segmenting the video to be denoised into frames, multiple image frames to be denoised can be accurately obtained, greatly improving the efficiency and accuracy of video denoising.

[0024] Step S102: Determine the motion probability map of each of the image frames to be denoised based on the preset motion probability recognition model.

[0025] The motion probability map includes the motion probability of each image region. The image region can be set according to the actual situation. This application embodiment does not make specific limitations on this. For example, the image region can be a single pixel; or the image region can be a region composed of nine adjacent pixels.

[0026] The motion probability recognition model is obtained by training a neural network model in advance based on multiple sample data. The sample data includes image frames and labeled motion probability maps. The image frames include the image frame to be denoised and the adjacent image frames of the image frame to be denoised. The adjacent image frames are the adjacent image frames of the image frame to be denoised and have completed motion probability recognition.

[0027] In some embodiments, such as Figure 2 As shown, the video noise reduction method includes steps S201 to S205.

[0028] Step S201: Obtain a sample dataset. The sample dataset includes multiple sample data, which includes sample image frames to be denoised, sample neighboring image frames, and labeled motion probability maps. The sample neighboring image frames are the adjacent image frames of the sample image frame to be denoised that have completed motion probability recognition.

[0029] In some embodiments, historical data is acquired, which includes the image frame to be denoised, neighboring image frames, and a motion probability map; the image frame to be denoised is designated as the sample image frame to be denoised, the neighboring image frame is designated as the sample neighboring image frame, and the motion probability map is labeled to obtain the labeled motion probability map; the sample image frame to be denoised, the neighboring image frame, and the labeled motion probability map are used as a sample data; the aforementioned steps are repeated to obtain a sample dataset.

[0030] Step S202: Obtain a preset neural network model and select a sample data from the sample dataset as the target sample data.

[0031] This neural network model includes, but is not limited to, convolutional neural network models, recurrent neural network models, and recurrent convolutional neural network models. For example... Figure 3 As shown, the preset neural network model includes a feature fusion layer 11, a feature extraction layer 12, and a probability output layer 13. The feature fusion layer 11 is used to stitch the image frame to be denoised and adjacent image frames to obtain a fused image frame. The feature extraction layer 12 is used to extract features from the fused image frame to obtain fused image features. The probability output layer 13 is used to perform motion probability recognition on the fused image features and output the result to obtain a motion probability map of the image frame to be denoised.

[0032] It should be noted that this feature extraction layer can use a strip convolutional network. By using a strip convolutional network, the computational power of the entire video noise reduction can be effectively reduced, and the efficiency of video noise reduction can be effectively improved.

[0033] In some embodiments, a sample data point is selected from the sample dataset as the target sample data to obtain the target sample data. This target sample data includes denoised image frames, neighboring image frames, and annotated motion probability maps.

[0034] Step S203: Using the preset neural network model, perform motion probability recognition on the sample image frame to be denoised and the adjacent image frames in the target sample data to obtain the predicted motion probability map of the sample image frame to be denoised.

[0035] The feature fusion layer stitches together the sample image frame to be denoised and the adjacent image frames to obtain the predicted fused image frame. The feature extraction layer extracts features from the predicted fused image frame to obtain the predicted fused image features. The probability output layer performs motion probability recognition on the predicted fused image features and outputs the predicted motion probability map of the sample image frame to be denoised. The predicted motion probability map includes the motion probability of each image region.

[0036] Step S204: Determine whether the preset neural network model has converged based on the predicted motion probability map and the labeled motion probability map.

[0037] Based on the predicted motion probability map and the labeled motion probability map, the model loss value of the preset neural network model is determined. If the model loss value is less than or equal to the preset loss value, the preset neural network model is determined to have converged; if the model loss value is greater than the preset loss value, the preset neural network model is determined to have not converged. The preset loss value can be set according to actual conditions, and this embodiment does not impose specific limitations on it. For example, the preset loss value can be set to 0.02.

[0038] In some embodiments, the method for determining the model loss value of a preset neural network model based on the predicted motion probability map and the labeled motion probability map can be as follows: calculate the similarity between the predicted motion probability map and the labeled motion probability map to obtain the current similarity; obtain the historical similarity, which is the average of the current similarities of each sample data that has been trained; calculate the average of the current similarity and the historical similarity to obtain the target similarity; subtract the target similarity from the unit value to determine the loss value. The method for calculating the similarity can be selected according to the actual situation, and this embodiment does not specifically limit it. For example, the cosine similarity between the predicted motion probability map and the labeled motion probability map can be calculated.

[0039] Step S205: If the preset neural network model does not converge, adjust the model parameters of the preset neural network model and continue to execute the step of selecting a sample data from the sample dataset as the target sample data until a converged motion probability recognition model is obtained.

[0040] If the model loss value is greater than the preset loss value, it is determined that the preset neural network model has not converged. The model parameters of the preset neural network model are adjusted, and the following steps are taken: Select a sample data from the sample dataset as the target sample data, and use the preset neural network model to perform motion probability recognition on the sample image frame to be denoised and the sample adjacent image frames in the target sample data to obtain the predicted motion probability map of the sample image frame to be denoised. Based on the predicted motion probability map and the labeled motion probability map, it is determined whether the preset neural network model has converged, until a converged motion probability recognition model is obtained.

[0041] In some embodiments, such as Figure 4 As shown, step S102 includes sub-steps S1021 to S1022.

[0042] Sub-step S1021: Obtain the neighboring image frames of each of the image frames to be denoised, wherein the neighboring image frames are the adjacent image frames of the image frames to be denoised that have completed motion probability recognition.

[0043] Obtain the adjacent image frames of each image frame to be denoised. These adjacent image frames are those adjacent to the image frame to be denoised and for which motion probability recognition has been completed. If the image frame to be denoised is the first frame of the video to be denoised, the adjacent image frames are empty frames.

[0044] For example, the video to be denoised includes a first image frame to be denoised, a second image frame to be denoised, a third image frame to be denoised, a fourth image frame to be denoised, a fifth image frame to be denoised, a sixth image frame to be denoised, a seventh image frame to be denoised, an eighth image frame to be denoised, a ninth image frame to be denoised, and a tenth image frame to be denoised. If the fifth image frame to be denoised is currently undergoing motion probability recognition, then the adjacent image frame is the image frame after the fourth image frame to be denoised has completed motion probability recognition.

[0045] Sub-step S1022: Perform motion probability recognition on the image frame to be denoised and the adjacent image frames using the preset motion probability recognition model to obtain the motion probability map of the image frame to be denoised.

[0046] In some embodiments, image preprocessing is performed on the image frame to be denoised, including mean filtering, black level correction, and normalization; and brightness adjustment of adjacent image frames based on a preset brightness gain value. The preset brightness gain value can be set according to actual conditions, and this embodiment does not specifically limit it. For example, the preset brightness gain value can be determined based on the exposure output value of the image acquisition device. By performing image preprocessing on the image frame to be denoised and adjusting the brightness of adjacent image frames, the efficiency and accuracy of video noise reduction can be effectively improved.

[0047] In some embodiments, sensor calibration data of the image acquisition device is acquired, and a first preset offset compensation formula and a second preset offset compensation formula are obtained. The first preset offset compensation formula is as follows: ,Should The image compensation value for the image frame to be denoised is... The filtered smoothing value of the image frame to be denoised is... The second preset offset compensation formula is as follows: This should be For the image frame to be denoised, after image compensation, this... The value is based on the output of the image acquisition device; image compensation is performed on the image frame to be denoised based on the first preset offset compensation formula and the second preset offset compensation formula to obtain the image frame to be denoised after image compensation.

[0048] It should be noted that by performing image preprocessing and image compensation on the image frames to be denoised, and by adjusting the brightness of adjacent image frames, the accuracy of video denoising can be effectively improved.

[0049] In some embodiments, a feature fusion layer stitches together the image frame to be denoised and the adjacent image frames to obtain a fused image frame; a feature extraction layer extracts features from the fused image frame to obtain fused image features; and a probability output layer performs motion probability recognition on the fused image features and outputs the results to obtain a motion probability map of the image frame to be denoised, which includes the motion probability of each image region. By using this motion probability recognition model to perform motion probability recognition on the image frame to be denoised and the adjacent image frames, the motion probability map of the image frame to be denoised can be accurately obtained.

[0050] By using a motion probability recognition model to perform motion probability recognition on the image frame to be denoised and adjacent image frames, it is possible to more accurately distinguish between moving and stationary regions, thereby achieving detailed protection and adaptive noise reduction of moving regions. This makes the edges of moving objects clearer, significantly reduces motion blur and bokeh, and improves the ability to restore details in dynamic scenes.

[0051] Step S103: Determine the denoising strategy for each image region in each image frame to be denoised based on the motion probability map of each image frame to be denoised.

[0052] The noise reduction strategy for image regions with a motion probability greater than or equal to a preset motion probability value is determined as a spatial domain noise reduction strategy; the noise reduction strategy for image regions with a motion probability less than the preset motion probability value is determined as a temporal domain averaging noise reduction strategy. The preset motion probability value can be set according to actual conditions, and this embodiment does not impose a specific limitation on it. For example, the preset motion probability value can be set to 60%. Appropriate noise reduction strategies can be accurately selected based on the motion probability of the image regions.

[0053] It should be noted that the spatial domain denoising strategy performs image region denoising based on the current frame image, while the temporal averaging denoising strategy performs denoising on the current frame image and the previous few frames. The specific denoising algorithms selected for the spatial domain denoising strategy and the temporal averaging denoising strategy can be chosen according to the actual situation, and this application embodiment does not impose specific limitations on this. By correcting moving image regions through the spatial domain denoising strategy and denoising still image regions through the temporal averaging denoising strategy, the accuracy and efficiency of video denoising can be effectively improved.

[0054] In some embodiments, a preset motion probability adjustment strategy is obtained; the motion probability values ​​in the motion probability map are adjusted according to the preset motion probability adjustment strategy. The preset motion probability adjustment strategy can be set according to actual conditions, and this application embodiment does not specifically limit it. For example, if the video to be denoised is a running video of an athlete, the preset motion probability adjustment strategy can be a strategy to increase the motion probability value. Adjusting the motion probability through scene-based adjustments can adapt to different scenes, thereby effectively improving the accuracy of video denoising.

[0055] Step S104: Perform noise reduction processing on the image frame according to the noise reduction strategy of each image region of each image frame to be denoised.

[0056] In some embodiments, the denoising strategy of each image region in each image frame to be denoised is used as input to a preset 3DNR digital denoising algorithm to perform denoising processing on each of the image frames to be denoised in the video to be denoised. By using the denoising strategy of each image region in each image frame to be denoised as input to the preset 3DNR digital denoising algorithm, the accuracy of video denoising can be effectively improved.

[0057] It should be noted that using 3DNR digital noise reduction algorithm to process image frames is a conventional technique and existing technical solutions can be referenced. Therefore, this application will not elaborate on this aspect.

[0058] By using a preset motion probability recognition model to identify the motion probability of image regions, and based on the identified motion probability map, a suitable noise reduction strategy can be accurately matched. Combined with a preset 3DNR digital noise reduction algorithm, video noise reduction is performed, which greatly improves the accuracy and reliability of video noise reduction.

[0059] The video denoising method provided in the above embodiments acquires a video to be denoised, which includes multiple image frames to be denoised; determines a motion probability map for each image frame to be denoised based on a preset motion probability recognition model, the motion probability map including the motion probability of each image region, the motion probability recognition model being pre-trained on a neural network model based on multiple sample data, the sample data including image frames and labeled motion probability maps; determines a denoising strategy for each image region in each image frame to be denoised based on the motion probability map of each image frame to be denoised; and performs denoising processing on the image frames according to the denoising strategies for each image region in each image frame to be denoised. In this application, the preset motion probability recognition model can accurately output the motion probability map of each image frame to be denoised, and based on the motion probability map of each image frame to be denoised, the denoising strategy for each image region in each image frame to be denoised can be accurately determined. Then, the image frames are denoised according to the denoising strategies for each image region, thus accurately completing the video denoising process and greatly improving the efficiency and accuracy of video denoising.

[0060] Please see Figure 5 , Figure 5 This is a schematic block diagram of a video noise reduction device provided in an embodiment of this application.

[0061] like Figure 5 As shown, the video noise reduction device 300 includes an acquisition module 310, a determination module 320, and a noise reduction module 330, wherein: The acquisition module 310 is used to acquire the video to be denoised, the video to be denoised including multiple image frames to be denoised; The determining module 320 is used to determine the motion probability map of each of the image frames to be denoised based on a preset motion probability recognition model. The motion probability map includes the motion probability of each image region. The motion probability recognition model is obtained by training a neural network model in advance based on multiple sample data. The sample data includes image frames and labeled motion probability maps. The determining module 320 is further configured to determine the denoising strategy for each image region in each image frame to be denoised based on the motion probability map of each image frame to be denoised. The noise reduction module 330 is used to perform noise reduction processing on the image frame according to the noise reduction strategy of each image region of the image frame to be denoised.

[0062] In some embodiments, such as Figure 6 As shown, the determining module 320 includes an acquisition submodule 321 and a generation module 322, wherein: The acquisition submodule 321 is used to acquire the adjacent image frames of each of the image frames to be denoised, wherein the adjacent image frames are the adjacent image frames of the image frames to be denoised that have completed motion probability recognition. The generation module 322 is used to perform motion probability recognition on the image frame to be denoised and the adjacent image frames through the preset motion probability recognition model to obtain the motion probability map of the image frame to be denoised.

[0063] In some embodiments, the generation module 322 is further configured to: The image frame to be denoised is subjected to image preprocessing, which includes mean filtering, black level correction, and normalization. The brightness of adjacent image frames is adjusted based on a preset brightness gain value.

[0064] In some embodiments, the generation module 322 is further configured to: The feature fusion layer performs image stitching on the image frame to be denoised and the adjacent image frames to obtain a fused image frame. The fused image features are obtained by extracting features from the fused image frame through the feature extraction layer. The motion probability map of the image frame to be denoised is obtained by performing motion probability recognition on the fused image features through the probability output layer and outputting the result. The motion probability map includes the motion probability of each image region.

[0065] In some embodiments, the determining module 320 is further configured to: The noise reduction strategy for image regions with a motion probability greater than or equal to a preset motion probability value is defined as the spatial domain noise reduction strategy. The noise reduction strategy for image regions with motion probability less than the preset motion probability value is determined to be the temporal average noise reduction strategy.

[0066] In some embodiments, the determining module 320 is further configured to: Obtain the preset motion probability adjustment strategy; The motion probability values ​​in the motion probability graph are adjusted according to the preset motion probability adjustment strategy.

[0067] In some embodiments, the noise reduction module 330 is further configured to: The denoising strategy of each image region of each image frame to be denoised is used as the input of a preset 3DNR digital denoising algorithm to perform denoising processing on each image frame to be denoised in the video to be denoised.

[0068] It should be noted that those skilled in the art will understand that, for the sake of convenience and brevity, the specific working process of the above-mentioned video noise reduction device can be referred to the corresponding process in the aforementioned video noise reduction method embodiments, and will not be repeated here.

[0069] Please see Figure 7 , Figure 7 This is a schematic block diagram of the structure of a computer device provided in an embodiment of this application.

[0070] like Figure 7 As shown, the computer device 400 includes a processor 402 and a memory 403 connected via a system bus 401, wherein the memory 403 may include a storage medium and internal memory.

[0071] The storage medium may store a computer program. This computer program includes program instructions that, when executed, cause the processor to perform any video noise reduction method.

[0072] Processor 402 provides computing and control capabilities to support the operation of the entire computer device.

[0073] Internal memory provides an environment for the execution of computer programs stored in the storage medium. When these computer programs are executed by the processor, the processor can perform any video noise reduction method.

[0074] Those skilled in the art will understand that Figure 7 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0075] It should be understood that processor 402 can be a Central Processing Unit (CPU), but it can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among these, the general-purpose processor can be a microprocessor or any conventional processor.

[0076] In one embodiment, the processor 402 is configured to run a computer program stored in a memory to perform the following steps: Acquire a video to be denoised, the video to be denoised including multiple image frames to be denoised; The motion probability map of each image frame to be denoised is determined based on a preset motion probability recognition model. The motion probability map includes the motion probability of each image region. The motion probability recognition model is obtained by training a neural network model in advance based on multiple sample data. The sample data includes image frames and labeled motion probability maps. Based on the motion probability map of each of the image frames to be denoised, a denoising strategy for each image region in each of the image frames to be denoised is determined; The image frame is denoised according to the denoising strategy of each image region of the image frame to be denoised.

[0077] In one embodiment, when the processor 402 determines the motion probability map of each of the image frames to be denoised based on a preset motion probability recognition model, it is configured to: Obtain the neighboring image frames of each of the image frames to be denoised, wherein the neighboring image frames are the adjacent image frames of the image frames to be denoised that have completed motion probability recognition; The motion probability recognition model is used to perform motion probability recognition on the image frame to be denoised and the adjacent image frames to obtain the motion probability map of the image frame to be denoised.

[0078] In one embodiment, before implementing the motion probability recognition of the image frame to be denoised and the adjacent image frames using the preset motion probability recognition model to obtain the motion probability map of the image frame to be denoised, the processor 402 is further configured to implement: The image frame to be denoised is subjected to image preprocessing, which includes mean filtering, black level correction, and normalization. The brightness of adjacent image frames is adjusted based on a preset brightness gain value.

[0079] In one embodiment, the preset motion probability recognition model includes a feature fusion layer, a feature extraction layer, and a probability output layer; when the processor 402 performs motion probability recognition on the image frame to be denoised and the adjacent image frames using the preset motion probability recognition model to obtain the motion probability map of the image frame to be denoised, it is configured to: The feature fusion layer performs image stitching on the image frame to be denoised and the adjacent image frames to obtain a fused image frame. The fused image features are obtained by extracting features from the fused image frame through the feature extraction layer. The motion probability map of the image frame to be denoised is obtained by performing motion probability recognition on the fused image features through the probability output layer and outputting the result. The motion probability map includes the motion probability of each image region.

[0080] In one embodiment, when implementing the denoising strategy for determining each image region in each image frame to be denoised based on the motion probability map of each image frame to be denoised, the processor 402 is configured to: The noise reduction strategy for image regions with a motion probability greater than or equal to a preset motion probability value is defined as the spatial domain noise reduction strategy. The noise reduction strategy for image regions with motion probability less than the preset motion probability value is determined to be the temporal average noise reduction strategy.

[0081] In one embodiment, before implementing the step of determining the denoising strategy for each image region in each image frame to be denoised based on the motion probability map of each image frame to be denoised, the processor 402 is further configured to implement: Obtain the preset motion probability adjustment strategy; The motion probability values ​​in the motion probability graph are adjusted according to the preset motion probability adjustment strategy.

[0082] In one embodiment, when the processor 402 performs the image frame denoising processing based on the denoising strategy for each image region of each image frame to be denoised, it is configured to: The denoising strategy of each image region of each image frame to be denoised is used as the input of a preset 3DNR digital denoising algorithm to perform denoising processing on each image frame to be denoised in the video to be denoised.

[0083] It should be noted that those skilled in the art will understand that, for the sake of convenience and brevity, the specific working process of the computer equipment described above can be referred to the corresponding process in the aforementioned video noise reduction method embodiments, and will not be repeated here.

[0084] This application also provides a computer-readable storage medium storing a computer program, the computer program including program instructions, and the method implemented when the program instructions are executed can refer to various embodiments of the video noise reduction method of this application.

[0085] The computer-readable storage medium can be an internal storage unit of the computer device described in the foregoing embodiments, such as a hard disk or memory of the computer device. The computer-readable storage medium can be non-volatile or volatile. Alternatively, the computer-readable storage medium can be an external storage device of the computer device, such as a plug-in hard disk, Smart Media Card (SMC), Secure Digital (SD) card, or Flash Card equipped on the computer device.

[0086] It should be understood that the terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the scope of the application. As used in this specification, unless the context clearly indicates otherwise, the singular forms “a,” “an,” and “the” are intended to include the plural forms.

[0087] It should also be understood that the term "and / or" as used in this specification refers to any combination and all possible combinations of one or more of the associated listed items, and includes such combinations. It should be noted that, herein, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or system. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or system that includes that element.

[0088] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments. The above descriptions are merely specific implementations of this application, but the protection scope of this application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this application, and these modifications or substitutions should all be covered within the protection scope of this application.

Claims

1. A video noise reduction method, characterized in that, include: Acquire a video to be denoised, the video to be denoised including multiple image frames to be denoised; The motion probability map of each image frame to be denoised is determined based on a preset motion probability recognition model. The motion probability map includes the motion probability of each image region. The motion probability recognition model is obtained by training a neural network model in advance based on multiple sample data. The sample data includes image frames and labeled motion probability maps. Based on the motion probability map of each of the image frames to be denoised, a denoising strategy for each image region in each of the image frames to be denoised is determined; The image frame is denoised according to the denoising strategy of each image region of the image frame to be denoised.

2. The video noise reduction method as described in claim 1, characterized in that, The process of determining the motion probability map of each frame of the image to be denoised based on a preset motion probability recognition model includes: Obtain the neighboring image frames of each of the image frames to be denoised, wherein the neighboring image frames are the adjacent image frames of the image frames to be denoised that have completed motion probability recognition; The motion probability recognition model is used to perform motion probability recognition on the image frame to be denoised and the adjacent image frames to obtain the motion probability map of the image frame to be denoised.

3. The video noise reduction method as described in claim 2, characterized in that, Before obtaining the motion probability map of the image frame to be denoised by performing motion probability recognition on the image frame to be denoised and the adjacent image frames using the preset motion probability recognition model, the method further includes: The image frame to be denoised is subjected to image preprocessing, which includes mean filtering, black level correction, and normalization. The brightness of adjacent image frames is adjusted based on a preset brightness gain value.

4. The video noise reduction method as described in claim 2, characterized in that, The preset motion probability recognition model includes a feature fusion layer, a feature extraction layer, and a probability output layer; The step of performing motion probability recognition on the image frame to be denoised and the adjacent image frames using the preset motion probability recognition model to obtain the motion probability map of the image frame to be denoised includes: The feature fusion layer performs image stitching on the image frame to be denoised and the adjacent image frames to obtain a fused image frame. The fused image features are obtained by extracting features from the fused image frame through the feature extraction layer. The motion probability map of the image frame to be denoised is obtained by performing motion probability recognition on the fused image features through the probability output layer and outputting the result. The motion probability map includes the motion probability of each image region.

5. The video noise reduction method as described in claim 1, characterized in that, The step of determining the denoising strategy for each image region in each image frame to be denoised based on the motion probability map of each image frame to be denoised includes: The noise reduction strategy for image regions with a motion probability greater than or equal to a preset motion probability value is defined as the spatial domain noise reduction strategy. The noise reduction strategy for image regions with motion probability less than the preset motion probability value is determined to be the temporal average noise reduction strategy.

6. The video noise reduction method as described in claim 5, characterized in that, Before determining the denoising strategy for each image region in each image frame to be denoised based on the motion probability map of each image frame to be denoised, the method further includes: Obtain the preset motion probability adjustment strategy; The motion probability values ​​in the motion probability graph are adjusted according to the preset motion probability adjustment strategy.

7. The video noise reduction method according to any one of claims 1-6, characterized in that, The step of performing image frame denoising processing based on the denoising strategy for each image region of each image frame to be denoised includes: The denoising strategy of each image region of each image frame to be denoised is used as the input of a preset 3DNR digital denoising algorithm to perform denoising processing on each image frame to be denoised in the video to be denoised.

8. A video noise reduction device, characterized in that, The video noise reduction device includes an acquisition module, a determination module, and a noise reduction module, wherein: The acquisition module is used to acquire the video to be denoised, which includes multiple image frames to be denoised. The determining module is used to determine the motion probability map of each of the image frames to be denoised based on a preset motion probability recognition model. The motion probability map includes the motion probability of each image region. The motion probability recognition model is obtained by training a neural network model in advance based on multiple sample data. The sample data includes image frames and labeled motion probability maps. The determining module is further configured to determine the denoising strategy for each image region in each image frame to be denoised based on the motion probability map of each image frame to be denoised. The noise reduction module is used to perform noise reduction processing on the image frame according to the noise reduction strategy of each image region of the image frame to be denoised.

9. A computer device, characterized in that, The computer device includes a processor, a memory, and a computer program stored in the memory and executable by the processor, wherein when the computer program is executed by the processor, it implements the steps of the video noise reduction method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, wherein when the computer program is executed by a processor, it implements the steps of the video noise reduction method as described in any one of claims 1 to 7.

Images