Video denoising method and device, computer readable medium and electronic device

Abstract

The present disclosure provides a video denoising method and device, a computer-readable medium and an electronic device, and relates to the field of image processing technology. The method includes: obtaining an input image frame, and determining the scene type of the input image frame; performing scene area division processing on the input image frame according to the scene type to obtain a scene area distribution map; performing image denoising on the input image frame based on the scene area distribution map to generate a denoised image; fusing the input image frame, the scene area distribution map and the denoised image to generate a target image frame, so as to generate a denoised video according to the target image frame. The present disclosure can perform regional denoising on an image in combination with the scene information contained in the image, avoid the problem of texture detail loss in some image areas or poor denoising in some image areas, improve the flexibility and rationality of denoising, and improve the denoising effect.

Description

Background Art

[0002] With the rapid development of science and technology, people are paying more and more attention to the clarity of videos or images. In daily life, due to the limitations of shooting conditions and the influence of various factors such as sending equipment, transmission equipment, and receiving equipment, the clarity of videos or images is often interfered by noise, which reduces the quality of videos or images and affects the visual effects of videos or images. Therefore, denoising videos or images is very critical and necessary to improve the quality of videos or images.

[0003] At present, in the relevant denoising schemes, the same denoising is generally performed on all areas of the entire image. However, performing the same denoising on all areas of the entire image may result in the loss of texture details in some image areas, or result in poor denoising in some image areas, resulting in poor image denoising effect. Summary of the invention

[0004] The purpose of the present disclosure is to provide a video denoising method, a video denoising device, a computer-readable medium and an electronic device, thereby at least to a certain extent avoiding the problem that denoising an image in related denoising schemes leads to loss of texture details in some image areas, or leads to poor denoising of some image areas and poor image denoising effect.

[0005] According to a first aspect of the present disclosure, a video denoising method is provided, comprising:

[0006] Acquire an input image frame, and determine a scene type of the input image frame;

[0007] Performing scene area division processing on the input image frame according to the scene type to obtain a scene area distribution map;

[0008] Performing image denoising on the input image frame based on the scene area distribution map to generate a denoised image;

[0009] The input image frame, the scene area distribution map and the denoised image are fused to generate a target image frame, so as to generate a denoised video according to the target image frame.

[0010] According to a second aspect of the present disclosure, a video denoising device is provided, comprising:

[0011] A scene type determination module, used to obtain an input image frame and determine the scene type of the input image frame;

[0012] A scene area distribution map generating module, used for performing scene area division processing on the input image frame according to the scene type to obtain a scene area distribution map;

[0013] A denoised image generation module, configured to perform image denoising on the input image frame based on the scene area distribution map to generate a denoised image;

[0014] The denoised video generation module is used to fuse the input image frame, the scene area distribution map and the denoised image to generate a target image frame, so as to generate a denoised video according to the target image frame.

[0015] According to a third aspect of the present disclosure, a computer-readable medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the above method is implemented.

[0016] According to a fourth aspect of the present disclosure, there is provided an electronic device, characterized in that it includes:

[0017] Processor; and

[0018] The memory is used to store one or more programs. When the one or more programs are executed by one or more processors, the one or more processors implement the above method.

[0019] A video denoising method provided by an embodiment of the present disclosure determines the scene type of an input image frame, performs scene area division processing on the input image frame according to the scene type, obtains a scene area distribution map, and performs image denoising on the input image frame based on the scene area distribution map to generate a denoised image; the input image frame, the scene area distribution map and the denoised image are fused to generate a target image frame, so as to generate a denoised video according to the target image frame. On the one hand, by guiding image denoising through the scene type corresponding to the image, it is possible to implement adaptive area division of images under different scene types, and perform image denoising in combination with area division information, effectively ensuring that different scene areas achieve different denoising strengths, avoiding the problem of image texture loss or weak denoising strength in some areas caused by denoising, improving the denoising effect, and improving the flexibility of the image denoising algorithm; on the other hand, the input image frame, the scene area distribution map and the denoised image are fused to implement further denoising of the image frame, further improve the quality of texture details of the denoised image, and improve the denoising effect.

[0020] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The drawings herein are incorporated into the specification and constitute a part of the specification, showing embodiments consistent with the present disclosure, and together with the specification, are used to explain the principles of the present disclosure. Obviously, the drawings described below are only some embodiments of the present disclosure, and for ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. In the drawings:

[0022] Figure 1 A schematic diagram showing an exemplary system architecture to which embodiments of the present disclosure may be applied;

[0023] Figure 2 A schematic diagram showing an electronic device to which the embodiments of the present disclosure can be applied;

[0024] Figure 3 A flowchart of a video denoising method in an exemplary embodiment of the present disclosure is schematically shown;

[0025] Figure 4 A flowchart for generating a scene area distribution map in an exemplary embodiment of the present disclosure is schematically shown;

[0026] Figure 5 A flowchart for generating a brightness area distribution map in an exemplary embodiment of the present disclosure is schematically shown;

[0027] Figure 6 A flowchart for enhancing image brightness information in an exemplary embodiment of the present disclosure is schematically shown;

[0028] Figure 7 A flowchart of performing smoothing filtering on image brightness information in an exemplary embodiment of the present disclosure is schematically shown;

[0029] Figure 8 A flowchart for generating a chromaticity region distribution map in an exemplary embodiment of the present disclosure is schematically shown;

[0030] Fig. 9 A flowchart for generating a detail level area distribution map in an exemplary embodiment of the present disclosure is schematically shown;

[0031] Fig.10 A flowchart for generating a denoised image corresponding to an input image frame in an exemplary embodiment of the present disclosure is schematically shown;

[0032] Fig.11 A flowchart of performing time domain denoising on an input image frame in an exemplary embodiment of the present disclosure is schematically shown;

[0033] Fig.12 The following is a schematic diagram schematically showing the composition of a video denoising device in an exemplary embodiment of the present disclosure. DETAILED DESCRIPTION

[0034] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in a variety of forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that the disclosure will be more comprehensive and complete and to fully convey the concepts of the example embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

[0035] In addition, the accompanying drawings are only schematic illustrations of the present disclosure and are not necessarily drawn to scale. The same reference numerals in the figures represent the same or similar parts, and their repeated description will be omitted. Some of the block diagrams shown in the accompanying drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities can be implemented in software form, or implemented in one or more hardware modules or integrated circuits, or implemented in different networks and / or processor devices and / or microcontroller devices.

[0036] Figure 1 A schematic diagram of a system architecture of an exemplary application environment in which a video denoising method and apparatus according to an embodiment of the present disclosure can be applied is shown.

[0037] like Figure 1 As shown, the system architecture 100 may include one or more of terminal devices 101, 102, 103, a network 104, and a server 105. The network 104 is used to provide a medium for communication links between the terminal devices 101, 102, 103 and the server 105. The network 104 may include various connection types, such as wired, wireless communication links or optical fiber cables, etc. The terminal devices 101, 102, 103 may be various electronic devices with image processing functions, including but not limited to desktop computers, portable computers, smart phones, tablet computers, etc. It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is only for illustration. According to the implementation requirements, there may be any number of terminal devices, networks and servers. For example, the server 105 may be a server cluster composed of multiple servers.

[0038] The video denoising method provided in the embodiment of the present disclosure is generally executed by the server 105, and accordingly, the video denoising device is generally disposed in the server 105. However, it is easy for a person skilled in the art to understand that the video denoising method provided in the embodiment of the present disclosure can also be executed by the terminal devices 101, 102, and 103, and accordingly, the video denoising device can also be disposed in the terminal devices 101, 102, and 103, which is not particularly limited in the present exemplary embodiment.

[0039] An exemplary embodiment of the present disclosure provides an electronic device for implementing a video denoising method, which may be Figure 1 The terminal device 101, 102, 103 or the server 105 in the embodiment of the present invention comprises at least a processor and a memory, wherein the memory is used to store executable instructions of the processor, and the processor is configured to execute the video denoising method by executing the executable instructions.

[0040] Below Figure 2 Taking the mobile terminal 200 in FIG. 1 as an example, the structure of the electronic device is exemplarily described. It should be understood by those skilled in the art that, in addition to the components specifically used for mobile purposes, Figure 2 The structure in the figure can also be applied to fixed type devices. In other embodiments, the mobile terminal 200 may include more or fewer components than shown in the figure, or combine some components, or split some components, or arrange the components differently. The components shown in the figure can be implemented in hardware, software, or a combination of software and hardware. The interface connection relationship between the components is only shown schematically and does not constitute a structural limitation of the mobile terminal 200. In other embodiments, the mobile terminal 200 may also adopt the same Figure 2 Different interface connection methods, or a combination of multiple interface connection methods.

[0041] like Figure 2 As shown, the mobile terminal 200 may specifically include: a processor 210, an internal memory 221, an external memory interface 222, a Universal Serial Bus (USB) interface 230, a charging management module 240, a power management module 241, a battery 242, an antenna 1, an antenna 2, a mobile communication module 250, a wireless communication module 260, an audio module 270, a speaker 271, a receiver 272, a microphone 273, an earphone interface 274, a sensor module 280, a display screen 290, a camera module 291, an indicator 292, a motor 293, a button 294, and a subscriber identification module (SIM) card interface 295, etc. The sensor module 280 may include a depth sensor 2801, a pressure sensor 2802, a gyroscope sensor 2803, etc.

[0042] The processor 210 may include one or more processing units, for example, the processor 210 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor and / or a neural network processor (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0043] NPU is a neural network (NN) computing processor. By drawing on the structure of biological neural networks, such as the transmission mode between neurons in the human brain, it can quickly process input information and can also continuously self-learn. Through NPU, applications such as intelligent cognition of the mobile terminal 200 can be realized, such as image recognition, face recognition, voice recognition, text understanding, etc.

[0044] The processor 210 is provided with a memory. The memory can store instructions for implementing six modular functions: detection instructions, connection instructions, information management instructions, analysis instructions, data transmission instructions, and notification instructions, and the execution is controlled by the processor 210.

[0045] The charging management module 240 is used to receive charging input from the charger. The power management module 241 is used to connect the battery 242, the charging management module 240 and the processor 210. The power management module 241 receives input from the battery 242 and / or the charging management module 240 to power the processor 210, the internal memory 221, the display screen 290, the camera module 291 and the wireless communication module 260.

[0046] The wireless communication function of the mobile terminal 200 can be implemented through antenna 1, antenna 2, mobile communication module 250, wireless communication module 260, modulation and demodulation processor and baseband processor. Among them, antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals; mobile communication module 250 can provide solutions for wireless communication including 2G / 3G / 4G / 5G applied on mobile terminal 200; modulation and demodulation processor can include modulator and demodulator; wireless communication module 260 can provide solutions for wireless communication including wireless local area network (WLAN) (such as wireless fidelity (Wi-Fi) network), Bluetooth (BT) and the like applied on mobile terminal 200. In some embodiments, antenna 1 of mobile terminal 200 is coupled with mobile communication module 250, and antenna 2 is coupled with wireless communication module 260, so that mobile terminal 200 can communicate with network and other devices through wireless communication technology.

[0047] The mobile terminal 200 implements the display function through a GPU, a display screen 290, and an application processor. The GPU is a microprocessor for image processing, which connects the display screen 290 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 210 may include one or more GPUs that execute program instructions to generate or change display information.

[0048] The mobile terminal 200 can realize the shooting function through the ISP, the camera module 291, the video codec, the GPU, the display screen 290 and the application processor, etc. Among them, the ISP is used to process the data fed back by the camera module 291; the camera module 291 is used to capture static images or videos; the digital signal processor is used to process digital signals, and can process other digital signals in addition to digital image signals; the video codec is used to compress or decompress digital videos, and the mobile terminal 200 can also support one or more video codecs.

[0049] The external memory interface 222 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the mobile terminal 200. The external memory card communicates with the processor 210 through the external memory interface 222 to implement a data storage function, such as storing music, video and other files in the external memory card.

[0050] The internal memory 221 can be used to store computer executable program codes, and the executable program codes include instructions. The internal memory 221 may include a program storage area and a data storage area. Among them, the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc. The data storage area may store data created during the use of the mobile terminal 200 (such as audio data, a phone book, etc.), etc. In addition, the internal memory 221 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash memory (Universal Flash Storage, UFS), etc. The processor 210 executes various functional applications and data processing of the mobile terminal 200 by running instructions stored in the internal memory 221 and / or instructions stored in a memory provided in the processor.

[0051] The mobile terminal 200 can implement audio functions such as music playing and recording through the audio module 270 , the speaker 271 , the receiver 272 , the microphone 273 , the earphone interface 274 and the application processor.

[0052] The depth sensor 2801 is used to obtain the depth information of the scene. In some embodiments, the depth sensor can be arranged in the camera module 291.

[0053] The pressure sensor 2802 is used to sense the pressure signal and can convert the pressure signal into an electrical signal. In some embodiments, the pressure sensor 2802 can be arranged on the display screen 290. There are many types of pressure sensors 2802, such as resistive pressure sensors, inductive pressure sensors, capacitive pressure sensors, etc.

[0054] The gyro sensor 2803 can be used to determine the motion posture of the mobile terminal 200. In some embodiments, the angular velocity of the mobile terminal 200 around three axes (i.e., x, y, and z axes) can be determined by the gyro sensor 2803. The gyro sensor 2803 can be used for shooting anti-shake, navigation, somatosensory game scenes, etc.

[0055] In addition, sensors with other functions may be set in the sensor module 280 according to actual needs, such as an air pressure sensor, a magnetic sensor, an acceleration sensor, a distance sensor, a proximity light sensor, a fingerprint sensor, a temperature sensor, a touch sensor, an ambient light sensor, a bone conduction sensor, etc.

[0056] The mobile terminal 200 may also include other devices that provide auxiliary functions. For example, the key 294 includes a power key, a volume key, etc., and the user can generate key signal input related to the user settings and function control of the mobile terminal 200 through key input. For example, the indicator 292, the motor 293, the SIM card interface 295, etc.

[0057] In the related technology, one denoising scheme is to perform feature extraction on the face area in the image, and then perform enhancement and denoising by histogram equalization; another technical scheme is to divide the image into regions by region division and dictionary retrieval, and perform denoising and fusion on different regions; another technical scheme is to perform simple region division by image brightness information, and perform denoising on different regions and fuse the output. However, none of these denoising schemes divide the image into regions in combination with scene recognition information, and perform adaptive denoising on different regions; image denoising without scene recognition information does not process individual scenes, and the denoising degree of each scene region is the same, resulting in the loss of texture details in some scene regions and poor denoising in some scenes, resulting in poor final image denoising effect.

[0058] Based on one or more problems in the related art, the video denoising method of the exemplary embodiment of the present disclosure is specifically described below by taking the server execution as an example.

[0059] Figure 3 The flow of a video denoising method in this exemplary embodiment is shown, which may include the following steps S310 to S340:

[0060] In step S310, an input image frame is acquired, and a scene type of the input image frame is determined.

[0061] In an exemplary embodiment, the input image frame refers to the image frame currently input in the video that needs to be denoised. Of course, the input image frame may also be a single input image, and this exemplary embodiment is not limited to this.

[0062] The scene type refers to the classification of the real scene presented by the image content in the image. For example, the scene type may include face scene, night scene, sky scene, beach scene, cloud scene, plant scene, bird scene, etc. Of course, the above scene types are only illustrative examples. The scene types in actual application scenarios are customized by the developer, and this example embodiment does not make any special limitations on this.

[0063] The scene type corresponding to the input image frame can be identified by a pre-built image scene recognition model. For example, the image scene recognition model can be a deep learning model such as a multi-label image classification model. Of course, the scene type corresponding to the input image frame can also be identified by other means. In this example embodiment, the recognition of the scene type of the input image frame is not limited to the image scene recognition model based on artificial intelligence (AI) technology.

[0064] In step S320, scene area division processing is performed on the input image frame according to the scene type to obtain a scene area distribution map.

[0065] In an exemplary embodiment, the scene area distribution map (SceneMap) can represent the probability that each pixel in the input image frame belongs to the scene area corresponding to the scene information (SceneInfo). The larger the value in the scene area distribution map, the closer it can be considered to be to the recognized scene.

[0066] In step S330, image denoising is performed on the input image frame based on the scene area distribution map to generate a denoised image.

[0067] In an exemplary embodiment, a denoised image refers to an image obtained by performing multi-dimensional denoising on an input image frame based on a scene area distribution map. For example, the denoised image may be an image obtained by performing spatial denoising on the input image frame based on a scene area distribution map, or an image obtained by performing temporal denoising on the input image frame based on a scene area distribution map. Of course, it may also be an image obtained by performing other types of denoising on the input image frame based on a scene area distribution map, and this exemplary embodiment does not specifically limit this.

[0068] In step S340, the input image frame, the scene area distribution map and the denoised image are fused to generate a target image frame, so as to generate a denoised video according to the target image frame.

[0069] In an exemplary embodiment, the target image frame refers to an image generated by fusing the input image frame, the scene area distribution map and the denoised image in different proportions in combination with scene information, that is, the final image obtained after denoising the input image frame. After denoising all input image frames corresponding to the video, multiple target image frames are obtained, and the multiple target image frames are spliced ​​together to obtain a denoised video, thereby completing the denoising of the original video.

[0070] Next, steps S310 to S340 are described in detail.

[0071] In an exemplary embodiment, before denoising the input frame image, the scene type of the input image frame may be determined first. Specifically, scene feature information of the input image frame may be extracted, and then the scene type of the input image frame may be determined based on the scene feature information.

[0072] Among them, scene feature information refers to information that can characterize the scene features in the input image frame. For example, a pre-trained multi-label image classification model can be used to identify the scene type corresponding to the input image frame. Then, the input image frame can be input into the multi-label image classification model to obtain multiple image tags (Tags) of the input image frame, and then the scene type of the input image frame can be determined according to the correspondence between the pre-set image tags and the scene type. For example, when the input image frame is input into the multi-label image classification model and 10 image tags are output, 7 image tags belong to scene type 1 and 3 image tags belong to scene type 2, then the scene type corresponding to the input image frame can be considered to be scene type 1.

[0073] For example, for the input image frame, the input image frame is input into the multi-label image classification model, and the output image label is {blue sky, white clouds, kite}, and the image labels {blue sky, white clouds, kite} all belong to the image label set corresponding to the sky scene type. Therefore, it can be considered that the scene type of the input image frame is a sky scene. Of course, this is only a schematic example, and this example embodiment does not make any special limitation to this.

[0074] In an exemplary embodiment, the Figure 4 The steps in the above code are used to divide the input image frame into scene areas according to the scene type to obtain the scene area distribution map. Figure 4 As shown, it may specifically include steps S410 to S440:

[0075] Step S410, performing brightness area division processing on the input image frame based on the scene type to obtain a brightness area distribution map;

[0076] Step S420, performing chromaticity region division processing on the input image frame based on the scene type to obtain a chromaticity region distribution map;

[0077] Step S430, performing detail level area division processing on the input image frame based on the scene type to obtain a detail level area distribution map; and

[0078] Step S440, obtaining a scene area distribution map by fusing any one of the brightness area distribution map, the chroma area distribution map and the detail level area distribution map or a plurality of area distribution maps.

[0079] Among them, steps S410 to S430 are scene area division processing methods. In actual applications, any one of the division processing methods in steps S410 to S430 can be selected according to actual conditions to implement scene area division processing for the input image frame according to the scene type. For example, the input image frame is divided into brightness areas only by the scene type, and the brightness area distribution map is used as the scene area distribution map of the input image frame; or a combination of any two of the division processing methods in steps S410 to S430 can be selected to implement scene area division processing for the input image frame according to the scene type. For example, the input image frame can be divided into brightness areas by scene type, and the input image frame can be divided into detail areas by scene type, and the brightness area distribution map and the detail area distribution map are fused, and the fused distribution map is used as the scene area distribution map of the input image frame; of course, the three division processing methods in steps S410 to S430 can also be combined to implement scene area division processing for the input image frame according to the scene type, and this example embodiment does not make any special limitation on this.

[0080] The luminance area distribution map (LumMap) refers to a distribution map obtained by dividing the luminance information (or the value of the Y channel) of the input image frame into regions according to the scene type. The luminance area distribution map can represent the probability that the luminance information of each pixel belongs to the scene area corresponding to the scene type. The larger the value in the luminance area distribution map, the closer the luminance information of the pixel is to the recognized scene.

[0081] The chromaticity area distribution map (ChromaMap) refers to a distribution map obtained by dividing the saturation information and hue information (or the value of the UV channel) of the input image frame into regions according to the scene type. The chromaticity area distribution map can represent the probability that the saturation information and hue information of each pixel point belongs to the scene area corresponding to the scene type. The larger the value in the chromaticity area distribution map, the closer the saturation information and hue information of the pixel point can be considered to be to the recognized scene.

[0082] The detail level regional distribution map (DetailMap) refers to a distribution map obtained by dividing the texture detail complexity of the input image frame into regions according to the scene type. The detail level regional distribution map can represent the texture complexity of each region of the input image frame. The larger the value in the detail level regional distribution map, the more complex the texture details of the image region can be considered.

[0083] By combining one or more processing methods in steps S410 to S430, any one or more of the brightness area distribution map, chromaticity area distribution map and detail level area distribution map corresponding to the input image frame can be obtained. Finally, by step S440, any one of the brightness area distribution map, chromaticity area distribution map and detail level area distribution map, or a fusion of multiple area distribution maps can be used as the scene area distribution map of the input image frame.

[0084] In an exemplary embodiment, step S410 may include: Figure 5 Steps S510 to S530 in the embodiment implement the brightness region division processing of the input image frame based on the scene type, referring to Figure 5 As shown, the details are as follows:

[0085] Step S510, extracting image brightness information of the input image frame in a first color space;

[0086] Step S520, performing enhancement processing on the image brightness information based on the scene type to obtain enhanced image brightness information; and

[0087] Step S530, performing smoothing and denoising processing on the enhanced image brightness information to obtain a brightness area distribution map.

[0088] Among them, the first color space refers to a color model that can extract the brightness information of an image. For example, the first color space can be a YUV color space (the Y channel represents brightness information, and the U channel and the V channel represent chromaticity information), or it can be a Lab color space (the L channel represents brightness information, the a channel includes colors ranging from dark green (low brightness value) to gray (medium brightness value) to bright pink (high brightness value), and the b channel includes colors ranging from bright blue (low brightness value) to gray (medium brightness value) to yellow (high brightness value). The Lab color space makes up for the color mode deficiencies of the RGB color space and the CMYK color space). Of course, the first color space can also be other types of color models that can extract image brightness information, and this example embodiment does not make any special limitations on this.

[0089] Before extracting the image brightness information, if it is detected that the color space of the input image frame is a color model such as the RGB color space that cannot extract brightness information, the color space of the input image frame can be converted into the first color space. The specific conversion method is the same as the prior art and will not be repeated here.

[0090] The image brightness information is enhanced in combination with the scene type, and the enhanced image brightness information is smoothed and denoised, which can preliminarily filter the noise in the brightness area distribution map and lay a good foundation for the subsequent denoising of the input image frame.

[0091] Specifically, you can Figure 6 The steps in step S520 are used to enhance the image brightness information. Figure 6 As shown, it may specifically include:

[0092] Step S610, obtaining a brightness threshold corresponding to the input image frame based on the scene type;

[0093] Step S620: determining a mapping curve according to the brightness threshold, and performing piecewise linear mapping on the image brightness information based on the mapping curve to obtain enhanced image brightness information.

[0094] Among them, the brightness threshold (LumLUT) refers to the threshold data corresponding to the current scene type selected from the pre-designed display look-up table (Look-Up-Table, LUT) corresponding to each scene type, and the mapping curve (CurveMapping) refers to a curve used to enhance the contrast of each part of the image in a piecewise linear mapping manner, that is, the mapping curve can be used to increase the brightness (grayscale) area of ​​interest in the image and relatively suppress the brightness (grayscale) area of ​​no interest.

[0095] The brightness threshold corresponding to the input image frame can be obtained from the display lookup table according to the scene type. The brightness threshold stored in the display lookup table corresponds to the number of points of the mapping curve. For example, this embodiment can use a 5-point mapping curve, and the brightness threshold of each scene type can be 5*2 groups of data, representing the input and output coordinate points (x1, y1), (x2, y2), (x3, y3), (x4, y4), (x5, y5) of the mapping curve, respectively. Of course, this is only a schematic example and should not impose any special limitations on this example embodiment. The mapping curve is formed by the acquired brightness threshold, and different brightness areas of the image brightness information in a specific scene are assigned values ​​through the mapping curve to achieve enhanced processing of the image brightness information.

[0096] Specifically, you can Figure 7 The steps in step S530 are used to implement the smooth denoising process of the enhanced image brightness information. Figure 7 As shown, it may specifically include:

[0097] Step S710, performing morphological processing on the enhanced image brightness information to obtain image brightness information after edge expansion, wherein the morphological processing includes image expansion processing and / or image erosion processing;

[0098] Step S720, performing smoothing filtering on the image brightness information after edge expansion to obtain a brightness area distribution map.

[0099] Among them, Morphological Image Processing refers to a series of image processing techniques that process the shape features of images. The basic idea of ​​morphological image processing is to use a special structuring element (SE) to measure or extract image components in the input image that are meaningful for expressing and depicting the shape of the region, so that subsequent recognition work can grasp the most essential (most discriminative) shape features of the target object and achieve the effect of edge expansion of the shape features in the image.

[0100] Specifically, morphological image processing can be image dilation or image erosion. Of course, morphological image processing can also be an opening operation (generally, an image is first eroded using a structural element and then dilated), a closing operation (generally, an image is first dilated using a structural element and then eroded), a white top-hat transformation (generally refers to the difference between the original image and the result of the opening operation) or a black top-hat transformation (generally refers to the difference between the result of the closing operation and the original image). The degree of this example embodiment is not particularly limited.

[0101] The image brightness information can be processed with different degrees of image dilation and image corrosion according to the pre-set morphological image processing templates for each scene type.

[0102] After morphological processing is performed on the image brightness information, the output image brightness information is smoothed and filtered to reduce noise interference in the output image brightness information. For example, the output image brightness information can be smoothed and filtered by a frequency domain filter function Lpfilter to obtain a brightness area distribution map corresponding to the input image frame, thereby reducing the sharpness of the brightness area distribution map.

[0103] In an exemplary embodiment, step S420 may include: Figure 8 Steps S810 to S840 in the embodiment implement the chromaticity region division processing of the input image frame based on the scene type, referring to Figure 8 As shown, the details are as follows:

[0104] Step S810, extracting saturation information and hue information of the input image frame in a second color space;

[0105] Step S820, fusing the saturation information and the hue information based on the scene type to obtain image chromaticity information;

[0106] Step S830, performing enhancement processing on the image chromaticity information based on the scene type to obtain enhanced image chromaticity information; and

[0107] Step S840, performing smoothing and denoising processing on the enhanced image chromaticity information to obtain a chromaticity area distribution map.

[0108] Among them, the second color space refers to a color model that can be used to extract saturation information and hue information in the input image frame. For example, the second color space can be an RGB color space or an HSV color space (the H channel can represent hue information Hue, the S channel can represent saturation information Saturation, and the V channel can represent value information Value). Of course, the second color space can also be other types of color models that can be used to extract saturation information and hue information in the input image frame, and this example embodiment does not limit this.

[0109] The saturation information and the hue information can be fused according to the current scene information corresponding to the scene type and combined in one picture to obtain the image chromaticity information. Then, the image chromaticity threshold can be determined from the display lookup table LUT according to the scene type of the input image frame, and the image chromaticity mapping curve can be determined according to the image chromaticity threshold. Based on the image chromaticity mapping curve, the image chromaticity information is piecewise linearly mapped to obtain enhanced image chromaticity information. The specific piecewise linear mapping process can refer to the processing of the brightness area distribution map, which will not be repeated here.

[0110] Of course, before fusing the saturation information and hue information based on the scene type, the saturation threshold and hue threshold can be determined from the display lookup table LUT according to the scene type of the input image frame, and the saturation mapping curve and the hue mapping curve can be determined respectively according to the saturation threshold and the hue threshold, and the saturation information can be piecewise linearly mapped based on the saturation mapping curve to obtain enhanced saturation information, and the hue information can be piecewise linearly mapped based on the hue mapping curve hue information to obtain enhanced hue information. The specific piecewise linear mapping process can refer to the processing of the brightness area distribution map, which will not be repeated here. Finally, the enhanced saturation information and the enhanced hue information can be combined according to the current scene information corresponding to the scene type as the enhanced image chromaticity information of the input image frame.

[0111] Furthermore, the image chromaticity information can be smoothed and denoised based on the scene type, that is, the enhanced image chromaticity information is subjected to morphological image processing and smoothing filtering to obtain a chromaticity area distribution map. The specific processing procedures of morphological image processing and smoothing filtering processing can refer to the morphological image processing and smoothing filtering processing of image brightness information, which will not be repeated here.

[0112] After the image chromaticity information is enhanced and smoothed for denoising, the noise interference of the chromaticity area distribution map can be effectively reduced, thereby ensuring the denoising effect of subsequent input image frames.

[0113] In an exemplary embodiment, step S430 may include: Fig. 9 Steps S910 to S930 in the embodiment implement the division of detail level regions of the input image frame based on the scene type, referring to Fig. 9 As shown, the details are as follows:

[0114] Step S910, extracting image brightness information of the input image frame in a first color space, and performing edge detection on the image brightness information to obtain image detail level information;

[0115] Step S920, performing enhancement processing on the image detail level information based on the scene type to obtain enhanced image detail level information; and

[0116] Step S930, performing smoothing and denoising processing on the enhanced image detail level information to obtain a detail level area distribution map.

[0117] Among them, edge detection refers to a detection method for extracting texture detail features in an image. The purpose of edge detection is to identify points in a digital image where brightness changes significantly. For example, edge detection can be performed on the image brightness information of an input image frame by combining edge-preserving filtering with an edge detection operator (such as a Sobel operator, a Kirsch operator, a Canny operator, etc.). Of course, edge detection can also be performed on the image brightness information by other edge detection methods, and this example embodiment is not limited to this.

[0118] Similarly, the obtained image detail information is also enhanced (piecewise linear mapping) and smoothed denoising (morphological processing and smoothing filtering processing) to obtain a detail level regional distribution map. The specific processing process refers to the processing process of the image brightness information, which will not be repeated here.

[0119] In an exemplary embodiment, step S330 may include: Fig.10 Steps S1010 to S1030 in the embodiment are used to implement denoising of the input image frame based on the scene area distribution map. Fig.10 As shown, it may specifically include:

[0120] Step S1010, performing spatial denoising on the input image frame based on the scene area distribution map to generate a spatial denoised image; and / or

[0121] Step S1020, performing time domain denoising on the input image frame based on the scene area distribution map to generate a time domain denoised image;

[0122] Step S1030: Using the spatial domain denoised image and / or the temporal domain denoised image as the denoised image.

[0123] Among them, spatial denoising refers to denoising the image content of a single frame in a video, mainly for static images, and temporal denoising refers to denoising the previous and next frames in a video, mainly for moving images. Performing spatial denoising and temporal denoising on the input image frames respectively, and using the spatial denoising images and temporal denoising images as denoising images for fusion processing can effectively reduce multi-dimensional noise in the video and improve the denoising effect of the video.

[0124] It should be noted that in the process of denoising the input image frame based on the scene area distribution map, you can choose to denoise only in the spatial domain, or choose to denoise only in the time domain, or choose to denoise by combining time domain denoising and spatial domain denoising. The specific selection can be made according to the actual situation, and this example embodiment does not make any special limitations on this.

[0125] Specifically, the spatial denoising filtering degree corresponding to the input image frame can be obtained according to the scene area distribution map; and the input image frame is spatially denoised based on the scene area distribution map and the spatial denoising filtering degree to generate a spatial denoised image.

[0126] Among them, the degree of spatial denoising filtering refers to the numerical value of the denoising intensity of each scene area set according to different scene types, and then adaptive spatial denoising can be implemented for different areas of the input image frame based on the scene area distribution map and the degree of spatial denoising filtering, avoiding the problem of texture loss in the image caused by denoising all areas in the input image frame according to the same denoising intensity.

[0127] Spatial denoising can adopt the non-local mean filtering algorithm (NLM). Of course, edge-preserving filtering, bilateral filtering, guided filtering and other spatial denoising methods can also be adopted. This example embodiment does not specifically limit this. Preferably, the denoising window size and denoising intensity of the spatial denoising algorithm can be adaptively adjusted according to the scene area distribution map to achieve better spatial denoising effect.

[0128] Specifically, step S1020 may include: Fig.11 Steps S1110 to S1130 in the embodiment of the present invention are used to implement time domain denoising of the input image frame based on the scene area distribution map, referring to Fig.11 As shown, it may specifically include:

[0129] Step S1110, selecting a reference image frame corresponding to the input image frame, and acquiring a maximum inter-frame fusion degree corresponding to the input image frame based on the scene area distribution map;

[0130] Step S1120, performing image alignment or motion region detection on the input image frame and the reference image frame to obtain a ghost protection distribution map;

[0131] Step S1130 , based on the scene area distribution map, the maximum inter-frame fusion degree and the ghost protection distribution map, inter-frame fusion is performed on the input image frame and the reference image frame to generate a time-domain denoised image.

[0132] The reference image frame refers to an image frame selected from a video to guide the input image frame to perform time domain denoising. For example, the reference image frame can be the first 4 frames connected to the input image frame in time, or the last 4 frames connected to the input image frame in time. Of course, it can also be the 2 frames before and after the input image frame connected in time. This example embodiment does not specifically limit this. The maximum inter-frame fusion degree refers to the fusion degree value for fusing the input image frame with the reference image frame, which is set according to different scene types, and is used to guide the fusion between the input image frame and the reference image frame.

[0133] By performing image alignment or motion region detection on the input image frame and the reference image frame, different image regions between the input image frame and the reference image frame, i.e., a ghost protection distribution map, can be obtained. Specifically, when the computing power of the server or the terminal device allows, the input image frame and the reference image frame can be aligned with a higher precision, and when the computing power of the server or the terminal device does not allow, the input image frame and the reference image frame can be detected with a lower precision.

[0134] Based on the scene area distribution map, the maximum inter-frame fusion degree and the ghost protection distribution map to guide the fusion ratio, the input image frame and the reference image frame can be inter-frame fused to obtain a time domain denoised image corresponding to the input image frame.

[0135] By setting the maximum inter-frame fusion degree according to the scene type or the scene area distribution map, and obtaining the ghost protection distribution map through image alignment or motion area detection, and then guiding the inter-frame fusion of the input image frame and the reference image frame through the scene area distribution map, the maximum inter-frame fusion degree and the ghost protection distribution map, the temporal noise of the input image frame can be effectively reduced, and the denoising effect of the input image frame can be further improved.

[0136] In summary, in this exemplary embodiment, the scene type of the input image frame is determined, the input image frame is processed by scene area division according to the scene type to obtain a scene area distribution map, and the input image frame is subjected to image denoising based on the scene area distribution map to generate a denoised image; the input image frame, the scene area distribution map and the denoised image are fused to generate a target image frame, so as to generate a denoised video according to the target image frame. On the one hand, by guiding image denoising through the scene type corresponding to the image, it is possible to achieve adaptive area division of images under different scene types, and perform image denoising in combination with the area division information, effectively ensuring that different scene areas achieve different denoising strengths, avoiding the problem of missing image texture or weak denoising strength in some areas due to denoising, improving the denoising effect, and improving the flexibility of the image denoising algorithm; on the other hand, the input image frame, the scene area distribution map and the denoised image are fused to achieve further denoising of the image frame, further improve the quality of the texture details of the denoised image, and improve the denoising effect.

[0137] It should be noted that the above figures are only schematic illustrations of the processes included in the method according to the exemplary embodiments of the present disclosure, and are not intended to be limiting. It is easy to understand that the processes shown in the above figures do not indicate or limit the time sequence of these processes. In addition, it is also easy to understand that these processes can be performed synchronously or asynchronously, for example, in multiple modules.

[0138] For further reference, Fig.12 As shown, in the embodiment of this example, a video denoising device 1200 is also provided, including a scene type determination module 1210, a scene area distribution map generation module 1220, a denoised image generation module 1230 and a denoised video generation module 1240. Among them:

[0139] The scene type determination module 1210 is used to obtain an input image frame and determine the scene type of the input image frame;

[0140] The scene area distribution map generation module 1220 is used to perform scene area division processing on the input image frame according to the scene type to obtain a scene area distribution map;

[0141] The denoised image generation module 1230 is used to perform image denoising on the input image frame based on the scene area distribution map to generate a denoised image;

[0142] The denoised video generation module 1240 is used to fuse the input image frame, the scene area distribution map and the denoised image to generate a target image frame, so as to generate a denoised video according to the target image frame.

[0143] In an exemplary embodiment, the scene type determination module 1210 may be used to:

[0144] Extracting scene feature information of the input image frame, wherein the scene feature information includes an image tag;

[0145] The scene type of the input image frame is determined according to the scene feature information.

[0146] In an exemplary embodiment, the scene area distribution map generation module 1220 may include one or more of the following distribution map generation units:

[0147] a brightness region distribution map generating unit, configured to perform brightness region division processing on the input image frame based on the scene type to obtain a brightness region distribution map;

[0148] a chromaticity region distribution map generating unit, configured to perform chromaticity region division processing on the input image frame based on the scene type to obtain a chromaticity region distribution map;

[0149] a detail level region distribution map generating unit, configured to perform detail level region division processing on the input image frame based on the scene type to obtain a detail level region distribution map; and

[0150] The scene area distribution map generating unit is used to obtain a scene area distribution map by fusing any one of the brightness area distribution map, the chroma area distribution map and the detail level area distribution map or a plurality of area distribution maps.

[0151] In an exemplary embodiment, the brightness area distribution map generating unit may be used to:

[0152] Extracting image brightness information of the input image frame in a first color space;

[0153] Performing enhancement processing on the image brightness information based on the scene type to obtain enhanced image brightness information; and

[0154] The enhanced image brightness information is smoothed and denoised to obtain a brightness area distribution map.

[0155] In an exemplary embodiment, the chromaticity region distribution map generation unit may be configured to:

[0156] Extracting saturation information and hue information of the input image frame in a second color space;

[0157] The saturation information and the hue information are merged based on the scene type to obtain image chromaticity information;

[0158] Performing enhancement processing on the image chromaticity information based on the scene type to obtain enhanced image chromaticity information; and

[0159] The enhanced image chromaticity information is smoothed and denoised to obtain a chromaticity area distribution map.

[0160] In an exemplary embodiment, the detail level region distribution map generation unit may be configured to:

[0161] Extracting image brightness information of the input image frame in a first color space, and performing edge detection on the image brightness information to obtain image detail level information;

[0162] Performing enhancement processing on the image detail level information based on the scene type to obtain enhanced image detail level information; and

[0163] The enhanced image detail level information is smoothed and denoised to obtain a detail level regional distribution map.

[0164] In an exemplary embodiment, the brightness area distribution map generating unit may also be used to:

[0165] Acquire a brightness threshold corresponding to the input image frame based on the scene type;

[0166] A mapping curve is determined according to the brightness threshold, and piecewise linear mapping is performed on the image brightness information based on the mapping curve to obtain enhanced image brightness information.

[0167] In an exemplary embodiment, the brightness area distribution map generating unit may also be used to:

[0168] Performing morphological processing on the enhanced image brightness information to obtain image brightness information after edge expansion, wherein the morphological processing includes image expansion processing and / or image erosion processing;

[0169] The brightness information of the image after edge expansion is smoothed and filtered to obtain a brightness area distribution map.

[0170] In an exemplary embodiment, the denoised image generation module 1230 may include a spatial denoised image generation unit and / or a temporal denoised image generation unit:

[0171] a spatial denoising image generating unit, configured to perform spatial denoising on the input image frame based on the scene area distribution map to generate a spatial denoising image; and / or

[0172] A time-domain denoised image generating unit, configured to perform time-domain denoising on the input image frame based on the scene area distribution map to generate a time-domain denoised image;

[0173] The denoised image generating unit is used to use the spatial domain denoised image and / or the temporal domain denoised image as the denoised image.

[0174] In an exemplary embodiment, the spatial domain denoised image generation unit may be used to:

[0175] Acquire the spatial domain denoising filtering degree corresponding to the input image frame according to the scene area distribution map;

[0176] The input image frame is subjected to spatial denoising based on the scene area distribution map and the degree of spatial denoising filtering to generate a spatial denoising image.

[0177] In an exemplary embodiment, the temporal denoised image generation unit may be used to:

[0178] Selecting a reference image frame corresponding to the input image frame, and acquiring a maximum inter-frame fusion degree corresponding to the input image frame based on the scene area distribution map;

[0179] Performing image alignment or motion region detection on the input image frame and the reference image frame to obtain a ghost protection distribution map;

[0180] Based on the scene area distribution map, the maximum inter-frame fusion degree and the ghost protection distribution map, the input image frame and the reference image frame are inter-frame fused to generate a time domain denoised image.

[0181] The specific details of each module in the above device have been described in detail in the implementation method of the method part. The undisclosed details can be found in the implementation method of the method part, so they will not be repeated here.

[0182] Those skilled in the art will appreciate that various aspects of the present disclosure may be implemented as systems, methods or program products. Therefore, various aspects of the present disclosure may be specifically implemented in the following forms, namely: complete hardware implementation, complete software implementation (including firmware, microcode, etc.), or a combination of hardware and software, which may be collectively referred to herein as "circuits", "modules" or "systems".

[0183] The exemplary embodiments of the present disclosure also provide a computer-readable storage medium on which a program product capable of implementing the above-mentioned method of the present specification is stored. In some possible implementations, various aspects of the present disclosure can also be implemented in the form of a program product, which includes a program code. When the program product is run on a terminal device, the program code is used to enable the terminal device to execute the steps according to various exemplary embodiments of the present disclosure described in the above-mentioned "Exemplary Method" section of the present specification, for example, it can execute Figures 3 to 11 Any one or more steps in .

[0184] It should be noted that the computer-readable medium shown in the present disclosure may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0185] In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in combination with an instruction execution system, apparatus, or device. In the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in combination with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium may be transmitted using any suitable medium, including but not limited to: wireless, wire, optical cable, RF, etc., or any suitable combination of the above.

[0186] In addition, program code for performing the operations of the present disclosure may be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving a remote computing device, the remote computing device may be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (e.g., through the Internet using an Internet service provider).

[0187] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary technical means in the art that are not disclosed in the present disclosure. The specification and examples are to be considered as exemplary only, and the true scope and spirit of the present disclosure are indicated by the claims.

[0188] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A video denoising method, characterized in that: include: Acquire an input image frame, and determine a scene type of the input image frame; Performing scene area division processing on the input image frame according to the scene type to obtain a scene area distribution map, wherein the scene area distribution map is used to represent the probability that each pixel point in the input image frame belongs to the scene area corresponding to the scene information; Performing image denoising on the input image frame based on the scene area distribution map to generate a denoised image; The input image frame, the scene area distribution map and the denoised image are fused to generate a target image frame, so as to generate a denoised video according to the target image frame; The performing scene area division processing on the input image frame according to the scene type includes one or more of the following division processing methods: performing brightness area division processing on the input image frame based on the scene type to obtain a brightness area distribution map; Performing chromaticity region division processing on the input image frame based on the scene type to obtain a chromaticity region distribution map; Performing detail level area division processing on the input image frame based on the scene type to obtain a detail level area distribution map; And a scene area distribution map is obtained by fusing any one area distribution map or multiple area distribution maps among the brightness area distribution map, the chromaticity area distribution map and the detail level area.

2. The method according to claim 1, characterized in that The determining the scene type of the input image frame comprises: Extracting scene feature information of the input image frame, wherein the scene feature information includes an image tag; The scene type of the input image frame is determined according to the scene feature information.

3. The method according to claim 1, characterized in that The step of performing brightness area division processing on the input image frame based on the scene type to obtain a brightness area distribution map includes: Extracting image brightness information of the input image frame in a first color space; Performing enhancement processing on the image brightness information based on the scene type to obtain enhanced image brightness information; and The enhanced image brightness information is smoothed and denoised to obtain a brightness area distribution map.

4. The method according to claim 1, characterized in that: The step of performing chromaticity region division processing on the input image frame based on the scene type to obtain a chromaticity region distribution map includes: Extracting saturation information and hue information of the input image frame in a second color space; The saturation information and the hue information are merged based on the scene type to obtain image chromaticity information; Performing enhancement processing on the image chromaticity information based on the scene type to obtain enhanced image chromaticity information; and The enhanced image chromaticity information is smoothed and denoised to obtain a chromaticity area distribution map.

5. The method according to claim 1, characterized in that The performing detail level area division processing on the input image frame based on the scene type to obtain a detail level area distribution map includes: Extracting image brightness information of the input image frame in a first color space, and performing edge detection on the image brightness information to obtain image detail level information; Performing enhancement processing on the image detail level information based on the scene type to obtain enhanced image detail level information; and The enhanced image detail level information is smoothed and denoised to obtain a detail level regional distribution map.

6. The method according to any one of claims 3 to 5, characterized in that: The step of performing enhancement processing on the image brightness information based on the scene type to obtain enhanced image brightness information includes: Acquire a brightness threshold corresponding to the input image frame based on the scene type; A mapping curve is determined according to the brightness threshold, and piecewise linear mapping is performed on the image brightness information based on the mapping curve to obtain enhanced image brightness information.

7. The method according to any one of claims 3 to 5, characterized in that: The step of performing smoothing and denoising on the enhanced image brightness information to obtain a brightness area distribution map includes: Performing morphological processing on the enhanced image brightness information to obtain image brightness information after edge expansion, wherein the morphological processing includes image expansion processing and / or image erosion processing; The brightness information of the image after edge expansion is smoothed and filtered to obtain a brightness area distribution map.

8. The method according to claim 1, characterized in that The step of performing image denoising on the input image frame based on the scene area distribution map to generate a denoised image includes: Performing spatial denoising on the input image frame based on the scene area distribution map to generate a spatial denoised image; and / or Performing time-domain denoising on the input image frame based on the scene area distribution map to generate a time-domain denoised image; The spatial domain denoised image and / or the temporal domain denoised image is used as the denoised image.

9. The method according to claim 8, characterized in that The performing spatial denoising on the input image frame based on the scene area distribution map to generate a spatial denoised image includes: Acquire the spatial domain denoising filtering degree corresponding to the input image frame according to the scene area distribution map; The input image frame is subjected to spatial denoising based on the scene area distribution map and the degree of spatial denoising filtering to generate a spatial denoising image.

10. The method according to claim 8, characterized in that The step of performing time domain denoising on the input image frame based on the scene area distribution map to generate a time domain denoised image includes: Selecting a reference image frame corresponding to the input image frame, and acquiring a maximum inter-frame fusion degree corresponding to the input image frame based on the scene area distribution map; Performing image alignment or motion region detection on the input image frame and the reference image frame to obtain a ghost protection distribution map; Based on the scene area distribution map, the maximum inter-frame fusion degree and the ghost protection distribution map, the input image frame and the reference image frame are inter-frame fused to generate a time domain denoised image.

11. A video denoising device, characterized in that: include: A scene type determination module, used to obtain an input image frame and determine the scene type of the input image frame; A scene area distribution map generating module, used to perform scene area division processing on the input image frame according to the scene type to obtain a scene area distribution map, wherein the scene area distribution map is used to characterize the probability that each pixel point in the input image frame belongs to the scene area corresponding to the scene information; A denoised image generation module, configured to perform image denoising on the input image frame based on the scene area distribution map to generate a denoised image; A denoising video generation module, used for fusing the input image frame, the scene area distribution map and the denoised image to generate a target image frame, so as to generate a denoised video according to the target image frame; The performing scene area division processing on the input image frame according to the scene type includes one or more of the following division processing methods: performing brightness area division processing on the input image frame based on the scene type to obtain a brightness area distribution map; Performing chromaticity region division processing on the input image frame based on the scene type to obtain a chromaticity region distribution map; Performing detail level area division processing on the input image frame based on the scene type to obtain a detail level area distribution map; And a scene area distribution map is obtained by fusing any one area distribution map or multiple area distribution maps among the brightness area distribution map, the chromaticity area distribution map and the detail level area.

12. A computer readable medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 10 is implemented.

13. An electronic device, characterized in that: include: processor; as well as A memory, configured to store executable instructions of the processor; The processor is configured to perform the method of any one of claims 1 to 10 by executing the executable instructions.

Images