Image Denoising Method, Device, Electronic Device and Computer Readable Storage Medium

Abstract

Embodiments of the present application disclose an image noise reduction method, apparatus, electronic device, and computer-readable storage medium. Among them, the image noise reduction method includes: obtaining a to-be-processed image and a historical frame image corresponding to the to-be-processed image, segmenting the to-be-processed image into different noise reduction regions, performing noise reduction on different noise reduction regions in the to-be-processed image using different noise reduction algorithms to obtain the to-be-processed image after noise reduction, and performing fusion noise reduction on the to-be-processed image after noise reduction using the historical frame image to obtain a target noise reduction image. Embodiments of the present application divide the to-be-processed image into regions, perform differential noise reduction on different regions using different noise reduction algorithms, and use the historical frame image to perform fusion noise reduction on the to-be-processed image after differential noise reduction to avoid image blurring. The use of differential noise reduction and multi-frame fusion noise reduction improves the effect of image noise reduction.

Description

Technical Field

[0001] This application relates to the field of image processing, and in particular, to an image noise reduction method, apparatus, electronic device, and computer-readable storage medium. Background Art

[0002] With the continuous development of intelligent terminal technology, the use of electronic devices has become more and more popular. Most electronic devices are equipped with cameras, and with the enhancement of the processing power of mobile terminals and the development of camera technology, users' requirements for the quality of captured images are also getting higher and higher. In order to obtain high-quality images, it is often necessary to perform noise reduction processing on the images.

[0003] Currently, when performing noise reduction processing on images, generally the pixels are averaged with the surrounding pixels, and the image noise is reduced after averaging. However, this noise reduction processing method also averages the pixels at the object edges and details in the image, which easily causes the image to be blurred and results in poor image noise reduction effect. Summary of the Invention

[0004] Embodiments of this application provide an image noise reduction method, apparatus, electronic device, and computer-readable storage medium, which can improve the effect of image noise reduction.

[0005] Embodiments of this application provide an image noise reduction method, wherein the image noise reduction method includes:

[0006] Obtain a to-be-processed image and a historical frame image corresponding to the to-be-processed image;

[0007] Segment the to-be-processed image into different noise reduction regions;

[0008] Perform noise reduction on different noise reduction regions in the to-be-processed image using different noise reduction algorithms to obtain a noise-reduced to-be-processed image;

[0009] Perform fusion noise reduction on the noise-reduced to-be-processed image using the historical frame image to obtain a target noise-reduced image.

[0010] Embodiments of this application also provide an image noise reduction apparatus, wherein the image noise reduction apparatus includes:

[0011] An obtaining module, configured to obtain a to-be-processed image and a historical frame image corresponding to the to-be-processed image;

[0012] A segmentation module, configured to segment the to-be-processed image into different noise reduction regions;

[0013] A noise reduction module, configured to perform noise reduction on different noise reduction regions in the to-be-processed image using different noise reduction algorithms to obtain a noise-reduced to-be-processed image;

[0014] A fusion module for fusing and denoising the denoised image to be processed with the historical frame image to obtain a target denoised image.

[0015] An embodiment of the present application also provides an electronic device. The electronic device includes a processor and a memory. A computer program is stored in the memory. The processor executes the steps in any of the image denoising methods provided by the embodiments of the present application by calling the computer program stored in the memory.

[0016] An embodiment of the present application also provides a computer-readable storage medium. A computer program is stored in the storage medium. When the computer program runs on a computer, the computer executes the steps in any of the image denoising methods provided by the embodiments of the present application.

[0017] In the embodiments of the present application, first, an image to be processed and a historical frame image corresponding to the image to be processed are obtained. Then, the image to be processed is segmented into different denoising regions, and different denoising algorithms are used for different denoising regions in the image to be processed to obtain a denoised image to be processed. Then, the denoised image to be processed is fused and denoised with the historical frame image to obtain a target denoised image. In the embodiments of the present application, the image to be processed is divided into regions, different denoising algorithms are used for different regions for differential denoising, and the denoised image to be processed after differential denoising is fused and denoised with the historical frame image to avoid image blurring. The differential denoising and multi-frame fusion denoising methods are used to improve the image denoising effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] To more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for the description of the embodiments. Obviously, the following drawings are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative efforts.

[0019] Figure 1 It is the first flowchart of the image denoising method provided by the embodiment of the present application.

[0020] Figure 2 It is the first schematic diagram of the image denoising method provided by the embodiment of the present application.

[0021] Figure 3 It is the second schematic diagram of the image denoising method provided by the embodiment of the present application.

[0022] Figure 4 It is the third schematic diagram of the image denoising method provided by the embodiment of the present application.

[0023] Figure 5 It is the second flowchart of the image denoising method provided by the embodiment of the present application.

[0024] Figure 6 This is the first structural schematic diagram of the image noise reduction device provided by the embodiment of the present application.

[0025] Figure 7 This is the second structural schematic diagram of the image noise reduction device provided by the embodiment of the present application.

[0026] Figure 8 This is the first structural schematic diagram of the electronic device provided by the embodiment of the present application.

[0027] Figure 9 This is the second structural schematic diagram of the electronic device provided by the embodiment of the present application.

[0028] Figure 10 This is the structural schematic diagram of the integrated circuit chip provided by the embodiment of the present application. Detailed implementation manners

[0029] Next, the technical solutions in the embodiments of the present application will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments of the present application, all the embodiments obtained by those skilled in the art without creative efforts fall within the protection scope of the present application.

[0030] The terms "first", "second", "third", etc. (if any) in the specification and claims of the present application and the above accompanying drawings are used to distinguish similar objects, and do not necessarily need to be used to describe a specific order or sequence. It should be understood that the described objects can be interchanged under appropriate circumstances. In addition, the terms "include" and "have" and any of their variations are intended to cover non-exclusive inclusion. For example, a process, method, or apparatus, electronic device, system that includes a series of steps does not necessarily need to be limited to those clearly listed steps, or modules and units, and may also include steps, modules, or units that are not clearly listed, or may also include other steps, modules, or units inherent to these processes, methods, apparatuses, electronic devices, or systems.

[0031] The embodiment of the present application provides an image noise reduction method, and this image noise reduction method is applied to an electronic device. The execution subject of this image noise reduction method can be the image noise reduction device provided by the embodiment of the present application, or an electronic device integrated with this image noise reduction device. This image noise reduction device can be implemented in a hardware or software manner. The electronic device can be a device with processing capabilities configured with a processor, such as a smart phone, a tablet computer, a handheld computer, a notebook computer, or a desktop computer, etc.

[0032] Please refer to Figure 1 ,Figure 1 FIG. 1 is a first flowchart of the image noise reduction method provided by the embodiments of the present application. The execution subject of the image noise reduction method may be the image noise reduction device provided by the embodiments of the present application, or an integrated circuit chip or an electronic device integrated with the image noise reduction device. The image noise reduction method provided by the embodiments of the present application may include the following steps:

[0033] 110. Obtain the image to be processed and the historical frame image corresponding to the image to be processed.

[0034] In one embodiment, start the camera to obtain the video stream of the scene to be photographed, and obtain multiple frames of raw images of the scene to be photographed. The raw image may be in the RAW (RAW Image Format, raw) format, that is, after the image sensor converts the captured light source signal into a digital signal, the unprocessed raw data. The specific number of multiple frames of raw images can be set according to actual needs, and the present application does not limit this. The image to be processed and the historical frame image corresponding to the image to be processed obtained by the embodiments of the present application may be taken from multiple frames of raw images of the same video stream, and for the same scene to be photographed, they have the same or similar image content.

[0035] Among them, multiple frames of raw images may have the same or different exposure parameters. For example, when obtaining the raw images, first determine the exposure parameters for normal exposure according to the automatic metering system of the camera, and then, on the basis of the exposure parameters for normal exposure, adjust the exposure parameters to increase or decrease the exposure degree, and then take pictures to obtain multiple frames of raw images with different exposure parameters. For example, the exposure parameters may include the exposure duration, and the exposure duration affects the exposure amount. By using different exposure durations for shooting, multiple frames of raw images with different exposure amounts of the same scene to be photographed can be obtained.

[0036] In one embodiment, the step of obtaining the image to be processed and the historical frame image corresponding to the image to be processed may include: obtaining multiple frames of raw images of the scene to be photographed; determining the image to be processed from the multiple frames of raw images, and obtaining the historical frame image corresponding to the image to be processed according to the determined image to be processed. The image to be processed may be any one of the multiple frames of raw images, and the historical frame image corresponding to the image to be processed may be the raw image before the image to be processed.

[0037] Alternatively, the historical frame image corresponding to the image to be processed may also be an image that has been subjected to noise reduction processing and obtained from the cache. For example, for any raw image before the image to be processed in the shooting order, use the image noise reduction method provided by the embodiments of the present application to perform noise reduction processing on the raw image, and cache the raw image after the noise reduction processing, and use the cache result as the historical frame image corresponding to the subsequent image to be processed.

[0038] It should be noted that the number of historical frame images corresponding to each image to be processed can be one, two, or more. Each original image or the cached result of the original image with a shooting order before the image to be processed in the cache can be used as the historical frame image corresponding to the image to be processed. As for how to specifically select the historical frame image, it can be set according to actual needs, and the present application does not limit this. For example, the cached result of the previous frame image adjacent to the image to be processed can be used as the historical frame image corresponding to the image to be processed, or the historical frame image corresponding to the image to be processed can be selected from the cache every n (n is a positive integer) frame images. For example, the cached results of the images separated from the image to be processed by 4 frames and 8 frames can be selected as the historical frame images corresponding to the image to be processed, and so on.

[0039] 120, segment the image to be processed into different noise reduction regions.

[0040] In one embodiment, the depth information of the image to be processed is obtained, and the image to be processed is segmented into different noise reduction regions according to the depth information.

[0041] Among them, the depth information of the image to be processed can exist in the form of at least one depth map, indicating the distance between the sensor (for example, an image sensor, a depth sensor) and the object (for example, the distance from the depth sensor to all objects in the image to be processed). Since different objects in the image to be processed have corresponding depth information respectively, different depth information can reflect the positions of different objects in the image to be processed. And the depth information corresponding to the same object constitutes a continuous depth interval. Therefore, for the regions corresponding to different depth intervals in the image to be processed, the degree of noise reduction processing can be different. For example, a certain (or certain) region corresponding to a depth interval can be focused on for noise reduction, and other regions corresponding to other depth information can be non-focused on for noise reduction. This (or these) region (or regions) for focused noise reduction is segmented into a focused noise reduction region, and other regions are segmented into non-focused noise reduction regions. Segmenting the image to be processed into different noise reduction regions according to the depth information can achieve differential noise reduction of the image and improve the smoothness of image noise reduction.

[0042] In one embodiment, the electronic device is provided with a light sensor. By calculating the phase difference between the near-infrared light emitted by the light sensor and its reflected light according to the phase parameters of the near-infrared light and its reflected light, the depth of each object in the image to be processed can be calculated, and thus the depth information of the image to be processed can be obtained.

[0043] In one embodiment, for the static region and the moving region in the image to be processed, different methods are respectively used to obtain the depth information. For example, the linear perspective method can be used to obtain the depth information of the static region, and the high-precision optical flow method based on the deformation theory can be used to obtain the motion vector and convert it into the depth information of the moving region.

[0044] In one embodiment, the preset depth interval can be set by the user according to needs. First, the focusing parameter when capturing the image to be processed can be obtained, and the focal plane when capturing the image to be processed can be determined according to the focusing parameter. The depth interval where the focal plane is located is determined as the preset depth interval. Among them, the depth interval where the focal plane is located can be the depth interval where a certain focusing object is located when the focusing object is captured. It can be understood that since the object is three-dimensional and the depth corresponding to the focal plane is a specific value, it is often impossible to include all the depth information of the object. Therefore, a depth interval is needed to summarize it.

[0045] Then, the image to be processed is segmented into different noise reduction regions according to the depth information. Optionally, the depth information of the image to be processed can be divided into at least two depth intervals including the preset depth interval, and the image to be processed is segmented into at least two different noise reduction regions according to the at least two depth intervals. Among them, at least two different noise reduction regions include a key noise reduction region, and the key noise reduction region is segmented from the image to be processed according to the preset depth interval, and the other regions are non-key noise reduction regions. There can be one or more non-key noise reduction regions, and the number of non-key noise reduction regions depends on the number of segmented depth intervals.

[0046] In the embodiment of the present application, the depth interval where the focusing object is located is used as the depth interval where the focal plane is located, and the depths within a certain range before and after the focal plane depth are included, so that the region where the focusing object is located can be correctly segmented as the key noise reduction region. The user can instruct the device through the focusing operation to perform key noise reduction on the focusing object, that is, the focusing parameter can be obtained based on the user's focusing operation.

[0047] In one embodiment, the image to be processed is segmented into two different noise reduction regions according to the depth information, namely the first noise reduction region and the second noise reduction region. Among them, the first noise reduction region is the key noise reduction region. According to the depth information, the region in the image to be processed where the depth is within the preset depth interval is segmented into the first noise reduction region, and the region in the image to be processed other than the first noise reduction region is segmented into the second noise reduction region.

[0048] In one embodiment, the image to be processed is segmented into multiple different noise reduction regions according to the depth information. Among the multiple different noise reduction regions, there is a first noise reduction region, and the first noise reduction region is the key noise reduction region. Multiple depth intervals corresponding to the image to be processed are divided according to the depth information. Among the multiple depth intervals, there is a preset depth interval. The image to be processed is segmented into multiple different noise reduction regions according to these multiple depth intervals. Among them, the preset depth interval where the focal plane is located corresponds to the first noise reduction region.

[0049] 130. Different noise reduction algorithms are used for different noise reduction regions in the image to be processed to obtain the noise-reduced image to be processed.

[0050] Among them, different denoising algorithms with different biases are adopted for different denoising regions in the image to be processed. For example, a denoising algorithm with a large denoising intensity is adopted for the key denoising region, and a generally fast denoising algorithm is used for other regions. On the one hand, this can speed up the denoising process. On the other hand, different degrees of denoising for different depths of the image to be processed can achieve the purpose of differential denoising, making the denoising effect of the image to be processed more natural.

[0051] In one embodiment, different denoising algorithms are adopted for different denoising regions in the image to be processed, which can also mean that for different denoising regions, the parameters in the denoising algorithm are changed, so that the denoising effects of different denoising regions are different.

[0052] In one embodiment, after obtaining the historical frame image corresponding to the image to be processed and the denoised image to be processed, before performing fusion denoising, if there are moving objects in the historical frame image and the image to be processed, it may cause the fused denoised image to shift. Therefore, the image denoising method may further include:

[0053] Detecting the moving regions in the historical frame image and the denoised image to be processed;

[0054] Aligning the moving regions according to the detection results.

[0055] For example, the third image is the image to be processed, and the first image and the second image are the corresponding historical frame images. The moving regions of the first image, the second image, and the third image can be detected respectively. The moving region can be a local tiny moving region in the image. For example, if the image includes a road and people or vehicles walking on the road, the moving region can be the people or vehicles in the image. After detecting the moving region, the pixel points of the moving region can be aligned. For example, the pixel point coordinates of the moving region in the first image and the second image can be adjusted to the corresponding coordinates of the pixel points of the moving region in the third image to achieve motion alignment.

[0056] 140. Using the historical frame image to perform fusion denoising on the denoised image to be processed to obtain the target denoised image.

[0057] In one embodiment, the historical frame images used to perform fusion denoising on the denoised image to be processed have different exposure degrees. When using the historical frame images to perform fusion denoising on the denoised image to be processed, the HDR (High-Dynamic Range) multi-frame synthesis method is adopted to obtain the target denoised image. Compared with the general synthesis method, using historical frame images with different exposure degrees to perform fusion denoising on the denoised image to be processed can retain more bright and dark details.

[0058] In one embodiment, the historical frame image has been denoised using the image denoising method provided by the embodiments of the present application. That is, different denoising regions have also been segmented from the historical frame image according to depth information. Based on the different denoising regions of the historical frame image and the image to be processed, the different denoising regions in the denoised image to be processed are placed in the historical frame image for matching, and the denoising regions of the historical frame image that are matched are segmented from the historical frame image and synthesized into the corresponding denoising regions of the denoised image to be processed.

[0059] For image noise, many noises are random noises, and their occurrence positions and frequencies are not fixed. That is to say, the same noise point rarely appears at the same position multiple times. Through fusion denoising, the noise-free points in the historical frame image can be fused into the image to be processed to correct the image to be processed. The final obtained target denoised image further improves the denoising effect on the basis of differential denoising.

[0060] In one embodiment, for the different denoising regions in the denoised image to be processed, different numbers of historical frame images can be used for fusion denoising. That is to say, the present application not only uses historical frame images to perform fusion denoising on the denoised image to be processed, but also the denoising degrees of different denoising regions are different. That is, there can be multiple historical frame images of the image to be processed, but for the multiple different denoising regions segmented from the image to be processed according to depth information, not all denoising regions use all the historical frame images for denoising. Only for the key denoising regions, all the historical frame images are used for denoising, while for other regions, the greater the depth difference from the key denoising region, the fewer the number of historical frame images used for fusion denoising.

[0061] In one embodiment, before using different numbers of historical frame images for fusion denoising for the different denoising regions in the denoised image to be processed, first obtain the depth intervals corresponding to the different denoising regions in the denoised image to be processed, determine the depth differences between the depth intervals corresponding to the different denoising regions in the denoised image to be processed and the depth interval where the focal plane is located. For each denoising region in the denoised image to be processed, determine the number of historical frame images used for fusion denoising according to the corresponding depth difference. Among them, the smaller the corresponding depth difference, the more the number of historical frame images used for fusion denoising. In this way, during fusion denoising, not the entire image uses a single standard for denoising, but the closer the depth is to the key denoising region, the more the number of historical frame images used for fusion denoising. And the more the number, the more noise points in this denoising region can be compensated for by using the historical frame images, and the better the denoising effect. The present application also makes a differential arrangement in fusion denoising, thereby ensuring the smoothness of image denoising and further improving the denoising effect of the image to be processed.

[0062] In one embodiment, after obtaining the target noise-reduced image, the target noise-reduced image is cached as the historical frame image corresponding to the subsequent image to be processed. That is to say, the image noise reduction method provided by the embodiments of the present application is not a noise reduction operation only for a single frame of image, but a continuous noise reduction operation during the acquisition of multiple frames of original images. The following will be described with reference to several drawings:

[0063] Please refer to Figure 2 , Figure 2 which is the first schematic diagram of the image noise reduction method provided by the embodiments of the present application. Figure 2 It shows the noise reduction process using the cached result of the adjacent previous frame image as the historical frame image corresponding to the image to be processed. Among them, images 11 to 15 are multiple frames of original images of the scene to be photographed. Images 11, 12, and 13 are the first frame, the second frame, and the third frame images respectively. Between image 13 and image 15, there are multiple frames of images not shown except image 14. The noise reduction in the figure may refer to the differential noise reduction method that uses depth information to adopt different noise reduction algorithms for different noise reduction regions as disclosed in the embodiments of the present application, and the fusion in the figure may refer to the fusion noise reduction method disclosed in the embodiments of the present application. The specific noise reduction and fusion methods can be seen in the description of the embodiments of the present application and will not be elaborated here.

[0064] Among them, for image 11, after differential noise reduction, the result of differential noise reduction of image 11 is directly cached; when performing noise reduction on image 12, the cached result of image 11 is fused with the differentially noise-reduced image 12 for noise reduction, and the result of fused noise reduction of image 12 (the target noise-reduced image) is cached; when performing noise reduction on image 13, the target noise-reduced image of image 12 in the cache is fused with the differentially noise-reduced image 13 for noise reduction, and the result of fused noise reduction of image 13 (the target noise-reduced image) is cached;...; until the noise reduction of the last image 15 is completed.

[0065] Except for the first frame image 11, when each image is used as the image to be processed, the historical frame image obtained from the cache is the result of noise reduction of the adjacent previous image using the image noise reduction method provided by the embodiments of the present application. After each image to be processed completes noise reduction using such a historical frame image, it will be used as the historical frame image corresponding to the subsequent image to be processed. In this way, each historical frame image used for fused noise reduction is the best noise reduction result of the previous original image, thereby ensuring that the noise reduction effect of each image to be processed will also reach the best. And, as the noise reduction progresses, the noise reduction quality from image 11 to image 15 will be improved step by step.

[0066] Please refer to Figure 3 , Figure 3 which is the second schematic diagram of the image noise reduction method provided by the embodiments of the present application. Figure 3The noise reduction process is shown in which the cached results of the adjacent first two frame images are used as the historical frame images corresponding to the image to be processed. Among them, images 21 to 26 are multiple frame original images of the scene to be photographed. Images 21, 22, 23, and 24 are the first frame, second frame, third frame, and fourth frame images respectively. There are multiple frames of images not shown between image 24 and image 26 except image 26. The noise reduction in the figure may refer to the differential noise reduction method that uses depth information to adopt different noise reduction algorithms for different noise reduction regions disclosed in the embodiments of the present application. The fusion in the figure may refer to the fusion noise reduction method disclosed in the embodiments of the present application. For specific noise reduction and fusion methods, please refer to the description of the embodiments of the present application and will not be elaborated here.

[0067] Among them, for images 21 and 22, after differential noise reduction, the results of differential noise reduction of images 21 and 22 are directly cached; when noise reduction is performed on image 23, the cached result of image 1, the cached result of image 22, and the differentially noise-reduced image 23 are fused for noise reduction, and the result of fused noise reduction of image 23 (the target noise-reduced image) is cached; when noise reduction is performed on image 24, the cached result of image 22 in the cache, the target noise-reduced image of image 23, and the differentially noise-reduced image 24 are fused for noise reduction, and the result of fused noise reduction of image 24 (the target noise-reduced image) is cached;...; until the noise reduction of the last image 26 is completed.

[0068] Except for the first frame image 21 and the second frame image 22, when each image is used as the image to be processed, the historical frame images obtained from the cache are the results of noise reduction of the adjacent first two frame images using the image noise reduction method provided in the embodiments of the present application. After each image to be processed completes noise reduction using such historical frame images, it will be used as the historical frame image corresponding to the subsequent image to be processed. In this way, each historical frame image used for fused noise reduction is the best noise reduction result of the first two frame original images, thereby ensuring that the noise reduction effect of each image to be processed will also reach the best. Moreover, as the noise reduction progresses, the noise reduction quality from image 21 to image 26 will be improved step by step.

[0069] Please continue to refer to Figure 4 , Figure 4This is the third schematic diagram of the image noise reduction method provided by the embodiments of the present application. In one embodiment, the images to be processed are selected from multiple frames of images at intervals of a preset number. That is, among the multiple frames of original images obtained by shooting the same scene to be photographed, only some of the original images adopt the image noise reduction method provided by the embodiments of the present application, and only the target noise reduction images of the images 31, 33, and 35 processed by the image noise reduction method provided by the embodiments of the present application will be cached. When reducing the noise of the image 35, only the historical frame image of the image 35 is selected from the cache. Therefore, the target noise reduction images cached by the images 31 and 33 can be used as the historical frame images of the image 35, while the unprocessed images 32 and 34 are not cached and do not participate in the noise reduction of the subsequent images to be processed. This is to ensure that the image details of each image to be processed are not exactly the same, which is beneficial to improving the effect of fusion noise reduction.

[0070] It should be noted that Figures 2 to 4 the selection method and selection quantity of the historical frame images listed in the appendix are all exemplary. Those skilled in the art should understand that the selection method and selection quantity of the historical frame images can be changed according to requirements. For example, the number of historical frame images corresponding to each image to be processed can be 3, or each image to be processed takes the cached results of all the previous original images as the corresponding historical frame images, and so on.

[0071] Please refer to Figure 5 , Figure 5 This is the second flow schematic diagram of the image noise reduction method provided by the embodiments of the present application. This image noise reduction method can be applied to the electronic device provided by the embodiments of the present application. The image noise reduction method provided by the embodiments of the present application may include the following steps:

[0072] 201. Obtain the image to be processed and the historical frame image corresponding to the image to be processed.

[0073] In one embodiment, start the camera, obtain the video stream of the scene to be photographed, and obtain multiple frames of original images of the scene to be photographed. The original image can be in RAW format, that is, after the image sensor converts the captured light source signal into a digital signal, the unprocessed original data. The specific number of multiple frames of original images can be set according to actual needs, and the present application does not limit this. The image to be processed and the historical frame image corresponding to the image to be processed obtained by the embodiments of the present application can be taken from multiple frames of original images of the same video stream, and for the same scene to be photographed, they have the same or similar image content.

[0074] Among them, the multiple frames of original images may have the same or different exposure parameters. For example, when acquiring the original images, first determine the exposure parameters for normal exposure according to the automatic metering system of the camera, and then, on the basis of the exposure parameters for normal exposure, adjust the exposure parameters to increase or decrease the exposure degree, and then take pictures to obtain multiple frames of original images with different exposure parameters. For example, the exposure parameters may include the exposure duration, and the exposure duration affects the exposure amount. By using different exposure durations during shooting, multiple frames of original images with different exposure amounts of the same scene to be shot can be obtained.

[0075] It should be noted that the number of historical frame images corresponding to each image to be processed can be one, two, or more. Each original image or the cached result of the original image with a shooting order before the image to be processed in the cache can be used as the historical frame image corresponding to the image to be processed. Specifically, how to select the historical frame images can be set according to actual needs, and this application does not limit this. For example, the cached result of the previous frame image adjacent to the image to be processed can be used as the historical frame image corresponding to the image to be processed, or the historical frame image corresponding to the image to be processed can be selected from the cache every n (n is a positive integer) frame images. For example, the cached results of the images separated from the image to be processed by 4 frames and 8 frames can be selected as the historical frame images corresponding to the image to be processed, and so on.

[0076] 202. Obtain the depth information of the image to be processed.

[0077] 203. Segment the image to be processed into different noise reduction regions according to the depth information.

[0078] For the description of step 203, reference can be made to the description of step 120 above, and details will not be repeated here.

[0079] 204. Use different noise reduction algorithms for different noise reduction regions in the image to be processed to obtain the image to be processed after noise reduction.

[0080] Among them, different noise reduction algorithms with different biases are used for different noise reduction regions in the image to be processed. For example, a noise reduction algorithm with a large noise reduction intensity is used for the key noise reduction regions, and a generally fast noise reduction algorithm is used for other regions. On the one hand, this can speed up the noise reduction processing speed. On the other hand, different degrees of noise reduction for different depths of the image to be processed can achieve the purpose of differential noise reduction, making the noise reduction effect of the image to be processed more natural.

[0081] In an embodiment, using different noise reduction algorithms for different noise reduction regions in the image to be processed may also mean changing the parameters in the noise reduction algorithm for different noise reduction regions, so that the noise reduction effects for different noise reduction regions are different.

[0082] 205. Identify feature points in the image to be processed after noise reduction and the historical frame image.

[0083] Among them, the feature points are the points with features in the image, which can be extreme points, or points with prominent attributes in some aspects. For example, the intersection of two lines, or the vertex of an angle, etc. The feature points in the image can reflect the positions and contours of various objects in the image.

[0084] Various feature point extraction algorithms can be used to identify feature points in the image to be processed after noise reduction and the historical frame image. For example, Harris (Harris corner detection), SIFT (Scale Invariant Feature Transform), etc. The feature point matching algorithm has good environmental adaptability and can achieve fast and accurate image stabilization of the device in various imaging environments while meeting real-time requirements.

[0085] 206. Based on the noise reduction regions where the feature points identified in each image are located, determine the feature points corresponding to different noise reduction regions in each image.

[0086] Whether it is the image to be processed after noise reduction or the historical frame image, different noise reduction regions have been divided. After identifying feature points, the feature points identified in the same noise reduction region will be attributed to the corresponding noise reduction region, obtaining a set of feature points corresponding to different noise reduction regions in each image, where each noise reduction region of each image corresponds to a set of feature points.

[0087] 207. For each noise reduction region in the image to be processed after noise reduction, match the corresponding noise reduction region in the historical frame image according to the feature points corresponding to different noise reduction regions in each image.

[0088] In one embodiment, when matching the corresponding noise reduction regions, determine the similarity of the two noise reduction regions based on the identified feature points. If the similarity of the two noise reduction regions is greater than or equal to the preset threshold, it is determined that the two noise reduction regions are successfully matched and there is a corresponding relationship between the two noise reduction regions; otherwise, it is determined that the two noise reduction regions fail to match.

[0089] 208. Synthesize the mutually matched noise reduction regions in the image to be processed after noise reduction and the historical frame image to perform fusion noise reduction on the image to be processed after noise reduction, obtaining the target noise reduction image.

[0090] In one embodiment, the historical frame images used for fusing and noise-reducing the to-be-processed image after noise reduction have different exposure degrees. When using the historical frame images to fuse and noise-reduce the to-be-processed image after noise reduction, the HDR (High-Dynamic Range) multi-frame synthesis method is adopted to obtain the target noise-reduced image. Compared with the general synthesis method, using historical frame images with different exposure degrees to fuse and noise-reduce the to-be-processed image after noise reduction can retain more bright and dark details.

[0091] In one embodiment, if the matching is successful, the noise-reduced regions matched in the historical frame images are synthesized into the corresponding noise-reduced regions of the to-be-processed image after noise reduction.

[0092] For image noise, many noises belong to random noises, and their occurrence positions and frequencies are not fixed, that is, the same noise point rarely appears at the same position multiple times. Through fusing and noise reduction, the noise-free points in the historical frame images can be fused into the to-be-processed image to correct the to-be-processed image. The finally obtained target noise-reduced image further improves the noise reduction effect on the basis of differential noise reduction.

[0093] In one embodiment, for different noise-reduced regions in the to-be-processed image after noise reduction, different numbers of historical frame images can be used for fusing and noise reduction. Before using different numbers of historical frame images to fuse and noise-reduce different noise-reduced regions in the to-be-processed image after noise reduction, first obtain the depth intervals corresponding to different noise-reduced regions in the to-be-processed image after noise reduction, determine the depth differences between the depth intervals corresponding to different noise-reduced regions in the to-be-processed image after noise reduction and the depth interval where the focal plane is located. For each noise-reduced region in the to-be-processed image after noise reduction, determine the number of historical frame images used for fusing and noise reduction according to the corresponding depth difference. Among them, the smaller the corresponding depth difference is, the more historical frame images are used for fusing and noise reduction. In this way, during fusing and noise reduction, not the whole image adopts a single standard for noise reduction, but the closer the depth is to the key noise-reduced region, the more historical frame images are used for fusing and noise reduction. And the more the number is, the more noise points in this noise-reduced region can be compensated by using the historical frame images, and the better the noise reduction effect is. The present application also makes a differential arrangement in fusing and noise reduction, thereby ensuring the smoothness of image noise reduction and further improving the noise reduction effect of the to-be-processed image.

[0094] 210. Cache the target noise-reduced image, and when noise-reducing subsequent to-be-processed images, use it as the historical frame image corresponding to the subsequent to-be-processed images.

[0095] The image denoising method provided by the embodiments of the present application is not a denoising operation only for a single frame of image, but a continuous denoising operation during the acquisition of multiple frames of original images. When each image is used as an image to be processed, the corresponding historical frame image obtained from the cache is the result of denoising the previous original images using the image denoising method provided by the embodiments of the present application. After each image to be processed is denoised using such historical frame images, it will, in turn, serve as the historical frame image corresponding to the subsequent image to be processed. In this way, each historical frame image used for fusion denoising is the best denoising result of the previous original images, thereby ensuring that the denoising effect of each image to be processed will also reach the best. Moreover, as the denoising progresses, the denoising quality of the subsequent original images will gradually improve.

[0096] As can be seen from the above, for the image denoising method provided by the embodiments of the present application, first, an image to be processed and the corresponding historical frame image of the image to be processed are obtained. Then, the image to be processed is segmented into different denoising regions, and different denoising algorithms are used for the different denoising regions in the image to be processed to perform denoising, obtaining the denoised image to be processed. Next, the historical frame image is used to perform fusion denoising on the denoised image to be processed, obtaining the target denoised image. The embodiments of the present application divide the image to be processed into regions, adopt different denoising algorithms for different regions to perform differential denoising, and use the historical frame image to perform fusion denoising on the differentially denoised image to be processed to avoid image blurring. By adopting the methods of differential denoising and multi-frame fusion denoising, the effect of image denoising is improved.

[0097] The embodiments of the present application also provide an image denoising device. Please refer to Figure 6 , Figure 6 which is the first structural schematic diagram of the image denoising device provided by the embodiments of the present application. The image denoising device 300 can be applied to an electronic device or an integrated circuit chip. The image denoising device 300 includes an acquisition module 301, a segmentation module 302, a denoising module 303, and a fusion module 304, as follows:

[0098] The acquisition module 301 is configured to acquire an image to be processed and the corresponding historical frame image of the image to be processed;

[0099] The segmentation module 302 is configured to segment the image to be processed into different denoising regions;

[0100] The denoising module 303 is configured to use different denoising algorithms for different denoising regions in the image to be processed to perform denoising, obtaining the denoised image to be processed;

[0101] The fusion module 304 is configured to use the historical frame image to perform fusion denoising on the denoised image to be processed, obtaining the target denoised image.

[0102] In one embodiment, when dividing the image to be processed into different noise reduction regions, the segmentation module 302 can be used to:

[0103] Obtain the depth information of the image to be processed;

[0104] Divide the image to be processed into different noise reduction regions according to the depth information.

[0105] In one embodiment, the different noise reduction regions include at least two different noise reduction regions. When dividing the image to be processed into different noise reduction regions according to the depth information, the segmentation module 302 can be used to:

[0106] Divide the depth information of the image to be processed to obtain at least two depth intervals including a preset depth interval;

[0107] Divide the image to be processed into at least two different noise reduction regions according to the at least two depth intervals.

[0108] In one embodiment, among the at least two different noise reduction regions, there is a key noise reduction region. When dividing the image to be processed into at least two different noise reduction regions according to the at least two depth intervals, the segmentation module 302 can be used to:

[0109] Segment out the key noise reduction region from the image to be processed according to the preset depth interval.

[0110] Please refer to Figure 7 , Figure 7 which is the second structural schematic diagram of the image noise reduction device 300 provided by the embodiment of the present application. In one embodiment, the image noise reduction device 300 further includes a cache module 305, and the cache module 305 is used to:

[0111] Cache the target noise reduction image, and use it as the historical frame image corresponding to the subsequent image to be processed when performing noise reduction on the subsequent image to be processed.

[0112] Please continue to refer to Figure 7 , in one embodiment, the image noise reduction device 300 further includes a first determination module 306, and the first determination module 306 is used to:

[0113] Obtain the focusing parameter when shooting the image to be processed;

[0114] Determine the focal plane when shooting the image to be processed according to the focusing parameter;

[0115] Determine the depth interval where the focal plane is located as the preset depth interval.

[0116] In one embodiment, when using the historical frame image to perform fusion noise reduction on the noise-reduced image to be processed, the fusion module 304 can be used to:

[0117] For different noise reduction regions in the to-be-processed image after noise reduction, different numbers of historical frame images are used for fusion noise reduction.

[0118] Please continue to refer to Figure 7 In one embodiment, the image noise reduction device 300 further includes a second determination module 307, and the second determination module 307 is configured to:

[0119] Obtain the depth intervals corresponding to different noise reduction regions in the to-be-processed image after noise reduction;

[0120] Determine the depth differences between the depth intervals corresponding to different noise reduction regions in the to-be-processed image after noise reduction and the depth interval where the focal plane is located;

[0121] For each noise reduction region in the to-be-processed image after noise reduction, determine the number of historical frame images used for fusion noise reduction according to the corresponding depth difference, where the smaller the corresponding depth difference, the more historical frame images are used for fusion noise reduction.

[0122] Please continue to refer to Figure 7 In one embodiment, the image noise reduction device 300 further includes an identification module 308 and a third determination module 309:

[0123] The identification module 308 is configured to perform feature point identification on the to-be-processed image and historical frame images; or perform feature point identification on the to-be-processed image after noise reduction and historical frame images;

[0124] The third determination module 309 is configured to determine the feature points corresponding to different noise reduction regions in each image based on the noise reduction regions where the identified feature points are located in each image.

[0125] In one embodiment, when using historical frame images to perform fusion noise reduction on the to-be-processed image after noise reduction, the fusion module 304 may be configured to:

[0126] For each noise reduction region in the to-be-processed image after noise reduction, match the corresponding noise reduction region in the historical frame images according to the feature points corresponding to different noise reduction regions in each image;

[0127] Synthesize the successfully matched noise reduction regions in the to-be-processed image after noise reduction and the historical frame images to perform fusion noise reduction on the to-be-processed image after noise reduction.

[0128] Among them, when matching the corresponding noise reduction regions, determine the similarity between two noise reduction regions based on the identified feature points. If the similarity of the feature point sets of the two noise reduction regions is greater than or equal to a preset threshold, it is determined that the two noise reduction regions are successfully matched and there is a corresponding relationship between the two noise reduction regions; otherwise, it is determined that the two noise reduction regions fail to match.

[0129] In one embodiment, when synthesizing the successfully matched noise reduction regions in the image to be processed after noise reduction and the historical frame image, the fusion module 304 can be used to:

[0130] If the matching is successful, the noise reduction region matched in the historical frame image is synthesized into the corresponding noise reduction region of the image to be processed after noise reduction.

[0131] For the specific implementation of each of the above modules, reference can be made to the previous embodiments, which will not be elaborated here.

[0132] As can be seen from the above, the image noise reduction device provided by the embodiments of the present application first obtains the image to be processed and the historical frame image corresponding to the image to be processed through the acquisition module 301, then the segmentation module 302 segments the image to be processed into different noise reduction regions, and the noise reduction module 303 uses different noise reduction algorithms for different noise reduction regions in the image to be processed to perform noise reduction, obtaining the image to be processed after noise reduction. The fusion module 304 performs fusion noise reduction on the image to be processed after noise reduction by using the historical frame image to obtain the target noise reduction image. The embodiments of the present application divide the image to be processed into regions, adopt different noise reduction algorithms for different regions to perform differential noise reduction, and use the historical frame image to perform fusion noise reduction on the image to be processed after differential noise reduction to avoid image blurring. By adopting the methods of differential noise reduction and multi-frame fusion noise reduction, the effect of image noise reduction is improved.

[0133] The embodiments of the present application also provide an electronic device. The electronic device can be a smart phone, a tablet computer, a game device, an AR (Augmented Reality) device, a vehicle, a vehicle peripheral obstacle detection device, an audio playback device, a video playback device, a notebook, a desktop computing device, a wearable device such as a watch, glasses, a helmet, an electronic bracelet, an electronic necklace, an electronic garment, etc.

[0134] Reference Figure 8 , Figure 8 is the first structural schematic diagram of the electronic device 400 provided by the embodiments of the present application. Among them, the electronic device 400 includes a processor 401 and a memory 402. A computer program is stored in the memory, and the processor executes the steps in any of the image noise reduction methods provided by the embodiments of the present application by calling the computer program stored in the memory. The processor 401 is electrically connected to the memory 402.

[0135] The processor 401 is the control center of the electronic device 400, connects various parts of the entire electronic device through various interfaces and lines, executes various functions of the electronic device and processes data by running or calling the computer program stored in the memory 402 and calling the data stored in the memory 402, so as to perform overall monitoring of the electronic device.

[0136] In this embodiment, the processor 401 in the electronic device 400 may load the instructions corresponding to the processes of one or more computer programs into the memory 402 according to the steps in the above image noise reduction method, and the processor 401 runs the computer programs stored in the memory 402, so as to implement the steps in the above image noise reduction method, for example:

[0137] Obtain the image to be processed and the historical frame image corresponding to the image to be processed;

[0138] Segment the image to be processed into different noise reduction regions;

[0139] Perform noise reduction on different noise reduction regions in the image to be processed using different noise reduction algorithms to obtain the image to be processed after noise reduction;

[0140] Perform fusion noise reduction on the image to be processed after noise reduction using the historical frame image to obtain the target noise reduction image.

[0141] Please continue to refer to Figure 9 , Figure 9 which is the second structural schematic diagram of the electronic device 400 provided in the embodiment of the present application. Among them, the electronic device 400 further includes: a display screen 403, a control circuit 404, an input unit 405, a sensor 406, and a power supply 407. Among them, the processor 401 is electrically connected to the display screen 403, the control circuit 404, the input unit 405, the sensor 406, and the power supply 407 respectively.

[0142] The display screen 403 can be used to display information input by the user or information provided to the user and various graphical user interfaces of the electronic device, and these graphical user interfaces can be composed of images, texts, icons, videos, and any combination thereof.

[0143] The control circuit 404 is electrically connected to the display screen 403 and is used to control the display screen 403 to display information.

[0144] The input unit 405 can be used to receive input digital, character information or user characteristic information (such as fingerprints), and generate keyboard, mouse, joystick, optical or trackball signal inputs related to user settings and function controls. For example, the input unit 405 may include a touch sensing module.

[0145] The sensor 406 is used to collect information of the electronic device itself, or information of the user, or information of the external environment. For example, the sensor 406 may include multiple sensors such as a distance sensor, a magnetic field sensor, a light sensor, an acceleration sensor, a fingerprint sensor, a Hall sensor, a position sensor, a gyroscope, an inertial sensor, an attitude sensor, a barometer, and a heart rate sensor.

[0146] The power supply 407 is used to supply power to various components of the electronic device 400. In some embodiments, the power supply 407 may be logically connected to the processor 401 through a power management system, so as to implement functions such as management of charging, discharging, and power consumption management through the power management system.

[0147] Although Figure 8 and Figure 9 are not shown in the figure, the electronic device 400 may further include a camera, a Bluetooth module, etc., which will not be elaborated here.

[0148] In this embodiment, the processor 401 in the electronic device 400 may load the instructions corresponding to the processes of one or more computer programs into the memory 402 according to the steps in the above translation method, and the processor 401 runs the computer programs stored in the memory 402, so as to implement the steps in the above image denoising method, for example:

[0149] Obtain the image to be processed and the historical frame image corresponding to the image to be processed;

[0150] Segment the image to be processed into different denoising regions;

[0151] Perform denoising on different denoising regions in the image to be processed using different denoising algorithms to obtain the denoised image to be processed;

[0152] Perform fusion denoising on the denoised image to be processed using the historical frame image to obtain the target denoised image.

[0153] In some cases, after obtaining the target denoised image, the processor 401 performs the following steps:

[0154] Cache the target denoised image, and use it as the historical frame image corresponding to the subsequent image to be processed when denoising the subsequent image to be processed.

[0155] In some cases, when segmenting the image to be processed into different denoising regions, the processor 401 performs the following steps:

[0156] Obtain the depth information of the image to be processed;

[0157] Segment the image to be processed into different denoising regions according to the depth information.

[0158] In some cases, before segmenting the image to be processed into different denoising regions according to the depth information, the processor 401 further performs the following steps:

[0159] Obtain the focusing parameter when shooting the object to be processed;

[0160] Determine the focal plane when shooting the image to be processed according to the focusing parameter;

[0161] Determine the depth interval where the focal plane is located as the preset depth interval.

[0162] In some cases, the different noise reduction regions include at least two different noise reduction regions. When the image to be processed is segmented into different noise reduction regions according to the depth information, the processor 401 performs the following steps:

[0163] Divide the depth information of the image to be processed to divide it into at least two depth intervals including the preset depth interval;

[0164] Segment the image to be processed into at least two different noise reduction regions according to the at least two depth intervals.

[0165] In some cases, among the at least two different noise reduction regions, there is a key noise reduction region. When the image to be processed is segmented into at least two different noise reduction regions according to the at least two depth intervals, the processor 401 performs the following steps:

[0166] Segment out the key noise reduction region from the image to be processed according to the preset depth interval.

[0167] In some cases, when fusing and reducing noise for the processed image after noise reduction using historical frame images, the processor 401 performs the following steps:

[0168] For different noise reduction regions in the processed image after noise reduction, use different numbers of historical frame images for fusing and reducing noise.

[0169] In some cases, before using different numbers of historical frame images for fusing and reducing noise for different noise reduction regions in the processed image after noise reduction, the processor 401 also performs the following steps:

[0170] Obtain the depth intervals corresponding to different noise reduction regions in the processed image after noise reduction;

[0171] Determine the depth difference between the depth intervals corresponding to different noise reduction regions in the processed image after noise reduction and the depth interval where the focal plane is located;

[0172] For each noise reduction region in the processed image after noise reduction, determine the number of historical frame images used for fusing and reducing noise according to the corresponding depth difference, where the smaller the corresponding depth difference, the more historical frame images used for fusing and reducing noise.

[0173] In some cases, when fusing and reducing noise for the processed image after noise reduction using historical frame images, the processor 401 performs the following steps:

[0174] For each noise reduction region in the to-be-processed image after noise reduction, according to the set of feature points corresponding to different noise reduction regions in each image, match the corresponding noise reduction region in the historical frame image;

[0175] Synthesize the noise reduction regions that match each other in the to-be-processed image after noise reduction and the historical frame image, so as to perform fusion noise reduction on the to-be-processed image after noise reduction.

[0176] Among them, when matching the corresponding noise reduction regions, determine the similarity of the two noise reduction regions based on the identified feature points. If the similarity of the sets of feature points of the two noise reduction regions is greater than or equal to the preset threshold, it is determined that the two noise reduction regions match successfully, and there is a corresponding relationship between the two noise reduction regions; otherwise, it is determined that the two noise reduction regions do not match.

[0177] In some cases, before matching the corresponding noise reduction regions in the historical frame image according to the set of feature points corresponding to different noise reduction regions in each image, the processor 401 also performs the following steps:

[0178] Perform feature point recognition on the to-be-processed image and the historical frame image; or perform feature point recognition on the to-be-processed image after noise reduction and the historical frame image;

[0179] Based on the noise reduction regions where the identified feature points are located in each image, determine the feature points corresponding to different noise reduction regions in each image.

[0180] In some cases, when synthesizing the successfully matched noise reduction regions in the to-be-processed image after noise reduction and the historical frame image, the processor 401 performs the following steps:

[0181] If the match is successful, synthesize the matched noise reduction region in the historical frame image into the corresponding noise reduction region of the to-be-processed image after noise reduction.

[0182] As can be seen from the above, the embodiments of the present application provide an electronic device, and the processor in the electronic device performs the following steps: obtain the to-be-processed image and the historical frame image corresponding to the to-be-processed image, divide the to-be-processed image into different noise reduction regions, perform noise reduction on different noise reduction regions in the to-be-processed image using different noise reduction algorithms to obtain the to-be-processed image after noise reduction, and perform fusion noise reduction on the to-be-processed image after noise reduction using the historical frame image to obtain the target noise reduction image. The embodiments of the present application divide the to-be-processed image into regions, perform differential noise reduction on different regions using different noise reduction algorithms, and use the historical frame image to perform fusion noise reduction on the to-be-processed image after differential noise reduction to avoid image blurring. The method of differential noise reduction and multi-frame fusion noise reduction improves the effect of image noise reduction.

[0183] The embodiments of the present application also provide an integrated circuit chip. The integrated circuit chip can be used in devices such as smartphones, tablet computers, gaming devices, AR (Augmented Reality) devices, automobiles, vehicle peripheral obstacle detection devices, audio playback devices, video playback devices, notebooks, desktop computing devices, and wearable devices such as watches, glasses, helmets, electronic bracelets, and electronic necklaces.

[0184] The integrated circuit chip provided by the embodiments of the present application is independent of the central processing unit and adopts a hardware acceleration technology. It allocates the work with a very large amount of calculation to dedicated hardware for processing to reduce the workload of the central processing unit, so that the central processing unit does not need to translate each pixel in the image layer by layer through software. Implementing the image noise reduction method provided by the embodiments of the present application in a hardware acceleration manner can achieve high-speed image noise reduction processing.

[0185] Reference Figure 10 , Figure 10 FIG. is a schematic structural diagram of the integrated circuit chip 500 provided by the embodiments of the present application. Among them, the integrated circuit chip 500 includes a processor 501, a memory 502, and an image noise reduction device 300. The processor 501 is electrically connected to the memory 502.

[0186] The processor 501 is the control center of the integrated circuit chip 500, and uses various interfaces and lines to connect various parts of the entire integrated circuit chip, and realizes front-end depth information acquisition and back-end multi-frame synthesis in the image noise reduction method provided by the embodiments of the present application.

[0187] In this embodiment, the image noise reduction device 300 is responsible for implementing the steps in the above image noise reduction method, such as:

[0188] Obtain the image to be processed and the historical frame image corresponding to the image to be processed;

[0189] Segment the image to be processed into different noise reduction regions;

[0190] Perform noise reduction on different noise reduction regions in the image to be processed using different noise reduction algorithms to obtain the image to be processed after noise reduction;

[0191] Perform fusion noise reduction on the image to be processed after noise reduction using the historical frame image to obtain the target noise reduction image.

[0192] The embodiments of the present application also provide a computer-readable storage medium, in which a computer program is stored. When the computer program runs on a computer, the computer executes the image noise reduction method in any of the above embodiments.

[0193] For example, in some embodiments, when the computer program runs on a computer, the computer executes the following steps:

[0194] Obtain the image to be processed and the historical frame image corresponding to the image to be processed;

[0195] Segment the image to be processed into different noise reduction regions;

[0196] Perform noise reduction on different noise reduction regions in the image to be processed using different noise reduction algorithms to obtain the image to be processed after noise reduction;

[0197] Use the historical frame image to perform fusion noise reduction on the image to be processed after noise reduction to obtain the target noise reduction image.

[0198] It should be noted that those of ordinary skill in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing relevant hardware through a computer program, and the computer program can be stored in a computer-readable storage medium. The storage medium may include, but is not limited to: read-only memory (ROM, Read Only Memory), random access memory (RAM, Random Access Memory), magnetic disk or optical disk, etc.

[0199] In the above embodiments, the descriptions of the various embodiments have their own emphases. For the parts not detailed in a certain embodiment, reference can be made to the detailed description of the translation method above, and details will not be repeated here.

[0200] The above has introduced in detail the image noise reduction method, device, electronic device and storage medium provided by the embodiments of the present application. Specific examples are used in this article to elaborate on the principle and implementation manner of the present application. The description of the above embodiments is only used to help understand the method and its core idea of the present application; at the same time, for those skilled in the art, according to the idea of the present application, there will be changes in the specific implementation manner and application scope. In summary, the content of this specification should not be construed as a limitation to the present application.

Claims

1. An image denoising method, characterized in that, Including: Obtaining an image to be processed and a historical frame image corresponding to the image to be processed; Segmenting the image to be processed into different noise reduction regions; Performing noise reduction on different noise reduction regions in the image to be processed using different noise reduction algorithms to obtain a noise-reduced image to be processed; Performing fusion noise reduction on the noise-reduced image to be processed using the historical frame image to obtain a target noise-reduced image, wherein, for different noise reduction regions in the noise-reduced image to be processed, different numbers of historical frame images are used for fusion noise reduction; Before performing fusion noise reduction on different noise reduction regions in the noise-reduced image to be processed using different numbers of historical frame images, it further includes: obtaining depth intervals corresponding to different noise reduction regions in the noise-reduced image to be processed; determining depth differences between the depth intervals corresponding to different noise reduction regions in the noise-reduced image to be processed and the depth interval where the focal plane is located; for each noise reduction region in the noise-reduced image to be processed, determining the number of historical frame images used for fusion noise reduction according to the corresponding depth difference.

2. The image noise reduction method according to claim 1, wherein The segmenting the image to be processed into different noise reduction regions includes: Obtaining depth information of the image to be processed; Segmenting the image to be processed into different noise reduction regions according to the depth information.

3. The image noise reduction method according to claim 2, wherein The different noise reduction regions include at least two different noise reduction regions, and the segmenting the image to be processed into different noise reduction regions according to the depth information includes: Dividing the depth information of the image to be processed to divide it into at least two depth intervals including a preset depth interval; Segmenting the image to be processed into at least two different noise reduction regions according to the at least two depth intervals.

4. The image noise reduction method according to claim 3, wherein Among the at least two different noise reduction regions, there is a key noise reduction region, and the segmenting the image to be processed into at least two different noise reduction regions according to the at least two depth intervals includes: Segmenting out the key noise reduction region from the image to be processed according to the preset depth interval.

5. The image noise reduction method according to claim 4, wherein Before segmenting the image to be processed into different noise reduction regions according to the depth information, it further includes: Obtaining a focusing parameter when shooting the image to be processed; Determining a focal plane when shooting the image to be processed according to the focusing parameter; Determining the depth interval where the focal plane is located as the preset depth interval.

6. The image noise reduction method according to claim 1, wherein The smaller the corresponding depth difference is, the more historical frame images are used for fusion noise reduction.

7. The image noise reduction method according to any one of claims 1 to 6, characterized in that The performing fusion noise reduction on the noise-reduced image to be processed using the historical frame image includes: For each noise reduction region in the noise-reduced image to be processed, matching corresponding noise reduction regions in the historical frame image according to feature points corresponding to different noise reduction regions in each image; Synthesizing the noise-reduced image to be processed and the mutually matching noise reduction regions in the historical frame image to perform fusion noise reduction on the noise-reduced image to be processed.

8. The image noise reduction method according to claim 7, wherein Before matching corresponding noise reduction regions in the historical frame image according to feature points corresponding to different noise reduction regions in each image, it further includes: Performing feature point recognition on the image to be processed and the historical frame image; or performing feature point recognition on the noise-reduced image to be processed and the historical frame image; Determine the feature points corresponding to different noise reduction regions in each image based on the noise reduction regions where the feature points identified in each image are located.

9. The image noise reduction method according to claim 8, wherein The method further includes: When matching corresponding noise reduction regions, determine the similarity between two noise reduction regions based on the identified feature points; If the similarity between two noise reduction regions is greater than or equal to a preset threshold, determine that the two noise reduction regions are successfully matched, and there is a corresponding relationship between the two noise reduction regions; Otherwise, determine that the two noise reduction regions fail to match.

10. The image noise reduction method according to claim 9, characterized in that, The synthesis of the successfully matched noise reduction regions in the noise-reduced image to be processed and the historical frame image includes: If the match is successful, synthesize the noise reduction region matched in the historical frame image into the corresponding noise reduction region of the noise-reduced image to be processed.

11. The image noise reduction method according to claim 1, characterized in that, After obtaining the target noise reduction image, it further includes: Cache the target noise reduction image, and when denoising subsequent images to be processed, use it as the historical frame image corresponding to the subsequent images to be processed.

12. An image noise reduction device, characterized in that, It includes: An acquisition module for acquiring an image to be processed and the historical frame image corresponding to the image to be processed; A segmentation module for segmenting the image to be processed into different noise reduction regions; A noise reduction module for denoising different noise reduction regions in the image to be processed using different noise reduction algorithms to obtain a noise-reduced image to be processed; A fusion module for performing fusion denoising on the noise-reduced image to be processed using the historical frame image to obtain a target noise reduction image, wherein, for different noise reduction regions in the noise-reduced image to be processed, different numbers of historical frame images are used for fusion denoising; A second determination module for, before using different numbers of historical frame images for fusion denoising of different noise reduction regions in the noise-reduced image to be processed, obtaining the depth intervals corresponding to different noise reduction regions in the noise-reduced image to be processed; determining the depth differences between the depth intervals corresponding to different noise reduction regions in the noise-reduced image to be processed and the depth interval where the focal plane is located; for each noise reduction region in the noise-reduced image to be processed, determining the number of historical frame images used for fusion denoising according to the corresponding depth difference.

13. An electronic device, wherein, The electronic device includes a processor and a memory, and a computer program is stored in the memory. The processor executes the steps in the image denoising method according to any one of claims 1 to 11 by calling the computer program stored in the memory.

14. A computer-readable storage medium, characterized in that, A computer program is stored in the storage medium, and when the computer program runs on a computer, the computer is caused to execute the steps in the image denoising method according to any one of claims 1 to 11.

Images