Image processing method, image processing device and storage medium

Abstract

The invention relates to an image processing method, an image processing device and a storage medium. The image processing method comprises the following steps: in response to a received image processing instruction, obtaining a to-be-processed denoised image and a corresponding original image; the signal-to-noise ratio of each pixel in the original image and the signal-to-difference ratio of each pixel are determined, the signal-to-difference ratio is the ratio of the pixel value of the pixel to the standard deviation, and the standard deviation is the arithmetic square root of the pixel value variance of the pixel; according to the signal-to-noise ratio and the signal-to-difference ratio, a fusion weight used for fusing the original image and the denoised image is determined, and the denoised image is an image obtained by denoising the original image; and according to the fusion weight, fusing the original image and the denoised image to obtain a target image. According to the invention, the details of the low-noise part of the image after noise reduction are effectively recovered, the details and texture of the texture part of the image are increased, and the smearing feeling caused by image noise reduction is reduced.

Description

Technical Field

[0001] The present disclosure relates to the field of image processing, and in particular to an image processing method, an image processing device and a storage medium. Background Art

[0002] When performing denoising on an image, the denoising algorithm will misprocess the high-frequency texture because it cannot distinguish the difference between the high-frequency texture and the signal-to-noise ratio, resulting in a smearing effect on the image after denoising.

[0003] In the related technology, the noise suppression strength is determined by using the difference before and after noise reduction, which relies on the effect of the noise reduction algorithm. It is also impossible to distinguish whether the suppressed part is noise or texture, which easily leads to excessive suppression of high-frequency texture areas and a smearing effect. Summary of the invention

[0004] In order to overcome the problems existing in the related art, the present disclosure provides an image processing method, an image processing device and a storage medium.

[0005] According to a first aspect of an embodiment of the present disclosure, there is provided an image processing method, comprising: in response to receiving an image processing instruction, obtaining a denoised image to be processed and a corresponding original image; determining a signal-to-noise ratio of each pixel in the original image and a signal-to-error ratio of each pixel, wherein the signal-to-error ratio is a ratio between a pixel value and a standard deviation of the pixel, and the standard deviation is an arithmetic square root of a variance of the pixel value of the pixel; determining a fusion weight for fusing the original image and the denoised image according to the signal-to-noise ratio and the signal-to-error ratio, wherein the denoised image is an image obtained by denoising the original image; and fusing the original image and the denoised image according to the fusion weight to obtain a target image.

[0006] In one embodiment, the signal-to-noise ratio represents the noise intensity of the original image, the difference between the signal-to-difference ratio and the signal-to-noise ratio represents the intensity of texture information in the original image whose pixel value change frequency is higher than a preset frequency threshold, and determining a fusion weight for fusing the original image and the denoised image based on the signal-to-noise ratio and the signal-to-difference ratio includes: determining a first fusion weight based on the signal-to-noise ratio of each pixel in the original image and a first threshold, the first fusion weight being used to fuse the original image and the denoised image based on the noise intensity of the original image; determining the difference between the signal-to-difference ratio and the signal-to-noise ratio of each pixel in the original image, and determining a second fusion weight based on the difference and the second threshold, the second fusion weight being used to fuse the original image and the denoised image based on the texture information intensity of the original image; and determining a third fusion weight based on the first fusion weight and the second fusion weight, the third fusion weight being used to fuse the original image and the denoised image based on the noise intensity and texture information intensity of the original image.

[0007] In one embodiment, the first fusion weight includes a preset fixed first fusion weight and a dynamic first fusion weight, the fixed first fusion weight is used to fuse the original image and the denoised image when the noise intensity is lower than a noise intensity threshold, and the dynamic first fusion weight is used to fuse the original image and the denoised image when the noise intensity is higher than the noise intensity threshold, and the first fusion weight is determined according to the signal-to-noise ratio of each pixel in the original image and a first threshold, including: in response to the signal-to-noise ratio being less than or equal to the first threshold, the fixed first fusion weight is determined as the first fusion weight, and there is a corresponding relationship between the first threshold and the noise intensity threshold; in response to the signal-to-noise ratio being greater than the first threshold, the dynamic first fusion weight is determined based on the signal-to-noise ratio and a preset first fusion weight determination method, and the dynamic first fusion weight is determined as the first fusion weight.

[0008] In one embodiment, the second fusion weight includes a preset fixed second fusion weight and a dynamic second fusion weight, the fixed second fusion weight is used to fuse the original image and the denoised image when the texture information intensity is lower than the information intensity threshold, and the dynamic second fusion weight is used to fuse the original image and the denoised image when the texture information intensity is higher than the information intensity threshold, and the determining the second fusion weight according to the difference and the second threshold includes: in response to the difference being less than or equal to the second threshold, determining the fixed second fusion weight as the second fusion weight, the second threshold corresponding to the information intensity threshold; in response to the difference being greater than the second threshold, determining the dynamic second fusion weight based on the signal-to-noise ratio, the signal-to-difference ratio and a preset second fusion weight determination method, and determining the dynamic second fusion weight as the first fusion weight.

[0009] In one implementation, determining the third fusion weight according to the first fusion weight and the second fusion weight includes: determining the third fusion weight based on the first fusion weight, the second fusion weight and a preset third fusion weight determination method.

[0010] In one embodiment, there is a corresponding relationship between each pixel of the original image and the denoised image, and the original image and the denoised image are fused according to the fusion weight to obtain a target image, including: in response to selecting a mode of image fusion based on noise intensity, a first weight of each pixel in the original image and a second weight of the corresponding pixel in the denoised image are allocated according to a first fusion weight corresponding to each pixel, the sum of the product of the pixel value of each pixel in the original image and the first weight and the product of the pixel value of the corresponding pixel in the denoised image and the second weight is determined as a fused pixel, and the image formed by the fused pixels is determined as the target image.

[0011] In one embodiment, the method further includes: in response to selecting a mode for image fusion based on texture information intensity, allocating a third weight to each pixel in the original image and a fourth weight to the corresponding pixel in the denoised image according to a second fusion weight corresponding to each pixel, determining the sum of the product of the pixel value of each pixel in the original image and the third weight and the product of the pixel value of the corresponding pixel in the denoised image and the fourth weight as a fused pixel, and determining the image formed by the fused pixels as a target image.

[0012] In one embodiment, the method also includes: in response to selecting a mode for image fusion based on the noise intensity and texture information intensity, allocating a fifth weight to each pixel in the original image and a sixth weight to the corresponding pixel in the denoised image according to a third fusion weight corresponding to each pixel, determining the sum of the product of the pixel value of each pixel in the original image and the fifth weight and the product of the pixel value of the corresponding pixel in the denoised image and the sixth weight as a fused pixel, and determining the image formed by the fused pixels as a target image.

[0013] According to a second aspect of an embodiment of the present disclosure, there is provided an image processing apparatus, comprising: an acquisition unit, for acquiring, in response to receiving an image processing instruction, a denoised image to be processed and a corresponding original image; a determination unit, for determining a signal-to-noise ratio of each pixel in the original image and a signal-to-error ratio of each pixel, wherein the signal-to-error ratio is a ratio between a pixel value and a standard deviation of the pixel, and the standard deviation is an arithmetic square root of a variance of the pixel value of the pixel; a processing unit, for determining a fusion weight for fusing the original image and the denoised image according to the signal-to-noise ratio and the signal-to-error ratio, wherein the denoised image is an image obtained by denoising the original image; and a fusion unit, for fusing the original image and the denoised image according to the fusion weight to obtain a target image.

[0014] In one embodiment, the signal-to-noise ratio represents the noise intensity of the original image, and the difference between the signal-to-difference ratio and the signal-to-noise ratio represents the intensity of texture information in the original image whose pixel value change frequency is higher than a preset frequency threshold. The processing unit determines a fusion weight for fusing the original image and the denoised image according to the signal-to-noise ratio and the signal-to-difference ratio in the following manner: determining a first fusion weight according to the signal-to-noise ratio of each pixel in the original image and a first threshold, the first fusion weight being used to fuse the original image and the denoised image based on the noise intensity of the original image; determining the difference between the signal-to-difference ratio and the signal-to-noise ratio of each pixel in the original image, and determining a second fusion weight according to the difference and the second threshold, the second fusion weight being used to fuse the original image and the denoised image based on the texture information intensity of the original image; determining a third fusion weight according to the first fusion weight and the second fusion weight, the third fusion weight being used to fuse the original image and the denoised image based on the noise intensity and texture information intensity of the original image.

[0015] In one embodiment, the first fusion weight includes a preset fixed first fusion weight and a dynamic first fusion weight, the fixed first fusion weight is used to fuse the original image and the denoised image when the noise intensity is lower than the noise intensity threshold, and the dynamic first fusion weight is used to fuse the original image and the denoised image when the noise intensity is higher than the noise intensity threshold, and the processing unit determines the first fusion weight according to the signal-to-noise ratio of each pixel in the original image and a first threshold in the following manner: in response to the signal-to-noise ratio being less than or equal to the first threshold, the fixed first fusion weight is determined as the first fusion weight, and the first threshold corresponds to the noise intensity threshold; in response to the signal-to-noise ratio being greater than the first threshold, the dynamic first fusion weight is determined based on the signal-to-noise ratio and the preset first fusion weight determination device, and the dynamic first fusion weight is determined as the first fusion weight.

[0016] In one embodiment, the second fusion weight includes a preset fixed second fusion weight and a dynamic second fusion weight, the fixed second fusion weight is used to fuse the original image and the denoised image when the texture information intensity is lower than the information intensity threshold, and the dynamic second fusion weight is used to fuse the original image and the denoised image when the texture information intensity is higher than the information intensity threshold, and the processing unit determines the second fusion weight according to the difference and the second threshold in the following manner: in response to the difference being less than or equal to the second threshold, the fixed second fusion weight is determined as the second fusion weight, and the second threshold corresponds to the information intensity threshold; in response to the difference being greater than the second threshold, the dynamic second fusion weight is determined based on the signal-to-noise ratio, the signal-to-difference ratio and the preset second fusion weight determination device, and the dynamic second fusion weight is determined as the first fusion weight.

[0017] In one implementation, the processing unit determines the third fusion weight according to the first fusion weight and the second fusion weight in the following manner: determining the third fusion weight based on the first fusion weight, the second fusion weight and a preset third fusion weight determination device.

[0018] In one embodiment, there is a corresponding relationship between each pixel of the original image and the denoised image, and the fusion unit fuses the original image and the denoised image according to the fusion weight in the following manner to obtain a target image: in response to selecting a mode of image fusion based on noise intensity, a first weight of each pixel in the original image and a second weight of the corresponding pixel in the denoised image are allocated according to a first fusion weight corresponding to each pixel, the sum of the product of the pixel value of each pixel in the original image and the first weight and the product of the pixel value of the corresponding pixel in the denoised image and the second weight is determined as a fused pixel, and the image formed by the fused pixels is determined as the target image.

[0019] In one embodiment, the fusion unit also fuses the original image and the denoised image according to the fusion weight to obtain a target image in the following manner: in response to selecting a mode for image fusion based on texture information intensity, a third weight of each pixel in the original image and a fourth weight of the corresponding pixel in the denoised image are allocated according to the second fusion weight corresponding to each pixel, the sum of the product of the pixel value of each pixel in the original image and the third weight and the product of the pixel value of the corresponding pixel in the denoised image and the fourth weight is determined as a fused pixel, and the image formed by the fused pixels is determined as the target image.

[0020] In one embodiment, the fusion unit also fuses the original image and the denoised image according to the fusion weight to obtain a target image in the following manner: in response to selecting a mode for image fusion based on the noise intensity and the texture information intensity, a fifth weight of each pixel in the original image and a sixth weight of the corresponding pixel in the denoised image are allocated according to a third fusion weight corresponding to each pixel, the sum of the product of the pixel value of each pixel in the original image and the fifth weight and the product of the pixel value of the corresponding pixel in the denoised image and the sixth weight is determined as a fused pixel, and the image formed by the fused pixels is determined as the target image.

[0021] According to a third aspect of an embodiment of the present disclosure, there is provided an image processing device, comprising: a processor; and a memory for storing processor executable instructions; wherein the processor is configured to: execute the image processing method described in the first aspect or any one of the embodiments of the first aspect.

[0022] According to a fourth aspect of an embodiment of the present disclosure, a storage medium is provided, in which instructions are stored. When the instructions in the storage medium are executed by a processor, the processor is enabled to execute the image processing method described in the first aspect or any one of the embodiments of the first aspect.

[0023] The technical solution provided by the embodiments of the present disclosure may include the following beneficial effects: obtaining the original image to be processed and the denoised image corresponding to the original image, determining the signal-to-error ratio and signal-to-noise ratio of the original image, and determining the fusion weight for fusing the original image and the denoised image based on the signal-to-error ratio and the signal-to-noise ratio, and fusing the original image and the denoised image according to the fusion weight to obtain the target image. Through the present disclosure, the details of the low-noise part of the denoised image are effectively restored, the details and texture of the image texture are increased, and the smearing caused by image denoising is reduced.

[0024] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0026] Figure 1A and Figure 1B It is a schematic diagram of a processing effect of a noise reduction method according to an exemplary embodiment of the present disclosure.

[0027] Figure 2 The figure is a flowchart of an image processing method according to an exemplary embodiment.

[0028] Figure 3 A signal-to-noise ratio mapping diagram corresponding to an original image is shown according to an exemplary embodiment of the present disclosure.

[0029] Figure 4 The diagram is a schematic diagram showing a method of determining a standard deviation of a pixel based on pixel values ​​of pixels within a window according to an exemplary embodiment of the present disclosure.

[0030] Figure 5 A signal-difference ratio mapping diagram corresponding to an original image is shown according to an exemplary embodiment of the present disclosure.

[0031] Figure 6 The invention is a flowchart showing a method for determining a fusion weight based on a signal-to-noise ratio and a signal-to-error ratio according to an exemplary embodiment.

[0032] Figure 7 The invention is a flowchart showing a method for determining a first fusion weight according to an exemplary embodiment.

[0033] Figure 8 It is a schematic diagram showing a corresponding relationship between a signal-to-noise ratio and a first fusion weight according to an exemplary embodiment.

[0034] Fig. 9 The invention is a flowchart showing a method for determining a second fusion weight according to an exemplary embodiment.

[0035] Fig.10 The diagram is a schematic diagram showing a corresponding relationship between a difference between a signal-to-error ratio and a signal-to-noise ratio and a second fusion weight according to an exemplary embodiment.

[0036] Fig.11 The present invention is a flowchart of a method for fusing an original image and a denoised image according to an exemplary embodiment.

[0037] Fig.12 is a flowchart of a method for fusing an original image and a denoised image according to yet another exemplary embodiment.

[0038] Fig.13 is a flowchart of a method for fusing an original image and a denoised image according to yet another exemplary embodiment.

[0039] Fig.14 It is a block diagram showing an image processing method according to an exemplary embodiment of the present disclosure.

[0040] Fig.15 is a block diagram of an image processing apparatus according to an exemplary embodiment.

[0041] Fig.16 It is a block diagram of a device for taking photos according to an exemplary embodiment. DETAILED DESCRIPTION

[0042] Here, exemplary embodiments will be described in detail, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present disclosure.

[0043] The method provided in the embodiments of the present disclosure can be applied to image processing during photo taking, image processing during video recording, and image processing of stored photos and videos.

[0044] When the terminal camera takes a photo, after adjusting the shooting parameters and pressing the shutter, the camera sensor converts the collected light signal into an electrical signal to generate a raw image (RAW image). However, the raw image directly obtained from the camera sensor has a lot of noise. Directly outputting the raw image as the target image will result in a large amount of noise in the target image. Therefore, after the camera sensor generates the raw image, it is necessary to perform noise reduction processing on the raw image to eliminate the noise in the image, such as Figure 1A The original image and Figure 1B As shown in the denoised image, the noise points in the denoised image are less than those in the original image, and the image performance is better. Image denoising is a basic technology in digital image processing. From a technical point of view, image denoising can be achieved by multi-frame fusion denoising and single-frame denoising. Single-frame denoising includes dictionary-based denoising methods, optimization model-based denoising methods, filtering-based denoising methods, and artificial intelligence (AI)-based denoising methods. However, while the above denoising methods process the noise, they also misprocess the inherent texture features of the image, resulting in some loss of details in the image after denoising, and a smearing feeling in the image after denoising, resulting in a poor final image quality.

[0045] In the related art, the quality of the denoised image is improved by post-processing the denoised image. The specific processing flow is as follows:

[0046] (1) Performing noise reduction processing on the original image (RAW image) based on a preset noise reduction method, and obtaining the noise suppression strength by making a difference between the images before and after noise reduction;

[0047] (2) Perform low-pass filtering on the original image and calculate the ratio of high-frequency and low-frequency information;

[0048] (3) Proportionally synthesize the noise suppression strength and high- and low-frequency information to obtain a mask image;

[0049] (4) The noise of the original image is suppressed by the noise suppression strength, and the fusion ratio of the original image and the low-frequency image is determined by the high- and low-frequency information. The fusion of the original image and the low-frequency image is performed according to the mask image.

[0050] However, the purpose of the above-mentioned related technologies is mainly to beautify the image and weaken some factors that affect the aesthetic quality of the original denoised image. It cannot eliminate the smearing feeling in the denoised image, and its suppression of high-frequency noise will further aggravate the smearing feeling of the denoised image. In addition, the above-mentioned related technologies determine the noise suppression intensity based on the difference before and after denoising. Its final processing result depends on the effect of the denoising algorithm. It cannot distinguish whether the suppressed part is noise or the inherent texture feature of the image, which will cause the high-frequency texture area to be suppressed too heavily and appear smearing. Furthermore, the above-mentioned related technologies fuse the low-frequency image and the noise-suppressed image according to the high and low frequency information, further weakening the content of the denoised image, which is not conducive to the purpose of restoring image details.

[0051] In view of this, the present disclosure provides an image processing method, which obtains a signal-to-noise ratio that characterizes the noise intensity of an original image, and obtains a signal-to-error ratio that characterizes the texture information of the original image. After obtaining the original image and performing noise reduction on the original image to obtain a noise-reduced image, the original image and the noise-reduced image are fused based on the obtained signal-to-noise ratio and signal-to-error ratio, while ensuring the noise reduction effect of the noise-reduced image, the texture information of the noise-reduced image is enhanced, the image details are restored, and the performance of the final imaged photo is improved.

[0052] In the disclosed embodiment, in the denoising stage of the original image, a preset noise model and a corresponding noise variance formula are used to denoise the original image.

[0053] In an exemplary embodiment of the present disclosure, a Poisson-Gaussian noise model is used as a noise model for image denoising. The first part of the noise model is Poisson noise related to the original image signal, and the second part is Gaussian background noise that is unrelated to the image signal and includes various image noises such as read noise and quantization noise. The expression of the noise model is: o =n p +n b , where n o is the comprehensive noise, n p is Poisson noise, n b is the Gaussian background noise. The general formula of the noise variance corresponding to the Poisson-Gaussian noise model is: in, is the variance of the noise in the original image, Luma is the expected value of the pixel (the pixel value when there is no noise), k is the Poisson noise parameter, and b is the variance of the Gaussian background noise.

[0054] In the embodiment of the present disclosure, when receiving the shooting instruction and collecting the original image, various shooting parameters during shooting are saved, including shutter time (shutter) s, sensitivity (ISO) g, image channel (such as R, Gr, Gb and B) c. In addition, the noise parameters under different shooting parameters can be obtained by pre-calibration, and the corresponding relationship table between the noise parameters and the shooting parameters is obtained: b = N b (g,s,c), k=N k (g,s,c). Where k is the Poisson noise parameter, N b is the mapping relationship between Poisson noise parameters and camera parameters, b is the variance of Gaussian background noise, N k is the mapping relationship between the variance of Gaussian background noise and the shooting parameters.

[0055] In an exemplary implementation of the present disclosure, after obtaining a photo-taking instruction, the photo-taking parameters are set as follows: the sensitivity g is {50, 100, 200, 400, 800, 1600, 3200, 6400}, the shutter time s is {0.00025, 0.0005, 0.001, 0.002, 0.004, ..., 0.1}, the image channel c is {0, 1, 2, 3}, and the calibration of the noise parameters corresponding to different photo-taking parameters is completed when the terminal leaves the factory.

[0056] In an exemplary embodiment of the present disclosure, the noise parameters are calibrated in the following manner: the lens of the terminal camera is blocked, and a pure dark image is captured at different combinations of sensitivity g and shutter time, and the Gaussian noise floor parameter b of each image channel corresponding to each pixel in the pure dark image is calibrated, and the Gaussian noise floor parameter b is expressed in the form of variance. A color card is captured under a standard light source environment to obtain a color card image, and the brightness and variance of different color card blocks on the color card image are counted, and fitting is performed based on the brightness and variance to obtain a function fitting line representing the Poisson noise parameter k.

[0057] In an embodiment of the present disclosure, in response to obtaining an original image and completing denoising of the original image, based on the saved photographing parameters, the parameters of the original image and the pre-calibrated noise parameters that correspond to the photographing parameters, the parameters of the original image are calculated to obtain parameters representing the noise intensity of the original image and an image representing the intensity of texture information of the original image, respectively, and the denoised image and the denoised image are fused based on the acquired parameters. The following embodiment of the present disclosure describes a post-processing method for the denoised image after denoising is completed.

[0058] Figure 2 FIG. 1 is a flowchart of an image processing method according to an exemplary embodiment. Figure 2 As shown, the method includes steps S101 to S104.

[0059] In step S101, in response to receiving an image processing instruction, a denoised image to be processed and a corresponding original image are acquired.

[0060] In step S102, the signal-to-noise ratio of each pixel and the signal-to-difference ratio of each pixel in the original image are determined.

[0061] The signal-to-error ratio is the ratio of the pixel value to the standard deviation, and the standard deviation is the arithmetic square root of the variance of the pixel values.

[0062] In step S103, a fusion weight for fusing the original image and the denoised image is determined according to the signal-to-noise ratio and the signal-to-error ratio.

[0063] The denoised image is an image obtained by denoising the original image.

[0064] In step S104, the original image and the denoised image are fused according to the fusion weight to obtain a target image.

[0065] In the embodiment of the present disclosure, based on the photographing parameters (sensitivity g, image channel c, exposure time s) stored during shooting, combined with the correspondence between the pre-set photographing parameters and the noise parameters, that is, the variance b of the Gaussian noise floor is = N b (g, s, c), Poisson noise parameter k = N k (g, s, c), determine the noise parameters of the original image (variance b of the Gaussian noise floor, Poisson noise parameter k), after obtaining the noise parameters, determine the noise standard deviation of each pixel in the original image based on the noise model of the original image and the obtained noise parameters, and determine the signal-to-noise ratio of each pixel based on the noise standard deviation of each pixel.

[0066] In an exemplary embodiment of the present disclosure, the noise standard deviation of each pixel in the original image is determined in the following manner:

[0067]

[0068] Among them, the subscript c corresponds to the image channel of the target pixel (including R, Gr, Gb, B), σ c (p) is the noise standard deviation, k c is the Poisson noise parameter under the corresponding image channel, I(p,c) is the pixel value under the corresponding image channel, b c is the variance of the Gaussian noise floor under the corresponding image channel.

[0069] In an exemplary embodiment of the present disclosure, the signal-to-noise ratio of each pixel in the original image is determined in the following manner:

[0070]

[0071] Where S(p) is the signal-to-noise ratio of the target pixel, I c (p) is the pixel value of the target pixel, σ c (p) is the standard deviation of the target pixel, and min is the operation of taking the minimum value. The standard deviation of the target pixel is a fixed value, and the pixel value of the target pixel has different values ​​under different determination methods. It can be understood that a single pixel in the original image includes multiple image channels, each of which has a different brightness value, and the brightness value ranges from 0 to 255. The present disclosure takes the brightness value corresponding to the image channel with the smallest total brightness value of multiple images as the pixel value of the target pixel, and obtains the signal-to-noise ratio of the target pixel. In an example, the target pixel includes four image channels, each of which has a different brightness value (R: 150, Gr: 250, Gb: 50, B: 200), then the pixel value of the target pixel is taken as 50. It can be understood that the brightness value of the target pixel in the present disclosure can also perform a weighted average operation on the brightness values ​​of the multiple channels constituting the pixel, and the obtained mean is used as the pixel value of the target pixel.

[0072] In the embodiment of the present disclosure, it is necessary to determine the signal-to-noise ratio of each pixel in the original image and save it. In an exemplary embodiment, Figure 3 As shown in the signal-to-noise ratio mapping diagram that corresponds to the original image, the present disclosure constructs a signal-to-noise ratio mapping diagram (signal-to-noise ratio map) that corresponds to the original image based on the signal-to-noise ratio of each pixel. The signal-to-noise ratio mapping diagram has a one-to-one correspondence with the pixels of the original image, and includes the signal-to-noise ratio of each pixel in the original image.

[0073] In the embodiment of the present disclosure, the signal-to-error ratio of the original image is determined in the following manner: the brightness values ​​of multiple image channels of each pixel in the original image are averaged to obtain a single-channel brightness (Luma) image L with only brightness values. The standard deviation of the window of the desired image L is calculated, that is, for each pixel, a window of n1*n1 (for example, 5*5, 3*3) is divided with it as the center, and the standard deviation of the pixel values ​​in the window is calculated as the standard deviation of the pixel point to obtain a standard deviation image. In an exemplary embodiment, Figure 4 As shown in the schematic diagram of determining the standard deviation of pixels based on the pixel values ​​in the window, the present disclosure divides the target pixel into 3*3 windows to obtain the standard deviation of each target pixel, for example Figure 4 In the area shown, the pixel values ​​of the middle pixels are 8, 4, 4, and 2, respectively, and the obtained standard deviations are 2.26, 2.06, 2.06, and 1.83. The standard deviation image is obtained by calculating the window standard deviation for each pixel in the ideal image. After obtaining the standard deviation image, a method similar to the above method of calculating the window standard deviation is used to perform box filtering on the standard deviation image to obtain the standard deviation δ of the original image. v(p), that is, the box filter window size is set to n2*n2(n1, n2, in one example, n1=5, n2=7), the values ​​corresponding to the multiple channels included in the window are averaged, and the obtained average is used as the standard deviation δ v (p), after obtaining the standard deviation of each pixel in the original image, the pixel value L(p) of each pixel in the ideal image is compared with the corresponding standard deviation σ v The ratio of (p) is determined as the signal difference ratio D(p) of the corresponding pixels in the original image, that is,

[0074] It can be understood that in the above embodiment, the brightness values ​​of multiple image channels of each pixel in the original image are averaged to obtain a single-channel brightness (Luma) image L with only brightness values. For the single-channel brightness (Luma) image L, obtaining the standard deviation of each pixel based on the pixel value of the single-channel brightness (Luma) image L is a simplified algorithm to reduce the amount of calculation. In one example, the algorithm can also be directly processed on the original image to obtain the brightness of each channel of the original image pixel, and the brightness value of each channel is processed by obtaining the window standard deviation and box filtering to obtain the standard deviation corresponding to each channel of each pixel, and the largest standard deviation is determined as the standard deviation of the pixel.

[0075] In the embodiment of the present disclosure, it is necessary to determine the signal-to-noise ratio of each pixel in the original image and save it. In an exemplary embodiment, Figure 5 The present invention is based on the obtained signal ratio of each pixel, and constitutes a signal ratio map corresponding to the original image. The signal ratio map has a one-to-one correspondence with the pixels of the original image, and includes the signal ratio of each pixel in the original image.

[0076] In the embodiment of the present disclosure, based on the photographing instruction or the original image, after the denoising of the original image is completed, the signal-to-noise ratio and signal-to-sound ratio of the original image are determined, and the fusion weight for fusing the original image and the denoised image is determined based on the signal-to-noise ratio and the signal-to-noise ratio, and the original image and the denoised image are fused according to the fusion weight to obtain the target image. Through the present disclosure, the details of the low-noise part of the denoised image are effectively restored, the details and texture of the image texture are increased, and the smearing caused by the image denoising is reduced.

[0077] In the embodiments of the present disclosure, the signal-to-noise ratio of the original image represents the noise intensity of the original image, and the difference between the signal-to-difference ratio and the signal-to-noise ratio of the original image represents the intensity of texture information in the original image whose pixel value change frequency is higher than a preset frequency threshold. Based on the signal-to-noise ratio and the signal-to-difference ratio, a fusion weight for fusing the original image and the denoised image can be determined. The following embodiments of the present disclosure use the signal-to-noise ratio and the signal-to-difference ratio to determine the fusion weight in the following manner.

[0078] Figure 6 FIG. 1 is a flow chart showing a method for determining fusion weights based on signal-to-noise ratio and signal-to-error ratio according to an exemplary embodiment. Figure 6 As shown, the method includes step S201, step S202A, step S202B and step S203.

[0079] In step S201, the signal-to-noise ratio and signal-to-difference ratio of the original image are determined.

[0080] In step S202A, a first fusion weight is determined according to the signal-to-noise ratio of each pixel in the original image and a first threshold.

[0081] The first fusion weight is used to fuse the original image and the denoised image based on the noise intensity of the original image.

[0082] In the embodiment of the present disclosure, the first fusion weight may be defined as a signal-to-noise ratio fusion weight.

[0083] In step S202B, the difference between the signal-to-difference ratio and the signal-to-noise ratio of each pixel in the original image is determined, and a second fusion weight is determined according to the difference between the signal-to-difference ratio and the signal-to-noise ratio and a second threshold.

[0084] In the embodiment of the present disclosure, the second fusion weight may be defined as a high-frequency fusion weight.

[0085] Among them, the second fusion weight is used to fuse the original image and the denoised image based on the texture information intensity of the original image.

[0086] In step S203, a third fusion weight is determined according to the first fusion weight and the second fusion weight.

[0087] The third fusion weight is used to fuse the original image and the denoised image based on the noise intensity and texture information intensity of the original image.

[0088] In the embodiment of the present disclosure, the third fusion weight may be defined as a comprehensive fusion weight or a noise backfilling fusion weight.

[0089] In the embodiment of the present disclosure, the first threshold is a threshold determined based on the noise intensity, and the first fusion weight can be adjusted based on the change of the noise intensity. Similarly, the second threshold is a threshold determined based on the texture information intensity, and the second fusion weight can be adjusted based on the change of the texture information intensity. Then, the third fusion weight can be adjusted based on the change of the noise intensity and the change of the texture information intensity. In summary, the fusion weight used for fusing the original image and the denoised image in the present disclosure can be adaptively adjusted, so that the fused target image can restore the texture information contained in the original image while ensuring the noise reduction effect, thereby improving the imaging effect.

[0090] In the disclosed embodiment, the high-frequency information of the image is the image information of the area where the pixel value in the image changes quickly and is higher than the change threshold. For example, at the boundary between different areas in the image, the pixels of the image will change significantly, so the image information of the area corresponding to the boundary line and the contour line in the image is the high-frequency information of the image. Similarly, the pixels at the position where noise appears in the image will also change significantly, so the image information corresponding to the noise area in the original image is also high-frequency information. In the disclosed embodiment, the difference between the signal-to-error ratio and the signal-to-noise ratio of each pixel in the original image represents the intensity of the texture information in the original image. The larger the difference between the signal-to-error ratio and the signal-to-noise ratio, the higher the texture information intensity of the corresponding pixel and the lower the noise intensity. Therefore, the second fusion weight determined based on the difference between the signal-to-error ratio and the signal-to-noise ratio can be defined as the high-frequency fusion weight. Based on the difference between the signal-to-error ratio and the signal-to-noise ratio, the weight of the original image and the weight of the denoised image are allocated when the images are fused.

[0091] In an embodiment of the present disclosure, the first fusion weight includes a preset fixed first fusion weight and a dynamic first fusion weight. The fixed first fusion weight is used to fuse the original image and the denoised image when the noise intensity is lower than the noise intensity threshold, and the dynamic first fusion weight is used to fuse the original image and the denoised image when the noise intensity is higher than the noise intensity threshold. The following embodiment of the present disclosure describes the method for determining the first fusion weight.

[0092] Figure 7 FIG. 1 is a flow chart showing a method for determining a first fusion weight according to an exemplary embodiment. Figure 7 As shown, the method includes step S301, step S302A and step S302B.

[0093] In step S301, the signal-to-noise ratio of each pixel in the original image is determined.

[0094] In step S302A, in response to the signal-to-noise ratio being less than or equal to the first threshold, a fixed first fusion weight is determined as the first fusion weight.

[0095] There is a corresponding relationship between the first threshold and the noise intensity threshold.

[0096] In step S302B, in response to the signal-to-noise ratio being greater than the first threshold, a dynamic first fusion weight is determined based on the signal-to-noise ratio and a preset first fusion weight determination method, and the dynamic first fusion weight is determined as the first fusion weight.

[0097] In the disclosed embodiment, the signal-to-noise ratio of the original image is negatively correlated with the noise intensity. Based on the corresponding relationship between the signal-to-noise ratio and the noise intensity, a first threshold, namely the signal-to-noise ratio threshold, is determined. By adaptively adjusting the first fusion weight associated with the noise intensity, it is ensured that the fused target image has a good noise reduction effect and the noise intensity is minimized.

[0098] In an exemplary embodiment of the present disclosure, Figure 8 The schematic diagram of the corresponding relationship between the signal-to-noise ratio and the first fusion weight is shown in FIG. The present disclosure adopts the following method to determine the first fusion weight, namely the signal-to-noise ratio fusion weight.

[0099]

[0100] Among them, W s (P) is the first fusion weight, i.e., the signal-to-noise ratio fusion weight, the preset constant a is the preset minimum signal-to-noise ratio fusion weight, the preset constant b is the preset maximum signal-to-noise ratio fusion weight, S(p) is the signal-to-noise ratio, σ s is a preset constant for controlling the incremental speed. When the signal-to-noise ratio is less than a certain first threshold t1, a fixed fusion weight a is taken; when the signal-to-noise ratio is greater than the first threshold t1, the signal-to-noise ratio fusion weight is increased synchronously based on the increase in the signal-to-noise ratio, and the upper limit of the signal-to-noise ratio fusion weight is b. Among them, σ s , t1, a, b are all adjustable parameters, and in practical applications, empirical values ​​are taken based on the results. In one example, σ s ,t1,a,b are 30, 32, 0.05, and 0.4 respectively.

[0101] In the embodiment of the present disclosure, the second fusion weight includes a preset fixed second fusion weight and a dynamic second fusion weight, the fixed second fusion weight is used to fuse the original image and the denoised image when the texture information intensity is lower than the information intensity threshold, and the dynamic second fusion weight is used to fuse the original image and the denoised image when the texture information intensity is higher than the information intensity threshold. The following embodiment of the present disclosure describes the method for determining the second fusion weight.

[0102] Fig. 9 FIG. 1 is a flowchart of a method for determining a second fusion weight according to an exemplary embodiment. As shown in FIG. 1 , the method includes step S401, step S402A and step S402B.

[0103] In step S401, the difference between the signal-to-difference ratio and the signal-to-noise ratio of each pixel in the original image is determined.

[0104] In step S402A, in response to the difference between the signal-to-difference ratio and the signal-to-noise ratio being less than or equal to the second threshold, a fixed second fusion weight is determined as the second fusion weight.

[0105] There is a corresponding relationship between the second threshold and the texture information intensity threshold.

[0106] In step S402B, in response to the difference between the signal-to-error ratio and the signal-to-noise ratio being greater than the second threshold, a dynamic second fusion weight is determined based on the signal-to-noise ratio, the signal-to-error ratio and a preset second fusion weight determination method, and the dynamic second fusion weight is determined as the first fusion weight.

[0107] In the disclosed embodiment, the size of the difference between the signal-to-error ratio and the signal-to-noise ratio of the original image is negatively correlated with the texture information intensity. Based on the correspondence between the difference between the signal-to-error ratio and the signal-to-noise ratio and the texture information intensity, a second threshold, i.e., the difference threshold between the signal-to-error ratio and the signal-to-noise ratio, is determined. By adaptively adjusting the second fusion weight associated with the texture information intensity, it is ensured that the fused target image has a good feature restoration effect, so that the texture information in the original image is preserved.

[0108] In an exemplary embodiment of the present disclosure, Fig.10 The schematic diagram of the corresponding relationship between the difference between the signal-to-difference ratio and the signal-to-noise ratio and the second fusion weight is shown in FIG. The present disclosure adopts the following method to determine the second fusion weight, namely the high-frequency fusion weight.

[0109]

[0110] Among them, W h (P) is the second fusion weight, i.e., the high-frequency fusion weight. The preset constant c is the preset minimum high-frequency fusion weight. The preset constant d is the preset maximum high-frequency fusion weight. δ(p) is the high-frequency information graph, corresponding to the difference between the signal-to-error ratio and the signal-to-noise ratio. σ h is a preset constant for controlling the incremental speed. When the difference between the signal error ratio and the signal noise ratio is less than the second threshold t2, a fixed fusion weight c is taken; when the difference between the signal error ratio and the signal noise ratio is greater than the second threshold t2, the high-frequency fusion weight is increased synchronously based on the increase of the signal noise ratio, and the upper limit of the high-frequency fusion weight is d. Among them, σ h ,t2,c,d are all adjustable parameters, and in practical applications, empirical values ​​are taken based on the effects. In one example, σ h ,t2,c,d are 20, 8, 0.05, and 0.3 respectively.

[0111] In the embodiment of the present disclosure, the third fusion weight is a comprehensive fusion weight, which is a weight determined based on the noise intensity and texture information intensity of the original image. The third fusion weight needs to be determined based on the first fusion weight associated with the noise intensity and based on the second fusion weight associated with the texture information intensity. The following embodiment of the present disclosure describes the method for determining the third fusion weight.

[0112] The present disclosure is an implementation of an embodiment, in which a third fusion weight is determined according to a first fusion weight and a second fusion weight, including: determining the third fusion weight based on the first fusion weight, the second fusion weight and a preset third fusion weight determination method.

[0113] In the disclosed embodiment, by adaptively adjusting the second fusion weight associated with the texture information intensity and the first fusion weight associated with the noise intensity, the denoising effect of the fused image is ensured while the texture information of the original image is effectively restored.

[0114] In an exemplary embodiment of the present disclosure, the third fusion weight is determined in the following manner:

[0115] W(P)=1-((1-W s )(1-W h ))

[0116] Among them, W(P)1 is the third fusion weight, that is, the comprehensive fusion weight, W s is the first fusion weight, namely the signal-to-noise ratio fusion weight, W h is the second fusion weight, i.e., the high-frequency fusion weight.

[0117] In the embodiment of the present disclosure, the first fusion weight, the second fusion weight and the third fusion weight are used to assign weights to the original image and the denoised image respectively during the image fusion process, and the original image and the denoised image are fused based on their respective assigned images to obtain a target image. The following embodiment of the present disclosure describes a method for fusing the original image and the denoised image based on the first fusion weight.

[0118] Fig.11 FIG. 1 is a flow chart showing a method for fusing an original image and a denoised image according to an exemplary embodiment. Fig.11 As shown, the method includes step S501 to step S502.

[0119] In step S501, a first fusion weight is obtained.

[0120] In step S502, in response to selecting a mode for image fusion based on noise intensity, a first weight of each pixel in the original image and a second weight of the corresponding pixel in the denoised image are allocated according to a first fusion weight corresponding to each pixel, and the sum of the product of the pixel value of each pixel in the original image and the first weight and the product of the pixel value of the corresponding pixel in the denoised image and the second weight is determined as the fused pixel, and the image formed by the fused pixels is determined as the target image.

[0121] In the disclosed embodiment, different fusion weights are selected based on different fusion modes to fuse the original images, weights are assigned to each pixel of the original image based on the fusion weights, and weights are assigned to each pixel of the denoised image, and based on the assigned weights and the pixel values ​​of the corresponding original image pixels and the pixel values ​​of the denoised image pixels, the pixel values ​​of each pair of corresponding original image pixels and image pixels are fused, and the image with pixel-by-pixel fusion is determined as the target image.

[0122] In an exemplary embodiment of the present disclosure, the weights for image fusion are selected in the following manner:

[0123]

[0124] Among them, W(P) is the target fusion weight (or detail backfill fusion weight), W s is the first fusion weight, namely the signal-to-noise ratio fusion weight, W h The second fusion weight is the high-frequency fusion weight. Strategy 1, strategy 2, and strategy 3 correspond to different image fusion modes respectively. Strategy 1 corresponds to the image fusion mode based on noise intensity and texture information intensity, strategy 2 corresponds to the image fusion mode based on noise intensity, and strategy 3 corresponds to the image fusion mode based on texture information intensity.

[0125] In the disclosed embodiment, image fusion (detail backfilling) is performed in the following manner to obtain a fused image (detail backfilled image).

[0126] I d (p)=(1-W(p))I(p)+W(p)I0(p) (1-2)

[0127] Among them, I d (p) is the fused image (detail backfill image), i.e., the target image; W(p) is the weight of the original image during the fusion process (corresponding to the first weight mentioned above), i.e., the target fusion weight (or detail backfill fusion weight); (1-W(p)) is the weight of the denoised image during the fusion process; I0(p) is the original image; and I(p) is the denoised image.

[0128] In the present disclosure, Figure 8 As shown, the first fusion weight (corresponding to W(P)=W in the above formula 1-1) s The situation of () is positively correlated with the signal-to-noise ratio of the original image, that is, positively correlated with the noise intensity of the original image. As the signal-to-noise ratio increases, the noise intensity representing the original image becomes lower, and the image information intensity inherent in the original image becomes higher. Therefore, when the signal-to-noise ratio of the original image increases, more weight is assigned to the pixels of the original image during image fusion, thereby restoring the image information.

[0129] The following embodiments of the present disclosure illustrate a method for fusing an original image and a denoised image based on a second fusion weight.

[0130] Fig.12 A flowchart of a method for fusing an original image and a denoised image is shown according to another exemplary embodiment. Fig.12 As shown, the method includes step S601 to step S602.

[0131] In step S601, a second fusion weight is obtained.

[0132] In step S602, in response to selecting a mode for image fusion based on texture information intensity, a third weight of each pixel in the original image and a fourth weight of the corresponding pixel in the denoised image are allocated according to the second fusion weight corresponding to each pixel, and the sum of the product of the pixel value of each pixel in the original image and the third weight and the product of the pixel value of the corresponding pixel in the denoised image and the fourth weight is determined as the fused pixel, and the image formed by the fused pixels is determined as the target image.

[0133] In the present disclosure, Fig.10 As shown, the second fusion weight (corresponding to W(P)=W in the above formula 1-1) h The difference between the signal-to-noise ratio and the signal-to-error ratio of the original image is positively correlated, that is, it is positively correlated with the texture strength of the original image. As the difference between the signal-to-noise ratio and the signal-to-error ratio increases, the higher the texture strength representing the original image, the lower the noise intensity of the original image. Therefore, when the difference between the signal-to-noise ratio and the signal-to-error ratio of the original image increases, more weight is assigned to the pixels of the original image during image fusion, thereby restoring the image information, retaining the inherent high-frequency information of the original image, and reducing the smearing feeling of the target image.

[0134] The following embodiments of the present disclosure illustrate a method for fusing an original image and a denoised image based on a third fusion weight.

[0135] Fig.13 A flowchart of a method for fusing an original image and a denoised image is shown according to another exemplary embodiment. Fig.13 As shown, the method includes step S701 to step S702.

[0136] In step S701 , a first fusion weight and a second fusion weight are obtained, and a third fusion weight is obtained based on the first fusion weight and the second fusion weight.

[0137] In step S702, in response to selecting a mode for image fusion based on noise intensity and texture information intensity, the fifth weight of each pixel in the original image and the sixth weight of the corresponding pixel in the denoised image are allocated according to the third fusion weight corresponding to each pixel, and the sum of the product of the pixel value of each pixel in the original image and the fifth weight and the product of the pixel value of the corresponding pixel in the denoised image and the sixth weight is determined as the fused pixel, and the image formed by the fused pixels is determined as the target image.

[0138] In the disclosed embodiment, by adaptively adjusting the second fusion weight associated with the texture information intensity and the first fusion weight associated with the noise intensity, the denoising effect of the fused image is ensured while the texture information of the original image is effectively restored.

[0139] An exemplary embodiment of the present disclosure shows a method for performing image processing during a photographing process, fusing an original image and a noise-reduced image, and obtaining a target image, such as Fig.14 As shown in the block diagram of an image processing method, the original image and the denoised image are fused in the following manner to obtain the target image: the camera noise parameter calibration is completed at the factory, and the corresponding relationship between the shooting parameters and the noise parameters is preset; the image is taken to obtain the exposure parameters and complete the denoising, and the noise parameters of the image are loaded using the exposure parameters to calculate the signal-to-noise ratio mapping map, the image variance is calculated on the original image to obtain the signal-to-error ratio mapping map, the detail back-addition weight is calculated according to the signal-to-noise ratio and the signal-to-error ratio, and the original image and the denoised image are fused. Through the present disclosure, the signal-to-noise ratio is used to backfill the image details, and the details of the low-noise part of the denoised image are effectively restored. The signal-to-noise ratio and the signal-to-error ratio distinguish high-frequency textures and noise, effectively increase the details and texture of the image texture, and reduce the smearing feeling caused by image denoising.

[0140] It is understandable that when storing a photographed image, the present disclosure may also store an original image (Raw image) corresponding to the photographed image. When it is necessary to improve the photographed image effect, the photographed image and the corresponding original image are fused by the image processing method of the present disclosure to obtain a target image. It is further understandable that the image processing method proposed in the present disclosure may also process a video frame by frame.

[0141] Based on the same concept, the embodiment of the present disclosure further provides an image processing device 100 .

[0142] It is understandable that, in order to realize the above functions, the image processing device 100 provided in the embodiment of the present disclosure includes hardware structures and / or software modules corresponding to the execution of each function. In combination with the units and algorithm steps of each example disclosed in the embodiment of the present disclosure, the embodiment of the present disclosure can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art may use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the technical solution of the embodiment of the present disclosure.

[0143] Fig.15 1 is a block diagram of an image processing apparatus 100 according to an exemplary embodiment. Fig.15 The device includes an acquisition unit 101, a determination unit 102, a processing unit 103 and a fusion unit 104.

[0144] The acquisition unit 101 is used for acquiring the denoised image to be processed and the corresponding original image in response to receiving the image processing instruction.

[0145] The determination unit 102 is used to determine the signal-to-noise ratio of each pixel in the original image and the signal-to-difference ratio of each pixel.

[0146] The signal-to-error ratio is the ratio of the pixel value to the standard deviation, and the standard deviation is the arithmetic square root of the variance of the pixel values.

[0147] The processing unit 103 is used to determine a fusion weight for fusing the original image and the denoised image according to the signal-to-noise ratio and the signal-to-error ratio.

[0148] The denoised image is an image obtained by denoising the original image.

[0149] The fusion unit 104 is used to fuse the original image and the denoised image according to a fusion weight to obtain a target image.

[0150] In one embodiment, the signal-to-noise ratio represents the noise intensity of the original image, and the difference between the signal-to-difference ratio and the signal-to-noise ratio represents the intensity of the texture information in the original image whose pixel value change frequency is higher than a preset frequency threshold. The processing unit 103 determines the fusion weight for fusing the original image and the denoised image according to the signal-to-noise ratio and the signal-to-difference ratio in the following manner: determining a first fusion weight according to the signal-to-noise ratio of each pixel in the original image and a first threshold, the first fusion weight being used to fuse the original image and the denoised image based on the noise intensity of the original image. determining the difference between the signal-to-difference ratio and the signal-to-noise ratio of each pixel in the original image, and determining a second fusion weight according to the difference and the second threshold, the second fusion weight being used to fuse the original image and the denoised image based on the texture information intensity of the original image. determining a third fusion weight according to the first fusion weight and the second fusion weight, the third fusion weight being used to fuse the original image and the denoised image based on the noise intensity and the texture information intensity of the original image.

[0151] In one embodiment, the first fusion weight includes a preset fixed first fusion weight and a dynamic first fusion weight, the fixed first fusion weight is used to fuse the original image and the denoised image when the noise intensity is lower than the noise intensity threshold, and the dynamic first fusion weight is used to fuse the original image and the denoised image when the noise intensity is higher than the noise intensity threshold, and the processing unit 103 determines the first fusion weight according to the signal-to-noise ratio of each pixel in the original image and the first threshold in the following manner: in response to the signal-to-noise ratio being less than or equal to the first threshold, the fixed first fusion weight is determined as the first fusion weight, and the first threshold corresponds to the noise intensity threshold. In response to the signal-to-noise ratio being greater than the first threshold, the dynamic first fusion weight is determined based on the signal-to-noise ratio and the preset first fusion weight determination device, and the dynamic first fusion weight is determined as the first fusion weight.

[0152] In one embodiment, the second fusion weight includes a preset fixed second fusion weight and a dynamic second fusion weight, the fixed second fusion weight is used to fuse the original image and the denoised image when the texture information intensity is lower than the information intensity threshold, and the dynamic second fusion weight is used to fuse the original image and the denoised image when the texture information intensity is higher than the information intensity threshold, and the processing unit 103 determines the second fusion weight according to the difference and the second threshold in the following manner: in response to the difference being less than or equal to the second threshold, the fixed second fusion weight is determined as the second fusion weight, and the second threshold corresponds to the information intensity threshold. In response to the difference being greater than the second threshold, the dynamic second fusion weight is determined based on the signal-to-noise ratio, the signal-to-difference ratio and the preset second fusion weight determination device, and the dynamic second fusion weight is determined as the first fusion weight.

[0153] In one implementation, the processing unit 103 determines the third fusion weight according to the first fusion weight and the second fusion weight in the following manner: determining the third fusion weight based on the first fusion weight, the second fusion weight and a preset third fusion weight determination device.

[0154] In one embodiment, there is a corresponding relationship between each pixel of the original image and the denoised image, and the fusion unit 104 fuses the original image and the denoised image according to the fusion weight in the following manner to obtain a target image: in response to selecting a mode of image fusion based on noise intensity, a first weight of each pixel in the original image and a second weight of the corresponding pixel in the denoised image are allocated according to a first fusion weight corresponding to each pixel, the sum of the product of the pixel value of each pixel in the original image and the first weight and the product of the pixel value of the corresponding pixel in the denoised image and the second weight is determined as a fused pixel, and an image formed by the fused pixels is determined as the target image.

[0155] In one embodiment, the fusion unit 104 also fuses the original image and the denoised image according to the fusion weight to obtain the target image in the following manner: in response to selecting a mode for image fusion based on texture information intensity, a third weight of each pixel in the original image and a fourth weight of the corresponding pixel in the denoised image are allocated according to the second fusion weight corresponding to each pixel, the sum of the product of the pixel value of each pixel in the original image and the third weight and the product of the pixel value of the corresponding pixel in the denoised image and the fourth weight is determined as the fused pixel, and the image formed by the fused pixels is determined as the target image.

[0156] In one embodiment, the fusion unit 104 also fuses the original image and the denoised image according to the fusion weight to obtain the target image in the following manner: in response to selecting a mode for image fusion based on noise intensity and texture information intensity, a fifth weight of each pixel in the original image and a sixth weight of the corresponding pixel in the denoised image are allocated according to the third fusion weight corresponding to each pixel, the sum of the product of the pixel value of each pixel in the original image and the fifth weight and the product of the pixel value of the corresponding pixel in the denoised image and the sixth weight is determined as the fused pixel, and the image formed by the fused pixels is determined as the target image.

[0157] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0158] Fig.16 2 is a block diagram of an apparatus 200 for image processing according to an exemplary embodiment. The apparatus 200 may be provided as a terminal. For example, the apparatus 200 may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, etc.

[0159] Reference Fig.16 , the device 200 may include one or more of the following components: a processing component 202 , a memory 204 , a power component 206 , a multimedia component 208 , an audio component 210 , an input / output (I / O) interface 212 , a sensor component 214 , and a communication component 216 .

[0160] The processing component 202 generally controls the overall operation of the device 200, such as operations associated with display, phone calls, data communications, camera operations, and recording operations. The processing component 202 may include one or more processors 220 to execute instructions to perform all or part of the steps of the above-described method. In addition, the processing component 202 may include one or more modules to facilitate interaction between the processing component 202 and other components. For example, the processing component 202 may include a multimedia module to facilitate interaction between the multimedia component 208 and the processing component 202.

[0161] The memory 204 is configured to store various types of data to support operations on the device 200. Examples of such data include instructions for any application or method operating on the device 200, contact data, phone book data, messages, pictures, videos, etc. The memory 204 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0162] The power component 206 provides power to the various components of the device 200. The power component 206 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power for the device 200.

[0163] The multimedia component 208 includes a screen that provides an output interface between the device 200 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundaries of the touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 208 includes a front camera and / or a rear camera. When the device 200 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera may receive external multimedia data. Each front camera and rear camera may be a fixed optical lens system or have a focal length and optical zoom capability.

[0164] The audio component 210 is configured to output and / or input audio signals. For example, the audio component 210 includes a microphone (MIC), and when the device 200 is in an operation mode, such as a call mode, a recording mode, and a speech recognition mode, the microphone is configured to receive an external audio signal. The received audio signal can be further stored in the memory 204 or sent via the communication component 216. In some embodiments, the audio component 210 also includes a speaker for outputting audio signals.

[0165] I / O interface 212 provides an interface between processing component 202 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include but are not limited to: a home button, a volume button, a start button, and a lock button.

[0166] The sensor assembly 214 includes one or more sensors for providing various aspects of the status assessment of the device 200. For example, the sensor assembly 214 can detect the open / closed state of the device 200, the relative positioning of components, such as the display and keypad of the device 200, the sensor assembly 214 can also detect the position change of the device 200 or a component of the device 200, the presence or absence of user contact with the device 200, the orientation or acceleration / deceleration of the device 200 and the temperature change of the device 200. The sensor assembly 214 can include a proximity sensor configured to detect the presence of a nearby object without any physical contact. The sensor assembly 214 can also include an optical sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 214 can also include an accelerometer, a gyroscope sensor, a magnetic sensor, a pressure sensor or a temperature sensor.

[0167] The communication component 216 is configured to facilitate wired or wireless communication between the device 200 and other devices. The device 200 can access a wireless network based on a communication standard, such as WiFi, 2G or 3G, or a combination thereof. In an exemplary embodiment, the communication component 216 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 216 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0168] In an exemplary embodiment, the apparatus 200 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors or other electronic components to perform the above method.

[0169] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 204 including instructions, and the instructions can be executed by the processor 220 of the device 200 to perform the above method. For example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0170] It is to be understood that in the present disclosure, "plurality" refers to two or more than two, and other quantifiers are similar. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the associated objects before and after are in an "or" relationship. The singular forms "a", "the" and "the" are also intended to include plural forms, unless the context clearly indicates other meanings.

[0171] It is further understood that the terms "first", "second", etc. are used to describe various information, but such information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other, and do not indicate a specific order or degree of importance. In fact, the expressions "first", "second", etc. can be used interchangeably. For example, without departing from the scope of the present disclosure, the first information can also be referred to as the second information, and similarly, the second information can also be referred to as the first information.

[0172] It will be further understood that the terms “center”, “longitudinal”, “lateral”, “front”, “back”, “up”, “down”, “left”, “right”, “vertical”, “horizontal”, “top”, “bottom”, “inside”, “outside”, etc., indicating orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings, are only for the convenience of describing the present embodiment and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation.

[0173] It can be further understood that, unless otherwise specified, “connection” includes a direct connection without other components between the two, and also includes an indirect connection with other components between the two.

[0174] It is further understood that, although the operations are described in a specific order in the drawings in the embodiments of the present disclosure, it should not be understood as requiring the operations to be performed in the specific order shown or in a serial order, or requiring the execution of all the operations shown to obtain the desired results. In certain environments, multitasking and parallel processing may be advantageous.

[0175] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. The present disclosure is intended to cover any variations, uses or adaptations of the present solution, which follow the general principles of the present disclosure and include common knowledge or customary technical means in the art that are not disclosed in the present disclosure.

[0176] It should be understood that the present disclosure is not limited to the precise structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the scope of the appended claims.

Claims

1. An image processing method, characterized in that: include: In response to receiving the image processing instruction, obtaining a noise-reduced image to be processed and a corresponding original image; Determine a signal-to-noise ratio and a signal-to-error ratio of each pixel in the original image, wherein the signal-to-error ratio is a ratio between a pixel value and a standard deviation of the pixel, and the standard deviation is an arithmetic square root of a variance of the pixel values ​​of the pixel; Determining a fusion weight for fusing the original image and a denoised image according to the signal-to-noise ratio and the signal-to-error ratio, wherein the denoised image is an image obtained by denoising the original image; The original image and the denoised image are fused according to the fusion weight to obtain a target image.

2. The method according to claim 1, characterized in that The signal-to-noise ratio represents the noise intensity of the original image, and the difference between the signal-to-difference ratio and the signal-to-noise ratio represents the intensity of texture information in the original image whose pixel value change frequency is higher than a preset frequency threshold. The step of determining a fusion weight for fusing the original image and the denoised image according to the signal-to-noise ratio and the signal-to-difference ratio comprises: Determining a first fusion weight according to a signal-to-noise ratio of each pixel in the original image and a first threshold, wherein the first fusion weight is used to fuse the original image and the denoised image based on the noise intensity of the original image; Determine the difference between the signal-to-difference ratio and the signal-to-noise ratio of each pixel in the original image, and determine a second fusion weight according to the difference and a second threshold, wherein the second fusion weight is used to fuse the original image and the denoised image based on the texture information intensity of the original image; A third fusion weight is determined according to the first fusion weight and the second fusion weight, and the third fusion weight is used to fuse the original image and the denoised image based on the noise intensity and texture information intensity of the original image.

3. The method according to claim 2, characterized in that The first fusion weight includes a preset fixed first fusion weight and a dynamic first fusion weight, wherein the fixed first fusion weight is used to fuse the original image and the denoised image when the noise intensity is lower than the noise intensity threshold, and the dynamic first fusion weight is used to fuse the original image and the denoised image when the noise intensity is higher than the noise intensity threshold. The determining a first fusion weight according to the signal-to-noise ratio of each pixel in the original image and a first threshold value includes: In response to the signal-to-noise ratio being less than or equal to a first threshold, determining the fixed first fusion weight as the first fusion weight, the first threshold corresponding to a noise intensity threshold; In response to the signal-to-noise ratio being greater than a first threshold, the dynamic first fusion weight is determined based on the signal-to-noise ratio and a preset first fusion weight determination method, and the dynamic first fusion weight is determined as the first fusion weight.

4. The method according to claim 2, characterized in that: The second fusion weight includes a preset fixed second fusion weight and a dynamic second fusion weight, wherein the fixed second fusion weight is used to fuse the original image and the denoised image when the texture information intensity is lower than the information intensity threshold, and the dynamic second fusion weight is used to fuse the original image and the denoised image when the texture information intensity is higher than the information intensity threshold. The determining the second fusion weight according to the difference and the second threshold comprises: In response to the difference being less than or equal to a second threshold, determining the fixed second fusion weight as the second fusion weight, the second threshold corresponding to the information strength threshold; In response to the difference being greater than a second threshold, the dynamic second fusion weight is determined based on the signal-to-noise ratio, the signal-to-difference ratio and a preset second fusion weight determination method, and the dynamic second fusion weight is determined as the first fusion weight.

5. The method according to claim 2, characterized in that: The determining a third fusion weight according to the first fusion weight and the second fusion weight includes: The third fusion weight is determined based on the first fusion weight, the second fusion weight and a preset third fusion weight determination method.

6. The method according to claim 1, characterized in that There is a corresponding relationship between each pixel of the original image and the denoised image, The fusing the original image and the denoised image according to the fusion weight to obtain a target image includes: In response to selecting a mode of image fusion based on noise intensity, a first weight of each pixel in the original image and a second weight of the corresponding pixel in the denoised image are allocated according to a first fusion weight corresponding to each pixel, a sum of the product of the pixel value of each pixel in the original image and the first weight and the product of the pixel value of the corresponding pixel in the denoised image and the second weight is determined as a fused pixel, and an image formed by the fused pixels is determined as a target image.

7. The method according to claim 6, characterized in that The method further comprises: In response to selecting a mode for image fusion based on texture information intensity, a third weight of each pixel in the original image and a fourth weight of the corresponding pixel in the denoised image are allocated according to a second fusion weight corresponding to each pixel, a sum of the product of the pixel value of each pixel in the original image and the third weight and the product of the pixel value of the corresponding pixel in the denoised image and the fourth weight is determined as a fused pixel, and an image formed by the fused pixels is determined as a target image.

8. The method according to claim 6, characterized in that The method further comprises: In response to selecting a mode for image fusion based on the noise intensity and the texture information intensity, a fifth weight of each pixel in the original image and a sixth weight of the corresponding pixel in the denoised image are allocated according to the third fusion weight corresponding to each pixel, and the sum of the product of the pixel value of each pixel in the original image and the fifth weight and the product of the pixel value of the corresponding pixel in the denoised image and the sixth weight is determined as a fused pixel, and an image formed by the fused pixels is determined as a target image.

9. An image processing device, characterized in that: include: An acquisition unit, configured to acquire a denoised image to be processed and a corresponding original image in response to receiving an image processing instruction; A determination unit, configured to determine a signal-to-noise ratio of each pixel in the original image and a signal-to-error ratio of each pixel, wherein the signal-to-error ratio is a ratio between a pixel value and a standard deviation of the pixel, and the standard deviation is an arithmetic square root of a variance of the pixel value of the pixel; a processing unit, configured to determine a fusion weight for fusing the original image and a denoised image according to the signal-to-noise ratio and the signal-to-difference ratio, wherein the denoised image is an image obtained by denoising the original image; A fusion unit is used to fuse the original image and the denoised image according to the fusion weight to obtain a target image.

10. An image processing device, characterized in that: include: processor: a memory for storing processor-executable instructions; Wherein, the processor is configured to: execute the image processing method according to any one of claims 1 to 8.

11. A storage medium, characterized in that: The storage medium stores instructions, and when the instructions in the storage medium are executed by a processor, the processor is enabled to execute the image processing method according to any one of claims 1 to 8.

Images