Image noise processing methods, apparatus, electronic devices and storage media

Abstract

This disclosure provides an image noise processing method, apparatus, electronic device, and storage medium. The method includes: acquiring at least two first sampled images of the same scene; determining pixel values ​​of pixels in the at least two first sampled images; wherein the pixel values ​​include random values ​​indicating the influence of random noise and non-uniform values ​​indicating the influence of non-uniform noise; determining residual noise information based on the difference between the pixel values ​​of corresponding pixels in two second sampled images; wherein the first sampled images include the second sampled images; and acquiring noise parameters for noise reduction based on the residual noise information.

Description

Technical Field

[0001] This disclosure relates to the field of computer technology, and in particular to an image noise processing method, apparatus, electronic device, and storage medium. Background Technology

[0002] A RAW image refers to the source image file output by the sensor; it is an unprocessed image. During the process of light passing through the sensor to obtain a RAW image, various random noises and noise caused by uneven variations between pixels are introduced. Examples include photon noise and dark noise introduced during sensor acquisition, noise introduced by the lenses of each photosensitive element, and noise introduced by the analog-to-digital converter, etc. Random noise can manifest as random spots in the image, severely affecting the subjective viewing experience.

[0003] In related technologies, image denoising or deep learning can be used to denoise images. For example, single-frame or multi-frame calibration methods can be used to calibrate image noise to obtain noise parameters, and then denoising can be performed based on these noise parameters. However, if a multi-frame calibration method is used, a relatively large number of images are required to statistically analyze the noise information, resulting in a very large number of statistical analyses. If a single-frame calibration method is used, the noise distribution among each pixel in the image will be uneven, affecting the accuracy of the final noise parameters. Summary of the Invention

[0004] To overcome the problems existing in related technologies, this disclosure provides an image noise processing method, apparatus, electronic device, and storage medium.

[0005] According to a first aspect of this disclosure, an image noise processing method is provided, comprising:

[0006] Acquire at least two first-sampled images from the same scene;

[0007] Determine the pixel values ​​of at least two pixels in the first sampled image; wherein the pixel values ​​include random values ​​indicating the portion affected by random noise and non-uniform values ​​indicating the portion affected by non-uniform noise;

[0008] Residual noise information is determined based on the difference in pixel values ​​of corresponding pixels in two second sampled images; wherein, the first sampled image includes the second sampled image;

[0009] Based on the residual noise information, noise parameters for noise reduction are obtained.

[0010] In some embodiments, determining the residual noise information based on the difference between the pixel values ​​of corresponding pixels in two second sampled images includes:

[0011] The pixel difference is determined based on the difference between the pixel values ​​of corresponding pixels in the two second sampled images;

[0012] The residual noise information is determined based on the variance of the pixel differences.

[0013] In some embodiments, the method includes:

[0014] The average pixel value of each pixel is determined based on the average pixel value of each corresponding pixel in at least two of the first sampled images.

[0015] All pixels corresponding to the same pixel mean are identified as a candidate pixel set;

[0016] If the number of pixels included in the candidate pixel set is greater than or equal to a predetermined number, the candidate pixel set is determined to be the target pixel set; wherein, the pixels included in the target pixel set are sampled pixels;

[0017] The determination of residual noise information based on the difference in pixel values ​​of corresponding pixels in two second sampled images includes:

[0018] The residual noise information corresponding to the target pixel set is determined based on the difference between the pixel values ​​of the pixels in the second sampled image corresponding to the sampled pixel.

[0019] In some embodiments, obtaining noise parameters for noise reduction based on the residual noise information includes:

[0020] Linear regression analysis is performed on the residual noise information corresponding to at least two sets of the target pixels to obtain the noise parameters.

[0021] In some embodiments, the method includes:

[0022] If the difference between the pixel values ​​of corresponding pixels in two first sampled images is less than or equal to a predetermined difference, the two first sampled images are determined to be the second sampled images.

[0023] or,

[0024] From at least two first sampled images, the two first sampled images with the smallest difference in the pixel values ​​of corresponding pixels are selected as the second sampled images.

[0025] In some embodiments, acquiring at least two first sampled images of the same scene includes:

[0026] Acquire at least two first sampled images acquired for at least one set of image acquisition parameters; wherein, the set of image acquisition parameters includes: exposure time and / or ISO sensitivity;

[0027] Determining the pixel values ​​of at least two pixels in the first sampled image includes:

[0028] Determine the pixel values ​​of at least two of the first sampled images under at least one of the image acquisition parameter sets;

[0029] The determination of residual noise information based on the difference in pixel values ​​of corresponding pixels in two second sampled images includes:

[0030] Based on the difference in noise information between two second sampled images under at least one set of image acquisition parameters, the residual noise information under at least one set of image acquisition parameters is determined;

[0031] The step of obtaining noise parameters for noise reduction based on the residual noise information includes:

[0032] Based on the residual noise information under at least one set of image acquisition parameters, at least one set of noise parameters for target image denoising is obtained;

[0033] The method further includes: constructing an image denoising model for denoising the target image based on the at least one set of noise parameters.

[0034] According to a second aspect of this disclosure, an image noise processing apparatus is provided, the apparatus comprising:

[0035] The acquisition module is configured to acquire at least two first sampled images from the same scene;

[0036] The determining module is configured to determine the pixel values ​​of at least two pixels in the first sampled image; wherein the pixel values ​​include random values ​​indicating the influence of random noise and non-uniform values ​​indicating the influence of non-uniform noise.

[0037] A first processing module is configured to determine residual noise information based on the difference between the pixel values ​​of corresponding pixels in two second sampled images; wherein the first sampled image includes the second sampled image.

[0038] The second processing module is configured to obtain noise parameters for noise reduction based on the residual noise information.

[0039] In some embodiments, the first processing module is configured to:

[0040] The pixel difference is determined based on the difference between the pixel values ​​of corresponding pixels in the two second sampled images;

[0041] The residual noise information is determined based on the variance of the pixel differences.

[0042] In some embodiments, the determining module is configured to:

[0043] The average pixel value of each pixel is determined based on the average pixel value of each corresponding pixel in at least two of the first sampled images.

[0044] All pixels corresponding to the same pixel mean are identified as a candidate pixel set;

[0045] If the number of pixels included in the candidate pixel set is greater than or equal to a predetermined number, the candidate pixel set is determined to be the target pixel set; wherein, the pixels included in the target pixel set are sampled pixels;

[0046] The first processing module is configured to determine the residual noise information corresponding to the target pixel set based on the difference between the pixel values ​​of the pixels in the second sampled image corresponding to the sampled pixel.

[0047] In some embodiments, the second processing module is configured to perform linear regression analysis on the residual noise information corresponding to at least two sets of target pixels to obtain the noise parameters.

[0048] In some embodiments, the determining module is configured to:

[0049] If the difference between the pixel values ​​of corresponding pixels in two first sampled images is less than or equal to a predetermined difference, the two first sampled images are determined to be the second sampled images.

[0050] or,

[0051] From at least two first sampled images, the two first sampled images with the smallest difference in the pixel values ​​of corresponding pixels are selected as the second sampled images.

[0052] In some embodiments, the acquisition module is configured to acquire at least two of the first sampled images acquired for at least one set of image acquisition parameters; wherein the set of image acquisition parameters includes: exposure time and / or ISO;

[0053] The determining module is configured to determine the pixel values ​​of at least two of the first sampled images under at least one of the image acquisition parameter sets;

[0054] The first processing module is configured to determine the residual noise information under at least one set of image acquisition parameters based on the difference in noise information between two second sampled images under at least one set of image acquisition parameters;

[0055] The second processing module is configured to obtain at least one set of noise parameters for target image denoising based on the residual noise information under at least one set of image acquisition parameters;

[0056] The apparatus further includes: a construction module; the construction module is configured to construct an image denoising model for denoising the target image based on the at least one set of noise parameters.

[0057] According to a third aspect of the present disclosure, an electronic device is provided, comprising:

[0058] processor;

[0059] Memory used to store processor-executable instructions;

[0060] The processor is configured to implement the image noise processing method described in any embodiment of this disclosure when running the executable instructions.

[0061] According to a fourth aspect of the present disclosure, a computer-readable storage medium is provided, the storage medium storing an executable program, wherein, when executed by a processor, the executable program implements the image noise processing method described in any embodiment of the present disclosure.

[0062] The technical solutions provided by the embodiments of this disclosure may include the following beneficial effects:

[0063] This embodiment of the present disclosure can acquire at least two first sampled images in the same scene; determine the pixel values ​​of the pixels in the at least two first sampled images; wherein the pixel values ​​include random values ​​indicating the influence of random noise and non-uniform values ​​indicating the influence of non-uniform noise; determine residual noise information based on the difference between the pixel values ​​of the corresponding pixels in the two second sampled images; wherein the first sampled image includes the second sampled image; and obtain noise parameters for noise reduction based on the residual noise information. Thus, on the one hand, this embodiment of the present disclosure only needs to select two or a relatively small number of second sampled images from the first sampled images to calculate the residual noise information, without needing to calculate the noise information of a relatively large number of sampled images to determine the noise parameters; thereby reducing the amount of data processing and computational complexity. On the other hand, this embodiment of the present disclosure can determine the residual noise information through the difference between the pixel values ​​of the two second sampled images; thereby at least partially eliminating the noise caused by the non-uniform noise influence between pixels, that is, compared to determining the noise parameters based on a single frame image, it can greatly reduce the influence of the non-uniform noise, thereby improving the accuracy of obtaining the noise parameters.

[0064] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0065] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure.

[0066] Figure 1 This is a flowchart illustrating an image noise processing method according to an exemplary embodiment of the present disclosure.

[0067] Figure 2 This is a flowchart illustrating an image noise processing method according to an exemplary embodiment of the present disclosure.

[0068] Figure 3 This is a flowchart illustrating an image noise processing method according to an exemplary embodiment of the present disclosure.

[0069] Figure 4 This is a flowchart illustrating an image noise processing method according to an exemplary embodiment of the present disclosure.

[0070] Figure 5 This is a schematic diagram illustrating a single-frame calibration method for obtaining noise information according to an exemplary embodiment of the present disclosure.

[0071] Figure 6 This is a schematic diagram illustrating a multi-frame calibration method for obtaining noise information according to an exemplary embodiment of the present disclosure.

[0072] Figure 7 This is a schematic diagram illustrating an image noise processing method for obtaining residual noise information according to an exemplary embodiment of the present disclosure.

[0073] Figure 8 This is a schematic diagram illustrating an embodiment of obtaining noise information according to an exemplary embodiment of the present disclosure.

[0074] Figure 9 This is a block diagram illustrating an image noise processing apparatus according to an exemplary embodiment of the present disclosure.

[0075] Figure 10 This is a block diagram illustrating an electronic device according to an exemplary embodiment of the present disclosure. Detailed Implementation

[0076] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present invention. Rather, they are merely examples of apparatuses and methods consistent with some aspects of the invention as detailed in the appended claims.

[0077] Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0078] To better understand the technical solutions described in any embodiment of this disclosure, firstly, some aspects of RAW images in the related art will be explained:

[0079] In one embodiment, the pixel value in a RAW image can be: f(x) = ap(x) + n; where f(x) can be the pixel value of each pixel in the image; x represents the amount of photons illuminating the pixel in the image sensor; ap(x) represents the pixel value corresponding to the first noise component in the noise; and n represents the pixel value corresponding to the second noise component in the noise. Here, p(x) can follow a Poisson distribution; for example, ap(x) ~ P(aλ, a 2 λ); where the mean of ap(x) can be aλ, and the variance of ap(x) can be a 2 λ. n can follow a Gaussian distribution; for example, n ~ N(0,b), where the mean of n can be 0 and the variance of n can be b. Here, noise includes, but is not limited to, at least one of the following: photon noise and dark noise introduced during sensor acquisition, noise introduced by the lens of each photosensitive element, noise introduced by the analog-to-digital converter, and noise introduced by the analog amplifier.

[0080] In one embodiment, the first noise component may refer to photon noise and dark noise introduced during the sensor process; the second noise component may be at least one of the noise introduced by each lens, the noise introduced by the analog-to-digital sensor, and the noise introduced by the analog amplifier.

[0081] In another embodiment, the first noise component may refer to noise that conforms to a Poisson distribution; the second noise component may refer to noise that conforms to a Gaussian distribution.

[0082] In one embodiment, the noise of the RAW image can be the variance of the pixel values; this noise can be: Var(f(x))=a×aλ+b=aE(f(x))+b; where Var(f(x)) represents the noise; λ represents the mean of the Poisson distribution p(x); and E(f(x)) is the mean of f(x). Here, a and b are noise parameters.

[0083] Figure 1 This is an exemplary embodiment of an image noise processing method disclosed herein.

[0084] The image noise processing method in this embodiment can be configured in an image noise processing device, which can be located in a server or an electronic device. This embodiment does not limit this.

[0085] like Figure 1 As shown, in some embodiments, the image noise processing method provided in this disclosure includes the following steps:

[0086] Step S11: Obtain at least two first sampled images from the same scene;

[0087] Step S13: Determine the pixel values ​​of at least two pixels in the first sampled image; wherein the pixel values ​​include random values ​​indicating the random noise-affected portion and non-uniform values ​​indicating the non-uniform noise-affected portion;

[0088] Step S15: Determine residual noise information based on the difference between the pixel values ​​of corresponding pixels in the two second sampled images; wherein, the first sampled image includes the second sampled image;

[0089] Step S17: Based on the residual noise information, obtain the noise parameters used for noise reduction.

[0090] The image noise processing method of this disclosure can be executed by an electronic device. The electronic device here can be various mobile or fixed devices. For example, the electronic device can be, but is not limited to, a server, computer, tablet computer, mobile phone, or wearable device.

[0091] It should be noted that the execution entity of the embodiments disclosed herein may be, in hardware, a central processing unit (CPU) in a server or electronic device, and in software, a related background service in a server or electronic device, without limitation.

[0092] In one embodiment, the electronic device may include an image acquisition module for acquiring images.

[0093] In another embodiment, the electronic device may not include an image acquisition module, but may communicate with other devices that include an image acquisition module. For example, the electronic device may receive image information from images sent by other devices, and the device includes an image acquisition module for acquiring images.

[0094] Here, the image acquisition module can be any element, module, device, or apparatus used for acquiring images. For example, the image acquisition module can be a camera. The image acquisition module can also include a sensor. It can also include a lens, an analog-to-digital converter, and / or an analog amplifier, etc.

[0095] Here, the same scene in step S11 can be, but is not limited to, a scene with the same lighting and / or the same image acquisition module settings. The same image acquisition module settings include setting the same set of image acquisition parameters for the image acquisition module. This set of image acquisition parameters can be, but is not limited to, exposure time and / or ISO. For example, the same scene in step S11 can be a scene with the same set of image acquisition parameters.

[0096] The scene in step S11 can also include the same target object. The same target object can refer to the same target object in the acquired images and the same surrounding environment of the target object.

[0097] The same scene in step S11 can also be, but is not limited to, continuously captured images. For example, using a tripod to fix an electronic device; the fixed electronic device continuously captures multiple first sampled images.

[0098] For example, step S11 may include: acquiring at least two consecutively acquired first sampled images. Alternatively, for example, step S11 may include: acquiring at least two consecutively acquired first sampled images of the same target object. Alternatively, for example, step S11 may include: acquiring at least two consecutively acquired first sampled images of the same scene. Alternatively, for example, step S11 may include: acquiring at least two consecutively acquired first sampled images of a static scene.

[0099] In one embodiment, the first sampled image, the second sampled image, and the target image (described below) can all be RAW images. Of course, in other embodiments, the first sampled image, the second sampled image, and the target image can also be any other image, as long as the image includes noise. Here, the second sampled image can be any image from the first sampled image.

[0100] Here, pixel values ​​are parameters used to characterize noise. A pixel value includes at least a random value and a non-uniform value. The random value indicates the pixel value affected by random noise. The non-uniform value indicates the pixel value affected by non-uniform noise. It is understood that the noise distribution among each pixel in an image is different. The noise portion that varies with pixel size is the non-uniform noise portion.

[0101] Here, noise can be, but is not limited to, at least one of the following: photon noise and / or dark noise introduced by the sensor of the image acquisition module, noise introduced by the lens, noise introduced by the analog-to-digital converter, and noise introduced by the analog amplifier, etc. Here, the random value can be the pixel value of the random noise affected by at least one of the above-mentioned noises; the non-uniform value can be the pixel value of the non-uniform noise affected by at least one of the above-mentioned noises.

[0102] Here, the random noise component in the noise can be the first noise component; the non-uniform noise component in the noise can be the second noise component.

[0103] In one embodiment, determining the pixel values ​​of pixels in at least two first sampled images in step S13 includes determining the pixel values ​​of at least a portion of pixels in at least two first sampled images. For example, the electronic device acquires the pixel values ​​of each pixel in at least two consecutively acquired first sampled images. Another example is that the electronic device acquires the pixel values ​​of each pixel in a first region of at least two consecutively acquired first sampled images. Yet another example is that the electronic device acquires the pixel values ​​of a predetermined number of pixels in at least two consecutively acquired first sampled images, where the predetermined number of pixels are randomly selected pixels.

[0104] In one embodiment, a method for determining the pixel value of a pixel in a first sampled image may be: obtaining the random value and non-uniformity value of each pixel in the first sampled image, and determining the pixel value of the pixel in the first sampled image based on the sum of the random value and non-uniformity value of each pixel in the first sampled image.

[0105] For example, the pixel value of a pixel in the first sampled image can be obtained by the following formula: f(x) = (a+m)p(x) + n; where f(x) is the pixel value of a pixel in the first sampled image; ap(x) + n is a random value; and mp(x) is a non-uniform value. Here, p(x) can follow a Poisson distribution; and n can follow a Gaussian distribution.

[0106] In one embodiment, the random value may also refer to the pixel value corresponding to the first noise component in the noise; the non-uniform value may also refer to the pixel value corresponding to the second noise component in the noise. Here, the first noise component may refer to the noise of the randomly affected portion of the noise; the second noise component may refer to the noise of the non-uniform affected portion.

[0107] In one embodiment, the sum of the non-uniform values ​​of all pixels in an image can be 0. For example, if an image has Q pixels, then some of these Q pixels have non-uniform values ​​that are positively shifted relative to random values, and some have non-uniform values ​​that are negatively shifted. It is sufficient that the total sum of the non-uniform values ​​of the Q pixels is 0. Thus, the mean of the non-uniform values ​​of the Q pixels is 0; for example, E(m) = 0. Based on the non-uniform values ​​and the mean of the non-uniform values ​​of the Q pixels, the variance of the non-uniform values ​​of the Q pixels can be derived as Var(m) = c.

[0108] In another embodiment, one way to determine the pixel value of a pixel in a second sampled image is to: obtain the random value and non-uniformity value of each pixel in the second sampled image, and determine the pixel value of the pixel in the second sampled image based on the sum of the random value and non-uniformity value of each pixel in the second sampled image.

[0109] Here, the first sampled image includes the second sampled image. For example, if there are three first sampled images, then the second sampled image is three or two of the three first sampled images. For example, if there are i first sampled images, then the second sampled image is j second sampled images among the i first sampled images; where j is less than or equal to i.

[0110] Here, since the first sampled image includes the second sampled image, the pixel values ​​of the pixels in the second sampled image can also be determined based on f(x) = (a+m)p(x)+n.

[0111] In some applications, noise information in a second sampled image can be determined based on the variance of the pixel values ​​of the pixels in that image. Here, noise information describes the noise in the image. The noise information includes: first noise information and / or second noise information. The first noise information can be noise information of the randomly influenced portion of the noise; the second noise information can be noise information of the non-uniformly influenced portion of the noise.

[0112] Here, one method for determining the noise information of the second sampled image based on the variance of the pixel values ​​is: Var(f(x))=cλ 2 +(a 2 +c)λ+b; where Var(f(x)) represents the noise of the second sampled image. Here, the pixel value of the second sampled image is f(x)=(a+m)p(x)+n, the mean of the non-uniformity is E(m)=0, the variance of the non-uniformity is Var(m)=c, and the mean of the pixel value of the second sampled image is E(f(x))=aλ; then the noise information of the second sampled image, that is, the variance of the pixel value of the second sampled image is Var(f(x))=cλ 2 +(a 2 +c)λ+b. Based on Var(f(x))=cλ 2 +(a 2 +c)λ+b also shows a quadratic relationship between noise information and the mean pixel value (i.e., E(f(x))=aλ); here, the non-uniformity of pixel values ​​also affects the magnitude of the quadratic term. Because of the quadratic term, this method of determining noise information results in a relatively large noise information value; this leads to significant errors in the noise parameters subsequently determined based on this noise information.

[0113] To address the issue of relatively large noise levels in the aforementioned application scenarios, leading to significant errors in noise parameter determination, this disclosure provides a method that uses two second sampled images acquired in the same scene. The pixel values ​​of corresponding pixels in the two second sampled images are subtracted to eliminate at least a portion of the noise affected by non-uniformity. Since the two second sampled images are acquired in the same scene, such as under the same lighting and / or camera settings, the non-uniformity noise affecting the corresponding pixels in the two second sampled images is essentially the same. Therefore, the noise affecting the non-uniformity can be at least partially eliminated by subtracting the pixel values ​​of the corresponding pixels in the two sampled images.

[0114] In some embodiments, step S15 includes:

[0115] The pixel difference is determined based on the difference between the pixel values ​​of corresponding pixels in the two second sampled images;

[0116] The residual noise information is determined based on the variance of the pixel differences.

[0117] For example, one method for determining the pixel difference based on the difference in pixel values ​​of corresponding pixels in two second sampled images is: Δf(x) = (a+m)×Δp(x)+Δn; where Δf(x) represents the pixel difference. Another method for determining residual noise information based on the variance of the pixel difference is: in, This represents the residual noise information. Thus, the obtained residual noise information has a linear relationship with the mean pixel value (i.e., the pixel mean E(f(x)) = aλ), relative to the noise information Var(f(x)) = cλ. 2 +(a 2 For the case of +c)λ+b, the quadratic term cλ is eliminated. 2 This can greatly reduce the noise in the affected areas caused by non-uniform noise.

[0118] In the example above, c is usually... 2 < 2 , i.e., c 2 Much smaller than a 2 Therefore, c can be ignored. 2 Then the residual noise information can be: Var(△f(x))=(a 2 )λ+b=a×E(f(x))+b. Of course, this residual noise information can also be Here, the residual noise information Var(△f(x)) or The noise parameters a and / or b are linearly related to the mean pixel value E(f(x)). This allows us to obtain relatively accurate noise parameters a and / or b.

[0119] It is understood that the methods described above for obtaining noise information and / or residual noise information can be implemented in various ways, and the parameters in obtaining the noise information and / or residual noise information are merely representative and not limited to specific values. No restrictions are placed on the specific formulas for obtaining noise information and / or residual noise information, nor on the coefficients before the mean of pixel values ​​in the noise information and / or residual noise information. It is understood that the noise information, due to the introduction of non-uniform noise, is a quadratic expression with a quadratic variance of the non-uniformity value; this non-uniformity value represents the pixel value of the non-uniform noise component. The residual noise information, determined based on the mean of the differences between corresponding pixel values ​​in two second sampled images, eliminates the quadratic variance of the non-uniformity value; thus, the residual noise information is a linear expression based on the mean of pixel values. In this way, the residual noise information can at least partially eliminate the noise caused by the non-uniform noise between pixels.

[0120] ​Here, obtaining the noise parameters for denoising in step S17 can be: obtaining noise parameters for denoising the target image. That is, the noise parameters can be used for denoising the target image. The target image can be an image from the same scene as the first sampled image and / or the second sampled image.

[0121] Here, in step S17, the residual noise information is a linear equation relating the pixel value mean. Therefore, the noise parameters used for noise reduction are the coefficients of this linear equation. For example, the residual noise information is Var(Δf(x))=(a 2 If λ+b=a×E(f(x))+b, then the noise parameters are a and / or b.

[0122] In this embodiment, at least two first sampled images of the same scene can be acquired, and the pixel values ​​of the pixels in the at least two first sampled images can be determined. The pixel values ​​include random values ​​indicating the influence of random noise and non-uniform values ​​indicating the influence of non-uniform noise. Residual noise information is determined based on the difference in pixel values ​​of corresponding pixels in two second sampled images. The first sampled images include the second sampled images. Noise parameters for noise reduction are obtained based on the residual noise information. Thus, on the one hand, this embodiment only needs to select two or a relatively small number of second sampled images from the first sampled images to calculate the residual noise information, without needing to calculate the noise information of a relatively large number of sampled images to determine the noise parameters; thereby reducing the amount of data processing and computational complexity. On the other hand, this embodiment can determine the residual noise information through the difference in pixel values ​​of two second sampled images; thereby at least partially eliminating the noise caused by the non-uniform noise influence between pixels. That is, compared to determining noise parameters based on a single frame image, it can greatly reduce the influence of non-uniform noise on the noise, thereby improving the accuracy of the noise parameters.

[0123] Furthermore, in this embodiment, compared to the prior art's method of obtaining noise parameters based on single-frame or multi-frame calibration, no specialized experimental conditions (e.g., no grayscale chart) are required. It only requires acquiring a first sampled image in the same scene (e.g., acquiring a first sampled image in a static scene) and obtaining the noise parameters based on the pixel values ​​of that first sampled image. Thus, this embodiment has the advantage of low requirements for experimental conditions.

[0124] In some embodiments, the image noise processing method further includes: constructing an image denoising model for the target image based on the noise parameters. Thus, in this embodiment, if relatively accurate noise parameters are obtained, they can be used to construct the image denoising model; thereby, a relatively high denoising effect can be achieved for the target image. This image denoising model can be applied to denoising methods such as deep learning.

[0125] In other embodiments, the image noise processing method further includes synthesizing a noisy image based on the noise parameters. This noisy image can also be used in denoising methods such as deep learning or neural networks; for example, the noisy image can be used to train a neural network.

[0126] In some embodiments, the image noise processing method further includes: randomly selecting two of the first sampled images from at least two first sampled images as the second sampled images. Thus, in this embodiment of the disclosure, randomly selecting two first sampled images as the second sampled images, i.e., as the images for subsequent calculation of residual noise information, can simplify the computational workload of calculating residual noise information.

[0127] like Figure 2 As shown, in some embodiments, the image noise processing method further includes:

[0128] Step S14: If the difference between the pixel values ​​of corresponding pixels in two first sampled images is less than or equal to a predetermined difference, the two first sampled images are determined to be the second sampled images.

[0129] Here, the electronic device can select two first sampled images from at least two first sampled images whose pixel values ​​of corresponding pixels are less than or equal to a predetermined difference as the second sampled images.

[0130] In one embodiment, step S14 may also include:

[0131] From at least two first sampled images, obtain the difference between the pixel values ​​of any two of the first sampled images;

[0132] If the difference between the pixel values ​​of any two first sampled images in a set is less than or equal to a predetermined difference, then the first sampled image in the set of any two first sampled images is determined to be the second sampled image.

[0133] For example, if there are four first sampled images, namely A1, A2, A3, and A4. If it is determined that the difference between the corresponding pixel values ​​of A1 and A2 is less than or equal to a predetermined difference, then A1 and A2 can be determined as second sampled images. Alternatively, if it is determined that the difference between the corresponding pixel values ​​of A2 and A4 is less than or equal to a predetermined difference, then A2 and A4 can be determined as second sampled images.

[0134] Thus, in this embodiment, two first sampled images whose corresponding pixel value difference is less than or equal to a predetermined difference can be selected as second sampled images from the first sampled images. That is, two first sampled images with relatively small noise differences can be selected as second images. When noise parameters are subsequently obtained based on the pixel values ​​of the second sampled images, the accuracy of the obtained noise parameters can be improved because they are based on noise parameters from two images with relatively small noise differences. Furthermore, since noise parameters can be obtained using only the two selected second sampled images, the computational workload for calculating noise parameters is also simplified.

[0135] In one embodiment, the image noise processing method further includes: selecting the two first sampled images with the smallest difference in pixel values ​​of corresponding pixel points from at least two first sampled images as the second sampled images.

[0136] For example, if there are three first sampled images, namely A1, A2, and A3, the electronic device determines that the difference in pixel values ​​between corresponding pixels of A1 and A2 is Q1, the difference in pixel values ​​between corresponding pixels of A2 and A3 is Q2, and the difference in pixel values ​​between corresponding pixels of A1 and A3 is Q3; if Q1 is less than Q2 and Q2 is less than Q3, then A1 and A2 are determined to be the second sampled images.

[0137] Thus, in this embodiment, the two first sampled images with the smallest difference in corresponding pixel values ​​can be selected as the second sampled images, meaning the two first sampled images with the smallest noise difference can be selected as the second sampled images. When subsequently obtaining noise parameters based on the pixel values ​​of the second sampled images, the accuracy of the obtained noise parameters can be greatly improved because they are based on images with relatively small noise differences. Furthermore, since the noise parameters can be obtained by selecting only two first sampled images, the computational workload for calculating the noise parameters is also simplified.

[0138] In some embodiments, the image noise processing method further includes:

[0139] From at least two first sampled images, select the two first sampled images with the smallest difference in color values ​​as the second sampled images; or...

[0140] From at least two first sampled images, two first sampled images whose color value difference is less than or equal to a predetermined color difference are selected as the second sampled images.

[0141] Here, the color values ​​of the first sampled image can be the color values ​​of at least a portion of the pixels in the first sampled image. The difference between the color values ​​of two first sampled images can be the difference between the color values ​​of corresponding pixels in the two first sampled images.

[0142] Here, color values ​​may include the pigment values ​​of the three pigments: red (R), green (G), and blue (B), and / or grayscale values.

[0143] Therefore, in this embodiment of the disclosure, other parameters of the first sampled image, such as color values, can be used to determine the two first sampled images with the smallest or relatively small difference among at least two first sampled images as the second sampled images. The noise parameters determined based on these two second sampled images can thus improve their accuracy.

[0144] In some embodiments, the image noise processing method further includes:

[0145] The average pixel value of each pixel is determined based on the average pixel value of each corresponding pixel in at least two of the first sampled images.

[0146] All pixels corresponding to the same pixel mean are identified as a candidate pixel set;

[0147] If the number of pixels included in the candidate pixel set is greater than or equal to a predetermined number, the candidate pixel set is determined to be the target pixel set; wherein, the pixels included in the target pixel set are sampled pixels;

[0148] Step S15 includes:

[0149] The residual noise information corresponding to the target pixel set is determined based on the difference between the pixel values ​​of the pixels in the second sampled image corresponding to the sampled pixel.

[0150] Here, the set of candidate pixels consists of two or more pixels.

[0151] Here, the predetermined quantity is greater than or equal to the first quantity. For example, the first quantity can be two, five, ten, or twenty, etc. In one embodiment, the predetermined quantity can be set arbitrarily, as long as the target candidate set is greater than or equal to two.

[0152] For example, the electronic device acquires three first sampled images, namely A1, A2 and A3; the pixel mean of the corresponding pixels in the three first sampled images can be f(x)' = f(x1) + f(x2) + f(x3); where f(x)' is the pixel mean; f(x1) is the pixel value of A1, f(x2) is the pixel value of A2, and f(x3) is the pixel value of A3.

[0153] In the above example, if the three first sampled images have one hundred corresponding pixels, ten of these pixels have the same pixel mean value, f(x1)'; five pixels have the same pixel mean value, f(x2)'; and eight pixels have the same pixel mean value, f(x3)'; and the pixel mean values ​​of the other pixels are all different. If the predetermined number is five, then the set of ten pixels with a pixel mean value of f(x1)', the set of five pixels with a pixel mean value of f(x2)', and the set of eight pixels with a pixel mean value of f(x3)' can each be a target pixel set. For example, the set of ten pixels with a pixel mean value of f(x1)' is the first target pixel set S1, the set of five pixels with a pixel mean value of f(x2)' is the second target pixel set S2, and the set of eight pixels with a pixel mean value of f(x3)' is the third target pixel set S3. Here, the ten pixels in the first target pixel set, the five pixels in the second target pixel set, and the eight pixels in the third target pixel set are all sampled pixels.

[0154] In some embodiments, determining the residual noise information corresponding to the target pixel set based on the difference between the pixel values ​​of pixels in the second sampled image corresponding to the sampled pixel includes:

[0155] Based on the difference between the pixel values ​​corresponding to the sampled pixel points in each target pixel set in the two second sampled images, a residual noise information is determined;

[0156] Obtain at least two residual noise information corresponding to at least two target pixel sets.

[0157] Based on the above example, the electronic device acquires the residual noise information of the first target pixel set. Obtain residual noise information from the second target pixel set And obtain the residual noise information of the third target pixel set. Should and All of these are residual noise information.

[0158] In some embodiments, obtaining noise parameters for noise reduction based on the residual noise information includes:

[0159] Linear regression analysis is performed on the residual noise information corresponding to at least two sets of the target pixels to obtain the noise parameters.

[0160] For example, a method for deriving noise parameters using the least squares approach is disclosed; specifically, the electronic device can obtain m residual noise information formulas based on the m target pixel set in the above example; if y represents z represents E(f(x)), then the formula for obtaining the residual noise information can be expressed as y=(a 2 λ+b=a×z+b. For example, the formula for the m residual noise information can be: If expressed using matrices and vectors, it can be: To obtain the optimal W, we can minimize the mean square error between ZW and Y, i.e. If we use the loss function J(W)=(Y-ZW) 2 Differentiating, we get: Here, since the number of m's is greater than or equal to 2, Z T Z is a full-rank matrix; and if we set the derivative of the above loss function to zero, then we can obtain the optimal solution W0. * =(Z T Z) -1 Z T Y.

[0161] For example, the following can be used and The noise parameters a and b are calculated using at least two of the formulas.

[0162] Thus, in this embodiment of the disclosure, at least two candidate pixel sets with the same pixel value greater than or equal to a predetermined number can be selected from at least the candidate pixel set as the target pixel set for determining residual noise information; this can reduce the amount of computation while obtaining relatively accurate residual noise information.

[0163] Since the number of candidate pixels is greater than or equal to a predetermined number, the number of pixels in the second sampled image corresponding to the candidate pixel set belongs to the same scene, which can eliminate more noise from the non-uniformity part of the second sampled image and thus obtain relatively accurate residual noise information.

[0164] Furthermore, in this embodiment of the present disclosure, the residual noise information can be determined by the difference between the pixel values ​​of the pixels in the second sampled image corresponding to the sampled pixels in the two target pixel sets, without the need to determine the residual noise information by the difference between the pixel values ​​of all corresponding pixels in the two second sampled images, thereby greatly reducing the computational complexity and the amount of computation.

[0165] like Figure 3 As shown, in some embodiments, S11 may include:

[0166] Step S111: Acquire at least two of the first sampled images for at least one set of image acquisition parameters; wherein, the set of image acquisition parameters includes: exposure time and / or ISO;

[0167] Step S13 includes:

[0168] Step S131: Determine the pixel values ​​of at least two of the first sampled images under at least one of the image acquisition parameter sets;

[0169] Step S15 includes:

[0170] Step S151: Determine the residual noise information under at least one of the image acquisition parameter sets based on the difference in noise information between two second sampled images under at least one of the image acquisition parameter sets;

[0171] Step S17 includes:

[0172] Step S171: Based on the residual noise information under at least one set of the image acquisition parameters, obtain at least one set of noise parameters for target image denoising;

[0173] Image noise processing methods also include:

[0174] Step S19: Based on the at least one set of noise parameters, construct an image denoising model for denoising the target image.

[0175] Here, the image acquisition parameter set can also be a set of parameters set by any other image acquisition module when acquiring images. For example, the image acquisition parameter set can also be aperture parameters.

[0176] Here, the exposure time can be any exposure time. For example, the exposure time can be... seconds, or In this embodiment of the disclosure, the occurrence of image overexposure can be reduced by setting an appropriate exposure time.

[0177] Here, ISO can be any ISO standard. For example, ISO can be ISO 100, ISO 200, or ISO 400, etc., which are internationally recognized standards.

[0178] In one embodiment, the noise parameters used for training in the image denoising model constructed in step S19 can be noise parameters corresponding to at least one set of image acquisition parameters.

[0179] In this embodiment, a set of noise parameters can be determined for different sets of image acquisition parameters; that is, different sets of image acquisition parameters can be set to determine noise parameters under different scenarios. The multiple sets of noise parameters obtained in this way can also enrich the settings of the image denoising model, adapting to more scenarios requiring denoising of the target image.

[0180] It should be noted that those skilled in the art will understand that the methods provided in the embodiments of this disclosure can be executed alone or together with some methods in the embodiments of this disclosure or some methods in related technologies.

[0181] To further explain any embodiment of this disclosure, a specific embodiment is provided below.

[0182] like Figure 4 As shown, this disclosure provides an image noise processing method, which can be executed by an electronic device; the method includes the following steps:

[0183] Step S21: Set an image acquisition parameter set for the image acquisition module in the electronic device, wherein the image acquisition parameter set includes: exposure time and ISO;

[0184] Step S22: The electronic device acquires three consecutive first sampled images based on the image acquisition module; the three first sampled images are images of the same target object in a static scene;

[0185] Step S23: The electronic device obtains the average pixel value of each pixel based on the average pixel value of each pixel corresponding to the three first sampled images; the pixels with the same average pixel value are used as a candidate pixel set; and a set of candidate pixels with a pixel value greater than or equal to a predetermined number are obtained from the candidate pixel set as the target pixel set.

[0186] Here, the set of candidate pixels can be: s Q ={(s Q ,t Q )}; where Q is the pixel mean; (s Q ,t Q The coordinates of the pixel mean Q. If the number of pixels corresponding to a pixel mean is greater than or equal to a predetermined number T, then the set of pixels corresponding to that pixel mean is determined as the target pixel set; the pixels in this target pixel set are the sampled pixels. Here, each target pixel set can be used as the sampling set S to be sampled.

[0187] Step S24: Among the three first sampled images, the electronic device selects the two first sampled images with the smallest difference in the pixel values ​​of the corresponding pixels as second sampled images; and obtains the pixel values ​​of the corresponding pixels in the sampling set S in the two second sampled images.

[0188] Here, the corresponding pixels in the sampling set S of the two second sampled images can be P1 and P2, respectively.

[0189] Step S25: The electronic device determines the residual noise information based on the variance of the mean of the pixel values ​​of corresponding pixels in the sampling set S of the two second sampled images; and determines the noise parameters based on the residual noise information and linear regression analysis.

[0190] Here, the residual noise information can be... The noise parameters are a and b.

[0191] Step S26: The electronic device repeats steps S21 to S25 above by setting different exposure times and / or ISOs to obtain noise parameters at different exposure times and / or ISOs.

[0192] Thus, by obtaining noise parameters through the embodiments of this disclosure, compared to obtaining noise parameters by single-frame calibration, the noise influence caused by non-uniform noise between pixels can be at least partially eliminated, thereby improving the accuracy of obtaining noise parameters. Moreover, compared to obtaining noise parameters by multi-frame calibration, relatively fewer first and / or second sampled images can be used for calculation, thereby reducing the amount of data processing and computational complexity.

[0193] For example, in some application scenarios, a comparative experiment yielded schematic diagrams of noise information obtained based on a single-frame calibration method, noise information obtained based on a multi-frame calibration method, and residual noise information obtained by the image noise processing method of this disclosure embodiment. For example... Figure 5 As shown, Figure 5 This diagram illustrates the acquisition of noise information based on a single-frame calibration method. Figure 6 As shown, Figure 6 A schematic diagram illustrating a multi-frame calibration method for acquiring noise information is shown. Figure 7 As shown, Figure 7 This diagram illustrates a method for obtaining residual noise information based on image noise processing. Figure 7 The two sampled images are consecutive images taken from the same scene, and there is a difference in pixel brightness between them. It is understandable that, compared to single-frame calibration methods, image noise processing methods for obtaining residual noise information at least partially eliminate the noise caused by non-uniform noise; and compared to multi-frame calibration methods, image processing methods for obtaining residual noise information significantly reduce the computational load.

[0194] Figure 8 A schematic diagram illustrating one method of obtaining noise information according to this disclosure is shown. Here, Figure 8This can be represented as the noise parameter 'a' obtained under different ISO settings. It is understandable that the noise parameters obtained by image noise processing methods are essentially similar to those obtained by multi-frame calibration methods, and the latter is simpler to operate.

[0195] It should be noted that those skilled in the art will understand that the methods provided in the embodiments of this disclosure can be executed alone or together with some methods in the embodiments of this disclosure or some methods in related technologies.

[0196] Figure 9 An image noise processing apparatus is shown as an exemplary embodiment of the present disclosure; such as Figure 9 As shown, the device includes:

[0197] The acquisition module 41 is configured to acquire at least two first sampled images in the same scene;

[0198] The determining module 42 is configured to determine the pixel values ​​of at least two pixels in the first sampled image; wherein the pixel values ​​include random values ​​indicating the influence of random noise and non-uniform values ​​indicating the influence of non-uniform noise.

[0199] The first processing module 43 is configured to determine residual noise information based on the difference between the pixel values ​​of corresponding pixels in two second sampled images; wherein the first sampled image includes the second sampled image.

[0200] The second processing module 44 is configured to obtain noise parameters for noise reduction based on the residual noise information.

[0201] In some embodiments, the first processing module 43 is configured to:

[0202] The pixel difference is determined based on the difference between the pixel values ​​of corresponding pixels in the two second sampled images;

[0203] The residual noise information is determined based on the variance of the pixel differences.

[0204] In some embodiments, the determining module 42 is configured to:

[0205] The average pixel value of each pixel is determined based on the average pixel value of each corresponding pixel in at least two of the first sampled images.

[0206] All pixels corresponding to the same pixel mean are identified as a candidate pixel set;

[0207] If the number of pixels included in the candidate pixel set is greater than or equal to a predetermined number, the candidate pixel set is determined to be the target pixel set; wherein, the pixels included in the target pixel set are sampled pixels;

[0208] The first processing module 43 is configured to determine the residual noise information corresponding to the target pixel set based on the difference between the pixel values ​​of the pixels in the second sampled image corresponding to the sampled pixel.

[0209] In some embodiments, the second processing module 44 is configured to perform linear regression analysis on the residual noise information corresponding to at least two sets of target pixels to obtain the noise parameters.

[0210] In some embodiments, the determining module 42 is configured to:

[0211] If the difference in pixel values ​​of corresponding pixels in two first sampled images is less than or equal to a predetermined difference, the two first sampled images are determined to be the second sampled images; or...

[0212] From at least two first sampled images, the two first sampled images with the smallest difference in the pixel values ​​of corresponding pixels are selected as the second sampled images.

[0213] In some embodiments, the acquisition module 41 is configured to acquire at least two of the first sampled images for at least one set of image acquisition parameters; wherein the set of image acquisition parameters includes: exposure time and / or ISO;

[0214] The determining module 42 is configured to determine the pixel values ​​of at least two of the first sampled images under at least one of the image acquisition parameter sets;

[0215] The first processing module 43 is configured to determine the residual noise information based on the difference in noise information between two second sampled images under at least one set of image acquisition parameters;

[0216] The second processing module 44 is configured to obtain at least one set of noise parameters for target image denoising based on the residual noise information under at least one set of image acquisition parameters;

[0217] The apparatus further includes: a construction module; the construction module is configured to construct an image denoising model for denoising the target image based on the at least one set of noise parameters.

[0218] Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0219] Embodiments of this disclosure also provide an electronic device, characterized in that it includes:

[0220] processor;

[0221] Memory used to store processor-executable instructions;

[0222] The processor is configured to implement the image noise processing method described in any embodiment of this disclosure when running the executable instructions.

[0223] The memory may include various types of storage media, which are non-temporary computer storage media that can continue to store information after the communication device loses power.

[0224] The processor can be connected to the memory via a bus or similar means to read executable programs stored in the memory, for example, to implement... Figures 1 to 4 At least one of the methods shown.

[0225] Embodiments of this disclosure also provide a computer-readable storage medium storing an executable program, wherein the executable program, when executed by a processor, implements the image noise processing method described in any embodiment of this disclosure. For example, implementing... Figures 1 to 4 At least one of the methods shown.

[0226] Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0227] Figure 10 This is a block diagram illustrating an electronic device 800 according to an exemplary embodiment. For example, the electronic device 800 may be a mobile phone, computer, digital broadcasting electronic device, messaging transceiver, game console, tablet device, medical device, fitness equipment, personal digital assistant, etc.

[0228] Reference Figure 10 The electronic device 800 may include one or more of the following components: a processing component 802, a memory 804, a power supply component 806, a multimedia component 808, an audio component 810, an input / output (I / O) interface 812, a sensor component 814, and a communication component 816.

[0229] Processing component 802 typically controls the overall operation of electronic device 800, such as operations associated with display, telephone calls, data communication, camera operation, and recording operations. Processing component 802 may include one or more processors 820 to execute instructions to complete all or part of the steps of the methods described above. Furthermore, processing component 802 may include one or more modules to facilitate interaction between processing component 802 and other components. For example, processing component 802 may include a multimedia module to facilitate interaction between multimedia component 808 and processing component 802.

[0230] Memory 804 is configured to store various types of data to support the operation of device 800. Examples of this data include instructions for any application or method operating on electronic device 800, contact data, phonebook data, messages, pictures, videos, etc. Memory 804 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0231] Power supply component 806 provides power to various components of electronic device 800. Power supply component 806 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to electronic device 800.

[0232] Multimedia component 808 includes a screen that provides an output interface between the electronic device 800 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touchscreen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensors may sense not only the boundaries of the touch or swipe action but also the duration and pressure associated with the touch or swipe operation. In some embodiments, multimedia component 808 includes a front-facing camera and / or a rear-facing camera. When the device 800 is in an operating mode, such as a shooting mode or a video mode, the front-facing camera and / or the rear-facing camera may receive external multimedia data. Each front-facing camera and rear-facing camera may be a fixed optical lens system or have focal length and optical zoom capabilities.

[0233] Audio component 810 is configured to output and / or input audio signals. For example, audio component 810 includes a microphone (MIC) configured to receive external audio signals when electronic device 800 is in an operating mode, such as call mode, recording mode, and voice recognition mode. The received audio signals may be further stored in memory 804 or transmitted via communication component 816. In some embodiments, audio component 810 also includes a speaker for outputting audio signals.

[0234] I / O interface 812 provides an interface between processing component 802 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include, but are not limited to, home buttons, volume buttons, power buttons, and lock buttons.

[0235] Sensor assembly 814 includes one or more sensors for providing state assessments of various aspects of electronic device 800. For example, sensor assembly 814 may detect the on / off state of device 800, the relative positioning of components such as the display and keypad of electronic device 800, changes in position of electronic device 800 or a component of electronic device 800, the presence or absence of user contact with electronic device 800, orientation or acceleration / deceleration of electronic device 800, and temperature changes of electronic device 800. Sensor assembly 814 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. Sensor assembly 814 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, sensor assembly 814 may also include an accelerometer, gyroscope, magnetometer, pressure sensor, or temperature sensor.

[0236] Communication component 816 is configured to facilitate wired or wireless communication between electronic device 800 and other devices. Electronic device 800 can access wireless networks based on communication standards, such as WiFi, 4G, or 5G, or combinations thereof. In one exemplary embodiment, communication component 816 receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, communication component 816 also includes a near-field communication (NFC) module to facilitate short-range communication. For example, the NFC module may be implemented based on radio frequency identification (RFID) technology, Infrared Data Association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

[0237] In an exemplary embodiment, the electronic device 800 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the methods described above.

[0238] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 804 including instructions, which can be executed by a processor 820 of an electronic device 800 to perform the above-described method. For example, the non-transitory computer-readable storage medium may be a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device, etc.

[0239] Other embodiments of the invention will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the invention that follow the general principles of the invention and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of the invention are indicated by the following claims.

[0240] It should be understood that the present invention is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of the invention is limited only by the appended claims.

Claims

1. An image noise processing method, characterized by, The method includes: Acquire at least two first-sampled images from the same scene; Determine pixel values ​​for at least two pixels in the first sampled image; wherein the pixel values ​​include random values ​​and non-uniform values; the random values ​​indicate the random noise component of at least one type of noise; and the non-uniform values ​​indicate the non-uniform noise component of at least one type of noise. Residual noise information is determined based on the difference in pixel values ​​of corresponding pixels in two second sampled images; wherein, the second sampled image is any image in the first sampled image; Based on the residual noise information, noise parameters for noise reduction are obtained.

2. The method of claim 1, wherein, The determination of residual noise information based on the difference in pixel values ​​of corresponding pixels in two second sampled images includes: The pixel difference is determined based on the difference between the pixel values ​​of corresponding pixels in the two second sampled images; The residual noise information is determined based on the variance of the pixel differences.

3. The method according to claim 1 or 2, characterized in that, The method further includes: The average pixel value of each pixel is determined based on the average pixel value of each corresponding pixel in at least two of the first sampled images. All pixels corresponding to the same pixel mean are identified as a candidate pixel set; If the number of pixels included in the candidate pixel set is greater than or equal to a predetermined number, the candidate pixel set is determined to be the target pixel set; wherein, the pixels included in the target pixel set are sampled pixels; The determination of residual noise information based on the difference in pixel values ​​of corresponding pixels in two second sampled images includes: The residual noise information corresponding to the target pixel set is determined based on the difference between the pixel values ​​of the pixels in the second sampled image corresponding to the sampled pixel.

4. The method of claim 3, wherein, The step of obtaining noise parameters for noise reduction based on the residual noise information includes: Linear regression analysis is performed on the residual noise information corresponding to at least two sets of the target pixels to obtain the noise parameters.

5. The method according to claim 1 or 2, characterized in that, The method includes: If the difference between the pixel values ​​of corresponding pixels in two first sampled images is less than or equal to a predetermined difference, the two first sampled images are determined to be the second sampled images. or, From at least two first sampled images, the two first sampled images with the smallest difference in the pixel values ​​of corresponding pixels are selected as the second sampled images.

6. The method according to claim 1 or 2, characterized in that, The acquisition of at least two first sampled images in the same scene includes: Acquire at least two first sampled images acquired for at least one set of image acquisition parameters; wherein the set of image acquisition parameters includes: exposure time and / or ISO sensitivity; Determining the pixel values ​​of at least two pixels in the first sampled image includes: Determine the pixel values ​​of at least two of the first sampled images under at least one of the image acquisition parameter sets; The determination of residual noise information based on the difference in pixel values ​​of corresponding pixels in two second sampled images includes: Based on the difference in noise information between two second sampled images under at least one set of image acquisition parameters, the residual noise information under at least one set of image acquisition parameters is determined; The step of obtaining noise parameters for noise reduction based on the residual noise information includes: Based on the residual noise information under at least one set of image acquisition parameters, at least one set of noise parameters for target image denoising is obtained; The method further includes: constructing an image denoising model for denoising the target image based on the at least one set of noise parameters.

7. An image noise processing apparatus, characterized in that, The device includes: The acquisition module is configured to acquire at least two first sampled images from the same scene; The determining module is configured to determine pixel values ​​of at least two pixels in the first sampled image; wherein the pixel values ​​include random values ​​and non-uniform values; the random values ​​indicate the random noise component in at least one type of noise; and the non-uniform values ​​indicate the non-uniform noise component in at least one type of noise. The first processing module is configured to determine residual noise information based on the difference between the pixel values ​​of corresponding pixels in two second sampled images; wherein the second sampled image is any image in the first sampled image. The second processing module is configured to obtain noise parameters for noise reduction based on the residual noise information.

8. The apparatus according to claim 7, characterized in that, The first processing module is configured as follows: The pixel difference is determined based on the difference between the pixel values ​​of corresponding pixels in the two second sampled images; The residual noise information is determined based on the variance of the pixel differences.

9. The apparatus according to claim 7 or 8, characterized in that, The determining module is configured as follows: The average pixel value of each pixel is determined based on the average pixel value of each corresponding pixel in at least two of the first sampled images. All pixels corresponding to the same pixel mean are identified as a candidate pixel set; If the number of pixels included in the candidate pixel set is greater than or equal to a predetermined number, the candidate pixel set is determined to be the target pixel set; wherein, the pixels included in the target pixel set are sampled pixels; The first processing module is configured to determine the residual noise information corresponding to the target pixel set based on the difference between the pixel values ​​of the pixels in the second sampled image corresponding to the sampled pixel.

10. The apparatus according to claim 9, characterized in that, The second processing module is configured to perform linear regression analysis on the residual noise information corresponding to at least two sets of the target pixels to obtain the noise parameters.

11. The apparatus according to claim 7 or 8, characterized in that, The determining module is configured as follows: If the difference between the pixel values ​​of corresponding pixels in two first sampled images is less than or equal to a predetermined difference, the two first sampled images are determined to be the second sampled images. or, From at least two first sampled images, the two first sampled images with the smallest difference in the pixel values ​​of corresponding pixels are selected as the second sampled images.

12. The apparatus according to claim 7 or 8, characterized in that, The acquisition module is configured to acquire at least two of the first sampled images for at least one set of image acquisition parameters; wherein the set of image acquisition parameters includes: exposure time and / or ISO sensitivity; The determining module is configured to determine the pixel values ​​of at least two of the first sampled images under at least one of the image acquisition parameter sets; The first processing module is configured to determine the residual noise information under at least one set of image acquisition parameters based on the difference in noise information between two second sampled images under at least one set of image acquisition parameters; The second processing module is configured to obtain at least one set of noise parameters for target image denoising based on the residual noise information under at least one set of image acquisition parameters; The apparatus further includes a construction module; the construction module is configured to construct an image denoising model for denoising the target image based on the at least one set of noise parameters.

13. An electronic device, characterized in that, include: processor; Memory used to store processor-executable instructions; The processor is configured to implement the image noise processing method according to any one of claims 1 to 6 when running the executable instructions.

14. A computer-readable storage medium, characterized in that, The readable storage medium stores an executable program, wherein the executable program, when executed by a processor, implements the image noise processing method according to any one of claims 1 to 6.

Images