Image denoising method and device, computer readable medium, and electronic device

Abstract

The present disclosure specifically relates to the field of image processing technology, and specifically to an image denoising method and device, a computer-readable medium and an electronic device. The method comprises: obtaining a current image and a corresponding reference image under preset exposure conditions, determining a first image parameter of the current image; querying a noise parameter lookup table according to the preset exposure conditions to obtain a second image parameter; comparing the first image parameter with the second image parameter to perform a first denoising process on the current image according to the image parameter comparison result to obtain a preliminary denoising image under the preset exposure conditions, and generating a denoising degree parameter according to the denoising result of the current image; performing a second fusion denoising process on the preliminary denoising image and the corresponding reference image based on the denoising degree parameter to obtain a fused image; performing a third fusion denoising process on the fused images under different preset exposure conditions to obtain a denoised image corresponding to the current image. The present disclosure can effectively eliminate the noise of the exposure image and ensure the image denoising effect.

Description

Technical Field

[0001] The present disclosure relates to the technical field of image processing, and in particular to an image denoising method, an image denoising device, a computer-readable medium and an electronic device. Background Art

[0002] In the field of image denoising, some solutions use graphics with different exposure effects to remove image noise, and use exposure length to control the detail expression of bright and dark areas. However, such technical solutions will inevitably cause residual noise in the fused image when performing image fusion.

[0003] It should be noted that the information disclosed in the above background technology section is only used to enhance the understanding of the background of the present disclosure, and therefore may include information that does not constitute the prior art known to ordinary technicians in the field. Summary of the invention

[0004] The present invention provides an image denoising method, an image denoising device, a computer-readable medium and an electronic device, which can effectively eliminate exposure image noise, ensure the denoising effect of the image, and avoid the loss of image details.

[0005] Other features and advantages of the present disclosure will become apparent from the following detailed description, or may be learned in part by the practice of the present disclosure.

[0006] According to a first aspect of the present disclosure, there is provided an image denoising method, comprising:

[0007] Acquire a current image and a corresponding reference image under preset exposure conditions, and determine a first image parameter of the current image;

[0008] According to the preset exposure condition, query the noise parameter lookup table to obtain the corresponding second image parameter;

[0009] Comparing the first image parameter with the second image parameter under the preset exposure condition, performing a first denoising process on the current image according to the image parameter comparison result to obtain a preliminary denoised image under the preset exposure condition, and generating a denoising degree parameter according to the denoising result of the current image;

[0010] Performing a second fusion denoising process on the preliminary denoised image under the preset exposure condition and the corresponding reference image based on the denoising degree parameter to obtain a fused image;

[0011] The fused images under different preset exposure conditions are subjected to a third fusion denoising process to obtain a denoised image corresponding to the current image.

[0012] According to a second aspect of the present disclosure, there is provided an image denoising device, comprising:

[0013] An image acquisition module, used to acquire a current image and a corresponding reference image under preset exposure conditions, and determine first image parameters of the current image and the reference image; wherein the preset exposure conditions include at least two different exposure conditions;

[0014] A noise parameter lookup table query module, used to query the noise parameter lookup table according to the preset exposure condition to obtain the corresponding second image parameter;

[0015] a first denoising processing module, configured to compare the first image parameter with the second image parameter under the preset exposure condition, to perform a first denoising process on the current image according to the image parameter comparison result to obtain a preliminary denoised image under the preset exposure condition, and to generate a denoising degree parameter according to the denoising result of the current image;

[0016] A second denoising processing module, configured to perform a second fusion denoising process on the preliminary denoised image under the preset exposure condition and the corresponding reference image based on the denoising degree parameter to obtain a fused image;

[0017] The third denoising processing module is used to perform a third fusion denoising process on the fused images under different preset exposure conditions to obtain a denoised image corresponding to the current image.

[0018] According to a third aspect of the present disclosure, a computer-readable medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the above-mentioned image denoising method is implemented.

[0019] According to a fourth aspect of the present disclosure, there is provided an electronic device, including:

[0020] one or more processors;

[0021] The storage device is used to store one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors implement the above-mentioned image denoising method.

[0022] An image denoising method provided by an embodiment of the present disclosure uses a noise parameter lookup table to obtain a second image parameter, so that the first image parameter and the second image parameter of the current image can be compared first, and a first denoising process can be performed based on the comparison result to eliminate the noise under the exposure environment and avoid the loss of image details, thereby obtaining a preliminary denoised image and a denoising degree parameter; the denoising degree parameter is then used to guide the preliminary denoised image and the reference image to perform image fusion, and a second denoising process is performed to eliminate random noise. Finally, the fused images under different exposure conditions are fused, and a third denoising process is performed to generate a denoised high dynamic range image.

[0023] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] The accompanying drawings herein are incorporated into the specification and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the specification are used to explain the principles of the present disclosure. Obviously, the accompanying drawings described below are only some embodiments of the present disclosure, and for ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without creative work.

[0025] Figure 1 A schematic diagram schematically illustrates an image denoising method in an exemplary embodiment of the present disclosure;

[0026] Figure 2 A schematic diagram schematically illustrates a method for constructing a noise parameter lookup table in an exemplary embodiment of the present disclosure;

[0027] Figure 3 A gray gradient image in an exemplary embodiment of the present disclosure is schematically shown;

[0028] Figure 4 Schematically showing a gradient grayscale grid image of a grid size in an exemplary embodiment of the present disclosure;

[0029] Figure 5 Schematically showing a gradient grayscale grid pattern image of another grid size in an exemplary embodiment of the present disclosure;

[0030] Figure 6 Schematically showing a gradient grayscale grid pattern image of another grid size in an exemplary embodiment of the present disclosure;

[0031] Figure 7 A schematic diagram schematically illustrates a flow chart of an image denoising method in an exemplary embodiment of the present disclosure;

[0032] Figure 8A schematic diagram schematically shows the composition of an image denoising device in an exemplary embodiment of the present disclosure;

[0033] Fig. 9 The structure diagram of an electronic device in an exemplary embodiment of the present disclosure is schematically shown. DETAILED DESCRIPTION

[0034] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in a variety of forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that the disclosure will be more comprehensive and complete and to fully convey the concepts of the example embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

[0035] In addition, the accompanying drawings are only schematic illustrations of the present disclosure and are not necessarily drawn to scale. The same reference numerals in the figures represent the same or similar parts, and their repeated description will be omitted. Some of the block diagrams shown in the accompanying drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities can be implemented in software form, or implemented in one or more hardware modules or integrated circuits, or implemented in different networks and / or processor devices and / or microcontroller devices.

[0036] In related technologies, when using multi-frame exposure image fusion denoising solutions, the detail expression of bright and dark areas is controlled according to the exposure length. In theory, long-exposure frame images have low noise in dark areas and overexposure in bright areas; while short-exposure frames have rich details in bright areas and high noise in dark areas; but in the image fusion process, it is inevitable that the dark area of ​​the short-exposure frame image and the bright area of ​​the long-exposure frame image will be selected for fusion, which will cause the problem of residual noise in the fused image.

[0037] In view of the above-mentioned shortcomings and deficiencies of the prior art, this exemplary embodiment provides an image denoising method, which performs denoising before fusing multiple frames of exposure images, thereby avoiding the problem of noise in the fused image. Figure 1 As shown in , the above-mentioned image denoising method may include the following steps:

[0038] S11, acquiring a current image and a corresponding reference image under a preset exposure condition, and determining a first image parameter of the current image;

[0039] S12, querying a noise parameter lookup table according to the preset exposure condition to obtain a corresponding second image parameter;

[0040] S13, comparing the first image parameter with the second image parameter under the preset exposure condition, performing a first denoising process on the current image according to the image parameter comparison result to obtain a preliminary denoised image under the preset exposure condition, and generating a denoising degree parameter according to the denoising result of the current image;

[0041] S14, performing a second fusion denoising process on the preliminary denoised image under the preset exposure condition and the corresponding reference image based on the denoising degree parameter to obtain a fused image;

[0042] S15, performing a third fusion denoising process on the fused images under different preset exposure conditions to obtain a denoised image corresponding to the current image.

[0043] In the image denoising method provided in this example implementation, on the one hand, the second image parameter is obtained by using a method of looking up a noise parameter lookup table, so that the first image parameter and the second image parameter of the current image can be compared first, and the first denoising process is performed according to the comparison result to remove the noise generated in the exposure environment, avoid the loss of image details, and obtain a preliminary denoised image and a denoising degree parameter; on the other hand, the denoising degree parameter is used to guide the preliminary denoised image and the reference image to perform image fusion, perform a second denoising process, and eliminate random noise. On the other hand, the fused images under different exposure conditions are finally fused, and a third denoising process is performed to generate a denoised high dynamic range image.

[0044] Below, each step of the image denoising method in this example implementation will be described in more detail with reference to the accompanying drawings and embodiments.

[0045] In this example implementation, for example, the above method can be applied to the server side, and the user can upload the captured image data, or the image frame data after the video data is decomposed, to the server side through the terminal device, so that the server side responds to the received image data and performs image denoising calculation. Alternatively, the above method can also be applied to smart terminal devices with the same computing power, such as smart terminals such as mobile phones, tablets or computers. After the user selects an existing frame of image, or takes a frame of image, the image denoising calculation can be started.

[0046] In step S11, a current image and a corresponding reference image under a preset exposure condition are acquired, and a first image parameter of the current image is determined.

[0047] In this example implementation, the above method is applied to a terminal device with a shooting function, such as a mobile phone equipped with a camera. Among them, the above preset exposure conditions may include: short exposure conditions, normal exposure conditions and overexposure conditions. For example, the above short exposure conditions, normal exposure conditions and overexposure conditions can be configured by controlling the exposure duration. For example, the overexposure duration can be 40ms, the normal exposure duration can be 20ms, and the short exposure duration can be 10ms. Alternatively, when the user uses the camera function to shoot an image, the corresponding overexposure duration and short exposure duration can be configured according to the normal exposure duration of the current scene. For example, in a night scene model, a portrait mode, or a general mode, different overexposure durations and short exposure durations can be configured according to the current normal exposure duration, that is, in different shooting modes or shooting scenes, different short exposure durations, normal exposure durations and overexposure durations can be configured for shooting modes and / or shooting scenes.

[0048] For example, when a user uses a camera to take photos, the image content currently captured by the lens can be cached in real time. For example, three consecutive frames of images can be cached; for each cached frame of images, the corresponding short exposure image, normal exposure image, and overexposure image can be obtained according to the preset short exposure condition, normal exposure condition, and overexposure condition. That is, for the image content in the preview interface, the short exposure condition, normal exposure condition, and overexposure condition are used to shoot respectively to obtain the corresponding short exposure image, normal exposure image, and overexposure image. That is, the three consecutive cached images correspond to nine images. Alternatively, only the normal exposure image corresponding to the normal exposure condition can be stored in the cache, and then the normal exposure image is calculated based on the short exposure condition and the overexposure condition to obtain the corresponding short exposure image and overexposure image.

[0049] When the user clicks the shutter to shoot and obtain the current image, the short exposure condition, normal exposure condition and overexposure condition are used to shoot at the same time. When the normally exposed current image is obtained, the corresponding short exposure image and overexposure image are also obtained. At the same time, the adjacent frame image before the current image can be used as a reference image. In some embodiments, only the images of the current image under three exposure conditions can be obtained. In other embodiments, the same short exposure condition, normal exposure condition and overexposure condition as the current image can be used at the same time to obtain a normally exposed reference image, a short exposure reference image and an overexposure reference image.

[0050] Alternatively, the current image may be a frame of an image in a video, and the reference image is a frame of an image before the current image, and the reference image is continuous with the current image. In this case, the previous reference image has been calculated once, and the reference image is the previous current image, and its image under short exposure or over exposure conditions can be cached.

[0051] Among them, the above-mentioned first image parameter can be an image variance parameter. After obtaining the current image, and the corresponding short-exposure image and over-exposure image; the reference image, and the corresponding short-exposure reference image and over-exposure reference image, the image variance corresponding to each image can be calculated. Among them, the variance of the image can be the average variance calculated based on the grayscale value of each pixel in the image; for example, variance = the grayscale value of each pixel minus the sum of the squares of the average grayscale value of the image divided by the total number of pixels. The first image parameter is used to represent the variance value corresponding to the grayscale value of the pixel in the current image under normal exposure conditions. The same is true for images under other exposure conditions.

[0052] Alternatively, in some embodiments, only the current image and the image variances corresponding to the corresponding short-exposure image and over-exposure image may be calculated. Alternatively, if the current image is the first frame of the video, the current image itself may be used as a reference image.

[0053] In step S12, a noise parameter lookup table is queried according to the preset exposure condition to obtain corresponding second image parameters.

[0054] In this example implementation, when acquiring the current image, the noise parameter lookup table can be queried according to three different exposure conditions to determine the second image parameters corresponding to the three different exposure conditions. The second image parameters can be image variance values ​​corresponding to grayscale values ​​under the three exposure conditions.

[0055] In this example implementation, specifically, the method may further include: pre-building the noise parameter lookup table. Specifically, the image parameters of the image corresponding to the target grayscale condition may be collected under preset exposure conditions, and the noise parameter lookup table may be built according to the statistical results of the image parameters.

[0056] As shown in reference image 2, constructing a noise parameter lookup table may specifically include:

[0057] Step S21, configuring any one or a combination of any multiple of a basic image sequence, a grayscale gradient image sequence, and a gradient grayscale checkered image sequence; wherein the basic image sequence includes a plurality of grayscale pure color basic images that change according to a preset grayscale interval, the grayscale gradient image sequence includes a plurality of grayscale gradient images with different numbers of grayscales, and the gradient grayscale checkered image sequence includes a plurality of frames of gradient grayscale checkered images with different grid sizes;

[0058] Step S22, collecting exposure images corresponding to the basic image, the grayscale gradient image, and the gradient grayscale checkered image under the preset exposure conditions, and calculating image parameters corresponding to each of the exposure images; wherein the image parameters include image mean parameters and image variance parameters;

[0059] Step S23: constructing the noise parameter lookup table based on the image parameters of each exposure image under the preset exposure condition.

[0060] For example, a first test scene can be set. For the above basic image sequence, the grayscale pure color image with a grayscale interval of 8 can be configured based on the image sequence. For example, 32 groups of grayscale pure color images can be configured. The grayscale value of the first group of grayscale pure color images is 0, and the grayscale value of the second group of grayscale pure color images is 8. The grayscale value interval of each group of images is 8. For short exposure conditions, normal exposure conditions, and overexposure conditions, 32 groups of images are collected under three different exposure conditions. 32 groups of grayscale pure color images, calculate the overall mean and variance of the image. Through the pure color grayscale images under different exposure conditions, the noise characteristics of different areas under different exposure conditions can be obtained.

[0061] Set the second test scene, for the above grayscale gradient image sequence, for example Figure 3 The grayscale gradient image shown in the figure can be configured with 2, 4, 8, 16, 32, and 64 gradient grayscale images. Under short exposure conditions, normal exposure conditions, and overexposure conditions, 6 images under three different exposure conditions are collected. 3 groups of images, and calculate the mean and variance of each image.

[0062] Set the third test scene, for the above-mentioned gradient grayscale grid image sequence, where grayscale grid images of different sizes can be configured, refer to Figure 4 , Figure 5 , Figure 6 As shown, for example, three grid sizes are configured: large, medium, and small. For example, the large grid size is 512 512, medium grid size is 256 256, small grid size is 128 128. The grayscale gradient rule can be 8, 16, or 32 steps. Alternatively, the grayscale of the grid can be randomly gradient. Under three different exposure conditions, grayscale grid images of three grid sizes are collected, and then the grayscale mean and variance of each image are calculated respectively.

[0063] Combining the image data of the above three types of images under three exposure conditions, the variance and mean corresponding to different grayscales are obtained. Then interpolation and averaging operations are performed to obtain the mean and variance of different grayscales from 0 to 255 under three exposure conditions. Through various forms of images, under three different exposure conditions, noise information under different exposure conditions can be obtained. Since the brightness of adjacent grayscales may affect the noise, different test scenes and a large number of sample images can be used for testing to obtain the noise characteristics under different exposure conditions.

[0064] Based on the mean and variance under the three exposure conditions calculated above, a noise parameter lookup table LUT can be constructed, which can also be called a noise model, to reflect the noise morphology under the three exposure conditions. In this noise model, a total of 256 noise parameters under long exposure, normal exposure and short exposure conditions are included. 2 items, total 3 256 2 items of data. Among them, 256 corresponds to the grayscale value, 2 corresponds to the variance value and mean, and 3 corresponds to three different exposure conditions.

[0065] In step S13, the first image parameter and the second image parameter under the preset exposure condition are compared, so as to perform a first denoising process on the current image according to the image parameter comparison result to obtain a preliminary denoised image under the preset exposure condition, and generate a denoising degree parameter according to the denoising result of the current image.

[0066] In this example implementation, under three exposure conditions, three noise variance maps, i.e., second image parameters, are obtained according to the variances under different grayscale conditions. According to the same calculation scale, the current image, as well as the corresponding short exposure image and long exposure image, and the corresponding image variances, i.e., first image parameters, can be calculated. Alternatively, in some embodiments, the second image parameter may also be a noise variance map composed of corresponding variance values ​​obtained by querying a noise parameter lookup table according to the grayscale value of each pixel in the current image under normal exposure conditions.

[0067] Under normal exposure conditions, the variance value of the first image parameter of the current image is compared pixel by pixel with the variance value of the second image parameter under normal exposure conditions, and spatial denoising is performed based on the comparison results of the variance values. For example, spatial denoising can be performed using a non-local flat denoising algorithm, a Gaussian filter, or a Laplacian pyramid filter, and the variance values ​​can be compared pixel by pixel. If the variance value of the current image is greater than the variance value of the noise model, it is considered to be a detail, and the degree of spatial filtering of the pixel is weakened. For example, the default user sets the spatial denoising intensity parameter to D1, and the variance difference between the variance calculated by comparing the current image pixel by pixel and the variance of the LUT lookup table is varDiff. The maximum value is set to varDiffMax based on the empirical value, and varDiff∈[0,varDiffMax]. Normalization degreePara=varDiff / varDiffMax, then the degree of weakening is D2=d1 (1-degreePara). The short-exposure image and the long-exposure image corresponding to the current image are calculated using the same method.

[0068] At the same time, when performing the first spatial denoising, the degree of spatial denoising is recorded pixel by pixel, normalized, and recorded as the spatial denoising degree map, that is, the denoising degree parameter.

[0069] In step S14, a second fusion denoising process is performed on the preliminary denoised image under the preset exposure condition and the corresponding reference image based on the denoising degree parameter to obtain a fused image.

[0070] In this example implementation, after the current image, short exposure image and long exposure image corresponding to normal exposure conditions, short exposure conditions and long exposure conditions are completely subjected to the first spatial denoising process to obtain preliminary denoised images under the three exposure conditions, they can be subjected to temporal denoising process together with the reference image under the same exposure condition to achieve the second fusion denoising process.

[0071] Specifically, for normal exposure conditions, the preliminary denoised image corresponding to the current image and the corresponding reference image are fused, and the denoising degree parameter under normal exposure conditions, that is, the spatial denoising degree map, is used to guide the inter-frame fusion ratio. For short exposure conditions, the preliminary denoised image corresponding to the short exposure image of the current image and the corresponding reference image under short exposure conditions are subjected to time domain denoising, and the spatial denoising degree map under short exposure conditions is used to guide the inter-frame fusion ratio. For long exposure conditions, the preliminary denoised image of the long exposure image of the current image and the corresponding reference image under long exposure conditions are subjected to time domain denoising, and the spatial denoising degree map under long exposure conditions is used to guide the inter-frame fusion ratio. The formula for guiding the inter-frame fusion ratio of time domain denoising using the spatial denoising degree map may include:

[0072] dTnr = (1-dSnr) paraAdjust

[0073] Among them, dSnr represents the degree of spatial denoising; dTnr represents the degree of temporal denoising; paraAdjust can be a user-set value or a default value, which is used to adjust the overall denoising degree of temporal denoising.

[0074] According to the above formula, the inter-frame fusion ratio of temporal denoising, that is, the degree of temporal denoising, can be calculated according to the above denoising degree parameter. By using this formula, when the denoising degree parameters of the three exposure conditions are obtained, the inter-frame fusion ratio of temporal denoising under the three exposure conditions can be determined.

[0075] In some embodiments, the reference image for performing time-domain fusion described above may be the original images acquired under three exposure conditions; alternatively, it may also be an image after spatial denoising. Perform time-domain denoising on the image after spatial denoising, use consecutive frame images for fusion denoising, and use the denoising degree parameter to guide the inter-frame fusion ratio. For the pixel points in the image, when the degree of spatial denoising is weak, the degree of time-domain denoising can be enhanced. Thus, random noise between frames can be effectively removed. For example, time-domain denoising may be to average the pixels whose pixel gray value difference between the current image and the reference image is less than a certain threshold as the gray value of the current image frame; or, it may be an algorithm that retains the larger value of the pixel gray value between the current image and the reference image as the gray value of the current image frame. Or, time-domain denoising can be performed based on a Gaussian pyramid filter.

[0076] In step S15, perform a third fusion denoising process on the fusion images under different preset exposure conditions to obtain the denoised image corresponding to the current image.

[0077] In this exemplary embodiment, after obtaining the three fusion images after time-domain denoising under short exposure conditions, normal exposure conditions, and overexposure conditions, the three fusion images can be subjected to high-dynamic range image fusion processing to complete the third fusion denoising process and achieve complete denoising of the current image. The fused denoised high-dynamic range image is generated, retaining the detailed information in the high region of the short exposure image and the detailed information in the low dark region of the long exposure image.

[0078] For example, the third fusion denoising process may include: configuring the first gray threshold thd1 = 64 and the second gray threshold thd2 = 196. For each pixel point in the image, let the gray value lumMap = Y. If lumMap < thd1, then configure the weight value mask = 0; if lumMap ≥ thd1 and lumMap < thd2, then configure the weight value mask = 1; if lumMap ≥ thd2 and lumMap < 255, then configure the weight value as mask = 2. Then the output image outImg of the high-dynamic range image fusion process = shortExpImg (mask == 0)+idExpImg (mask == 1)+longExpImg (mask == 2); where shortExpImg is the short exposure image, idExpImg is the normal exposure image, and longExpImg is the long exposure image.

[0079] The image denoising method provided in the embodiment of the present disclosure tests various grayscale noise images under different exposure levels in advance, summarizes the noise characteristics under each exposure level, constructs a noise parameter table in the form of a noise model, provides the noise fluctuation range under each exposure condition, and clarifies the noise characteristics. Figure 7 As shown in the figure, under normal exposure conditions, short exposure (underexposure) conditions, and long exposure (overexposure) conditions, images are collected, and the first-level spatial denoising degree is guided by the noise model to eliminate the noise of each exposure image, avoid detail loss, and obtain the spatial denoising degree map. Then, the spatial denoising degree map is used to guide the second-level temporal denoising fusion of each exposure image under equal exposure conditions, and the purpose of eliminating random noise is achieved by superimposing the processed frame after spatial denoising and the reference frame. Finally, the third-level image fusion under each exposure condition is performed to generate a denoised high dynamic range image. Through the logic of three-level denoising, spatial denoising and temporal denoising are combined, and the degree of the two denoising methods is coordinated to reduce the noise in the fused image. The denoising-processed image fused with the high dynamic range image has higher quality and can show more image details.

[0080] It should be noted that the above figures are only schematic illustrations of the processes included in the method according to an exemplary embodiment of the present invention, and are not intended to be limiting. It is easy to understand that the processes shown in the above figures do not indicate or limit the time sequence of these processes. In addition, it is also easy to understand that these processes can be performed synchronously or asynchronously, for example, in multiple modules.

[0081] For further reference, Figure 8 As shown, in the embodiment of this example, an image denoising device 80 is also provided, comprising: an image acquisition module 801, a noise parameter lookup table query module 802, a first denoising processing module 803, a second denoising processing module 804 and a third denoising processing module 805.

[0082] The image acquisition module 801 can be used to obtain a current image and a corresponding reference image under preset exposure conditions, and determine first image parameters of the current image and the reference image; wherein the preset exposure conditions include at least two different exposure conditions.

[0083] The noise parameter lookup table query module 802 may be configured to query a noise parameter lookup table according to the preset exposure condition to obtain a corresponding second image parameter.

[0084] The first denoising processing module 803 can be used to compare the first image parameter with the second image parameter under the preset exposure condition, to perform a first denoising process on the current image according to the image parameter comparison result to obtain a preliminary denoised image under the preset exposure condition, and to generate a denoising degree parameter according to the denoising result of the current image.

[0085] The second denoising processing module 804 may be configured to perform a second fusion denoising process on the preliminary denoised image under the preset exposure condition and the corresponding reference image based on the denoising degree parameter to obtain a fused image.

[0086] The third denoising processing module 805 may be used to perform a third fusion denoising process on the fused images under different preset exposure conditions to obtain a denoised image corresponding to the current image.

[0087] In an example of the present disclosure, the preset exposure conditions include: a short exposure condition, a normal exposure condition, and an overexposure condition.

[0088] In an example of the present disclosure, the first image parameter and the second image parameter include image variance parameters;

[0089] The first denoising processing module 803 can be used to compare the first image parameter with the second image parameter under the preset exposure condition to obtain an image variance difference, and guide the current image to perform spatial denoising processing according to the image variance difference and obtain a preliminary denoised image.

[0090] In an example of the present disclosure, the first denoising processing module 803 can be used to determine the corresponding denoising strength according to the variance difference between the first image parameter and the second image parameter of each pixel in the current image, and construct the denoising degree parameter based on the denoising strength.

[0091] In an example of the present disclosure, the second denoising processing module 804 can be used to guide the fusion ratio between the preliminary denoised image and the corresponding reference image based on the above-mentioned denoising degree parameter, and perform time domain denoising on the preliminary denoised image to obtain the fused image.

[0092] In an example of the present disclosure, the device 80 may include a noise parameter lookup table building module.

[0093] The noise parameter lookup table construction module can be used to pre-construct the noise parameter lookup table, including: under preset exposure conditions, collecting image parameters of an image corresponding to a target grayscale condition, and constructing the noise parameter lookup table according to statistical results of the image parameters.

[0094] In an example of the present disclosure, the noise parameter lookup table construction module may include: configuring any one or a combination of any multiple of a basic image sequence, a grayscale gradient image sequence, and a gradient grayscale plaid image sequence; wherein the basic image sequence includes a plurality of grayscale pure color basic images that change at a preset grayscale interval, the grayscale gradient image sequence includes a plurality of grayscale gradient images with different numbers of grayscales, and the gradient grayscale plaid image sequence includes a plurality of frames of gradient grayscale plaid images with different grid sizes; collecting exposure images corresponding to the basic image, the grayscale gradient image, and the gradient grayscale plaid image under the preset exposure conditions, and calculating image parameters corresponding to each of the exposure images; wherein the image parameters include image mean parameters and image variance parameters; and constructing the noise parameter lookup table based on the image parameters of each exposure image under the preset exposure conditions.

[0095] The specific details of each module in the above-mentioned image denoising device have been described in detail in the corresponding image denoising method, so they will not be repeated here.

[0096] It should be noted that, although several modules or units of the device for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiments of the present disclosure, the features and functions of two or more modules or units described above can be embodied in one module or unit. On the contrary, the features and functions of one module or unit described above can be further divided into multiple modules or units to be embodied.

[0097] Fig. 9 A schematic diagram of an electronic device suitable for implementing an embodiment of the present invention is shown.

[0098] It should be noted that Fig. 9 The electronic device 500 shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0099] like Fig. 9 As shown, the electronic device 500 includes a central processing unit (CPU) 501, which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM) 502 or the program loaded from the storage part 508 to the random access memory (RAM) 503. Various programs and data required for system operation are also stored in the RAM 503. The CPU 501, the ROM 502 and the RAM 503 are connected to each other through a bus 504. The input / output (I / O) interface 505 is also connected to the bus 504.

[0100] The following components are connected to the I / O interface 505: an input section 506 including a keyboard, a mouse, etc.; an output section 507 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker; a storage section 508 including a hard disk, etc.; and a communication section 509 including a network interface card such as a LAN (Local Area Network) card, a modem, etc. The communication section 509 performs communication processing via a network such as the Internet. A drive 510 is also connected to the I / O interface 505 as needed. A removable medium 511, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 510 as needed so that a computer program read therefrom is installed into the storage section 508 as needed.

[0101] In particular, according to an embodiment of the present invention, the process described below with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present invention includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program contains program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication part 509, and / or installed from a removable medium 511. When the computer program is executed by a central processing unit (CPU) 501, various functions defined in the system of the present application are executed.

[0102] Specifically, the electronic device may be a smart mobile terminal device such as a mobile phone, a tablet computer or a laptop computer, or may be a smart terminal device such as a desktop computer.

[0103] It should be noted that the computer-readable medium shown in the embodiment of the present invention may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a flash memory, an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present invention, a computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, device or device. In the present invention, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, which may send, propagate, or transmit programs for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0104] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present invention. In this regard, each box in the flow chart or block diagram can represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0105] The units involved in the embodiments of the present invention may be implemented by software or hardware, and the units described may also be arranged in a processor. The names of these units do not, in some cases, limit the units themselves.

[0106] It should be noted that, as another aspect, the present application also provides a computer-readable medium, which may be included in the electronic device described in the above embodiments; or it may exist independently without being installed in the electronic device. The above computer-readable medium carries one or more programs, and when the above one or more programs are executed by an electronic device, the electronic device implements the method described in the following embodiments. For example, the electronic device may implement the following Figure 1 or Figure 2 The steps shown.

[0107] In addition, the above-mentioned figures are only schematic illustrations of the processes included in the method according to an exemplary embodiment of the present invention, and are not intended to be limiting. It is easy to understand that the processes shown in the above-mentioned figures do not indicate or limit the time sequence of these processes. In addition, it is also easy to understand that these processes can be performed synchronously or asynchronously, for example, in multiple modules.

[0108] In addition, the above-mentioned figures are only schematic illustrations of the processes included in the method according to an exemplary embodiment of the present invention, and are not intended to be limiting. It is easy to understand that the processes shown in the above-mentioned figures do not indicate or limit the time sequence of these processes. In addition, it is also easy to understand that these processes can be performed synchronously or asynchronously, for example, in multiple modules.

[0109] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary technical means in the art that are not disclosed in the present disclosure. The specification and examples are to be considered as exemplary only, and the true scope and spirit of the present disclosure are indicated by the claims.

[0110] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. An image denoising method, characterized in that: include: Acquire a current image and a corresponding reference image under preset exposure conditions, and determine a first image parameter of the current image; The reference image is an image before the current image; According to the preset exposure conditions, a noise parameter lookup table is queried to obtain corresponding second image parameters; wherein, under each of the preset exposure conditions, the second image parameters are three noise variance maps obtained according to the variance under different grayscale conditions; the preset exposure conditions include: short exposure condition, normal exposure condition and overexposure condition; Comparing the first image parameter with the second image parameter under the preset exposure condition, performing a first denoising process on the current image according to the image parameter comparison result to obtain a preliminary denoised image under the preset exposure condition, and generating a denoising degree parameter according to the denoising result of the current image; Performing a second fusion denoising process on the preliminary denoised image under the preset exposure condition and the corresponding reference image based on the denoising degree parameter to obtain a fused image; Performing a third fusion denoising process on the fused images under different preset exposure conditions to obtain a denoised image corresponding to the current image; The first image parameter and the second image parameter include image variance parameters; The comparing the first image parameter with the second image parameter under the preset exposure condition to perform a first denoising process on the current image according to the image parameter comparison result to obtain a preliminary denoised image under the preset exposure condition includes: The first image parameter and the second image parameter under the preset exposure condition are compared to obtain an image variance difference, and according to the image variance difference, the current image is guided to perform spatial domain denoising processing and obtain a preliminary denoised image.

2. The image denoising method according to claim 1, characterized in that: The generating a denoising degree parameter according to the denoising result of the current image includes: The corresponding denoising intensity is determined according to the variance difference between the first image parameter and the second image parameter of each pixel in the current image, and the denoising degree parameter is constructed based on the denoising intensity.

3. The image denoising method according to claim 1, characterized in that: The performing a second fusion denoising process on the preliminary denoised image under the preset exposure condition and the corresponding reference image based on the denoising degree parameter to obtain a fused image includes: Based on the denoising degree parameter, the fusion ratio between the preliminary denoised image and the corresponding reference image is guided, and the preliminary denoised image is subjected to time domain denoising to obtain the fused image.

4. The image denoising method according to claim 1, characterized in that: The method further includes: pre-building the noise parameter lookup table, including: Under preset exposure conditions, image parameters of an image corresponding to a target grayscale condition are collected, and the noise parameter lookup table is constructed according to statistical results of the image parameters.

5. The image denoising method according to claim 4, characterized in that: The method of collecting image parameters of an image corresponding to a target grayscale condition under a preset exposure condition and constructing the noise parameter lookup table according to statistical results of the image parameters includes: Configure any one or any combination of a basic image sequence, a grayscale gradient image sequence, and a gradient grayscale checkered image sequence; wherein the basic image sequence includes a plurality of grayscale pure color basic images that change at preset grayscale intervals, the grayscale gradient image sequence includes a plurality of grayscale gradient images with different numbers of grayscales, and the gradient grayscale checkered image sequence includes a plurality of frames of gradient grayscale checkered images with different grid sizes; Under the preset exposure conditions, exposure images corresponding to the basic image, the grayscale gradient image, and the gradient grayscale checkered image are collected, and image parameters corresponding to each of the exposure images are calculated; wherein the image parameters include image mean parameters and image variance parameters; The noise parameter lookup table is constructed based on the image parameters of each exposure image under the preset exposure condition.

6. An image denoising device, characterized in that: include: An image acquisition module, used to acquire a current image and a corresponding reference image under a preset exposure condition, and determine first image parameters of the current image and the reference image; wherein the preset exposure condition includes at least two different exposure conditions; and the reference image is an image before the current image; A noise parameter lookup table query module, used to query the noise parameter lookup table according to the preset exposure conditions to obtain corresponding second image parameters; wherein, under each of the preset exposure conditions, the second image parameters are three noise variance maps obtained according to the variance under different grayscale conditions; the preset exposure conditions include: short exposure condition, normal exposure condition and overexposure condition; a first denoising processing module, configured to compare the first image parameter with the second image parameter under the preset exposure condition, to perform a first denoising process on the current image according to the image parameter comparison result to obtain a preliminary denoised image under the preset exposure condition, and to generate a denoising degree parameter according to the denoising result of the current image; A second denoising processing module, configured to perform a second fusion denoising process on the preliminary denoised image under the preset exposure condition and the corresponding reference image based on the denoising degree parameter to obtain a fused image; A third denoising processing module, used for performing a third fusion denoising process on the fused images under different preset exposure conditions to obtain a denoised image corresponding to the current image; The first image parameter and the second image parameter include image variance parameters; The comparing the first image parameter with the second image parameter under the preset exposure condition to perform a first denoising process on the current image according to the image parameter comparison result to obtain a preliminary denoised image under the preset exposure condition is configured as follows: The first image parameter and the second image parameter under the preset exposure condition are compared to obtain an image variance difference, and according to the image variance difference, the current image is guided to perform spatial domain denoising processing and obtain a preliminary denoised image.

7. A computer readable medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the image denoising method according to any one of claims 1 to 5 is implemented.

8. An electronic device, characterized in that: include: one or more processors; A storage device for storing one or more programs, when the one or more programs are executed by the one or more processors, the one or more processors implement the image denoising method according to any one of claims 1 to 5.

Images