A method, apparatus and readable storage medium for removing image artifacts

Abstract

The present disclosure provides a method, device and readable storage medium for removing image artifacts, applied to the field of image processing, comprising: acquiring N long-exposure images collected by a camera; calculating the probability of each pixel point in each non-reference frame image being located in an artifact region according to the noise variance coefficient of the camera and the N long-exposure images; calculating the weight of the corresponding pixel point in the corresponding non-reference frame image according to the probability of each pixel point in each non-reference frame image being located in the artifact region; and synthesizing a weighted image of the N long-exposure images according to the weight of each pixel point in N-1 non-reference frame images. In the present disclosure, the noise variance coefficient of the camera is set according to the performance of different cameras, and the probability of each pixel point in each non-reference frame image being located in the artifact region is calculated using the corresponding noise variance coefficient, thereby effectively removing the artifact region in the synthesized image and improving the picture quality effect.

Description

Technical Field

[0001] This disclosure relates to the field of image processing technology, and in particular to a method, apparatus and readable storage medium for removing image artifacts. Background Technology

[0002] In high dynamic range (HDR) image compositing, multiple long-exposure images with the same brightness are combined into a single image to improve image quality, increase detail, and reduce noise. However, long exposures are more prone to camera shake, causing the subject to shift or become blurred within the frame. These artifacts cannot be corrected through image registration, and directly involving these images in the fusion process can produce artifacts in the final composite image.

[0003] In the method of detecting artifact regions by image difference, the clearest frame is first selected as the reference frame from all frames. The reference frame is used to register other frames, and the difference between other frames and the reference frame is calculated. By setting a threshold, artifact regions are filtered out. If the difference is greater than this threshold, it is considered an artifact region. This region does not participate in subsequent image fusion, thereby mitigating artifacts to a certain extent.

[0004] The method for detecting artifact regions by image difference is highly sensitive to the threshold setting. If the threshold is set too low, flat areas without information are easily detected as artifact regions, and these regions will not participate in the fusion, affecting the multi-frame noise reduction effect. If the threshold is set too high, artifact regions with small differences will be missed, resulting in artifacts in the final image. Summary of the Invention

[0005] In view of this, the present disclosure provides a method, apparatus and readable storage medium for removing image artifacts.

[0006] According to a first aspect of the present disclosure, a method for removing image artifacts is provided, comprising:

[0007] Acquire N long-exposure images captured by the camera; the N long-exposure images include one reference frame image and N-1 non-reference frame images;

[0008] The probability of each pixel in each non-reference frame image being located in the artifact region is calculated based on the noise variance coefficient of the camera and the N long exposure images.

[0009] The weight of each pixel in the corresponding non-reference frame image is calculated based on the probability that each pixel is located in the artifact region.

[0010] The weighted image of the N long exposure images is synthesized based on the weight of each pixel in the N-1 non-reference frame images.

[0011] In one embodiment, calculating the probability that each pixel in each non-reference frame image is located in the artifact region based on the noise variance coefficient of the camera and the N long exposure images includes:

[0012] The noise characterization map corresponding to each non-reference frame image is determined based on the noise variance coefficient of the camera and each non-reference frame image.

[0013] The probability that each pixel in each non-reference frame image is located in the artifact region is calculated based on the noise characterization map and the reference frame image.

[0014] In one embodiment, calculating the probability that each pixel in each non-reference frame image is located in an artifact region based on the noise characterization map and the reference frame image includes:

[0015] Determine the difference map between each non-reference frame image and the reference frame image;

[0016] Based on the noise characterization map and the gap map, calculate the probability that each pixel in each non-reference frame image is located in the artifact region.

[0017] In one embodiment, determining the noise characterization map corresponding to each non-reference frame image based on the noise variance coefficient of the camera and each non-reference frame image includes:

[0018] The value of the corresponding pixel in the noise characterization map is determined based on the gray value of the pixel in each non-reference frame image and the noise variance coefficient.

[0019] In one embodiment, the noise variance coefficient includes a first coefficient and a second coefficient;

[0020] The step of determining the value of the corresponding pixel in the noise characterization map based on the grayscale value of the pixel in each non-reference frame image and the noise variance coefficient includes:

[0021] Calculate the product of the gray value of a pixel in the non-reference frame image and the first coefficient, and the sum of the product and the second coefficient;

[0022] The sum and are used as the values ​​of the corresponding pixels in the noise characterization diagram.

[0023] In one embodiment, calculating the probability that each pixel in the non-reference frame image is located in an artifact region based on the noise characterization map and the difference map includes:

[0024] The values ​​corresponding to the same pixel in the noise characterization map and the difference map are input into the probability density function, and the output value of the probability density function is used as the probability that the corresponding pixel in the non-reference frame image is located in the artifact region.

[0025] In one embodiment, calculating the probability that each pixel in the non-reference frame image is located in an artifact region based on the noise characterization map and the difference map further includes:

[0026] Input the values ​​corresponding to the same pixel in the noise characterization map and the difference map into the probability density function to determine the output value of the probability density function;

[0027] When the output value is greater than or equal to a preset probability, the difference between the output value and an adjustment value is used as the probability that the corresponding pixel in the non-reference frame image is located in the artifact region.

[0028] When the output value is less than a preset probability, the sum of the output value and an adjustment value is used as the probability that the corresponding pixel in the non-reference frame image is located in the artifact region.

[0029] The adjustment value is negatively correlated with the imaging quality of the camera.

[0030] In one embodiment, the probability of each pixel in each non-reference frame image being located in an artifact region is negatively correlated with the weight of the corresponding pixel in the corresponding non-reference frame image.

[0031] According to a second aspect of the present disclosure, an apparatus for removing image artifacts is provided, comprising:

[0032] The acquisition module is used to acquire N long exposure images captured by the camera; the N long exposure images include one reference frame image and N-1 non-reference frame images;

[0033] The first calculation module is used to calculate the probability that each pixel in each non-reference frame image is located in the artifact region based on the noise variance coefficient of the camera and the N long exposure images;

[0034] The second calculation module is used to calculate the weight of the corresponding pixel in the corresponding non-reference frame image based on the probability that each pixel in each non-reference frame image is located in the artifact region.

[0035] The synthesis module is used to synthesize a weighted image of the N long exposure images based on the weight of each pixel in the N-1 non-reference frame images.

[0036] In one embodiment, the first computing module includes:

[0037] The first determining module is used to determine the noise characterization map corresponding to each non-reference frame image based on the noise variance coefficient of the camera and each non-reference frame image;

[0038] The third calculation module calculates the probability that each pixel in each non-reference frame image is located in the artifact region based on the noise characterization map and the reference frame image.

[0039] In one embodiment, the third computing module further includes:

[0040] The second determining module is used to determine the difference map between each non-reference frame image and the reference frame image;

[0041] The fourth calculation module is further configured to calculate the probability that each pixel in each non-reference frame image is located in the artifact region based on the noise characterization map and the gap map.

[0042] In one embodiment, the first determining module is further configured to determine a noise characterization map corresponding to each non-reference frame image using the following method based on the noise variance coefficient of the camera and each non-reference frame image:

[0043] The value of the corresponding pixel in the noise characterization map is determined based on the gray value of the pixel in each non-reference frame image and the noise variance coefficient.

[0044] In one embodiment, the noise variance coefficient includes a first coefficient and a second coefficient;

[0045] The step of determining the value of the corresponding pixel in the noise characterization map based on the grayscale value of the pixel in each non-reference frame image and the noise variance coefficient includes:

[0046] Calculate the product of the gray value of a pixel in the non-reference frame image and the first coefficient, and the sum of the product and the second coefficient;

[0047] The sum and are used as the values ​​of the corresponding pixels in the noise characterization diagram.

[0048] In one embodiment, the fourth computing module includes:

[0049] The first function call module is used to input the values ​​corresponding to the same pixel in the noise characterization map and the difference map into the probability density function, and use the output value of the probability density function as the probability that the corresponding pixel in the non-reference frame image is located in the artifact region.

[0050] In one embodiment, the fourth computing module includes:

[0051] The second function call module is used to input the values ​​corresponding to the same pixel in the noise characterization map and the difference map into the probability density function, and determine the output value of the probability density function.

[0052] An adjustment module is used to use the difference between the output value and an adjustment value as the probability that the corresponding pixel in the non-reference frame image is located in the artifact region when the output value is greater than or equal to a preset probability.

[0053] When the output value is less than a preset probability, the sum of the output value and an adjustment value is used as the probability that the corresponding pixel in the non-reference frame image is located in the artifact region.

[0054] The adjustment value is negatively correlated with the imaging quality of the camera.

[0055] In one embodiment, the probability of each pixel in each non-reference frame image being located in an artifact region is negatively correlated with the weight of the corresponding pixel in the corresponding non-reference frame image.

[0056] This disclosure provides an apparatus for removing image artifacts, comprising:

[0057] processor;

[0058] Memory used to store processor-executable instructions;

[0059] The processor is configured to execute executable instructions in the memory to implement the steps of the method.

[0060] This disclosure provides a non-transitory computer-readable storage medium having executable instructions stored thereon, which, when executed by a processor, implement the steps of any of the methods described herein.

[0061] The technical solutions provided by the embodiments of this disclosure may include the following beneficial effects: setting the corresponding noise variance coefficient for the camera according to the performance of different cameras, using the corresponding noise variance coefficient to calculate the probability that each pixel in each non-reference frame image is located in the artifact region, thereby determining the weight of each pixel in each non-reference frame image, using the weight to weight the gray value of the same pixel to obtain a more accurate artifact region, thereby effectively removing the artifact region in the synthesized image and improving the image quality.

[0062] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0063] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure.

[0064] Figure 1 This is a flowchart of a method for removing image artifacts according to an example embodiment;

[0065] Figure 2 This is a flowchart of another method for removing image artifacts according to an example embodiment;

[0066] Figure 3This is a flowchart of another method for removing image artifacts according to an example embodiment;

[0067] Figure 4 This is a diagram illustrating the effect of removing image artifacts according to an example embodiment.

[0068] Figure 5 This is a structural diagram of an apparatus for removing image artifacts according to an example embodiment;

[0069] Figure 6 This is a structural diagram of an apparatus for removing image artifacts according to an example embodiment. Detailed Implementation

[0070] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with those in this disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of the embodiments in this disclosure as detailed in the appended claims.

[0071] The terminology used in this disclosure is for the purpose of describing particular embodiments only and is not intended to be limiting of the present disclosure. The singular forms “a,” “the,” and “the” as used in this disclosure and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any and all possible combinations of one or more of the associated listed items.

[0072] Considering the different imaging quality of different cameras (sensors), images captured by cameras with poor imaging quality have a low signal-to-noise ratio and more noise. When calculating the difference between the reference frame and the non-reference frame, the two images will have a large difference in the non-artifact area due to the noise, thus misjudging the non-artifact area as an artifact area.

[0073] This disclosure provides a method for removing image artifacts, applied to a mobile terminal. This mobile terminal has a camera and can be a mobile phone, tablet computer, smart device, etc.

[0074] Reference Figure 1 , Figure 1 This is a flowchart illustrating a method for removing image artifacts according to an exemplary embodiment. Figure 1 As shown, this method includes:

[0075] Step S11: Acquire N long exposure images captured by the camera; the N long exposure images include one reference frame image and N-1 non-reference frame images;

[0076] Step S12: Calculate the probability that each pixel in each non-reference frame image is located in the artifact region based on the noise variance coefficient of the camera and the N long exposure images;

[0077] Step S13: Calculate the weight of the corresponding pixel in the corresponding non-reference frame image based on the probability that each pixel in each non-reference frame image is located in the artifact region;

[0078] Step S14: Synthesize the weighted image of the N long exposure images according to the weight of each pixel in the N-1 non-reference frame images.

[0079] In one embodiment, before step S11, there is a step S10, which determines the noise variance coefficient of the camera. Specifically, this includes determining the noise variance coefficient of the camera through a noise calibration method, so that the noise variance coefficient reflects the imaging quality of the camera.

[0080] In one embodiment, the N long exposure images in step S11 are N long exposure images with the same brightness. These N long exposure images with the same brightness are images that have undergone image registration.

[0081] In this embodiment, a corresponding noise variance coefficient is set for the performance of different cameras. The probability of each pixel in each non-reference frame image being located in the artifact region is calculated using the corresponding noise variance coefficient, thereby determining the weight of each pixel in each non-reference frame image. The gray value of the same pixel is weighted using the weight to obtain a more accurate artifact region, thereby effectively removing the artifact region in the synthesized image and improving the image quality.

[0082] This disclosure provides a method for removing image artifacts, applied to a mobile terminal. (Refer to...) Figure 2 , Figure 2 This is a flowchart illustrating a method for removing image artifacts according to an exemplary embodiment. Figure 2 As shown, this method includes:

[0083] Step S11: Acquire N long exposure images captured by the camera; the N long exposure images include one reference frame image and N-1 non-reference frame images;

[0084] Step S12-1: Determine the noise characterization map corresponding to each non-reference frame image based on the noise variance coefficient of the camera and each non-reference frame image;

[0085] Step S12-2: Calculate the probability that each pixel in each non-reference frame image is located in the artifact region based on the noise characterization map and the reference frame image.

[0086] Step S13: Calculate the weight of the corresponding pixel in the corresponding non-reference frame image based on the probability that each pixel in each non-reference frame image is located in the artifact region;

[0087] Step S14: Synthesize the weighted image of the N long exposure images according to the weight of each pixel in the N-1 non-reference frame images.

[0088] In this embodiment of the disclosure, the noise variance coefficient of the camera is used to determine the noise characterization map corresponding to each non-reference frame image, which is used to represent the noise distribution in the non-reference frame image, thereby calculating the probability that each pixel in each non-reference frame image is located in the artifact region through the noise characterization map.

[0089] In one embodiment, step S12-1, determining the noise characterization map corresponding to each non-reference frame image based on the noise variance coefficient of the camera and each non-reference frame image, includes: determining the value of the corresponding pixel in the noise characterization map based on the gray value of the pixel in each non-reference frame image and the noise variance coefficient.

[0090] For example, the noise variance coefficient includes a first coefficient a and a second coefficient b.

[0091] Determining the value of the corresponding pixel in the noise characterization map based on the gray value of the pixel in each non-reference frame image and the noise variance coefficient includes: calculating the product of the gray value of the pixel in the non-reference frame image and the first coefficient, and the sum of the product and the second coefficient; and using the sum as the value of the corresponding pixel in the noise characterization map.

[0092] For example: the grayscale value of the pixel at position (x, y) in the i-th non-reference frame image is Y. i (x, y), the noise representation map corresponding to the i-th non-reference frame image is N. i (x,y), calculate the noise characterization map N according to formula (1). i The value of the corresponding pixel at (x, y):

[0093] N i (x,y)=a·Y i (x,y)+b (1)

[0094] In this embodiment, the noise variance value is represented by the linear transformation result of the gray values ​​in the pixels, and the noise variance value is used to construct a noise characterization map, thereby effectively reflecting the distribution of noise.

[0095] This disclosure provides a method for removing image artifacts, applied to a mobile terminal. (Refer to...) Figure 3 , Figure 3 This is a flowchart illustrating a method for removing image artifacts according to an exemplary embodiment. Figure 3 As shown, this method includes:

[0096] Step S11: Acquire N long exposure images captured by the camera; the N long exposure images include one reference frame image and N-1 non-reference frame images;

[0097] Step S12-1': Determine the noise characterization map corresponding to each non-reference frame image based on the noise variance coefficient of the camera; determine the difference map between each non-reference frame image and the reference frame image;

[0098] Step S12-2': Calculate the probability that each pixel in each non-reference frame image is located in the artifact region based on the noise characterization map and the gap map.

[0099] Step S13: Calculate the weight of the corresponding pixel in the corresponding non-reference frame image based on the probability that each pixel in each non-reference frame image is located in the artifact region;

[0100] Step S14: Synthesize the weighted image of the N long exposure images according to the weight of each pixel in the N-1 non-reference frame images.

[0101] The method for determining the noise characterization map corresponding to each non-reference frame image based on the noise variance coefficient of the camera in step S12-1' is the same as the method in step S12-1, and will not be described again here.

[0102] In one embodiment, determining the difference map between each non-reference frame image and the reference frame image in step S12-1' includes: calculating the difference in grayscale values ​​of the same pixels in each non-reference frame image and the reference frame image, for example: calculating the absolute value of the difference in grayscale values ​​of the same pixels in each non-reference frame image and the reference frame image.

[0103] For example: the grayscale value of the pixel at position (x, y) in the i-th non-reference frame image is Y. i (x, y), where the grayscale value of the pixel at position (x, y) in the reference frame image is Y. ref (x,y),

[0104] The difference diagram D between the i-th non-reference frame image and the reference frame image is calculated according to formula (2). i (x,y).

[0105] D i (x,y)=|Y ref (x,y)-Yi (x,y)| (2)

[0106] In one embodiment, step S12-2', calculating the probability that each pixel in the non-reference frame image is located in the artifact region based on the noise characterization map and the difference map, includes:

[0107] The values ​​corresponding to the same pixel in the noise characterization map and the difference map are input into the probability density function, and the output value of the probability density function is used as the probability that the corresponding pixel in the non-reference frame image is located in the artifact region.

[0108] For example: Calculate the probability P of the pixel at position (x, y) in the i-th non-reference frame image being located in the artifact region according to formula (3). i (x,y):

[0109] P i (x,y)=R(N i (x,y),D i (x,y)) (3)

[0110] Where R is the probability density function.

[0111] In one embodiment, step S12-2', calculating the probability that each pixel in the non-reference frame image is located in the artifact region based on the noise characterization map and the difference map, includes:

[0112] Input the values ​​corresponding to the same pixel in the noise characterization map and the difference map into the probability density function to determine the output value of the probability density function.

[0113] When the output value is greater than or equal to a preset probability, the difference between the output value and an adjustment value is used as the probability that the corresponding pixel in the non-reference frame image is located in the artifact region.

[0114] When the output value is less than a preset probability, the sum of the output value and an adjustment value is used as the probability that the corresponding pixel in the non-reference frame image is located in the artifact region.

[0115] The adjustment value is negatively correlated with the imaging quality of the camera.

[0116] For example:

[0117] When the output value is greater than or equal to a preset probability, as shown in formula (4), the difference between the output value and an adjustment value is taken as the probability that the corresponding pixel in the non-reference frame image is located in the artifact region:

[0118] P i (x,y)'=P i (x,y)-k (4)

[0119] When the output value is less than a preset probability, as shown in formula (5), the sum of the output value and an adjustment value is taken as the probability that the corresponding pixel in the non-reference frame image is located in the artifact region:

[0120] P i (x,y)'=P i (x,y)+k (5)

[0121] Where k is an adjustment value, for example, k is a value between 0 and 0.5.

[0122] The adjustment value k is negatively correlated with the camera's image quality; that is, a smaller k value indicates better image quality, and a larger k value indicates poorer image quality. For example, if a camera has good image quality, and image artifact removal is performed on N long-exposure images captured by this camera, the value of k is set to 0.1. If a camera has poor image quality, and image artifact removal is performed on N long-exposure images captured by this camera, the value of k is set to 0.4. The camera's image quality can be represented by an image quality index.

[0123] In this embodiment, formulas (4) and (5) are used to adjust the probability calculated using the probability density function in order to make corresponding corrections for different cameras, thereby making it applicable to various cameras with different imaging qualities.

[0124] This disclosure provides a method for removing image artifacts, applied to a mobile terminal. This method includes... Figure 1 , Figure 2 or Figure 3 The method shown, and:

[0125] The probability of each pixel in a non-reference frame image being located in an artifact region is negatively correlated with the weight of the corresponding pixel in the corresponding non-reference frame image.

[0126] For example: the probability P that the pixel at position (x, y) in the i-th non-reference frame image is located in the artifact region. i (x, y), the weight W corresponding to the pixel at position (x, y) in the i-th non-reference frame image. i (x,y)=1-P i (x,y).

[0127] In this embodiment, a value negatively correlated with the probability of a pixel in a non-reference frame image being located in an artifact region is used as the weight of that pixel in the non-reference frame image. This means that the higher the probability of the pixel being located in an artifact region, the lower its weight; conversely, the lower the probability of the pixel being located in an artifact region, the higher its weight. Therefore, when combining N long-exposure images into a single image, the weight of pixels with a lower probability of being located in artifact regions is increased, while the weight of pixels with a lower probability of being located in non-artifact regions is decreased, effectively removing artifact regions from the synthesized image.

[0128] This disclosure provides a method for removing image artifacts, applied to a mobile terminal. This method includes... Figure 1 , Figure 2 or Figure 3 The method shown, and:

[0129] Step S14, which involves synthesizing a weighted image of the N long-exposure images based on the weight of each pixel in the N-1 non-reference frame images, includes:

[0130] Set the weight of each pixel in the reference frame image to 1.

[0131] For example: the weight W corresponding to the pixel at position (x, y) in the i-th non-reference frame image. i (x,y)=1-P i (x, y), where i ranges from 1 to N-1. The weight W corresponds to the pixel at position (x, y) in the reference image frame. N (x, y). Calculate the weighted image Y of N long-exposure images according to formula (6). f (x,y):

[0132]

[0133] In one example, such as Figure 4 In the schematic diagram of the processing results shown, the lower image is a composite image obtained using the processing method of the present disclosure embodiment. Compared with the composite image obtained using the prior art processing method in the upper image, the clarity of the person's legs is significantly improved.

[0134] This disclosure provides an apparatus for removing image artifacts, applied to a mobile terminal. (See also...) Figure 5 , Figure 5 This is a structural diagram illustrating an apparatus for removing image artifacts according to an exemplary embodiment. Figure 5 As shown, this device includes:

[0135] The acquisition module 51 is used to acquire N long exposure images captured by the camera; the N long exposure images include one reference frame image and N-1 non-reference frame images;

[0136] The first calculation module 52 is used to calculate the probability that each pixel in each non-reference frame image is located in the artifact region based on the noise variance coefficient of the camera and the N long exposure images.

[0137] The second calculation module 53 is used to calculate the weight of the corresponding pixel in the corresponding non-reference frame image based on the probability that each pixel in each non-reference frame image is located in the artifact region.

[0138] The synthesis module 54 is used to synthesize a weighted image of the N long exposure images according to the weight of each pixel in the N-1 non-reference frame images.

[0139] This disclosure provides an apparatus for removing image artifacts, applied to a mobile terminal, including... Figure 5 The apparatus shown, and:

[0140] The first calculation module 52 includes:

[0141] The first determining module is used to determine the noise characterization map corresponding to each non-reference frame image based on the noise variance coefficient of the camera and each non-reference frame image;

[0142] The third calculation module calculates the probability that each pixel in each non-reference frame image is located in the artifact region based on the noise characterization map and the reference frame image.

[0143] This disclosure provides an apparatus for removing image artifacts, applied to a mobile terminal, including... Figure 5 The apparatus shown, and:

[0144] The third calculation module also includes:

[0145] The second determining module is used to determine the difference map between each non-reference frame image and the reference frame image;

[0146] The fourth calculation module is further configured to calculate the probability that each pixel in each non-reference frame image is located in the artifact region based on the noise characterization map and the gap map.

[0147] In one embodiment, the first determining module is further configured to determine a noise characterization map corresponding to each non-reference frame image using the following method based on the noise variance coefficient of the camera and each non-reference frame image:

[0148] The value of the corresponding pixel in the noise characterization map is determined based on the gray value of the pixel in each non-reference frame image and the noise variance coefficient.

[0149] In one embodiment, the noise variance coefficient includes a first coefficient and a second coefficient;

[0150] The step of determining the value of the corresponding pixel in the noise characterization map based on the grayscale value of the pixel in each non-reference frame image and the noise variance coefficient includes:

[0151] Calculate the product of the gray value of a pixel in the non-reference frame image and the first coefficient, and the sum of the product and the second coefficient;

[0152] The sum and are used as the values ​​of the corresponding pixels in the noise characterization diagram.

[0153] In one implementation,

[0154] The fourth calculation module includes:

[0155] The first function call module is used to input the values ​​corresponding to the same pixel in the noise characterization map and the difference map into the probability density function, and use the output value of the probability density function as the probability that the corresponding pixel in the non-reference frame image is located in the artifact region.

[0156] This disclosure provides an apparatus for removing image artifacts, applied to a mobile terminal, including... Figure 5 The apparatus shown, and:

[0157] The fourth calculation module includes:

[0158] The second function call module is used to input the values ​​corresponding to the same pixel in the noise characterization map and the difference map into the probability density function, and determine the output value of the probability density function.

[0159] An adjustment module is used to use the difference between the output value and an adjustment value as the probability that the corresponding pixel in the non-reference frame image is located in the artifact region when the output value is greater than or equal to a preset probability.

[0160] When the output value is less than a preset probability, the sum of the output value and an adjustment value is used as the probability that the corresponding pixel in the non-reference frame image is located in the artifact region.

[0161] The adjustment value is negatively correlated with the imaging quality of the camera.

[0162] This disclosure provides an apparatus for removing image artifacts, applied to a mobile terminal, including... Figure 5 The apparatus shown, and:

[0163] The probability of each pixel in a non-reference frame image being located in an artifact region is negatively correlated with the weight of the corresponding pixel in the corresponding non-reference frame image.

[0164] This disclosure provides an apparatus for removing image artifacts, comprising:

[0165] processor;

[0166] Memory used to store processor-executable instructions;

[0167] The processor is configured to execute executable instructions in the memory to implement the steps of the method for removing image artifacts.

[0168] This disclosure provides a non-transitory computer-readable storage medium storing executable instructions that, when executed by a processor, implement the steps of the method for removing image artifacts.

[0169] Figure 6 This is a block diagram illustrating an apparatus 600 for removing image artifacts according to an exemplary embodiment. For example, apparatus 600 may be a mobile phone, computer, digital broadcasting terminal, messaging device, game console, tablet device, medical device, fitness equipment, personal digital assistant, etc.

[0170] Reference Figure 6 The device 600 may include one or more of the following components: a processing component 602, a memory 604, a power supply component 606, a multimedia component 608, an audio component 610, an input / output (I / O) interface 612, a sensor component 614, and a communication component 616.

[0171] Processing component 602 typically controls the overall operation of device 600, such as operations associated with display, telephone calls, data communication, camera operation, and recording. Processing component 602 may include one or more processors 620 to execute instructions to perform all or part of the steps of the methods described above. Furthermore, processing component 602 may include one or more modules to facilitate interaction between processing component 602 and other components. For example, processing component 602 may include a multimedia module to facilitate interaction between multimedia component 608 and processing component 602.

[0172] Memory 604 is configured to store various types of data to support the operation of device 600. Examples of this data include instructions for any application or method operating on device 600, contact data, phonebook data, messages, pictures, videos, etc. Memory 604 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0173] Power supply component 606 provides power to the various components of device 600. Power supply component 606 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to device 600.

[0174] Multimedia component 608 includes a screen that provides an output interface between the device 600 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touchscreen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensors may sense not only the boundaries of the touch or swipe action but also the duration and pressure associated with the touch or swipe operation. In some embodiments, multimedia component 608 includes a front-facing camera and / or a rear-facing camera. When the device 600 is in an operating mode, such as a shooting mode or a video mode, the front-facing camera and / or the rear-facing camera may receive external multimedia data. Each front-facing camera and rear-facing camera may be a fixed optical lens system or have focal length and optical zoom capabilities.

[0175] Audio component 610 is configured to output and / or input audio signals. For example, audio component 610 includes a microphone (MIC) configured to receive external audio signals when device 600 is in an operating mode, such as call mode, recording mode, and voice recognition mode. The received audio signals may be further stored in memory 604 or transmitted via communication component 616. In some embodiments, audio component 610 also includes a speaker for outputting audio signals.

[0176] I / O interface 612 provides an interface between processing component 602 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include, but are not limited to, home buttons, volume buttons, power buttons, and lock buttons.

[0177] Sensor assembly 614 includes one or more sensors for providing status assessments of various aspects of device 600. For example, sensor assembly 614 may detect the on / off state of device 600, the relative positioning of components such as the display and keypad of device 600, changes in the position of device 600 or a component of device 600, the presence or absence of user contact with device 600, the orientation or acceleration / deceleration of device 600, and temperature changes of device 600. Sensor assembly 614 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. Sensor assembly 614 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, sensor assembly 614 may also include an accelerometer, a gyroscope, a magnetometer, a pressure sensor, or a temperature sensor.

[0178] Communication component 616 is configured to facilitate wired or wireless communication between device 600 and other devices. Device 600 can access wireless networks based on communication standards, such as WiFi, 4G, or 5G, or combinations thereof. In one exemplary embodiment, communication component 616 receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, communication component 616 also includes a near-field communication (NFC) module to facilitate short-range communication. For example, the NFC module may be implemented based on radio frequency identification (RFID) technology, Infrared Data Association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

[0179] In an exemplary embodiment, the apparatus 600 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the methods described above.

[0180] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 604 including instructions, which can be executed by a processor 620 of the device 600 to perform the above-described method. For example, the non-transitory computer-readable storage medium may be a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device, etc.

[0181] Other embodiments of the embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the embodiments of this disclosure that follow the general principles of the embodiments of this disclosure and include common knowledge or customary techniques in the art not disclosed in the embodiments. The specification and embodiments are to be considered exemplary only, and the true scope and spirit of the embodiments of this disclosure are indicated by the following claims.

[0182] It should be understood that the embodiments in this disclosure are not limited to the precise structures described above and shown in the accompanying drawings, and various combinations, substitutions, modifications, and changes can be made to the method steps or terminal components disclosed in this application without departing from its scope. All such combinations, substitutions, modifications, and changes are considered to be included within the scope of this disclosure. The scope of protection claimed in this disclosure is limited by the appended claims.

[0183] It should be noted that, in this disclosure, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

Claims

1. A method for removing image artifacts, characterized in that, include: Acquire N long-exposure images captured by the camera; the N long-exposure images include one reference frame image and N-1 non-reference frame images; The probability of each pixel in each non-reference frame image being located in the artifact region is calculated based on the noise variance coefficient of the camera and the N long exposure images. The weight of each pixel in the corresponding non-reference frame image is calculated based on the probability that each pixel is located in the artifact region. A weighted image of the N long-exposure images is synthesized based on the weight of each pixel in the N-1 non-reference frame images; The step of calculating the probability that each pixel in each non-reference frame image is located in the artifact region based on the noise variance coefficient of the camera and the N long exposure images includes: The noise characterization map corresponding to each non-reference frame image is determined based on the noise variance coefficient of the camera and each non-reference frame image. The probability that each pixel in each non-reference frame image is located in the artifact region is calculated based on the noise characterization map and the reference frame image.

2. The method as described in claim 1, characterized in that, The step of calculating the probability that each pixel in each non-reference frame image is located in the artifact region based on the noise characterization map and the reference frame image includes: Determine the difference map between each non-reference frame image and the reference frame image; Based on the noise characterization map and the gap map, calculate the probability that each pixel in each non-reference frame image is located in the artifact region.

3. The method as described in claim 1 or 2, characterized in that, The step of determining the noise characterization map corresponding to each non-reference frame image based on the noise variance coefficient of the camera and each non-reference frame image includes: The value of the corresponding pixel in the noise characterization map is determined based on the gray value of the pixel in each non-reference frame image and the noise variance coefficient.

4. The method as described in claim 3, characterized in that, The noise variance coefficient includes a first coefficient and a second coefficient; The step of determining the value of the corresponding pixel in the noise characterization map based on the grayscale value of the pixel in each non-reference frame image and the noise variance coefficient includes: Calculate the product of the gray value of a pixel in the non-reference frame image and the first coefficient, and the sum of the product and the second coefficient; The sum and are used as the values ​​of the corresponding pixels in the noise characterization diagram.

5. The method as described in claim 2, characterized in that, The step of calculating the probability that each pixel in the non-reference frame image is located in the artifact region based on the noise characterization map and the difference map includes: The values ​​corresponding to the same pixel in the noise characterization map and the difference map are input into the probability density function, and the output value of the probability density function is used as the probability that the corresponding pixel in the non-reference frame image is located in the artifact region.

6. The method as described in claim 2, characterized in that, The step of calculating the probability that each pixel in the non-reference frame image is located in the artifact region based on the noise characterization map and the difference map further includes: Input the values ​​corresponding to the same pixel in the noise characterization map and the difference map into the probability density function to determine the output value of the probability density function; When the output value is greater than or equal to a preset probability, the difference between the output value and an adjustment value is used as the probability that the corresponding pixel in the non-reference frame image is located in the artifact region. When the output value is less than a preset probability, the sum of the output value and an adjustment value is used as the probability that the corresponding pixel in the non-reference frame image is located in the artifact region. The adjustment value is negatively correlated with the imaging quality of the camera.

7. The method as described in claim 1, characterized in that, The probability of each pixel in a non-reference frame image being located in an artifact region is negatively correlated with the weight of the corresponding pixel in the corresponding non-reference frame image.

8. An apparatus for removing image artifacts, characterized in that, include: The acquisition module is used to acquire N long exposure images captured by the camera; the N long exposure images include one reference frame image and N-1 non-reference frame images; The first calculation module is used to calculate the probability that each pixel in each non-reference frame image is located in the artifact region based on the noise variance coefficient of the camera and the N long exposure images; The second calculation module is used to calculate the weight of the corresponding pixel in the corresponding non-reference frame image based on the probability that each pixel in each non-reference frame image is located in the artifact region. The synthesis module is used to synthesize a weighted image of the N long exposure images based on the weight of each pixel in the N-1 non-reference frame images; The first computing module includes: The first determining module is used to determine the noise characterization map corresponding to each non-reference frame image based on the noise variance coefficient of the camera and each non-reference frame image; The third calculation module calculates the probability that each pixel in each non-reference frame image is located in the artifact region based on the noise characterization map and the reference frame image.

9. The apparatus as claimed in claim 8, characterized in that, The third calculation module also includes: The second determining module is used to determine the difference map between each non-reference frame image and the reference frame image; The fourth calculation module is used to calculate the probability that each pixel in each non-reference frame image is located in the artifact region based on the noise characterization map and the gap map.

10. The apparatus as claimed in claim 8 or 9, characterized in that, The first determining module is further configured to determine a noise characterization map corresponding to each non-reference frame image using the following method based on the noise variance coefficient of the camera and each non-reference frame image: The value of the corresponding pixel in the noise characterization map is determined based on the gray value of the pixel in each non-reference frame image and the noise variance coefficient.

11. The apparatus as claimed in claim 10, characterized in that, The noise variance coefficient includes a first coefficient and a second coefficient; The step of determining the value of the corresponding pixel in the noise characterization map based on the grayscale value of the pixel in each non-reference frame image and the noise variance coefficient includes: Calculate the product of the gray value of a pixel in the non-reference frame image and the first coefficient, and the sum of the product and the second coefficient; The sum and are used as the values ​​of the corresponding pixels in the noise characterization diagram.

12. The apparatus as claimed in claim 9, characterized in that, The fourth calculation module includes: The first function call module is used to input the values ​​corresponding to the same pixel in the noise characterization map and the difference map into the probability density function, and use the output value of the probability density function as the probability that the corresponding pixel in the non-reference frame image is located in the artifact region.

13. The apparatus as claimed in claim 9, characterized in that, The fourth calculation module includes: The second function call module is used to input the values ​​corresponding to the same pixel in the noise characterization map and the difference map into the probability density function, and determine the output value of the probability density function. An adjustment module is used to, when the output value is greater than or equal to a preset probability, take the difference between the output value and an adjustment value as the probability that the corresponding pixel in the non-reference frame image is located in the artifact region; and when the output value is less than the preset probability, take the sum of the output value and an adjustment value as the probability that the corresponding pixel in the non-reference frame image is located in the artifact region. The adjustment value is negatively correlated with the imaging quality of the camera.

14. The apparatus as claimed in claim 8, characterized in that, The probability of each pixel in a non-reference frame image being located in an artifact region is negatively correlated with the weight of the corresponding pixel in the corresponding non-reference frame image.

15. An apparatus for removing image artifacts, characterized in that, include: processor; Memory used to store processor-executable instructions; The processor is configured to execute executable instructions in the memory to implement the steps of the method for removing image artifacts according to any one of claims 1 to 7.

16. A non-transitory computer-readable storage medium having executable instructions stored thereon, characterized in that, When executed by a processor, the executable instructions implement the steps of the method for removing image artifacts as described in any one of claims 1 to 7.

Images