Image noise reduction method and device, electronic equipment and storage medium

Abstract

The invention relates to an image noise reduction method and device, equipment and a storage medium, and relates to the technical field of image noise reduction. The method comprises the following steps: acquiring a to-be-processed first image; determining depth information of the first image; calculating a noise correction parameter based on the depth information; and based on the noise correction parameter, performing noise reduction processing on the first image to obtain a noise-reduced image. The method comprises the steps of obtaining a to-be-processed first image; determining depth information of the first image; calculating a noise correction parameter based on the depth information; and based on the noise correction parameter, performing noise reduction processing on the first image to obtain a noise-reduced image. The noise correction parameter is calculated by using the image depth information, and three-dimensional space information is considered during noise reduction processing, so that noise and real details can be distinguished more accurately, and the image noise reduction effect is improved.

Description

Technical Field

[0001] This disclosure relates to the field of image denoising technology, and in particular to an image denoising method, apparatus, electronic device and storage medium. Background Technology

[0002] Digital images are susceptible to device and environmental noise during processing and transmission, including additive, multiplicative, and quantization noise, which causes random variations in image brightness and color, reducing sharpness. Therefore, noise reduction is crucial for improving image quality and sharpness.

[0003] In related technologies, some noise reduction algorithms are used to remove noise and restore the original information. However, for original images with a lot of detail, the noise reduction algorithms in these technologies have difficulty accurately distinguishing between noise and real details, resulting in unsatisfactory image noise reduction effects.

[0004] It should be noted that the information disclosed in the background section above is only used to enhance the understanding of the background of this disclosure, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0005] This disclosure provides an image noise reduction method, apparatus, electronic device, and storage medium.

[0006] According to a first aspect of the present disclosure, an image noise reduction method is provided, comprising:

[0007] Obtain the first image to be processed;

[0008] Determine the depth information of the first image;

[0009] Based on the depth information, noise correction parameters are calculated;

[0010] Based on the noise correction parameters, the first image is denoised to obtain a denoised image.

[0011] In some embodiments of this disclosure, acquiring a first image to be processed includes:

[0012] Obtain the pixel values ​​of multiple pixels in the first image;

[0013] Based on the noise correction parameters, the first image is denoised to obtain a denoised image, including:

[0014] The pixel value of the target pixel is adjusted based on the noise correction parameters to obtain the noise-reduced pixel value corresponding to the target pixel.

[0015] The first image includes the target pixel.

[0016] In some embodiments of this disclosure, noise correction parameters are calculated based on the depth information, including:

[0017] Based on the target pixel, a correction region in the first image is obtained, the correction region including a plurality of consecutive pixels centered on the target pixel;

[0018] The noise correction parameters are calculated based on the pixel value of each pixel in the correction region and the depth information corresponding to each pixel.

[0019] In some embodiments of this disclosure, adjusting the pixel value of a target pixel based on the noise correction parameters to obtain the noise-reduced pixel value corresponding to the target pixel includes:

[0020] Based on the pixel value of the target pixel, the noise correction parameters, and the pixel value of each pixel in the correction region, filtering calculations are performed to obtain the noise-reduced pixel value corresponding to the target pixel.

[0021] In some embodiments of this disclosure, the denoised pixel value corresponding to the target pixel is obtained by filtering calculation based on the pixel value of the target pixel, the noise correction parameters, and the pixel value of each pixel in the correction region, including:

[0022] Based on the pixel value of the target pixel, the noise correction parameters, and the pixel value of each pixel in the correction region, bilateral filtering is performed to obtain the noise-reduced pixel value corresponding to the target pixel.

[0023] The noise correction parameters are calculated using the following formula, based on the pixel value of each pixel in the correction region and the corresponding depth information:

[0024]

[0025] Among them, w t (i,j,k) represents the noise correction parameter corresponding to the pixel with coordinates (i,j), and the depth value of the pixel is k;

[0026] The coordinates of the target pixel are (x, y), and the corresponding depth value is z;

[0027] σ t Represents the neighborhood factor in three-dimensional space.

[0028] In some embodiments of this disclosure, the denoised pixel value corresponding to the target pixel is obtained by filtering calculation based on the pixel value of the target pixel, the noise correction parameters, and the pixel value of each pixel in the correction region, including:

[0029] Based on the pixel value of the target pixel, the noise correction parameters, and the pixel value of each pixel in the correction region, Gaussian filtering is performed to obtain the noise-reduced pixel value corresponding to the target pixel.

[0030] The noise correction parameters are calculated using the following formula, based on the pixel value of each pixel in the correction region and the corresponding depth information:

[0031]

[0032] Where G(m,n,σ) represents the coordinates of a pixel (m,n), the corresponding depth value p, and the corresponding noise correction parameter;

[0033] The coordinates of any pixel within the correction area are (i,j), and the depth value of this pixel is k;

[0034] σ represents the standard deviation of the Gaussian function;

[0035] σ t Represents the neighborhood factor in three-dimensional space.

[0036] In some embodiments of this disclosure, the denoised pixel value corresponding to the target pixel is obtained by filtering calculation based on the pixel value of the target pixel, the noise correction parameters, and the pixel value of each pixel in the correction region, including:

[0037] Based on the pixel value of the target pixel, the noise correction parameters, and the pixel value of each pixel in the correction region, nonlocal mean filtering is performed to obtain the noise-reduced pixel value corresponding to the target pixel.

[0038] The noise correction parameters are calculated using the following formula, based on the pixel value of each pixel in the correction region and the corresponding depth information:

[0039]

[0040] Where, N X This represents the neighborhood window centered at the target pixel (x,y), which is also the correction region.

[0041] N I This represents a neighborhood window centered at pixel (i,j).

[0042] This represents the square of the weighted Euclidean distance between pixel values ​​within two neighboring windows, and 'a' represents the standard deviation of the Gaussian kernel, used to control the smoothness of the weights.

[0043] h represents the filtering parameter, which controls the decay rate of the weights;

[0044] Z(I) represents the normalization factor, ensuring that the sum of all weights is 1. Specifically,

[0045] σ t This represents the three-dimensional spatial neighborhood factor, which is a pre-set constant.

[0046] The depth value corresponding to the target pixel is z, and the depth value of the pixel with coordinates (i,j) is k.

[0047] In some embodiments of this disclosure, determining the depth information of the first image includes:

[0048] Determine the disparity map corresponding to the first image;

[0049] Obtain the intrinsic and extrinsic parameters corresponding to the first image;

[0050] Based on the disparity map and the intrinsic and extrinsic parameters, the depth value corresponding to each pixel in the first image is calculated.

[0051] According to a second aspect of the present disclosure, an image noise reduction apparatus is provided, comprising:

[0052] An image acquisition unit is used to acquire a first image to be processed.

[0053] A depth information determination unit is used to determine the depth information of the first image;

[0054] The correction parameter calculation unit is used to calculate noise correction parameters based on the depth information;

[0055] The noise reduction processing unit is used to perform noise reduction processing on the first image based on the noise correction parameters to obtain a noise-reduced image.

[0056] According to a third aspect of the present disclosure, an image noise reduction apparatus is provided, comprising:

[0057] processor;

[0058] Memory used to store processor-executable instructions;

[0059] The processor is configured to implement the image noise reduction method described in the first aspect above.

[0060] According to a fourth aspect of the present disclosure, a non-transitory computer-readable storage medium is provided, which, when instructions in the storage medium are executed by a processor of a mobile terminal, enables the mobile terminal to perform the image noise reduction method described in the first aspect.

[0061] The technical solutions provided by the embodiments of this disclosure may include the following beneficial effects:

[0062] This disclosure involves acquiring a first image to be processed; determining the depth information of the first image; calculating noise correction parameters based on the depth information; and performing noise reduction processing on the first image based on the noise correction parameters to obtain a denoised image. By utilizing image depth information to calculate the noise correction parameters, three-dimensional spatial information is considered during the noise reduction process, enabling a more accurate distinction between noise and real details, thereby improving the image noise reduction effect.

[0063] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0064] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure. It is obvious that the drawings described below are merely some embodiments of this disclosure, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort.

[0065] Figure 1 This is a flowchart illustrating an image noise reduction method according to some embodiments of the present disclosure.

[0066] Figure 2 This is a flowchart illustrating one implementation of step S104 according to some embodiments of the present disclosure.

[0067] Figure 3 This is a flowchart illustrating one implementation of step S106 according to some embodiments of the present disclosure.

[0068] Figure 4 This is a flowchart illustrating one implementation of step S108 according to some embodiments of the present disclosure.

[0069] Figure 5 This is a block diagram of an image noise reduction apparatus according to some embodiments of the present disclosure.

[0070] Figure 6 This is a block diagram illustrating an electronic device according to an exemplary embodiment of the present disclosure. Detailed Implementation

[0071] Some embodiments of this disclosure will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. Various changes, modifications, and equivalents of the methods, apparatus, and / or systems described herein will become apparent upon understanding this disclosure. For example, the order of operations described herein is merely illustrative and is not limited to those orders set forth herein, but can be changed as will become apparent upon understanding this disclosure, except for operations that must be performed in a particular order. Furthermore, for clarity and brevity, descriptions of features known in the art may be omitted.

[0072] The embodiments described in the following examples of this disclosure are not representative of all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this disclosure as detailed in the appended claims.

[0073] The specific implementation methods of the embodiments of this disclosure will now be described in detail with reference to the accompanying drawings.

[0074] Figure 1 This is a flowchart illustrating an image noise reduction method according to some embodiments of the present disclosure, such as... Figure 1 As shown, the image noise reduction method can be applied to electronic devices, including but not limited to smartphones, wearable devices, and smart tablets with shooting functions, as well as server-side devices such as local servers and cloud servers, which can be deployed in a computer or a computer cluster composed of multiple computers.

[0075] Figure 1 The image denoising method shown includes the following steps.

[0076] In step S102, the first image to be processed is obtained.

[0077] It should be noted that the first image is the image to be denoised. Step S102 specifically includes: obtaining the pixel values ​​of multiple pixels in the first image, specifically the pixel value of each pixel in the first image.

[0078] In step S104, the depth information of the first image is determined.

[0079] In some exemplary embodiments of this disclosure, the first image to be processed is a two-dimensional image, that is, an image depicted on a two-dimensional plane. The first image may be a two-dimensional image containing depth information, such as a two-dimensional image taken using stereo vision, a depth map, or a two-dimensional image obtained using monocular depth estimation.

[0080] It should be noted that the depth information of the first image includes the depth value corresponding to each pixel in the first image, representing the distance of each pixel to the camera (or observer), and providing the three-dimensional structure of the scene to be captured in the first image. In specific implementations, the depth information of the first image can be determined through binocular vision, structured light, time-of-flight, laser scanning, etc.

[0081] In step S106, noise correction parameters are calculated based on depth information.

[0082] It should be noted that in related technologies, the noise correction parameters for noise reduction using two-dimensional information are only related to the image information in the two-dimensional plane. In this embodiment, the noise correction parameters are calculated based on depth information and the pixel values ​​of multiple pixels in the first image; that is, noise reduction is performed on the two-dimensional image using three-dimensional spatial information.

[0083] In the exemplary embodiments of this disclosure, noise correction parameters refer to parameters used to adjust and optimize the denoising algorithm in image denoising processing, and may include: threshold, filter parameters, weights, window length, etc. It is understood that the noise correction parameters vary depending on the denoising algorithm used.

[0084] In step S108, the first image is denoised based on the noise correction parameters to obtain the denoised image.

[0085] It should be noted that, based on the noise correction parameters, the first image is denoised using the corresponding denoising algorithm to obtain the denoised image.

[0086] In some exemplary embodiments of this disclosure, step S108 includes: adjusting the pixel value of the target pixel based on noise correction parameters to obtain the denoised pixel value corresponding to the target pixel; wherein the first image includes the target pixel.

[0087] As can be seen from the above steps, the image denoising method provided in this embodiment of the present disclosure acquires a first image to be processed; determines the depth information of the first image; calculates noise correction parameters based on the depth information; and performs denoising processing on the first image based on the noise correction parameters to obtain a denoised image. By using image depth information to calculate the noise correction parameters, three-dimensional spatial information is considered during the denoising process, which can more accurately distinguish between noise and real details, thereby improving the denoising effect on two-dimensional images.

[0088] In some exemplary embodiments of this disclosure, such as Figure 2 The diagram shown is a flowchart illustrating the implementation process of step S104 provided in some exemplary embodiments of this disclosure, including the following steps.

[0089] In step S202, the disparity map corresponding to the first image is determined.

[0090] It should be noted that the depth value of each pixel in the first image can be determined by using a stereo camera to capture images. Stereo matching results are obtained by comparing the images captured by the two cameras, and a disparity map corresponding to the first image is determined based on these results. The disparity map represents the difference in pixel position of the same object in the left and right images, i.e., the disparity. The magnitude of the disparity is closely related to the object's depth information; the larger the disparity, the closer the object is to the camera.

[0091] In some embodiments of this disclosure, the first image is a two-dimensional image captured by the main camera of a binocular camera, and the second image is a two-dimensional image captured by the secondary camera of the binocular camera corresponding to the same scene. Stereo matching is performed by comparing local features (such as grayscale, texture, etc.) in the first and second images. Specifically, the choice of matching algorithm has a significant impact on the accuracy of depth information, and algorithms that can be used include SGBM (Semi-Global Block Matching) and BM (Block Matching). Based on the results of stereo matching, a disparity map is calculated.

[0092] In step S204, the intrinsic and extrinsic parameters corresponding to the first image are obtained.

[0093] In some exemplary embodiments of this disclosure, the stereo camera is calibrated to obtain its intrinsic parameters (such as focal length, principal point coordinates, etc.) and extrinsic parameters (such as rotation matrix and translation vector), as well as distortion parameters. Calibration ensures that the captured images have accurate geometric relationships. After calibration, stereo correction is performed to eliminate rotation and translation differences between the two cameras, ensuring perfect row alignment of the two images.

[0094] In step S206, the depth value corresponding to each pixel in the first image is calculated based on the disparity map and intrinsic and extrinsic parameters.

[0095] It should be noted that the depth information of each pixel can be calculated based on the disparity map and the intrinsic and extrinsic parameters of the stereo camera. This can be achieved using a depth calculation formula, such as Z = fb / d, where Z is the depth value corresponding to the pixel in the image, f is the camera focal length, b is the baseline distance (i.e., the distance between the two cameras), and d is the disparity at the pixel in the image.

[0096] Those skilled in the art will understand that the depth information acquisition process in steps S202 to S206 described above is merely an example, illustrating one implementation of determining the depth information of a first image using a binocular camera, and is not intended to limit the scope of protection of the embodiments of this disclosure. Other methods can also be used to determine the depth; for example, depth information of each point in the first image can be obtained using a depth sensor, which will not be elaborated upon here.

[0097] In some exemplary embodiments of this disclosure, such as Figure 3 The diagram shown is a flowchart illustrating the implementation process of step S106 provided in some exemplary embodiments of this disclosure, including the following steps.

[0098] In step S302, the correction region in the first image is obtained based on the target pixel.

[0099] It should be noted that the correction region includes multiple consecutive pixels centered on the target pixel. The principle of the denoising algorithm used in image denoising is to estimate the pixel value of the denoised target pixel based on the pixel values ​​of other pixels within a certain range around the target pixel. Therefore, the correction region corresponding to the target pixel is determined based on the target pixel to determine the noise correction parameters.

[0100] In step S304, noise correction parameters are calculated based on the pixel value of each pixel in the correction region and the depth information corresponding to each pixel.

[0101] It should be noted that the noise correction parameters differ depending on the noise reduction algorithm used, and consequently, the calculation formulas also differ. For example, when using a Gaussian filtering algorithm, the noise correction parameter is a Gaussian weight parameter. Specifically, the noise correction parameter is calculated based on the pixel value and corresponding depth value of each pixel in the correction region, as well as the pixel value and corresponding depth value of the target pixel.

[0102] Accordingly, the implementation process of step S108 is as follows: Figure 4 As shown, the steps include the following.

[0103] In step S402, based on the pixel value of the target pixel, the noise correction parameters, and the pixel value of each pixel in the correction region, filtering calculations are performed to obtain the noise-reduced pixel value corresponding to the target pixel.

[0104] In some exemplary embodiments of this disclosure, the implementation process of step S402 includes: performing bilateral filtering calculation based on the pixel value of the target pixel, noise correction parameters, and the pixel value of each pixel in the correction region to obtain the denoised pixel value corresponding to the target pixel. That is, the bilateral filtering algorithm is used to calculate the denoised pixel value corresponding to the target pixel.

[0105] Optionally, the denoised pixel value f(x,y) corresponding to the target pixel is calculated according to the following formula:

[0106]

[0107] Where (i,j)∈S x,y This represents any pixel in the correction region centered at the target pixel (x,y) and with a size of (2N+1)×(2N+1), and the pixel coordinates are (i,j).

[0108] g(i,j) represents the pixel value of the pixel with coordinates (i,j);

[0109] w t (i,j,k) represents the noise correction parameter corresponding to the pixel with coordinates (i,j), also known as the spatial neighborhood weight parameter, and the depth value of the pixel is k;

[0110] w r (i,j) represents the planar neighborhood weight parameter corresponding to the pixel with coordinates (i,j);

[0111] w s (i,j) represents the brightness similarity weight parameter corresponding to the pixel with coordinates (i,j).

[0112] It should be noted that the planar neighborhood weight parameter is calculated according to the following formula:

[0113]

[0114] The brightness similarity weight parameter is calculated according to the following formula:

[0115]

[0116] Where, σ s σ represents the two-dimensional planar neighborhood factor. r These represent the brightness similarity factor, which is a pre-set constant.

[0117] Accordingly, the noise correction parameters are calculated based on the pixel value and the corresponding depth information of each pixel in the correction region, according to the following formula:

[0118]

[0119] Where, σ t This represents the neighborhood factor in three-dimensional space, which is a pre-set constant.

[0120] In the above embodiments of this disclosure, a bilateral filter is used to perform noise reduction processing on the first image. As can be seen from the above formula, the bilateral filter is controlled by parameters, which may include: the filter width N, the two-dimensional plane neighborhood factor σ. s , brightness similarity factor σ r 3D spatial neighborhood factor σ t The intensity of noise reduction can be adjusted by controlling the above parameters in this embodiment.

[0121] Those skilled in the art will understand that the above parameters are preset and the specific values ​​are not limited. They can be set according to the results of multiple experiments in advance to adjust the noise reduction intensity and obtain values ​​that result in good noise reduction effect.

[0122] In some exemplary embodiments of this disclosure, the implementation process of step S402 includes: performing Gaussian filtering calculation based on the pixel value of the target pixel, noise correction parameters, and the pixel value of each pixel in the correction region to obtain the noise-reduced pixel value corresponding to the target pixel. That is, the Gaussian filtering algorithm is used to calculate the noise-reduced pixel value corresponding to the target pixel.

[0123] Optionally, the denoised pixel value I′(x,y) corresponding to the target pixel is calculated according to the following formula:

[0124]

[0125] Where (i,j)∈S x,y This represents any pixel in the correction region centered at the target pixel (x,y) and generally in the form of a rectangle, with the pixel coordinates being (i,j).

[0126] I(i,j) represents the pixel value of the pixel with coordinates (i,j);

[0127] G(ix,xy,σ) represents the value of the Gaussian function at the pixel with coordinates (ix,xy), also known as the Gaussian weight coefficient, and is also a noise correction parameter.

[0128] Accordingly, the noise correction parameter G(m,n,σ) is calculated based on the pixel value of each pixel in the correction region and the corresponding depth information, according to the following formula:

[0129]

[0130] Where σ represents the standard deviation of the Gaussian function; t Represents the neighborhood factor in three-dimensional space;

[0131] G(m,n,σ) represents the Gaussian function value at the pixel with coordinates (m,n), which is the corresponding noise correction parameter;

[0132] The coordinates of any pixel within the correction area are (i,j), and the depth value of this pixel is k.

[0133] In the above embodiments of this disclosure, when performing Gaussian image denoising, for a target pixel, pixels within a certain range around it, that is, pixels within the correction area, are taken, and weight coefficients of these pixels are calculated according to the Gaussian function based on depth information. These weighted pixel values ​​are then added together to obtain the pixel value of the denoised target pixel.

[0134] Those skilled in the art will understand that the size of the above-mentioned correction area and the value of the three-dimensional spatial neighborhood factor are preset, and the specific values ​​are not limited. They can be set according to the results of multiple experiments in advance to adjust the noise reduction intensity and obtain values ​​that can achieve good noise reduction effect.

[0135] In some exemplary embodiments of this disclosure, the implementation process of step S402 includes: performing non-local mean filtering calculation based on the pixel value of the target pixel, noise correction parameters, and the pixel value of each pixel in the correction region to obtain the denoised pixel value corresponding to the target pixel. That is, a non-local mean filtering algorithm is used to calculate the denoised pixel value corresponding to the target pixel.

[0136] Optionally, the denoised pixel value NL[v](x,y) corresponding to the target pixel is calculated according to the following formula:

[0137] NL[v](x,y)=∑ (i,j)∈s w(X,I)·v(i,j) (7)

[0138] Where (i,j)∈S represents any pixel in the correction region, and the pixel coordinates are (i,j); the correction region is the set of all pixel positions in the image;

[0139] w(X,I) represents the weighting coefficient between the target pixel (x,y) and the pixel (i,j), which depends on the similarity between them, and is also the noise correction parameter.

[0140] v(i,j) represents the pixel value of pixel (i,j).

[0141] Accordingly, the noise correction parameter w(X,I) is calculated based on the pixel value of each pixel in the correction region and the corresponding depth information of each pixel, according to the following formula:

[0142]

[0143] Where, N X This represents the neighborhood window centered at the target pixel (x,y), which is also the correction region.

[0144] N I This represents a neighborhood window centered at pixel (i,j).

[0145] This represents the square of the weighted Euclidean distance between pixel values ​​within two neighboring windows, and 'a' represents the standard deviation of the Gaussian kernel, used to control the smoothness of the weights.

[0146] h represents the filtering parameter, which controls the decay rate of the weights;

[0147] Z(I) represents the normalization factor, ensuring that the sum of all weights is 1. Specifically,

[0148] σ t This represents the three-dimensional spatial neighborhood factor, which is a pre-set constant.

[0149] The depth value corresponding to the target pixel is z, and the depth value of the pixel with coordinates (i,j) is k.

[0150] In the embodiments of this disclosure, a nonlocal mean filter is used to denoise the first image. As can be seen from the above formula, the nonlocal mean filter estimates the pixel value of the target pixel by weighted averaging of the pixel values ​​of all pixels in the image, where the weights depend on the similarity between the target pixel and other pixels. Furthermore, the weights are determined based on the depth value of the pixel, taking into account not only the spatial location of the pixel but also the similarity between pixels. Therefore, it can preserve the details and texture information of the image while denoising.

[0151] In some embodiments of this disclosure, a nonlocal mean filter is used to perform noise reduction processing on the first image. As can be seen from the above formula, the nonlocal mean filter is controlled by parameters, which may include: the size of the neighborhood window (that is, the size of the correction region), the standard deviation of the Gaussian kernel, the filtering parameters, and the three-dimensional spatial neighborhood factor, etc. The values ​​of the above parameters can be selected and adjusted according to the specific image processing task to obtain a nonlocal mean filter with good noise reduction effect.

[0152] Those skilled in the art will understand that the above parameters are preset and the specific values ​​are not limited. They can be set according to the results of multiple experiments in advance to adjust the noise reduction intensity and obtain values ​​that result in good noise reduction effect.

[0153] In the embodiments of this disclosure, by introducing depth information to calculate the relationship between the center point (target pixel) and its corresponding neighboring points in three-dimensional space, it is possible to more accurately determine whether the region to which the target pixel belongs is a flat region or an edge region. In flat regions, the denoising algorithm provided in this disclosure enhances the denoising effect; while in edge regions, the denoising algorithm provided in this disclosure weakens the denoising to maintain edge clarity. Because it utilizes depth information in three-dimensional space to denoise two-dimensional images, but is not limited to the judgment of the two-dimensional plane, the accuracy of the denoising process is improved, and the precision of denoising is significantly enhanced.

[0154] This disclosure comprehensively considers information from multiple dimensions of an image, helping to distinguish between noise and real details, thereby more accurately identifying and removing noise, resulting in more significant noise reduction and a substantial improvement in image quality. By processing image information more finely, it can better preserve the image's edge structure while removing noise, making the edges of the image clearer and sharper. For images with complex textures or similar objects, considering multi-dimensional information improves the adaptability of the noise reduction method to complex scenes, enabling it to maintain good noise reduction performance under various conditions and exhibiting better robustness. This leads to better noise reduction results in remote sensing image processing, computer vision, video processing, and other scenarios, benefiting subsequent remote sensing applications, model performance, and improving video quality and viewing experience.

[0155] The following are embodiments of the apparatus disclosed herein, which can be used to execute embodiments of the method disclosed herein. For details not disclosed in the apparatus embodiments of this disclosure, please refer to the embodiments of the method disclosed herein.

[0156] Figure 5 This is a block diagram of an image noise reduction apparatus according to some embodiments of the present disclosure. (Refer to...) Figure 5 The device includes: an image acquisition unit 501, a depth information determination unit 502, a correction parameter calculation unit 503, and a noise reduction processing unit 504.

[0157] Image acquisition unit 501 is used to acquire a first image to be processed;

[0158] The depth information determination unit 502 is used to determine the depth information of the first image;

[0159] The correction parameter calculation unit 503 is used to calculate noise correction parameters based on depth information;

[0160] The noise reduction processing unit 504 is used to perform noise reduction processing on the first image based on noise correction parameters to obtain a noise-reduced image.

[0161] In some exemplary embodiments of this disclosure, the image acquisition unit 501 is configured to acquire pixel values ​​of a plurality of pixels in a first image. Correspondingly, the noise reduction processing unit 504 is configured to adjust the pixel values ​​of a target pixel based on noise correction parameters to obtain the noise-reduced pixel values ​​corresponding to the target pixel; the first image includes the target pixel.

[0162] In some exemplary embodiments of this disclosure, the depth information determination unit 502 is configured to:

[0163] Determine the disparity map corresponding to the first image;

[0164] Obtain the intrinsic and extrinsic parameters corresponding to the first image;

[0165] Based on the disparity map and intrinsic and extrinsic parameters, the depth value corresponding to each pixel in the first image is calculated.

[0166] In some exemplary embodiments of this disclosure, the correction parameter calculation unit 503 is configured as follows:

[0167] Based on the target pixel, obtain the correction region in the first image. The correction region includes multiple consecutive pixels centered on the target pixel.

[0168] The noise correction parameters are calculated based on the pixel value of each pixel in the correction region and the depth information corresponding to each pixel.

[0169] Accordingly, the noise reduction processing unit 504 is configured to perform filtering calculations based on the pixel value of the target pixel, noise correction parameters, and the pixel value of each pixel in the correction region to obtain the noise-reduced pixel value corresponding to the target pixel.

[0170] In some exemplary embodiments of this disclosure, the noise reduction processing unit 504 is configured to: perform bilateral filtering calculation based on the pixel value of the target pixel, noise correction parameters, and the pixel value of each pixel in the correction region to obtain the noise-reduced pixel value corresponding to the target pixel.

[0171] In some exemplary embodiments of this disclosure, the noise reduction processing unit 504 is configured to: perform Gaussian filtering calculation based on the pixel value of the target pixel, noise correction parameters, and the pixel value of each pixel in the correction region to obtain the noise-reduced pixel value corresponding to the target pixel.

[0172] In some exemplary embodiments of this disclosure, the noise reduction processing unit 504 is configured to: perform non-local mean filtering calculation based on the pixel value of the target pixel, noise correction parameters, and the pixel value of each pixel in the correction region to obtain the noise-reduced pixel value corresponding to the target pixel.

[0173] Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0174] Figure 6 This is a block diagram illustrating an electronic device according to an exemplary embodiment of the present disclosure. For example, device 600 may be a mobile phone, computer, digital broadcasting terminal, messaging device, game console, tablet device, medical device, fitness device, personal digital assistant, etc.

[0175] Reference Figure 6 The device 600 may include one or more of the following components: a processing component 602, a memory 604, a power supply component 606, a multimedia component 608, an audio component 610, an input / output (I / O) interface 612, a sensor component 614, and a communication component 616.

[0176] Processing component 602 typically controls the overall operation of device 600, such as operations associated with display, telephone calls, data communication, camera operation, and recording. Processing component 602 may include one or more processors 620 to execute instructions to perform all or part of the steps of the methods described above. Furthermore, processing component 602 may include one or more modules to facilitate interaction between processing component 602 and other components. For example, processing component 602 may include a multimedia module to facilitate interaction between multimedia component 608 and processing component 602.

[0177] Memory 604 is configured to store various types of data to support the operation of device 600. Examples of this data include instructions for any application or method operating on device 600, contact data, phonebook data, messages, pictures, videos, etc. Memory 604 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0178] Power supply assembly 606 provides power to various components of device 600. Power supply assembly 606 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to device 600.

[0179] Multimedia component 608 includes a screen that provides an output interface between the device 600 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touchscreen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensors may sense not only the boundaries of the touch or swipe action but also the duration and pressure associated with the touch or swipe operation. In some embodiments, multimedia component 608 includes a front-facing camera and / or a rear-facing camera. When the device 600 is in an operating mode, such as a shooting mode or a video mode, the front-facing camera and / or the rear-facing camera may receive external multimedia data. Each front-facing camera and rear-facing camera may be a fixed optical lens system or have focal length and optical zoom capabilities.

[0180] Audio component 610 is configured to output and / or input audio signals. For example, audio component 610 includes a microphone (MIC) configured to receive external audio signals when device 600 is in an operating mode, such as call mode, recording mode, and voice recognition mode. The received audio signals may be further stored in memory 604 or transmitted via communication component 616. In some embodiments, audio component 610 also includes a speaker for outputting audio signals.

[0181] I / O interface 612 provides an interface between processing component 602 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include, but are not limited to, home buttons, volume buttons, power buttons, and lock buttons.

[0182] Sensor assembly 614 includes one or more sensors for providing status assessments of various aspects of device 600. For example, sensor assembly 614 may detect the on / off state of device 600, the relative positioning of components such as the display and keypad of device 600, changes in the position of device 600 or a component of device 600, the presence or absence of user contact with device 600, the orientation or acceleration / deceleration of device 600, and temperature changes of device 600. Sensor assembly 614 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. Sensor assembly 614 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, sensor assembly 614 may also include an accelerometer, a gyroscope, a magnetometer, a pressure sensor, or a temperature sensor.

[0183] Communication component 616 is configured to facilitate wired or wireless communication between device 600 and other devices. Device 600 can access wireless networks based on communication standards, such as WiFi, 3G, 4G, 5G, other communication standards, or combinations thereof. In some embodiments of this disclosure, communication component 616 receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In some embodiments of this disclosure, communication component 616 further includes a near-field communication (NFC) module to facilitate short-range communication. For example, the NFC module may be implemented based on radio frequency identification (RFID) technology, Infrared Data Association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

[0184] In some embodiments of this disclosure, the apparatus 600 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the methods described above.

[0185] In some embodiments of this disclosure, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 604 including instructions that can be executed by a processor 620 of device 600 to perform the above-described method. For example, the non-transitory computer-readable storage medium may be a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device, etc.

[0186] In some embodiments of this disclosure, a non-transitory computer-readable storage medium, when instructions in the storage medium are executed by a processor of a mobile terminal, enables the mobile terminal to perform an image noise reduction method, the method comprising:

[0187] Obtain the first image to be processed;

[0188] Determine the depth information of the first image;

[0189] Based on depth information, noise correction parameters are calculated;

[0190] Based on the noise correction parameters, the first image is denoised to obtain the denoised image.

[0191] In some embodiments of this disclosure, a computer program product is also provided, including a computer program / instructions that, when executed by a processor, implement an image noise reduction method, the method comprising:

[0192] Obtain the first image to be processed;

[0193] Determine the depth information of the first image;

[0194] Based on depth information, noise correction parameters are calculated;

[0195] Based on the noise correction parameters, the first image is denoised to obtain the denoised image.

[0196] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the following claims.

[0197] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.

Claims

1. An image denoising method, characterized in that, include: Obtain the first image to be processed; Determine the depth information of the first image; Based on the depth information, noise correction parameters are calculated; Based on the noise correction parameters, the first image is denoised to obtain a denoised image.

2. The image denoising method according to claim 1, characterized in that, Obtain the first image to be processed, including: Obtain the pixel values ​​of multiple pixels in the first image; Based on the noise correction parameters, the first image is denoised to obtain a denoised image, including: The pixel value of the target pixel is adjusted based on the noise correction parameters to obtain the noise-reduced pixel value corresponding to the target pixel. The first image includes the target pixel.

3. The image denoising method according to claim 2, characterized in that, Based on the depth information, noise correction parameters are calculated, including: Based on the target pixel, a correction region in the first image is obtained, the correction region including a plurality of consecutive pixels centered on the target pixel; The noise correction parameters are calculated based on the pixel value of each pixel in the correction region and the depth information corresponding to each pixel.

4. The image denoising method according to claim 3, characterized in that, The pixel value of the target pixel is adjusted based on the noise correction parameters to obtain the noise-reduced pixel value corresponding to the target pixel, including: Based on the pixel value of the target pixel, the noise correction parameters, and the pixel value of each pixel in the correction region, filtering calculations are performed to obtain the noise-reduced pixel value corresponding to the target pixel.

5. The image denoising method according to claim 4, characterized in that, Based on the pixel value of the target pixel, the noise correction parameters, and the pixel value of each pixel in the correction region, filtering calculations are performed to obtain the noise-reduced pixel value corresponding to the target pixel, including: Based on the pixel value of the target pixel, the noise correction parameters, and the pixel value of each pixel in the correction region, bilateral filtering is performed to obtain the noise-reduced pixel value corresponding to the target pixel. The noise correction parameters are calculated using the following formula, based on the pixel value of each pixel in the correction region and the corresponding depth information: Among them, w t (i,j,k) represents the noise correction parameter corresponding to the pixel with coordinates (i,j), and the depth value of the pixel is k; The coordinates of the target pixel are (x, y), and the corresponding depth value is z; σ t Represents the neighborhood factor in three-dimensional space.

6. The image denoising method according to claim 4, characterized in that, Based on the pixel value of the target pixel, the noise correction parameters, and the pixel value of each pixel in the correction region, filtering calculations are performed to obtain the noise-reduced pixel value corresponding to the target pixel, including: Based on the pixel value of the target pixel, the noise correction parameters, and the pixel value of each pixel in the correction region, Gaussian filtering is performed to obtain the noise-reduced pixel value corresponding to the target pixel. The noise correction parameters are calculated using the following formula, based on the pixel value of each pixel in the correction region and the corresponding depth information: Where G(m,n,σ) represents the coordinates of a pixel (m,n), the corresponding depth value p, and the corresponding noise correction parameter; The coordinates of any pixel within the correction area are (i,j), and the depth value of this pixel is k; σ represents the standard deviation of the Gaussian function; σ t Represents the neighborhood factor in three-dimensional space.

7. The image denoising method according to claim 4, characterized in that, Based on the pixel value of the target pixel, the noise correction parameters, and the pixel value of each pixel in the correction region, filtering calculations are performed to obtain the noise-reduced pixel value corresponding to the target pixel, including: Based on the pixel value of the target pixel, the noise correction parameters, and the pixel value of each pixel in the correction region, nonlocal mean filtering is performed to obtain the noise-reduced pixel value corresponding to the target pixel. The noise correction parameters are calculated using the following formula, based on the pixel value of each pixel in the correction region and the corresponding depth information: Where, N X This represents the neighborhood window centered at the target pixel (x,y), which is also the correction region. N I This represents a neighborhood window centered at pixel (i,j). This represents the square of the weighted Euclidean distance between pixel values ​​within two neighboring windows, and 'a' represents the standard deviation of the Gaussian kernel, used to control the smoothness of the weights. h represents the filtering parameter, which controls the decay rate of the weights; Z(I) represents the normalization factor, ensuring that the sum of all weights is 1. Specifically, σ t This represents the three-dimensional spatial neighborhood factor, which is a pre-set constant. The depth value corresponding to the target pixel is z, and the depth value of the pixel with coordinates (i,j) is k.

8. The image denoising method according to claim 1, characterized in that, Determine the depth information of the first image, including Determine the disparity map corresponding to the first image; Obtain the intrinsic and extrinsic parameters corresponding to the first image; Based on the disparity map and the intrinsic and extrinsic parameters, the depth value corresponding to each pixel in the first image is calculated.

9. An image noise reduction device, characterized in that, include: An image acquisition unit is used to acquire a first image to be processed. A depth information determination unit is used to determine the depth information of the first image; The correction parameter calculation unit is used to calculate the noise correction parameters based on the depth information; The noise reduction processing unit is used to perform noise reduction processing on the first image based on the noise correction parameters to obtain a noise-reduced image.

10. An image noise reduction device, characterized in that, include: processor; Memory used to store processor-executable instructions; The processor is configured to implement the image noise reduction method according to any one of claims 1 to 8.

11. A non-transitory computer-readable storage medium, wherein when instructions in the storage medium are executed by a processor of a mobile terminal, the mobile terminal is enabled to perform an image noise reduction method according to any one of claims 1 to 8.

Images