Light field denoising method and device based on spectral concentration degree and reparameterization

Abstract

The application discloses a light field denoising method and device based on spectrum concentration degree and introducing reparameterization, which comprises the following steps: step 1, calculating the reparameterization light field double-plane distance D by minimizing the concentration degree measurement function re ; step 2, reparameterizing the noisy light field according to the reparameterization light field double-plane distance D re , and outputting the reparameterized noisy light field; and step 3, denoising the reparameterized noisy light field by using a hyperfan filter, and outputting the denoised reparameterization light field. The application can have a better denoising effect, and can better maintain scene edges and reflection information and the like.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of computational imaging and digital image processing, and particularly relates to a light field denoising method and device based on spectrum concentration degree and re-parameterization. BACKGROUND

[0002] Light field data will introduce noise in the acquisition process due to environmental light, detector material properties and other factors, which seriously affects the quality of subsequent light field imaging applications such as depth estimation, so it is necessary to denoise the light field first. Using a 4D hyperfan filter based on the spectrum branch set structure to denoise in the frequency domain, the denoising of light field data can be achieved by filtering out the noise energy outside the spectrum branch set. The characteristics of the hyperfan structure of the light field spectrum are determined by the scene geometry and the parameterization form of the light field, and the characteristics of the spectrum branch set structure affect the denoising effect of the filter. SUMMARY

[0003] The purpose of the present application is to provide a light field denoising method based on spectrum concentration degree and re-parameterization, which can introduce re-parameterization to the noisy light field, concentrate the spectrum branch set, and thus obtain better filtering denoising effect.

[0004] To achieve the above purpose, the present application provides a light field denoising method based on spectrum concentration degree and re-parameterization, which comprises:

[0005] Step 1, calculate the re-parameterized light field double-plane distance D by the minimum concentration degree metric function provided by the following formula (1) re ;

[0006] f(D)=αf1(D)+f2(D) (1)

[0007] In the formula, α is a weight parameter, f1(D) is a metric function of the angle between the two boundaries of the spectrum branch set, which is described as the following formula (2) and (3) according to the change of scene depth, and f2(D) is a symmetry degree metric function of the spectrum branch set, which is described as the following formula (4);

[0008] 1) The maximum depth of the scene Z max and the minimum depth of the scene Z min satisfy the following formula (2) and (3) at the same time:

[0009]

[0010] Where D is the initial double-plane distance of the light field, A and B are both intermediate parameters used to simplify the formula, A=Z max +Z min ,

[0011] ​2) the maximum depth of the scene Z max and the minimum depth of the scene Z min simultaneously satisfy

[0012]

[0013]

[0014] Step 2, according to the re-parameterization light field double plane distance D re re-parameterize the noisy light field, and output the re-parameterized noisy light field;

[0015] Step 3, use the hyperfan filter to denoise the re-parameterized noisy light field, and output the denoised re-parameterized light field.

[0016] Further, the method for obtaining the concentration degree metric function in step 1 comprises:

[0017] Step 11a, calculate the spectrum branch set two boundary angle Δθ by using the following formula (5), and take Δθ as the spectrum branch set two boundary angle metric function;

[0018]

[0019] wherein, and are the slopes of the spectrum branch set two boundaries, respectively;

[0020] Step 12a, for a fixed scene, define the spectrum branch set two boundary angle Δθ as the spectrum branch set two boundary angle metric function f1(D);

[0021] Step 13a, define the absolute value |k1+k2| of the slopes of the spectrum branch set two boundaries as the symmetry degree of the spectrum branch set, and for a fixed scene, define the spectrum branch set symmetry degree as the spectrum branch set symmetry degree metric function f2(D).

[0022] Further, the method for determining the weight parameter α in step 1 comprises:

[0023] Step 11b, select a plurality of weight parameters α, and use the concentration degree metric function f(D) provided by formula (1) to calculate the double plane distance D α corresponding to each weight parameter α, and then calculate the corresponding denoising result peak signal-to-noise ratio and structural similarity;

[0024] ​Step 12b: Compare the peak signal-to-noise ratio (PSNR) of the denoised results calculated under all weight parameters α with the structural similarity. Use the weight parameter α corresponding to the highest PSNR and structural similarity as the adjusted weight parameter α. opt .

[0025] Furthermore, in step 1, the biplane spacing D of the reparameterized optical field is calculated. re The methods include:

[0026] By minimizing f(D) under the weighting parameter α, the appropriate repetition parameter optical field biplane spacing D is calculated. re .

[0027] Furthermore, in step 2, the noisy optical field is reparameterized according to the following equation (6):

[0028]

[0029] In the formula, x re Here, u represents the image coordinates after reparameterization, and x represents the image coordinates before reparameterization.

[0030] The present invention also provides an optical field denoising device based on spectral concentration degree to introduce reparameterization, which includes:

[0031] The reparameterized optical field biplane spacing calculation unit is used to calculate the reparameterized optical field biplane spacing D using the minimization concentration metric function provided by equation (1). re ;

[0032] f(D)=αf1(D)+f2(D) (1)

[0033] In the formula, α is the weight parameter, f1(D) is the measurement function of the angle between the two boundaries of the spectral support, which is described by the following formulas (2) and (3) according to the change of scene depth, and f2(D) is the measurement function of the symmetry of the spectral support, which is described by the following formula (4).

[0034] 1) Maximum scene depth Z max and the minimum depth Z of the scene min Simultaneously satisfy In the following circumstances:

[0035]

[0036] Where D is the initial two-plane spacing of the light field, and A and B are intermediate parameters used to simplify the formula, A = Z max +Z min ,

[0037] 2) Maximum scene depth Zmax and scene minimum depth Z min simultaneously satisfy In the case of:

[0038]

[0039]

[0040] a reparameterization unit configured to calculate a reparameterized light field dual-plane distance D according to the reparameterization light field dual-plane distance D re reparameterize the noisy light field to output a reparameterized noisy light field;

[0041] a denoising unit configured to denoise the reparameterized noisy light field using a hyperfan filter to output a denoised reparameterized light field.

[0042] Further, the method for obtaining the minimization concentration degree metric function in the reparameterization light field dual-plane distance calculation unit comprises:

[0043] Step 11a, calculate the spectrum branch set two boundary angle Δθ using the following formula (5), and take Δθ as the spectrum branch set two boundary angle metric function;

[0044]

[0045] wherein, and are the slopes of the spectrum branch set two boundaries, respectively;

[0046] Step 12a, for a fixed scene, define the spectrum branch set two boundary angle Δθ as the spectrum branch set two boundary angle metric function f1(D);

[0047] Step 13a, define the absolute value |k1+k2| of the slopes of the spectrum branch set two boundaries as the symmetry degree of the spectrum branch set, and for a fixed scene, define the spectrum branch set symmetry degree as the spectrum branch set symmetry degree metric function f2(D).

[0048] Further, the method for determining the weight parameter α in the reparameterization light field dual-plane distance calculation unit comprises:

[0049] Step 11b, select a plurality of weight parameters α, and use the minimization concentration degree metric function f(D) provided by formula (1) to calculate the dual-plane distance D corresponding to each weight parameter α α , and then calculate the corresponding denoising result peak signal-to-noise ratio and structural similarity;

[0050] Step 12b, compare the peak signal-to-noise ratio and the structural similarity of the denoising results calculated under all weight parameters α, and take the weight parameter α corresponding to the maximum peak signal-to-noise ratio and the structural similarity as the weight parameter α obtained by debugging opt .

[0051] Further, the method for calculating the re-parameterized light field dual-plane distance D re of the re-parameterization unit includes:

[0052] Minimizing f(D) under the weight parameter α, the appropriate re-parameterized light field dual-plane distance D re is calculated.

[0053] Further, the re-parameterization unit re-parameters the noisy light field according to the following formula (6):

[0054]

[0055] In the formula, x re is the image point coordinate after re-parameterization, u is the viewpoint coordinate, and x is the image point coordinate before re-parameterization.

[0056] Since the present application explores the spectral branch set structure characteristics suitable for the denoising problem, thereby determining the light field re-parameterization mode, the noisy light field is re-parameterized before filtering denoising, which has good denoising effect in vision and peak signal-to-noise ratio (PSNR) and structural similarity (SSIM) quantitative evaluation index, and can better maintain scene edges and reflection information. BRIEF DESCRIPTION OF DRAWINGS

[0057] Figure 1 is a flowchart of the light field denoising method based on the spectral concentration degree and re-parameterization provided by the embodiment of the present application.

[0058] Figure 2 is a schematic diagram of the slope and angle of the spectral branch set structure determined by the light field dual-plane distance. DETAILED DESCRIPTION

[0059] The present application will be described in detail below in combination with the drawings and embodiments.

[0060] As Figure 1 shown, the light field denoising method based on the spectral concentration degree and re-parameterization includes:

[0061] Step 1, calculate the re-parameterized light field dual-plane distance D re by minimizing the concentration degree measurement function provided by the following formula (1):

[0062] f(D)=αf1(D)+f2(D) (1)

[0063] where α is a weight parameter, f1(D) is a measure function of the angle between two boundaries of the spectral dispersion, which is described as the following equations (2) and (3) according to the change of the scene depth, and f2(D) is a measure function of the symmetry degree of the spectral dispersion.

[0064] the maximum depth of the scene Z max and the minimum depth of the scene Z min satisfy In the case of min increasing from Z max , f1(D) first increases and then decreases:

[0065]

[0066] where D is the initial dual-plane distance of the light field, and A and B are intermediate parameters used for simplifying the formula, A = Z max + Z min ,

[0067] 2) the maximum depth of the scene Z max and the minimum depth of the scene Z min satisfy In the case of

[0068]

[0069] The symmetry degree of the spectral dispersion refers to the symmetry degree of the spectral dispersion about the ω x axis. Figure 2 The closer the sum of the slopes k1 and k2 of the two boundaries of the spectral dispersion is to 0, the more symmetrical the structure of the spectral dispersion is. Therefore, the absolute value |k1+k2| of the sum of the slopes of the two boundaries of the spectral dispersion is defined as the symmetry degree of the spectral dispersion. For a fixed scene, the symmetry degree of the spectral dispersion can be arranged as a function of the dual-plane distance D, which is defined as the measure function f2(D) of the symmetry degree of the spectral dispersion, as shown in the following equation (4). The smaller f2(D) is, the higher the symmetry degree of the spectral dispersion is.

[0070]

[0071] Step 2, reparameterize the noisy light field according to the dual-plane distance D re of the reparameterized light field. The existing technology can also be used to reparameterize the light field data according to the change relationship of the pixel coordinates before and after reparameterization by using the linear interpolation method.

[0072] Step 3: Use a Hyperfan filter to denoise the noisy light field after reparameterization, outputting the denoised reparameterized light field. The spectral support of the light field is improved, resulting in better denoising performance in both visual and quantitative indicators, while better preserving scene edges and reflections. Denoising can also be performed using existing filters; however, unlike this embodiment, using existing filters requires adjusting the filter parameters based on the spectral structure characteristics before and after reparameterization.

[0073] In one embodiment, the method for obtaining the minimization concentration measure function in step 1 includes:

[0074] Step 11a: Directly calculate the angle between the two boundaries of the spectral support as the spectral support boundary angle measure function f1(D), and use the maximum scene depth Z. max Minimum depth Z min Furthermore, the slope of the spectral support boundary can be obtained from the biplane spacing D. like Figure 2 As shown, the angle Δθ between the two boundaries of the spectral support is calculated using the following formula (5), and Δθ is used as the angular quantity function of the angle between the two boundaries of the spectral support.

[0075]

[0076] in, and These are the slopes of the two boundaries of the spectral support, respectively.

[0077] Step 12a: For a fixed scene, the included angle Δθ between the two boundaries of the spectral support is defined as the metric function f1(D) of the included angle between the two boundaries of the spectral support. The smaller f1(D) is, the smaller the included angle between the two boundaries of the optical field spectral support.

[0078] Step 13a: Define the absolute value of the sum of the slopes of the two boundaries of the spectral support, |k1+k2|, as the degree of symmetry of the spectral support. For a fixed scenario, the degree of symmetry of the spectral support is defined as the spectral support symmetry measurement function f2(D).

[0079] In one embodiment, the weight parameter α in step 1 can be obtained through empirical tuning, specifically including:

[0080] Step 11b: Select multiple weight parameters α, and use the minimization concentration measure function f(D) provided by equation (1) to calculate the biplane spacing D corresponding to each weight parameter α. α Before denoising, the image plane is reparameterized to a distance of D from the viewpoint plane. α At the location, the peak signal-to-noise ratio and structural similarity are calculated to quantitatively evaluate the denoising effect. Therefore, it is also necessary to calculate the corresponding peak signal-to-noise ratio and structural similarity of the denoising result.

[0081] Step 12b: Compare the peak signal-to-noise ratio (PSNR) of the denoised results calculated under all weight parameters α with the structural similarity. Use the weight parameter α corresponding to the highest PSNR and structural similarity as the adjusted weight parameter α. opt .

[0082] In one embodiment, step 1 calculates the biplane spacing D of the reparameterized optical field. re The methods specifically include:

[0083] Under the weight parameter α, preferably under the weight parameter α opt By minimizing f(D), the appropriate repetition parameter optical field biplane spacing D is calculated. re .

[0084] In addition to the above embodiments, an exhaustive method can also be used to randomly try different biplane spacings to obtain a reparameterized optical field biplane spacing that yields better denoising results. Of course, compared to the above embodiments, directly obtaining a suitable reparameterized optical field biplane spacing based on the target spectral structure characteristics that are beneficial for denoising, i.e., spectral concentration, greatly reduces the number of attempts and computational load.

[0085] In one embodiment, in step 2, the noisy optical field is reparameterized according to equation (6) above:

[0086]

[0087] In the formula, x re The image coordinates are reparameterized. The same ray can be represented by the viewpoint coordinates u and the pixel coordinates x.

[0088] This invention also provides an optical field denoising device based on spectral concentration, which includes a reparameterized optical field dual-plane spacing calculation unit, a reparameterization unit, and a denoising unit, wherein:

[0089] The reparameterized optical field biplane spacing calculation unit is used to calculate the reparameterized optical field biplane spacing D using the minimization concentration measure function provided by equation (1) above. re .

[0090] The reparameterized unit is used to adjust the biplane spacing D of the reparameterized optical field. re The noisy light field is reparameterized, and the reparameterized noisy light field is output.

[0091] The denoising unit is used to denoise the reparameterized noisy light field using a hyperfan filter and output the denoised reparameterized light field.

[0092] In one embodiment, the method for obtaining the minimization of the concentration degree metric function in the reparameterization light field dual-plane distance calculation unit includes:

[0093] Step 11a, calculate the spectrum branch set two boundary angle Δθ by the following formula (5), and take Δθ as the spectrum branch set two boundary angle metric function;

[0094]

[0095] Wherein, and are the slopes of the spectrum branch set two boundaries, respectively;

[0096] Step 12a, for a fixed scene, define the spectrum branch set two boundary angle Δθ as the spectrum branch set two boundary angle metric function f1(D);

[0097] Step 13a, define the absolute value |k1+k2| of the slopes of the spectrum branch set two boundaries as the symmetry degree of the spectrum branch set, and for a fixed scene, define the spectrum branch set symmetry degree as the spectrum branch set symmetry degree metric function f2(D).

[0098] In one embodiment, the method for determining the weight parameter α in the reparameterization light field dual-plane distance calculation unit includes:

[0099] Step 11b, select a plurality of weight parameters α, and use the minimization of the concentration degree metric function f(D) provided by formula (1) to calculate the dual-plane distance D corresponding to each weight parameter α α , and then calculate the corresponding denoising result peak signal-to-noise ratio and structural similarity;

[0100] Step 12b, compare the denoising result peak signal-to-noise ratio and structural similarity calculated under all weight parameters α, and take the weight parameter α corresponding to the maximum denoising result peak signal-to-noise ratio and structural similarity as the weight parameter α obtained by debugging opt .

[0101] In one embodiment, the method for calculating the reparameterization light field dual-plane distance D re in the reparameterization light field dual-plane distance calculation unit includes:

[0102] Minimize f(D) under the weight parameter α to obtain a suitable reparameterization light field dual-plane distance D re .

[0103] In one embodiment, the reparameterization unit reparameterizes the noisy light field according to the above formula (6).

[0104] The light field denoising method based on the spectrum concentration degree and the reparameterization provided by the application has good denoising effect in vision and quantitative evaluation indexes of peak signal-to-noise ratio (PSNR) and structural similarity (SSIM), and can well maintain scene edges and information such as reflection

[0105] Finally, it should be pointed out that: the above examples are only used to illustrate the technical solutions of the present application, but not to limit it. Those skilled in the art should understand that the technical solutions described in the foregoing embodiments can be modified, or some technical features can be replaced by equivalents; these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for light field denoising based on spectral concentration degree and reparameterization, characterized in that, Comprising: Step 1, compute the reparametrized light field biplane separation D by minimizing the concentration measure function provided by the following formula (1) re ; f(D) = alpha * f1(D) + f2(D) (1) In the formula, alpha is a weight parameter, f1(D) is a measure function of the angle between two boundaries of the spectral branch set, which is described as the following formula (2) and (3) according to the change of scene depth, and f2(D) is a measure function of the symmetry degree of the spectral branch set, which is described as the following formula (4); 1) scene maximum depth Z max and scene minimum depth Z min simultaneously satisfying in the case where: where D is the initial dual-plane separation of the light field, A and B are intermediate parameters used to simplify the formula, A = Z max +Z min , 2) scene maximum depth Z max and scene minimum depth Z min simultaneously satisfying in the case where: Step 2, reparameterize the light field according to the reparameterization light field bi-plane distance D re reparameterize the noisy light field, output the reparameterized noisy light field; Step 3, using a hyperfan filter to denoise the re-parameterized noisy light field, and outputting the denoised re-parameterized light field.

2. The method of claim 1, wherein the method is based on spectral flatness. The method for obtaining the minimization concentration measure function in step 1 comprises: Step 11a, calculating the angle Δθ between two boundaries of the spectral branch set by the following formula (5), and taking Δθ as the measure function of the angle between two boundaries of the spectral branch set; wherein and respectively are the slopes of the two boundaries of the spectral bin. Step 12a, for a fixed scene, defining the angle Δθ between two boundaries of the spectral branch set as the measure function f1(D) of the angle between two boundaries of the spectral branch set; Step 13a, defining the absolute value |k1+k2| of the sum of the slopes of the two boundaries of the spectral branch set as the symmetry degree of the spectral branch set, and for a fixed scene, defining the symmetry degree of the spectral branch set as the measure function f2(D) of the symmetry degree of the spectral branch set.

3. The light field denoising method based on spectral concentration degree introducing reparameterization according to claim 1 or 2, characterized in that, The method for determining the weight parameter alpha in step 1 comprises: Step 11b, select a plurality of weight parameters a, and use the minimization of the concentration degree metric function f(D) provided by formula (1) to calculate the double plane distance D corresponding to each weight parameter a α The corresponding denoising result peak signal-to-noise ratio and structural similarity are calculated again; Step 12b, compare the peak signal-to-noise ratio and structural similarity of the de-noising results calculated by all weight parameters a, and take the weight parameter a corresponding to the maximum peak signal-to-noise ratio and structural similarity as the weight parameter a obtained by debugging opt .

4. The light field denoising method based on spectral concentration degree introducing reparameterization according to claim 3, wherein, The method of calculating the reparameterized light field biplane separation D in step 1 re includes: Minimizing f(D) under the weight parameter a, the appropriate re-parameterized light field dual-plane separation D is calculated re .

5. The light field denoising method based on spectral flatness induced reparameterization according to any one of claims 1-3, wherein, In step 2, the noisy light field is re-parameterized according to the following formula (6): where x re are the reparameterized image point coordinates, u are the viewpoint coordinates, and x are the pre-reparameterized image point coordinates.

6. An optical field denoising device based on introducing reparameterization according to the degree of spectral concentration, characterized in that, Comprising: A reparameterized light field dual-plane distance calculation unit for calculating a reparameterized light field dual-plane distance D by minimizing a concentration measure function provided by the following equation (1) re ; f(D) = alpha * f1(D) + f2(D) (1) In the formula, alpha is a weight parameter, f1(D) is a measure function of the angle between two boundaries of the spectral branch set, which is described as the following formula (2) and (3) according to the change of scene depth, and f2(D) is a measure function of the symmetry degree of the spectral branch set, which is described as the following formula (4); 1) scene maximum depth Z max and scene minimum depth Z min simultaneously satisfying in the case where: where D is the initial dual-plane separation of the light field, A and B are intermediate parameters used to simplify the formula, A = Z max +Z min , 2) scene maximum depth Z max and scene minimum depth Z min simultaneously satisfying in the case where: a reparameterization unit configured to reparameterize the light field according to a reparameterization light field bi-plane distance D re reparameterize the noisy light field, output a reparameterized noisy light field; The denoising unit is configured to use a hyperfan filter to denoise the re-parameterized noisy light field, and output the denoised re-parameterized light field.

7. The apparatus for denoising light field based on reparameterization introduced by spectral centrality degree according to claim 6, wherein, The method for obtaining the minimization concentration measure function in the re-parameterized light field double-plane distance calculation unit comprises: Step 11a, calculating the angle Δθ between two boundaries of the spectral branch set by the following formula (5), and taking Δθ as the measure function of the angle between two boundaries of the spectral branch set; wherein and respectively are the slopes of the two boundaries of the spectral bin. Step 12a, for a fixed scene, defining the angle Δθ between two boundaries of the spectral branch set as the measure function f1(D) of the angle between two boundaries of the spectral branch set; Step 13a, defining the absolute value |k1+k2| of the sum of the slopes of the two boundaries of the spectral branch set as the symmetry degree of the spectral branch set, and for a fixed scene, defining the symmetry degree of the spectral branch set as the measure function f2(D) of the symmetry degree of the spectral branch set.

8. The device for denoising light field based on reparameterization introduced by spectral centrality degree according to claim 5 or 6, characterized in that, The method for determining the weight parameter alpha in the re-parameterized light field double-plane distance calculation unit comprises: Step 11b, select a plurality of weight parameters a, and use the minimization of the concentration degree metric function f(D) provided by formula (1) to calculate the double plane distance D corresponding to each weight parameter a α The corresponding denoising result peak signal-to-noise ratio and structural similarity are calculated again; Step 12b, compare the peak signal-to-noise ratio and structural similarity of the de-noising results calculated by all weight parameters a, and take the weight parameter a corresponding to the maximum peak signal-to-noise ratio and structural similarity as the weight parameter a obtained by debugging opt .

9. The apparatus for denoising light field based on reparameterization introduced by spectral centrality degree according to claim 8, wherein, Computing a reparameterized light field biplane distance D in a reparameterized light field biplane distance computing unit re The method comprises: Minimizing f(D) under the weight parameter a, the appropriate re-parameterized light field dual-plane separation D is calculated re .

10. The device for denoising of light field based on reparameterization introduced by spectral centrality degree according to any one of claims 6-9, wherein, The re-parameterization unit re-parameters the noisy light field according to the following formula (6): where x re is the reparameterized image point coordinate, u is the viewpoint coordinate, and x is the pre-reparameterized image point coordinate.

Images