Unsupervised light field parallax estimation method and device integrated with data statistical prior

Abstract

The invention discloses an unsupervised light field parallax estimation method and device integrated with data statistical prior. The method comprises the following steps: S0, learning on a parallax data set and representing data prior of an ideal parallax image; the method comprises the following steps: S1, calculating an initial parallax estimated value # imgabs0 # S2 according to a viewpoint plane light field sub-aperture image LF, and calculating fusion data statistics prior loss according to parallax images d0 in scenes corresponding to # imgabs1 # and # imgabs2 #; s3, transforming the sub-aperture image of the LF by using # imgabs3 # to obtain adaptive matching loss; s4, taking the difference information of the real center sub-aperture image as a weight, and obtaining edge perception smooth loss; and S5, carrying out weighted summation on the loss in S1, S2 and S3, and jointly returning back to the parallax estimation network to guide parameter updating of the parallax estimation network. According to the unsupervised optical field parallax estimation method integrated with data statistical prior, important technical support is provided for computational imaging and three-dimensional scene reconstruction.

Description

Technical Field

[0001] The present invention relates to the technical fields of computational imaging, machine vision, and digital image processing, and particularly to an unsupervised light field disparity estimation method and device incorporating data statistical prior knowledge. Background Art

[0002] For the unsupervised light field disparity estimation method incorporating data statistical prior knowledge, the performance of existing disparity estimation methods either depends on manually designed algorithms, making it difficult to fully exploit more light field information; or depends on training on a large number of light field data datasets containing ground-truth disparities. In practical applications, it is difficult to collect a large amount of light field data with ground-truth disparities, and there is a problem of insufficient generalization in supervised light field disparity estimation using the ground-truth as labels for training. More information contained in high-quality disparity maps awaits excavation.

[0003] The present invention introduces data statistical prior knowledge into the disparity estimation task of light field data, constructs a loss incorporating data statistical prior knowledge using the prior information learned from high-quality disparity maps, and constructs an adaptive matching loss with anti-occlusion function using the spatial-angular coupling relationship of light field data. It also introduces the edge information of the central view into the disparity map iteration process and uses an edge-aware smoothing loss to achieve a robust disparity estimation effect. Summary of the Invention

[0004] The purpose of the present invention is to provide an unsupervised light field disparity estimation method and device incorporating data statistical prior knowledge to overcome or at least mitigate at least one of the above-mentioned defects of the prior art.

[0005] To achieve the above purpose, the present invention provides an unsupervised light field disparity estimation method incorporating data statistical prior knowledge, which includes:

[0006] S0, using a conditional generative model to learn and characterize the data prior knowledge of the ideal disparity map on a disparity dataset

[0007] S1, according to the acquired sub-aperture images LF of the viewpoint plane light field, calculating an initial disparity estimate through Equation (1) provided by a disparity estimation module

[0008]

[0009] In the formula, F and D respectively represent the light field feature extraction module and the light field disparity recovery module of the disparity estimation module. The light field feature extraction module uses a max pooling layer with a kernel of a preset size to gradually downsample and extract the feature F(LF) from the LF. The light field disparity recovery module is used to gradually upsample the extracted feature F(LF) to the original spatial size using a transposed convolutional layer. As the spatial size decreases, the number of features increases accordingly. At each number of features, a convolutional layer and two residual blocks are used for feature extraction, and the features at the same scale are skipped to enhance the information flow. Finally, in the upsampling branch, two convolutional layers are applied to output the disparity map D(F(LF)) at each scale;

[0010] S2. According to that in step S1 and the disparity map d0 in the corresponding scenario calculate the fused data statistical prior loss E1 through a loss module that incorporates data statistical priors;

[0011] S3. Utilize that in step S1 Transform the sub-aperture image L of the LF in step S1 using formula (4), and interpolate to obtain the virtual central sub-aperture image u,v Then calculate the distance between and the true central sub-aperture image I of the LF 0,0 Finally, perform a Gaussian transform on the distance using formula (6) to obtain the adaptive matching loss E2; where (u, v), (x, y), and U×V respectively represent the angular coordinate index, intermediate coordinate index, and viewpoint set of the LF, and σ is the standard deviation of the Gaussian function;

[0012]

[0013] where (u, v), (x, y), and U×V respectively represent the angular coordinate index, intermediate coordinate index, and viewpoint set of the LF, and σ is the standard deviation of the Gaussian function;

[0014] S4. Use the differential information of the true central sub-aperture image I 0,0 as the weight, and calculate the weighted total variation of the current disparity map through formula (7) to obtain the edge-aware smoothing loss E3;

[0015]

[0016] where H×W represents the set of spatial coordinates of the LF in step S1, and w x , w y are the differentials of I 0,0 along the x and y directions respectively;

[0017] S5. Weightedly sum the losses calculated in S1, S2, and S3 to obtain the total loss Loss, and jointly backpropagate Loss to the disparity estimation network to guide the update of the parameters of the disparity estimation network:

[0018] Loss = λ1E1 + E2 + λ2E3 (8)

[0019] Wherein, λ1 and λ2 are weight parameters.

[0020] Furthermore, in step S4, w x = exp(-γ|Δ x I 0,0 |), w y = exp(-γ|Δ y I 0,0 |), and γ is the edge weight.

[0021] Furthermore, in step S2, the disparity map d0 is generated by iterative denoising starting from the noise d T ~ N(0, I) through Equation (3) provided by the conditional generation model:[[]]

[0022]

[0023] Wherein, d t represents the disparity map obtained by denoising at the t-th step, α t represents the noise coefficient at the t-th step, represents the cumulative product of the noise coefficients in the previous t steps, α i represents the noise coefficient at the i-th step, and ε η , η respectively represent the denoising network and its parameters trained on the external dataset.

[0024] Furthermore, the loss calculation module that incorporates data statistical priors in step S2 includes a variational autoencoder LE, a U-Net denoising module, a decoder, and a loss calculation module;

[0025] The variational autoencoder LE is used to receive the output encoded features in step S1

[0026] The U-Net denoising module is used to receive the features and output the denoised feature vector Z d ; wherein, the features are obtained by concatenating with the noise image Z ∈ sampled from the noise that follows a Gaussian distribution;

[0027] The decoder is used to receive the feature vector Z d and output the disparity map d0 corresponding to the scene;

[0028] The loss calculation module is used to calculate The distance to d0, which serves as the prior loss E1 for fused data statistics.

[0029] The present invention also provides an unsupervised light field disparity estimation device incorporating data statistics prior, which includes:

[0030] A conditional generative model, which is used to learn and represent the data prior of the ideal disparity map on the disparity dataset

[0031] A disparity estimation module, which is used to calculate the initial disparity estimate according to the acquired sub-aperture image LF of the viewpoint plane light field through Equation (1) provided by the disparity estimation module

[0032]

[0033] In the formula, F and D respectively represent the light field feature extraction module and the light field disparity recovery module of the disparity estimation module. The light field feature extraction module uses the max pooling layer with a preset-size kernel to gradually downsample and extract the feature F(LF) from LF. The light field disparity recovery module is used to gradually upsample the extracted feature F(LF) to the original spatial size using the transposed convolutional layer. As the spatial size decreases, the number of features increases accordingly; at each number of features, one layer of convolution and two residual blocks are used for feature extraction, and the features at the same scale are skipped to enhance the information flow; finally, in the upsampling branch, two convolutional layers are applied to output the disparity map D(F(LF)) at each scale;

[0034] A loss calculation module incorporating data statistics prior, which is used to calculate the prior loss E1 of fused data statistics through the loss module incorporating data statistics prior according to the disparity estimation module's and the disparity map d0 in the corresponding scenario ;

[0035] An adaptive matching loss calculation module, which is used to transform the sub-aperture image L of LF of the disparity estimation module using Equation (4), interpolate to obtain the virtual central sub-aperture image u,v and then calculate the distance to the real central sub-aperture image I of LF 0,0 ; Finally, perform Gaussian transformation on the distance using Equation (6) to obtain the adaptive matching loss E2;

[0036]

[0037] where (u, v), (x, y), and U×V respectively represent the angular coordinate index, the intermediate coordinate index, and the viewpoint set of LF, and σ is the standard deviation of the Gaussian function;

[0038] An edge-aware smoothing loss calculation module, which is used to use the differential information of the real central sub-aperture image I 0,0 as a weight, and calculate the weighted total variation of the current disparity map through Equation (7) to obtain the edge-aware smoothing loss E3;

[0039]

[0040] where H×W represents the set of spatial coordinates of the LF of the disparity estimation module, and w x , w y are the differentials of I 0,0 along the x and y directions respectively;

[0041] A disparity estimation network, which is used to weighted sum the losses calculated in the disparity estimation module, the loss calculation module integrating data statistical prior, and the adaptive matching loss calculation module to obtain the total loss Loss, and jointly back-propagate Loss to the disparity estimation network to guide the parameter update of the disparity estimation network:

[0042] Loss = λ1E1 + E2 + λ2E3 (8)

[0043] where λ1 and λ2 are weight parameters.

[0044] Further, in the edge-aware smoothing loss calculation module, w x = exp(-γ|Δ x I 0,0 |), w y = exp(-γ|Δ y I 0,0 |), and γ is the edge weight.

[0045] Further, in the disparity estimation module, the disparity map d0 is generated iteratively by denoising starting from the noise d T ~N(0, I) according to Equation (3) provided by the conditional generation model:

[0046]

[0047] where d t represents the disparity map obtained by denoising at the t-th step, α t represents the noise coefficient at the t-th step, represents the cumulative product of the noise coefficients in the previous t steps, α i represents the noise coefficient at the i-th step, and ε η , η respectively represent the denoising network and its parameters trained on an external dataset.

[0048] Furthermore, the loss module incorporating data statistical prior includes a variational autoencoder LE, a U-Net denoising module, a pre-trained decoder, and a loss calculation module;

[0049] The variational autoencoder LE is used to receive the output of step S1 and encode the features

[0050] The U-Net denoising module is used to receive the features and output the denoised feature vector Z d ; where the features are concatenated with the noise image Z sampled from noise obeying a Gaussian distribution ∈ ;

[0051] The decoder is used to receive the feature vector Z d and output the disparity map d0 for the corresponding scene;

[0052] The loss calculation module is used to calculate the distance between

[0053] and d0 as the loss E1 of the fused data statistical prior.

[0054] 1. By using the unsupervised light field disparity estimation method incorporating data statistical prior provided by the present invention, it is possible to estimate the light field disparity based on an unsupervised deep learning method, providing important technical support for computational imaging and three-dimensional scene reconstruction.

[0055] 2. The present invention can learn the mapping from light field to disparity without relying on a large number of light field data sets with ground truth disparities. It introduces the statistical prior information in the high-quality disparity data set into the unsupervised learning framework in a regularization manner to guide the optimization of network parameters. At the same time, by transforming the traditional matching cost through the Gaussian probability density function, the influence of occlusions, texture deficiencies, etc. on the disparity estimation is effectively reduced. The present invention can meet the requirements of unsupervised learning-based light field disparity estimation. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] Figure 1 is a schematic flowchart of the unsupervised light field disparity estimation method incorporating data statistical prior according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0057] In the drawings, the same or similar reference numerals are used to represent the same or similar elements or elements having the same or similar functions. The embodiments of the present invention will be described in detail below with reference to the drawings.

[0058] In the description of the present invention, the orientation or positional relationship indicated by terms such as "center", "longitudinal", "lateral", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", etc. is based on the orientation or positional relationship shown in the drawings. It is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be construed as a limitation on the protection scope of the present invention.

[0059] As Figure 1 shown, the unsupervised light field disparity estimation method incorporating data statistical prior provided by the embodiment of the present invention includes:

[0060] S0, using a conditional generative model, learning and characterizing the data prior of the ideal disparity map on a high-quality disparity dataset For example, for ideal disparity map data with different spatial resolutions, input it into the conditional generative model, and optimize the parameters of the model so that the model can achieve the effect of generating a high-quality disparity map.

[0061] S1, according to the acquired viewpoint plane light field sub-aperture image LF, through Equation (1) provided by the disparity estimation module Net, obtain the initial disparity estimate

[0062]

[0063] For example, select the publicly available light field dataset HCI-4D, and collect the viewpoint plane equally spaced light field sub-aperture image LF ∈ R 3×U×V×H×W , where R represents the set of real numbers, 3 represents the number of channels, U×V represents the angular resolution, and H×W represents the spatial resolution. The angular resolution of each scene in this dataset is 9×9, and the spatial resolution is 512×512, that is, the angular domain resolution U×V = 9×9, and the dimension of each sub-aperture image in the spatial domain is 3×H×W = 3×512×512. The light field data can be expressed as LF ∈ R 3×9×9×512×512 ,

[0064] As a preferred embodiment of the parallax estimation module Net, it includes: a light field feature extraction module F and a light field parallax estimation module D. Among them, the light field feature extraction module F uses a max-pooling layer with a kernel size of 2×2 to gradually downsample and extract features from the light field stack. The light field parallax recovery module uses a transposed convolutional layer to gradually upsample the extracted features to the original spatial size. As the spatial size decreases, the number of features increases from 64, 128, 256 to 512; at each number of features, one convolutional layer and two residual blocks are used for feature extraction, and features at the same scale are skipped to enhance the information flow; finally, in the upsampling branch, two convolutional layers are applied to output the parallax map at each scale. The specific process is as follows:

[0065] S1.1, Input the light field data LF, and obtain the feature vector f1 through the first convolutional module (ReLU). Pass f1 through the max-pooling module and the ReLU module to obtain the feature vector f2. Pass f2 through the max-pooling module and the ReLU module to obtain the feature vector f3. Pass f3 through the max-pooling module and the ReLU module to obtain the feature vector f4. Pass f4 through the max-pooling module and the ReLU module to obtain the feature vector f5.

[0066] S1.2, Pass f5 through the upsampling module to obtain d4. Concatenate f4 and d4 to obtain the feature concat[f4, d4]. Pass concat[f4, d4] through the upsampling module to obtain d3. Concatenate f3 and d3 to obtain the feature concat[f3, d3]. Pass concat[f3, d3] through the upsampling module to obtain d2. Concatenate f2 and d2 to obtain the feature concat[f2, d2]. Pass concat[f2, d2] through the upsampling module to obtain d1. Pass d1 through a 1×1 convolutional layer to obtain the parallax estimate.

[0067] S2, According to the initial parallax estimate in step S1 and the corresponding parallax map d0 in the scene, calculate the initial parallax map by incorporating the formula (2) provided by the loss module with data statistical prior, and obtain the distance between the sampling-generated parallax map d0, thereby obtaining the fused data statistical prior loss E1;

[0068]

[0069] wherein, is the two-norm of the vector.

[0070] In the above embodiment, the fused data statistical prior loss E1 can also be calculated It can be replaced by other distances that can measure the distance between two images, such as the Manhattan distance or Mahalanobis Distance between and d0.

[0071] S3. Using the initial disparity estimate Respectively transform each side view image of the LF in step S1, and interpolate to obtain a virtual central sub-aperture image Then calculate The distance from the true central sub-aperture image I of the LF in step S1 0,0 of Finally, according to the distance Perform Gaussian transformation to obtain the adaptive matching loss E2;

[0072]

[0073] Among them, U×V represents the viewpoint set of the LF in step S1, (x, y) is the spatial coordinate index of the LF in step S1, (u, v) is the angular coordinate index of the LF in step S1, σ is the standard deviation of the Gaussian function, and its value can be adjusted according to the characteristics of the dataset itself. If the scene contains a large amount of noise, a larger value can be selected. If there are multiple occlusions in the scene, a smaller value can be selected. Specifically, the selected value can be adjusted according to multiple experiments, or multiple σ values can be selected to calculate E2 and then take the average. Select the σ value that makes the experimental results optimal.

[0074] In the above embodiment, the adaptive matching loss E2 can also be The Manhattan distance or Mahalanobis Distance between and I 0,0 It can be replaced by other distances that can measure the distance between two images.

[0075] For example, input the disparity estimate in step S1 into module A, and input the surrounding sub-aperture images in the light field data LF ∈ R 3 ×9×9×512×512 in S1. Through disparity transfer by formula (4), transform them to the position of the central sub-aperture view to obtain new data Calculate the distance between the transformed data and the central sub-aperture one by one according to formula (6), and calculate the loss E2 ∈ R after Gaussian transformation according to formula (6)

[0076] S4. Use the differential information of the central view I 0,0 as the weight, and calculate the weighted total variation of the current disparity map through formula (7) to obtain the edge-aware smoothing loss E3 ∈ R;

[0077]

[0078] in, is the disparity map obtained in S1, H×W represents the spatial coordinate set of LF in step S1, w x 、w y I 0,0 Difference along the x and y directions.

[0079] S5, the weighted sum of the losses calculated in S1, S2, and S3 is jointly fed back to the disparity estimation network to guide the parameter update of the disparity estimation network:

[0080] Loss=λ1E1+E2+λ2E3 (8)

[0081] Where λ1 and λ2 are weight parameters. Their specific values can vary depending on the constraints and contribute to the overall loss. A larger λ1 emphasizes the role of the statistical prior for high-quality datasets, while a larger λ2 emphasizes the role of the local smoothness prior for parallax.

[0082] In one embodiment, in step S4, w x =exp(-γ|Δ x I 0,0 |), w y =exp(-γ|Δ y I 0,0 |), γ is the edge weight, set to 150. Alternative solutions: ω x =f(Δ x I 0,0 ), where the function f(·) can take a negative exponential function or other monotonically decreasing functions. The purpose is to make the weight smaller at the edge of the central sub-aperture image and reduce the smoothness requirement in the loss function.

[0083] In one embodiment, in step S2, the sampling formula of the disparity map d0 generated by conditional generative model sampling is as follows:

[0084]

[0085] From the noise d T Starting from N(0,I), d0 is generated by iterative denoising through formula (3), where d0 is the disparity map generated by sampling the conditional generation model, and d t is the disparity map obtained by denoising in the tth step, α t is the noise coefficient at step t, α tThe network weights learned by training a conditional generative model on an external high-quality disparity dataset are automatically updated based on the fused data statistical prior loss E1 in Equation (2) during the training process of the model, rather than being preset or manually adjusted by humans. Specifically, this parameter characterizes the mapping relationship of the model from the input to the generated d0 of the output; is the product of the noise coefficients α i in the first t steps, and ε η is the denoising network trained on an external dataset.

[0086] For example, input the disparity map output in step S1 into the conditional generation model, sample random noise d Figure 1 that follows a Gaussian distribution and has the same dimension as the input disparity T ∈R 512×512 , and gradually iterate by Equation (3) to output the disparity map d0 that incorporates the data statistical prior, where d0 ∈ R 512×512 , and then calculate the loss E1 ∈ R by Equation (2).

[0087] In one embodiment, to measure the gap between the current estimated disparity and the high-quality disparity map, the loss calculation module that incorporates the data statistical prior in step S2 includes a variational autoencoder LE, a U-Net denoising module, a pre-trained decoder, and a loss calculation module, where:

[0088] The variational autoencoder LE is used to receive the initial disparity estimate in step S1 and output the encoded features

[0089] The U-Net denoising module is used to receive the features and output the denoised feature vector Z d ; where the features are obtained by concatenating with the noise image Z ∈ sampled from the noise that follows a Gaussian distribution.

[0090] The decoder is used to receive the feature vector Z d and output the high-quality disparity map d0 for the corresponding scene.

[0091] The loss calculation module can calculate the Euclidean distance between and d0 according to Equation (2) as the fused data statistical prior loss E1.

[0092] Using the unsupervised light field disparity estimation method that incorporates the data statistical prior provided by the present invention, the light field disparity can be estimated based on the unsupervised deep learning method, providing important technical support for computational imaging and three-dimensional scene reconstruction.

[0093] An embodiment of the present invention further provides an unsupervised light field parallax estimation device incorporating data statistical prior, which includes a conditional generative model, a parallax estimation module Net, a loss calculation module incorporating data statistical prior, an edge-aware smoothing loss calculation module, and a parallax estimation network, where:

[0094] The conditional generative model is used to learn and represent the data prior of the ideal parallax map on the parallax dataset

[0095]

[0096] The parallax estimation module is used to calculate the initial parallax estimation value according to the acquired sub-aperture image LF of the viewpoint plane light field through formula (1) provided by the parallax estimation module.

[0097] The loss calculation module incorporating data statistical prior is used to calculate the fused data statistical prior loss E1 according to the and the corresponding parallax map d0 in the corresponding scenario through the loss module incorporating data statistical prior.

[0098] The adaptive matching loss calculation module is used to transform the sub-aperture image L of LF of the parallax estimation module by using formula (4), interpolate to obtain the virtual central sub-aperture image u,v and then calculate the distance from 0,0 to the true central sub-aperture image I of LF Finally, perform Gaussian transformation on the distance by using formula (6) to obtain the adaptive matching loss E2.

[0099] The edge-aware smoothing loss calculation module is used to use the differential information of the true central sub-aperture image I 0,0 as the weight, and calculate the weighted total variation of the current parallax map through formula ( (7) to obtain the edge-aware smoothing loss E3.

[0100] The parallax estimation network is used to sum the losses calculated in the parallax estimation module, the loss calculation module incorporating data statistical prior, and the adaptive matching loss calculation module with weights to obtain Loss, and jointly back-propagate Loss to the parallax estimation network to guide the parameter update of the parallax estimation network.

[0101] In one embodiment, in the edge-aware smoothing loss calculation module, w x = exp(-γ|Δ x I 0,0 |), w y = exp(-γ|Δ y I 0,0|), where γ is the edge weight.

[0102] In one embodiment, the loss module incorporating data statistical prior includes a variational autoencoder LE, a U-Net denoising module, a pre-trained decoder, and a loss calculation module. The variational autoencoder LE is used to receive the initial disparity estimation of step S1 and output the encoded features The U-Net denoising module is used to receive the features and output the denoised feature vector Z d . Among them, the features are concatenated with the noise image Z sampled from the noise obeying the Gaussian distribution ∈ . The decoder is used to receive the feature vector Z d and output a high-quality disparity map d0 for the corresponding scene. The loss calculation module is used to calculate the Euclidean distance between and d0 as the loss E1 of the integrated data statistical prior.

[0103] Using the unsupervised light field disparity estimation method incorporating data statistical prior provided by the present invention, the light field estimation method based on deep learning does not depend on a large-scale high-resolution training data set.

[0104] Finally, it should be pointed out that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than limiting them. Those of ordinary skill in the art should understand that the technical solutions recorded in the foregoing embodiments can be modified, or some of the technical features can be equivalently replaced; these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. An unsupervised light field parallax estimation method incorporating prior data statistics, characterized in that, Including: S0, utilize a conditional generative model to learn and characterize the data prior of the ideal disparity map on the disparity dataset S1. According to the sub-aperture image LF of the viewpoint plane light field collected, calculate the initial disparity estimate using Equation (1) provided by the disparity estimation module In the formula, F and D respectively represent the light field feature extraction module and the light field disparity recovery module of the disparity estimation module. The light field feature extraction module uses a max pooling layer with a preset kernel size to gradually downsample and extract the feature F(LF) from the LF. The light field disparity recovery module is used to gradually upsample the extracted feature F(LF) to the original spatial size using a transposed convolutional layer. As the spatial size decreases, the number of features increases accordingly. At each number of features, one convolutional layer and two residual blocks are used for feature extraction, and the features at the same scale are skipped to enhance the information flow. Finally, in the upsampling branch, two convolutional layers are applied to output the disparity map D(F(LF)) at each scale. S2. According to the and the parallax map d0 in the corresponding scenario of calculate the fused data statistical prior loss E1 through a loss module that incorporates data statistical priors; S3, using step S1 The sub-aperture image L of LF in step S1 is obtained by using formula (4). u,v Perform transformation and interpolation to obtain the virtual central sub-aperture image Recalculate The true central subaperture image I of LF 0,0 distance Finally, use formula (6) to calculate the distance Perform Gaussian transform to obtain adaptive matching loss E2; Among them, (u, v), (x, y), and U×V respectively represent the angular coordinate index, intermediate coordinate index, and viewpoint set of the LF, and σ is the standard deviation of the Gaussian function. S4, using the differential information of the real central sub-aperture image I 0,0 as the weight, calculate the weighted total variation of the current disparity map through Equation (7) to obtain the edge-aware smoothing loss E3; where, H×W represents the set of spatial coordinates of the LF in step S1, and w x , w y are the differences of I 0,0 in the x and y directions, respectively; S5. Weightedly sum up the losses calculated in S1, S2, and S3 to obtain the total loss Loss, and jointly backpropagate Loss to the disparity estimation network to guide the update of the parameters of the disparity estimation network: Loss = λ1E1 + E2 + λ2E3 (8) Among them, λ1 and λ2 are weight parameters.

2. The unsupervised light field parallax estimation method incorporating prior data statistics as described in claim 1, wherein, In step S4, w x = exp(-γ|Δ x I 0,0 |), w y = exp(-γ|Δ y I 0,0 |), where γ is the edge weight.

3. The unsupervised light field parallax estimation method incorporating prior data statistics as described in claim 1, characterized in that, In step S2, the disparity map d0 is iteratively denoised and generated starting from the noise d T ~N(0, I) through Equation (3) provided by the conditional generation model: Among them, d t represents the disparity map obtained by denoising in the t-th step, and α t represents the noise coefficient in the t-th step, represents the product of the noise coefficients in the previous t steps, α i represents the noise coefficient in the i-th step, and ε η and η respectively represent the denoising network and its parameters obtained by training on an external dataset.

4. The unsupervised light field parallax estimation method incorporating prior data statistics as described in any one of claims 1-3, characterized in that, The loss calculation module incorporating data statistical prior in step S2 includes a variational autoencoder LF, a U-Net denoising module, a decoder, and a loss calculation module. The variational autoencoder LE is used to receive the encoded features output in step S1 The U-Net denoising module is used to receive features and output the denoised feature vector Z d ; where the features are concatenated with the noise image Z sampled from noise following a Gaussian distribution ∈ to obtain; The decoder is used to receive the feature vector Z d , and output the disparity map d0 under the corresponding scenario; A loss calculation module for calculating the distance to d0 as the prior loss E1 for fused data statistics.

5. An unsupervised light field parallax estimation device incorporating prior data statistics, characterized in that, Including: A conditional generative model for learning and representing the data prior of an ideal disparity map on a disparity dataset A parallax estimation module, which is configured to calculate an initial parallax estimation value according to the acquired sub-aperture images LF of the viewpoint plane light field by using Equation (1) provided by the parallax estimation module In the formula, F and D respectively represent the light field feature extraction module and the light field disparity recovery module of the disparity estimation module. The light field feature extraction module uses a max pooling layer with a preset kernel size to gradually downsample and extract the feature F(LF) from the LF. The light field disparity recovery module is used to gradually upsample the extracted feature F(LF) to the original spatial size using a transposed convolutional layer. As the spatial size decreases, the number of features increases accordingly. At each number of features, one convolutional layer and two residual blocks are used for feature extraction, and the features at the same scale are skipped to enhance the information flow. Finally, in the upsampling branch, two convolutional layers are applied to output the disparity map D(F(LF)) at each scale. A loss calculation module incorporating data statistics prior, which is used to calculate the fused data statistics prior loss E1 through the loss module incorporating data statistics prior according to the and the corresponding disparity map d0 in the corresponding scenario; An adaptive matching loss calculation module, which is used to transform the sub-aperture image L of the LF of the disparity estimation module by using Equation (4), and interpolate to obtain a virtual central sub-aperture image u,v Then calculate the distance between the virtual central sub-aperture image and the true central sub-aperture image I of the LF 0,0 Finally, perform Gaussian transformation on the distance by using Equation (6) to obtain the adaptive matching loss E2; ​ Among them, (u, v), (x, y), and U×V respectively represent the angular coordinate index, intermediate coordinate index, and viewpoint set of the LF, and σ is the standard deviation of the Gaussian function. Edge-aware smoothing loss calculation module, which is used to use the differential information of the real central sub-aperture image I 0,0 as a weight, and calculate the weighted total variation of the current disparity map through Equation (7) to obtain the edge-aware smoothing loss E3; where, H×W represents the set of spatial coordinates of the LF of the disparity estimation module, w x , w y are the differences of I 0,0 along the x and y directions, respectively; The disparity estimation network is used to weightedly sum up the losses calculated in the disparity estimation module, the loss calculation module incorporating data statistical prior, and the adaptive matching loss calculation module to obtain the total loss Loss, and jointly backpropagate Loss to the disparity estimation network to guide the update of the parameters of the disparity estimation network: Loss = λ1E1 + E2 + λ2E3 (8) Among them, λ1 and λ2 are weight parameters.

6. The unsupervised light field parallax estimation device incorporating prior data statistics as described in claim 5, characterized in that In the edge-aware smoothing loss calculation module, w x = exp(-γ|Δ x I 0,0 |), w y = exp(-γ|Δ y I 0,0 |), where γ is the edge weight.

7. The unsupervised light field parallax estimation device integrating prior data statistics according to claim 5, wherein In the parallax estimation module, the parallax map d0 is iteratively denoised and generated starting from the noise d T ~N(0, I) according to Equation (3) provided by the conditional generation model: Among them, d t represents the disparity map obtained by denoising at the t-th step, and α t represents the noise coefficient at the t-th step, represents the cumulative product of the noise coefficients in the previous t steps, α i represents the noise coefficient at the i-th step, and ε η and η respectively represent the denoising network and its parameters trained on the external dataset.

8. The unsupervised light field parallax estimation device incorporating prior data statistics as described in any one of claims 5-7, characterized in that The loss module incorporating data statistical prior includes a variational autoencoder LE, a U-Net denoising module, a pre-trained decoder, and a loss calculation module. The variational autoencoder LE is used to receive the encoded features output in step S1 The U-Net denoising module is used to receive features and output the denoised feature vector Z d ; among which, the features are obtained by concatenating with the noise image Z sampled from noise following a Gaussian distribution ∈ ; The decoder is used to receive the feature vector Z d , and output the disparity map d0 under the corresponding scenario; A loss calculation module, which is used to calculate the distance to d0 as the prior loss E1 of the fusion data statistics.

Images