A light field camera calibration method, system, device, medium, and product

Abstract

The application discloses a light field camera calibration method, system, device, medium and product, relates to the field of camera calibration, and comprises the following steps: acquiring white images shot by a to-be-calibrated light field camera at multiple aperture scale values; determining initial values of internal parameters of the to-be-calibrated light field camera and high-confidence center points according to the white images; determining corner point blur radii of a calibration board image and optimized corner point coordinates based on a geometric observation mode of a virtual depth domain according to the initial values of the internal parameters; performing iterative optimization on a total cost function by using a Levenberg-Marquardt algorithm according to the initial values of the internal parameters, the high-confidence center points, the corner point blur radii and the optimized corner point coordinates, so as to obtain optimized internal parameters of the camera and optimized external parameters of the camera, and to complete calibration of the light field camera; and the total cost function is determined according to corner point re-projection errors, blur radius re-projection errors and micro-image center re-projection errors. The application improves the accuracy of light field camera calibration.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of camera calibration, and particularly to a light field camera calibration method, system, device, medium and product. BACKGROUND

[0002] The calibration method of a focusing light field camera generally includes three key steps: initial estimation of calibration parameters, feature extraction, and nonlinear optimization. However, when facing the calibration of a multi-focal light field camera, the two steps of feature extraction and nonlinear optimization still have problems such as uneven quality of feature points, high complexity of optimization problems, and sensitivity to initial estimation values. The specific reasons are analyzed as follows:

[0003] (1) Quality problem of feature points.

[0004] At present, the corner detector in the more advanced method mainly includes a saddle point detection method and a template matching method. The main idea of the saddle point detection method is that when there is a corner point in the micro image, the corner point can be found at the "saddle point / maximum minimum point" in the intensity domain. The two axes are not necessarily orthogonal, but determine the direction of the saddle seat centering on the corner point, and this is the two maximum and minimum cumulative intensity lines passing through the image. The search of these lines determines two potential saddle axes, and the corner point is located at their intersection. The main idea of the template matching method is to describe the appearance characteristics (such as intensity distribution or geometric shape) of the corner point through a pre-defined template, adjust the template parameters and calculate the similarity (such as cross-correlation, mean square error) to best fit the target corner point and calculate its coordinate value.

[0005] For a corner point in a real three-dimensional world, the micro image formed by the multi-focal light field camera will show different degrees of blurring phenomenon. The existing corner point detector uses the intensity change of the image as a feature to extract the corner point. When the micro image is more blurred, the intensity change at the corner point will be distributed in the pixel area within a certain range centering on the real corner point. Therefore, the corner point coordinates extracted by the existing corner point detector are inaccurate or even wrong, which further leads to poor accuracy of the calculated camera parameters. It is worth noting that the greater the blurring degree of the micro image, the greater the intensity change distribution area, and the more likely the accuracy of the extracted corner point coordinates is poor.

[0006] (2) Sensitivity of nonlinear optimization to initial estimation value.

[0007] Nonlinear optimization is to optimize the initial estimates of camera intrinsic and extrinsic parameters to minimize the reprojection error. Existing methods usually use nonlinear optimization algorithms such as Levenberg-Marquardt algorithm or Powell method to optimize all parameters at once. These nonlinear optimization algorithms usually rely on the initial estimate value to adjust the parameters iteratively to approximate the optimal solution of the objective function. However, this way of optimizing all parameters in a single process may be sensitive to the initial estimate value. The main reasons include the following three points: first, due to the complexity of the nonlinear function, different initial estimates may cause the nonlinear optimization algorithm to iterate along different paths, resulting in different solutions; second, there may be multiple extreme points in the nonlinear optimization problem, including local optimum and global optimum. If the initial point is not properly selected, the algorithm may fall into a local optimal solution and fail to find a global optimal solution. This is because the nonlinear function may have a flat gradient or steep curvature near the local optimal solution, making it difficult for the algorithm to escape the local optimal region; finally, if the initial point is not properly selected, the nonlinear optimization algorithm may not converge to the optimal solution due to the accumulation of calculation errors, rounding errors and other factors during the iteration process. SUMMARY

[0008] The purpose of the present application is to provide a light field camera calibration method, system, device, medium and product to improve the accuracy of light field camera calibration.

[0009] To achieve the above-mentioned purpose, the present application provides the following solutions:

[0010] In a first aspect, the present application provides a light field camera calibration method, comprising:

[0011] Obtaining a plurality of white images taken by a light field camera to be calibrated at a plurality of aperture scale values;

[0012] According to the plurality of white images, determining the initial value of the intrinsic parameter of the light field camera to be calibrated and the high-confidence center point;

[0013] According to the initial value of the intrinsic parameter, based on the geometric observation mode of the virtual depth domain, determining the corner point blur radius and the optimized corner point coordinates of the calibration board image; the calibration board image is an image taken by the light field camera to be calibrated from multiple angles on a black and white checkerboard;

[0014] According to the initial value of the intrinsic parameter of the light field camera to be calibrated, the high-confidence center point, the corner point blur radius and the optimized corner point coordinates of the calibration board image, the Levenberg-Marquardt algorithm is used to iteratively optimize the total cost function to obtain the optimized camera intrinsic parameter and the optimized camera extrinsic parameter, and the calibration of the light field camera is completed; the total cost function is determined according to the corner point reprojection error, the blur radius reprojection error and the micro-image center reprojection error.

[0015] According to the specific embodiments provided in the application, the application has the following technical effects:

[0016] The application provides a light field camera calibration method, system, device, medium and product. A plurality of white images photographed by a to-be-calibrated light field camera at a plurality of aperture scale values are acquired. An initial value of an intrinsic parameter of the to-be-calibrated light field camera and a high-confidence center point are determined according to the plurality of white images. A corner point blur radius of a calibration board image and an optimized corner point coordinate are determined according to the initial value of the intrinsic parameter, and based on a geometric observation mode of a virtual depth domain. The initial value of the intrinsic parameter of the to-be-calibrated light field camera, the high-confidence center point, the corner point blur radius of the calibration board image and the optimized corner point coordinate are used to perform iterative optimization on a total cost function by using a Levenberg-Marquardt algorithm, so as to obtain an optimized camera intrinsic parameter and an optimized camera extrinsic parameter, and to complete the calibration of the light field camera. The total cost function is determined according to a corner point re-projection error, a blur radius re-projection error and a micro-image center re-projection error. In the application, the initial value of the intrinsic parameter of the to-be-calibrated camera is determined by using the white image, so as to avoid falling into local optimization. In addition, the geometric observation mode of the virtual depth domain is used to perform step-by-step optimization on adjacent blur micro-image corner points, so as to improve the quality of the extraction of the corner point coordinate, and to improve the accuracy of the calibration of the light field camera. BRIEF DESCRIPTION OF DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed in the embodiments will be briefly introduced as follows. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can also be obtained by those skilled in the art without creative labor.

[0018] Figure 1 A flowchart of a light field camera calibration method provided by an embodiment of the present application is shown in the figure.

[0019] Figure 2 A pixel error curve graph of the extracted coordinates and the real coordinates of 60 corner points selected in R2 in the present application is shown in the figure.

[0020] Figure 3 A pixel error curve graph of the extracted coordinates and the real coordinates of 60 corner points selected in R5 in the present application is shown in the figure.

[0021] Figure 4 A display graph of four types of corner point coordinates in R2 in the present application is shown in the figure.

[0022] Figure 5 An enlarged view of the green square box in the figure. Figure 4

[0023] Figure 6 An enlarged view of the green square box in the figure.​Figure 5 Zoom-in view of the medium red square;

[0024] Figure 7 A display diagram of four types of corner point coordinates in R5 of the present application;

[0025] Figure 8 A display diagram of Figure 7 Zoom-in view of the medium green square;

[0026] Figure 9 A display diagram of Figure 8 Zoom-in view of the medium red square;

[0027] Figure 10 A structural schematic diagram of a computer device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0028] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the scope of protection of the present application.

[0029] The above purposes, features and advantages of the present application can be more obvious and easy to understand. The present application will be further described in detail below with reference to the drawings and specific embodiments.

[0030] As a basic and important primary step in light field three-dimensional reconstruction, the accuracy of the camera parameters calculated by light field camera calibration directly determines the reconstruction quality of the three-dimensional model. In the face of multi-focal light field cameras, existing methods have great challenges in the accuracy and sensitivity of the calibration parameters. The present application proposes a light field camera calibration method. First, a high-credibility center point and radius are estimated using multiple white images under different aperture values, and the initial value of the internal parameter is calculated according to the linear relationship between the aperture and the radius based on the defocusing theory. Second, a geometric observation mode of virtual depth domain is constructed, and the blurring degree of the same corner point is quantified and classified. The clear-to-blurred corner point coordinates are optimized in turn according to the step-by-step diffusion mode. Then, the micro-image center light projection equation is constructed to assist the optimization of the micro-lens array parameters, and an adaptive credibility weight considering blurring is established to construct a global nonlinear optimization. Finally, the public data set and the actual data set are used for experimental verification and analysis. Compared with the famous methods of Labussière, Nousias and Liu, the present method performs best in terms of accuracy index and calibration sensitivity of corner feature extraction, calibration of internal and external parameters when facing multi-focal light field cameras, which indicates the advancement and robustness of the present method.

[0031] In one exemplary embodiment, as Figure 1As shown, a light field camera calibration method is provided, comprising the following steps:

[0032] S1: Obtain multiple white images captured by a light field camera to be calibrated at multiple aperture scale values.

[0033] S2: Determine the initial value of the intrinsic parameters of the light field camera to be calibrated and a high-confidence center point according to the multiple white images.

[0034] As an optional implementation, S2 specifically comprises:

[0035] S21: Detect the micro image centers in all the white images at an aperture scale value using the sub-pixel precision intensity centroid method; one white image corresponds to multiple micro image centers.

[0036] S22: Take the median value of the micro image centers as the high-confidence center point of the light field camera to be calibrated.

[0037] S23: Based on the micro image centers and the median value of the micro image centers, screen all the white images at an aperture scale value to obtain screened white images.

[0038] As an optional implementation, S23 specifically comprises:

[0039] S231: Calculate the Euclidean distance between each micro image center in each white image and the median value of the micro image centers.

[0040] S232: According to the Euclidean distance, degrade the micro lens array into a grid image to generate an error map.

[0041] S233: Determine the similarity between the error map and the corresponding white image using the structural similarity index.

[0042] S234: If the similarity is less than the similarity error, eliminate the white image corresponding to the error map to obtain the screened white images.

[0043] S24: According to the micro image in the center region of the screened white image, determine the micro image radius in combination with the covariance matrix.

[0044] S25: According to the micro image radius and the actual aperture value, determine the linear parameters of the linear relationship between the micro image radius and the actual aperture value in the defocus theory using the least square method.

[0045] S26: Determine the initial value of the intrinsic parameters of the light field camera to be calibrated according to the linear parameters and the factory parameters of the light field camera to be calibrated.

[0046] In practical applications, multiple white images at different aperture scale values ​​are used to estimate the high-confidence center point and radius. Based on the defocus theory, the linear relationship between the actual aperture value and the micro-image radius is fitted to calculate the initial value of the intrinsic parameters.

[0047] (1) Install the light diffuser on the main lens and adjust the camera aperture to different aperture scale values ​​{N0, N1, ..., N}. m}, take multiple white images I0, I1, ..., I at the same aperture scale value. j N m →{I0,I1,...,I j The intensity centroid method with sub-pixel precision is used to detect the aperture scale value N. m The micro-image center in all white images below, where white image I j With micro-image center C k,l The relationship is I j →{C 0,0 C 0,1 ,...,C k,l The aperture scale value N is obtained by taking the median value of the relationship between multiple white images and the center of the micro-image. m Median of the center of the lower micro-image A high-confidence center point is obtained.

[0048] (2) For the same aperture scale value N m For all white images below, calculate the micro-image center C corresponding to each microlens (k,l). k,l With median The Euclidean distance D(k,l) is used to degenerate the microlens array into a raster image to generate an error map, where one pixel represents a microlens and the pixel value is a normalized range of 0 to 255 for D(k,l). Using the Structural Similarity Index (SSIM), the similarity between error maps is evaluated by comparing pixel values, brightness, contrast, and structural information, and white images corresponding to error maps with low similarity are filtered out and removed. For each retained white image, the covariance matrix is ​​constructed using the second-order central moment for the microimage in the center region of the white image, and the square root of the largest eigenvalue of the covariance matrix is ​​solved. The radius R of the microimage is calculated using formula (1):

[0049]

[0050] Where, σ xx Let σ be the variance of the microimage in the x-direction. yy Let σ be the variance of the microimage in the y-direction. xy σ represents the variance of the micro-image in the x and y directions. yxThe variance of the micro image in the y direction and the x direction.

[0051] (3) Using the correspondence between the lens aperture scale value AV and the actual aperture value N Calculate the actual aperture value N corresponding to each white image. According to the linear relationship between the micro image radius R and the actual aperture value N in the defocus theory:

[0052] R i (N -1 )=m·N -1 +q i (2)

[0053] Where i represents the type of microlens, R i represents the radius of the micro image of type i, and m and q i are linear parameters. The linear equation is fitted by least squares estimation using the micro image radius and the actual aperture value, and the linear parameters m and q i are solved.

[0054] (4) Using the m and q i solved above and the camera factory parameters (focal length F of the main lens, sensor size s), further calculate the initial value of the intrinsic parameter of the light field camera to be calibrated.

[0055] S3: According to the initial value of the intrinsic parameter, determine the corner blur radius of the calibration board image and the optimized corner coordinates based on the geometric observation mode in the virtual depth domain; the calibration board image is an image captured by the light field camera to be calibrated from multiple angles on a black and white checkerboard.

[0056] As an optional implementation, S3 specifically includes:

[0057] S31: Determine the initial corner coordinates of each of the calibration board images using a template matching algorithm.

[0058] S32: According to each of the initial corner coordinates, determine a plurality of corner clusters using a density-based clustering algorithm.

[0059] S33: Determine the corner blur radius according to the initial value of the intrinsic parameter and the linear parameter.

[0060] S34: According to the corner blur radius, classify the initial corners in each corner cluster to obtain a clear corner set, a sub-clear corner set, and a blurred corner set.

[0061] S35: Optimize the initial corners in the clear corner set, the sub-clear corner set, and the blurred corner set using the geometric observation mode in the virtual depth domain to obtain the optimized corner coordinates.

[0062] In practical applications, based on the linear parameters m, q iand the initial value of the internal reference, the same corner blurring imaging degree is quantified and classified, and the geometric observation mode of the virtual depth domain is constructed. The corner coordinates from clear to blur are optimized in turn in the form of step-by-step diffusion, and the specific process is as follows:

[0063] (1) Black and white checkerboard is used as a calibration board, and a multi-focus focusing type plenoptic camera is used to collect light field original images (i.e. calibration board images) of the black and white checkerboard from multiple angles. For each light field original image, a template matching algorithm is used to locate the corner points, and DBSCAN (Density-Based Spatial Clustering of Applications with Noise) is used to cluster the detected corner point coordinates within a certain distance into clusters to correspond to the same corner point on the object space black and white checkerboard. Based on the linear parameters m, q i , the corner blurring radius is calculated:

[0064]

[0065] Wherein, r represents the corner blurring radius, Δc represents the diameter of the micro image, is the proportionality coefficient between the micro image and the micro lens, D is the micro lens array to the main lens distance; d is the micro lens array to the sensor distance, v represents the virtual depth, q' i is a linear constant, q' i =q i +Δc / 2, and one micro lens type corresponds to one q' i .

[0066] According to the size of the corner blurring radius, the corner points in each corner point cluster are classified into clear corner point set, sub-clear corner point set and blurred corner point set.

[0067] (2) All corner points in the corner point cluster correspond to the same virtual image point. Using two or more corner point coordinates I i and I j , the relative distance (virtual depth) v between the micro lens array and the virtual image is calculated by the triangulation relationship.

[0068]

[0069] Wherein, M i is the center of the i-th micro lens; M j is the center of the j-th micro lens; |M j -M i | is the Euclidean distance (baseline) between the center of the i-th micro lens and the center of the j-th micro lens; I i is the i-th corner point in the micro image; I j is the j-th corner point in the micro image; |I i-I j | Represents a corner point I in a micro-image i With corner point I j Euclidean distance, |M i -M j |-|I i -I j |For corner point I i With corner point I j The parallax. After estimating the virtual depth, a corner point I is used. i Coordinates based on epipolar and geometric constraints will I i Back-projected onto the micro-image containing any corner point within the corner cluster, and the back-projected coordinates I are obtained. j ' This forms a geometric observation pattern for the virtual depth domain:

[0070]

[0071] Where v is the virtual depth domain; C i C is the center of the i-th micro-image; j The center of the j-th micro-image; Let I be the unit direction vector from the j-th micro-image center to the i-th micro-image center, i.e., the epipolar direction; j ' Let be the corner point after back-projection of the i-th corner point in the micro-image.

[0072] (3) Select a subset of corner points that are close to the center of the micro-image from the set of sharp corner points. Use the coordinates of this subset of corner points to perform pairwise forward intersection to obtain virtual depth values, and take the mean as the virtual depth of the cluster. Based on the geometric observation mode of the virtual depth domain, backproject all sharp corner points into the micro-image where the set of second-sharp corner points is located and take the median as the optimized corner point coordinates. Fit the pixel blocks around the corner points and take the local maximum as the refined corner point coordinates. Then, using the set of sharp corner points and the optimized set of second-sharp corner points as known data, use the same method to optimize the corner point coordinates of the blurred corner point set.

[0073] S4: Based on the initial intrinsic values ​​of the light field camera to be calibrated, the high confidence center point, the corner blur radius of the calibration board image, and the optimized corner coordinates, the Levenberg-Marquardt algorithm is used to iteratively optimize the total cost function to obtain the optimized camera intrinsic and extrinsic parameters, thus completing the calibration of the light field camera; the total cost function is determined based on the corner reprojection error, the blur radius reprojection error, and the micro-image center reprojection error.

[0074] In practical applications, based on the high reliability center point and the initial value of the internal parameter, the optimized corner point coordinate and the corner point blur radius, the micro image center light ray projection equation is constructed to assist in optimizing the micro lens array parameter, and the blur adaptive reliability weight is established, so as to construct the global nonlinear optimization, optimize the internal parameter and external parameter of the light field camera to be calibrated, and complete the calibration of the light field camera to be calibrated.

[0075] (1) The micro image of different focal length micro lens presents different blur, and the feature point in the micro image space is represented by the center and radius, that is, p = [x, y, p, 1] T , and the projection model is constructed:

[0076]

[0077]

[0078] Wherein, x is the coordinate of the corner point on the x axis; y is the coordinate of the corner point on the y axis; p is the blur radius of the corner point; P(i, k, l) represents the projection matrix of the micro lens with the number (k, l) and the type i considering the blur degree, which is formula (7), wherein, P(k, l) is the virtual image point projection matrix of the micro lens with the number (k, l) considering the blur radius, Kf i ) is the thin lens projection matrix of a given focal length. T u (k, l) is the pose of the micro lens (k, l) expressed in the camera coordinate system. The function represents the modeling of the lateral distortion. T c is the pose of the main lens relative to the world coordinate system. p w = [x, y, z, 1] T represents the three-dimensional coordinates of the feature point in the real world. The internal parameter of the light field camera to be calibrated consists of 16+i parameters, including: the focal length F of the main lens and its five lateral distortion coefficients Q1, Q2, Q3, P1 and P2, the sensor translation d and (u0, v0), wherein u0 is the x axis coordinate of the image principal point, v0 is the y axis coordinate of the image principal point, the pose of the micro lens array includes three rotation angles θ x , θ y , θ z and three translation distances t x , t y , D, D is the distance between the micro lens array and the main lens, the pitch of the micro lens is Δμ, the focal length of the i-th micro lens is f(i), i = 0, 1, 2...

[0079] (2) The high reliability center point and the initial estimated micro lens center and the main lens center are used to reversely build the light diffuser reflected light. Based on such light, the center light ray projection model is established, and the cost function of the observation value and the re-projection calculation value is calculated:

[0080]

[0081] where MI denotes the set of parameters related to the microlens array to be optimized, C k,l denotes the center of the micro-image, denotes the object point on the diffuse reflector in the real world. The Levenberg-Marquardt algorithm is used for iterative optimization to obtain the optimized values of the parameters related to the microlens array, MI, which mainly include: the distance D of the microlens array to the main lens; the distance d of the microlens array to the sensor; the rotation angle θ and the translation distance t of the microlens array relative to the main lens.

[0082] (3) The optimized corner point coordinates are used to calculate the weight of the corner point according to the corner point blur radius, and the weighted average gravity center is calculated. The initial extrinsic parameters are estimated by fixing the intrinsic parameters and using the P3P method. Based on the center ray assisted microlens array parameter optimization, the intrinsic parameters and the initial extrinsic parameters are established, the intrinsic parameters are fixed, the cost function is constructed using the clear corner point set, and the extrinsic parameters are optimized:

[0083]

[0084] where E denotes the set of extrinsic parameters to be optimized, denotes the corner point coordinates detected in the nth frame of the calibration board image; denotes the corner point coordinates of the checkerboard in the real world;∏ k,l (k, l) microlens. The Levenberg-Marquardt algorithm is used for iterative optimization.

[0085] (4) After optimization, the object points are re-projected into the micro-image, and the re-projection error of each corner point is calculated. If the re-projection error of the corner point is greater than a certain threshold, the corner point is directly excluded; otherwise, it is retained. For the corner points that meet the threshold, the corner point blur radius r is recalculated. A total cost function is constructed for global nonlinear optimization. In one iteration, the intrinsic and extrinsic parameters are optimized. The total cost function is mainly composed of the corner point re-projection error, the blur radius re-projection error, and the micro-image center re-projection error:

[0086] The total cost function is:

[0087]

[0088] where T is the set of camera intrinsic parameters to be optimized and camera extrinsic parameters to be optimized; Θ(MI) is the cost function of the micro-image center re-projection error. denotes the corner point cost function considering the blur perception feature, The normalized weight corresponding to the quantized blur degree of the corner point is weighted by a weighting function:

[0089]

[0090] wherein, is the blur radius of the corner point in the nth frame of the calibration plate image, and a is the minimum weight value, generally set to 0.4; represents the minimum value of the blur radius in the nth frame.

[0091] The effectiveness of the proposed method is verified from the aspects of the accuracy of corner point feature extraction, the accuracy of calibration and robustness. The experimental equipment, experimental data, experimental results and analysis are as follows.

[0092] 1. Experimental equipment:

[0093] The light field cameras used in the experiment mainly include Raytrix R12, Raytrix R32 and HR260, and the detailed parameters are as follows:

[0094] The main lens used by Raytrix R12 is Nikon AF Nikkor F / 1.8D with a focal length of 50 mm; the microlens array has a total of 176x152 microlenses with three focal lengths arranged in cross; the sensor is Basler boA4000-KC with a pixel size of 0.0055 mm and a resolution of 4080x3068.

[0095] The main lens used by Raytrix R32 is Nikon AF Nikkor F / 2 with a focal length of 35 mm; the microlens array has a total of 194x126 microlenses with three focal lengths arranged in cross; the sensor is Basler boA6500-36CC with a pixel size of 0.0032 mm and a resolution of 6560x4948.

[0096] The main lens of HR260 has a focal length of 105 mm; the microlens array has a total of 65x50 microlenses with three focal lengths arranged in cross; the sensor has a pixel size of 0.0037 mm and a resolution of 6240x4168.

[0097] 2. Experimental data:

[0098] The light field images used in the experiment mainly include simulation data and light field camera shooting data. Among them, the simulation data is mainly used for experimental verification of the accuracy of corner point feature extraction; the light field camera shooting data is mainly used for experimental verification of the accuracy of calibration and robustness.

[0099] Simulation data.

[0100] The simulation data is mainly based on a total of 5 groups of representative data sets simulated by ray tracing method, and the detailed parameters are shown in Table 1. The simulation data can obtain the corner point coordinates of the original image before camera calibration, so the simulation data can be used to measure the accuracy of the corner point extraction method.

[0101] In Table 1, Resolution represents the resolution of the image; D mi represents the radius of the micro image, in pixels; F is the focal length of the main lens, in millimeters; D is the distance from the main lens to the microlens array, in millimeters; d is the distance from the microlens array to the sensor, in millimeters; s xy is the pixel size of the sensor, in microns; f(i) is the focal length of different types of microlenses contained in the microlens array, i = 0, 1, 2, in millimeters.

[0102] Table 1: Statistics of simulated light field image data sets

[0103]

[0104] Light field camera shooting data.

[0105] The light field camera shooting data mainly includes public data sets and field collected data sets. Among them, the public data set is mainly collected by Raytrix R12; the field collected data set is collected by Raytrix R32 and HR260 two cameras. The light field image used in the experiment is shot at different focusing distances. The detailed introduction of the data set is shown in Table 2, wherein R12-S1, R12-S2 and R12-S3 are collected by Raytrix R12; R32-S1, R32-S2 and R32-S3 are collected by Raytrix R32; HR260-S1, HR260-S2 and HR260-S3 are collected by HR260. H refers to the focusing distance, all distances are in millimeters, Scale refers to the square size of the checkerboard, and the depth distance is set with reference to the linear motion workbench.

[0106] Table 2: Statistics of original image data sets collected by three light field cameras

[0107]

[0108]

[0109] 3、Experimental results:

[0110] (1) Corner feature extraction results.

[0111] The quality of the corner feature extraction from the different blur level micro images in the multi-focal focusing type light field image is one of the main problems solved by the present application. In order to verify the effectiveness of the method in the aspect of the accuracy of the feature corner points, 5 groups of light field simulation image data sets are used to compare and analyze the present application method with the classic method of Nousias and Noury from the quantitative aspect.

[0112] Table 3 lists the performance of the extracted corner points and the true value corner points in the error mean, error standard deviation, error maximum value, error minimum value and other indicators on the R1-R5 data sets. Figure 2 and Figure 3 It is shown that R2 and R5 are randomly selected from R1-R5 data sets, and further randomly selected from the two groups of images. The pixel error between the extracted corner points and the true value corner points is statistically selected from 60 corner points.

[0113] Combined with the experimental results of Table 3 and Figures 2-3 It can be seen that: (1) the overall performance of the present application method is the best in the four indicators, as shown in Table 3, which shows the error mean (Mean), error standard deviation (std), error maximum value (max), and error minimum value (min) in turn. Type1 represents the corner points in the clearest image; Type2 represents the corner points in the second clearest image; Type3 represents the corner points in the most blurred image; All represents the corner points in the three types of images. Especially in the error mean and error standard deviation, the values obtained by the present application method are the smallest. The results in Figures 2-3 It is also confirmed that the pixel error value obtained by the present application method is smaller than that of the other two methods. This is mainly due to the adjacent blur information guided corner feature extraction strategy proposed in the present application. It mainly uses the geometric relationship between the epipolar lines and the virtual depth space to construct a multiple constraint model, and based on this, a geometric observation mode in the virtual depth domain is constructed. Then the blur imaging degree of multiple corner points in the cluster is quantified and classified, and the clear to blurred corner point coordinates are iteratively filtered and optimized in the order of gradual diffusion. Therefore, the present application method refines the blurred corner point coordinates, effectively improving the extraction quality of the corner feature. (2) The Noury method and the present application method appear the same value in the error minimum value. This is mainly because the first step of the adjacent blur information guided corner feature extraction strategy proposed in the present application uses the corner point coordinates detected by the template matching algorithm introduced by the Noury method. The error minimum value of the corner point coordinates is mainly the detection result of the clearest corner point. The adjacent blur information guided corner feature extraction strategy proposed in the present application uses the coordinates of these clearest corner points to construct the geometric observation of the virtual depth domain to optimize other corner point coordinates.

[0114] Table 3 shows the pixel error statistics table of the extracted corner points and the true value

[0115]

[0116]

[0117] To more intuitively demonstrate the error between the extracted corner coordinates and the actual corner coordinates, local areas are randomly selected from the two sets of simulated light field images, R2 and R5, for detailed display, such as... Figures 4-9 As shown. From Figures 4-9 As can be seen, the corner coordinates extracted by the method of this application are basically closest to the true corner coordinates, regardless of whether the micro-image is clear or blurry. This also proves the effectiveness and advancement of the method of this application in corner coordinate extraction.

[0118] (2) Verification of the accuracy of the calibration internal parameters.

[0119] In the absence of ground truth values, relevant methods typically use reprojection error (RMSE) as the primary indicator of the accuracy of calibration intrinsic parameters. Five publicly available datasets (R12-S1, R12-S2, R12-S3, HR260-S1, and HR260-S3) and field-acquired datasets were selected as experimental data. The method described in this application was quantitatively compared and analyzed with well-known methods such as those of Labussière, Nousias, and Liu. The camera intrinsic parameter results calibrated by the four methods are shown in Tables 4–8, and the reprojection error is shown in Table 9. In Table 9, horizontal lines indicate calculations that did not consider this parameter. The unit in Table 9 is pixels. In addition to showing the reprojection error at corner points, the results from this application and the Labussière method also show the reprojection error of the blur radius, i.e., the value in parentheses. It is worth noting that the Nousias method calculates the intrinsic parameters for each type of microlens. The light field image data used in the experiment were all captured by a multifocal focusing light field camera composed of three different types of microlenses. Therefore, the results of the Nousias method in Tables 4 to 9 will be displayed as NOUS1, NOUS2, and NOUS3.

[0120] As can be seen from Tables 4-9, the intrinsic parameters estimated by the method of the present application are the most complete among the methods of Labussiere, Nousias and Liu. Especially in the case of limited camera parameters given, only the focal length of the main lens. Although the re-projection errors of the methods of Nousias and Liu are not the same, especially the re-projection error of the method of Liu is better, but the estimated value of the focal length of the main lens is obviously different from the given value, so the accuracy of the estimated intrinsic parameters is questionable; the value of the focal length of the main lens estimated by the method of the present application and the method of Labussiere is the closest to the given value, and the re-projection error of the method of the present application is the smallest, basically around 0.5 pixels, but the re-projection error of the method of Labussiere is the largest, up to 2.080 pixels. Therefore, the comprehensive performance of the method of the present application in calibrating intrinsic parameters is the best. This is mainly because the method of the present application constructs the quantification and grading rules of the micro-image sharpness considering the blurred perceptual plenoptic features, uses the geometric observation mode of the virtual depth domain, and effectively improves the extraction quality of the corner features. The decoupling and stacking optimization strategy considering multi-view information is proposed, the micro-image center light projection equation is constructed to assist in optimizing the parameters of the microlens array, the adaptive light reliability weight considering blurred plenoptic features is established, and the step-by-step solving global nonlinear optimization function is constructed, which effectively avoids local optimum and is more likely to converge to the global optimal solution.

[0121] Table 4 Initial intrinsic parameters of R12-S1 dataset and optimized parameters obtained by five methods

[0122]

[0123] Table 5 Initial intrinsic parameters of R12-S2 dataset and optimized parameters obtained by five methods

[0124]

[0125]

[0126] Table 6 Initial intrinsic parameters of R12-S3 dataset and optimized parameters obtained by five methods

[0127]

[0128]

[0129] Table 7 Initial intrinsic parameters of HR260-S1 dataset and optimized parameters obtained by five methods

[0130]

[0131] Table 8 Initial intrinsic parameters of HR260-S3 dataset and optimized parameters obtained by five methods

[0132]

[0133] Table 9. Statistics on Corner Point Reprojection Errors

[0134] Datasets Our Labussière NOUS1 NOUS2 NOUS3 Liu R12-S1 0.527(0.080) 0.856(0.083) 0.773 0.667 0.958 0.691 R12-S2 0.421(0.086) 0.674(0.183) 0.538 0.519 0.593 0.330 R12-S3 0.336(0.040) 0.738(0.041) 1.287 0.681 0.411 0.338 HR260-S1 0.572(0.257) 1.827(0.293) 1.624 1.432 1.316 0.681 HR260-S3 0.515(0.305) 2.080(0.258) 2.032 1.785 2.323 0.551

[0135] (3) Evaluation of the accuracy of calibration external parameters.

[0136] Reprojection error and z-axis displacement error are two main indicators for evaluating the accuracy of light field camera calibration parameters. Nine publicly available datasets and field-acquired datasets were selected to quantitatively compare and analyze the proposed method with renowned methods from Labussière, Nousias, and Liu. The camera intrinsic parameters and reprojection errors calibrated by the four methods are shown in Table 4.

[0137] (4) Robustness of calibration.

[0138] To further verify the effectiveness of the proposed method on nonlinear optimization problems sensitive to initial conditions, nine sets of publicly available datasets and field-collected datasets were used as experimental data. The proposed method was also compared and analyzed with well-known methods such as those of Labussière, Nousias, and Liu.

[0139] As shown in Tables 10 to 14, four sets of experimental results of calibration failures of other methods are presented. The diagonal line indicates calibration failure, and the horizontal line indicates that the solution of the parameter was not considered. In Table 14, the unit is pixels. In addition to showing the reprojection error of the corner point, the reprojection error of the blur radius is also shown in the results of the method of this application and the Labussière method, i.e., the value in parentheses. It can be seen from the table that (1) whether in the data set collected by the Raytrix R32 camera or the HR260 camera, the existing mainstream methods all showed calibration failure results. The maximum reprojection error of the method of this application is only about 1 pixel, and all are successfully calibrated; (2) compared with the given camera main lens focal length parameter, the main lens focal length value estimated by the Nousias and Liu methods is still significantly different from the given value and the reprojection error is large, even reaching 6 pixels. The main lens focal length value estimated by the Labussière method is close to the given value, but the reprojection error, which is the indicator of the accuracy of the intrinsic parameter, is too large, with a maximum of 20.049 pixels. Therefore, the robustness performance of the method of this application is the best. This is mainly because the method in this application introduces fuzzy perception all-light features, constructs a progressively diffused corner optimization model to achieve high-quality extraction of corner coordinates, and establishes a decoupled, stacked intrinsic and extrinsic parameter optimization strategy to ensure the accuracy and robustness of the calibration.

[0140] Table 10 shows the initial intrinsic parameters of the R32-S1 dataset and the optimized parameters obtained by solving it using four methods.

[0141]

[0142]

[0143] Table 11. Initial intrinsic parameters of R32-S2 dataset and optimized parameters obtained by four methods

[0144]

[0145]

[0146] Table 12. Initial intrinsic parameters of R32-S3 dataset and optimized parameters obtained by four methods

[0147]

[0148] Table 13. Initial intrinsic parameters of HR260-S2 dataset and optimized parameters obtained by four methods

[0149]

[0150]

[0151] Table 14. Statistics of corner reprojection error

[0152] Datasets Our Labussière NOUS1 NOUS2 NOUS3 Liu R32-S1 0.791(0.221) 13.071(6.343) \ \ 2.325 1.591 R32-S2 0.687(0.073) 12.454(6.418) \ \ 1.532 0.672 R32-S3 0.605(0.056) 11.872(6.481) \ \ 4.418 2.436 HR260-S2 1.250(0.371) 20.049(12.866) 3.296 19.175 3.514 5.849

[0153] The light field camera calibration method proposed in the application realizes high-quality corner point coordinate extraction and robust calibration by introducing a blur-aware plenoptic feature to construct a step-by-step diffusion corner point optimization model and a decoupled cascaded intrinsic and extrinsic parameter optimization strategy. A decoupled cascaded nonlinear optimization strategy considering multi-view information is proposed to construct a micro-image center light projection equation to assist in optimizing the micro-lens array parameters, establish an adaptive light ray reliability weight considering blur plenoptic features, and construct a hierarchical decoupled global nonlinear optimization function to reduce the sensitivity to initial values in the optimization process. Through experimental verification and analysis on public data sets and real shot data sets, the following conclusions are drawn:

[0154] (1) Quantitative evaluation of corner feature extraction: Compared with the Labussière, Nousias, Liu, etc. methods, the method of the application has the best overall performance in terms of pixel error indicators such as mean error, standard deviation of error, maximum error and minimum error. This is because the proposed virtual depth domain geometric observation mode is used to optimize the corner points of adjacent blur micro-images step by step, which positively contributes to improving the quality of corner point coordinate extraction.

[0155] (2) Calibration of intrinsic parameters: The method of the present application estimates the most complete and closest initial camera parameters to the given values, and the reprojection error of the method of the present application is the smallest. Although the reprojection errors of the methods of Nousias and Liu are not consistent, especially the reprojection error of the method of Liu is better, but the focal length of the main lens estimated by the two methods is significantly different from the given value.

[0156] (3) Calibration of extrinsic parameters: Nine data sets obtained by three types of cameras mounted on a precision control mobile platform were used for experimental verification. Compared with the existing methods (Labussière, Nousias, Liu), the method of the present application is the best in terms of extrinsic parameter error, average error and standard deviation, and there is no obvious performance fluctuation, which shows excellent stability. The method of the present application maintains consistent accuracy level under different camera configurations, verifying its robustness. In addition, the method of the present application shows stable performance regardless of the distance, and the extrinsic parameter error does not change with the increase of the shooting distance.

[0157] (4) Robustness: Nine public data sets and real shot data sets were selected as experimental data. Compared with the methods of Labussière, Nousias, Liu, etc., the method of the present application successfully completed the calibration, and the focal length of the main lens calculated was closest to the given parameter, and the maximum reprojection error was about 1 pixel. While the mainstream methods all failed to calibrate, and the maximum reprojection error can reach about 20 pixels.

[0158] Based on the same inventive concept, the embodiments of the present application also provide a light field camera calibration system for implementing the light field camera calibration method described above. The implementation scheme of the system for solving the problem is similar to the implementation scheme described in the above method, so the specific limitations in the following light field camera calibration system embodiments can be referred to the limitations of the light field camera calibration method in the above text, which will not be repeated here.

[0159] In one exemplary embodiment, a light field camera calibration system is provided, comprising:

[0160] An image acquisition module is configured to acquire a plurality of white images shot by a light field camera to be calibrated at a plurality of aperture scale values.

[0161] An initial value determination module is configured to determine initial values of intrinsic parameters and high-confidence center points of the light field camera to be calibrated according to the plurality of white images.

[0162] An angle point optimization module is configured to determine a blur radius of an angle point of a calibration board image and an optimized angle point coordinate based on a geometric observation mode of a virtual depth domain according to the initial values of the intrinsic parameters; the calibration board image is an image of a black-and-white checkerboard shot by the light field camera to be calibrated from multiple angles.

[0163] a calibration module, configured to: perform iterative optimization on a total cost function by using a Levenberg-Marquardt algorithm according to the initial value of the intrinsic parameter of the light field camera to be calibrated, the high-confidence center point, the blur radius of the corner point of the calibration board image, and the optimized corner point coordinate, to obtain the optimized camera intrinsic parameter and the optimized camera extrinsic parameter, and complete the calibration of the light field camera; the total cost function is determined according to the corner point re-projection error, the blur radius re-projection error, and the micro-image center re-projection error.

[0164] In an exemplary embodiment, a computer device is provided, comprising a memory and a processor, the memory storing a computer program, and the processor implementing the above light field camera calibration method when executing the computer program.

[0165] In an exemplary embodiment, a computer readable storage medium is provided, storing a computer program, and the computer program implementing the above light field camera calibration method when executed by a processor.

[0166] In an exemplary embodiment, a computer program product is provided, comprising a computer program, and the computer program implementing the above light field camera calibration method when executed by a processor.

[0167] In an exemplary embodiment, a computer device is provided, which can be a server or a terminal, and an internal structure diagram of the computer device can be as shown in Figure 10 The computer device comprises a processor, a memory, an input / output interface (I / O), and a communication interface. The processor, the memory, and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. The processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device comprises a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operating system and the computer program in the non-volatile storage medium to run. The input / output interface of the computer device is configured to exchange information between the processor and external devices. The communication interface of the computer device is configured to communicate with external terminals through a network connection. The computer program is executed by the processor to implement a light field camera calibration method.

[0168] Those skilled in the art can understand that Figure 10 The structure shown in the above-mentioned figure is only a block diagram of part of the structure related to the scheme of the present application, and does not constitute a limitation on the computer device to which the scheme of the present application is applied. The specific computer device can comprise more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0169] It should be noted that the user information (including but not limited to user equipment information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in the present application are all information and data authorized by the user or authorized by all parties, and the collection, use and processing of related data need to comply with relevant regulations.

[0170] It can be understood by those skilled in the art that all or part of the processes in the above-mentioned embodiment methods can be completed by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer readable storage medium. When the computer program is executed, it can include the processes of the above-mentioned embodiments. Any reference to memory, database or other medium used in the embodiments provided by the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical storage, high-density embedded non-volatile memory, resistive memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration but not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc.

[0171] The database involved in the embodiments provided by the present application can include at least one of a relational database and a non-relational database. The non-relational database can include a distributed database based on blockchain, etc., without being limited thereto. The processor involved in the embodiments provided by the present application can be a general-purpose processor, a central processing unit, a graphics processing unit, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., without being limited thereto.

[0172] Any technical features in the above embodiments can be combined, and for the sake of brevity, not all possible combinations are described above, however, it should be understood that the application encompasses all possible combinations of the technical features described above.

[0173] The principles and implementation manners of the present application are described herein by using specific examples, and the above embodiments are only used to help understand the method of the present application and its core idea; meanwhile, according to the idea of the present application, the specific implementation manners and application scopes will be changed by those skilled in the art. In conclusion, the content of the present specification should not be understood as a limitation of the present application.

Claims

1. A method for calibrating a light field camera, the method comprising: The method comprises the following steps: acquiring a plurality of white images photographed by a to-be-calibrated light field camera at a plurality of aperture scale values; determining initial intrinsic parameters and high-confidence center points of the to-be-calibrated light field camera according to the plurality of white images; determining corner point blur radii and optimized corner point coordinates of a calibration board image according to the initial intrinsic parameters and based on a geometric observation mode in a virtual depth domain; the calibration board image is an image of a black-and-white checkerboard photographed by the to-be-calibrated light field camera from multiple angles; determining corner point blur radii and optimized corner point coordinates of a calibration board image according to the initial intrinsic parameters and based on a geometric observation mode in a virtual depth domain, specifically comprising: determining initial corner point coordinates of the calibration board image by using a template matching algorithm; determining a plurality of corner point clusters by using a density-based clustering algorithm according to the initial corner point coordinates; determining corner point blur radii according to the initial intrinsic parameters and linear parameters; classifying initial corner points in each corner point cluster according to the corner point blur radii to obtain a clear corner point set, a sub-clear corner point set and a blur corner point set; optimizing initial corner points in the clear corner point set, the sub-clear corner point set and the blur corner point set by using the geometric observation mode in the virtual depth domain to obtain optimized corner point coordinates; iteratively optimizing a total cost function by using a Levenberg-Marquardt algorithm according to the initial intrinsic parameters, the high-confidence center points, the corner point blur radii and the optimized corner point coordinates of the calibration board image of the to-be-calibrated light field camera to obtain optimized camera intrinsic parameters and optimized camera extrinsic parameters, thereby completing calibration of the light field camera; the total cost function is determined according to corner point re-projection errors, blur radius re-projection errors and micro-image center re-projection errors.

2. The light field camera calibration method of claim 1, wherein, determining initial intrinsic parameters and high-confidence center points of the to-be-calibrated light field camera according to the plurality of white images, specifically comprising: detecting micro-image centers in all the white images at an aperture scale value by using a sub-pixel precision intensity centroid method; one white image corresponds to a plurality of micro-image centers; taking a median value of the micro-image centers as the high-confidence center points of the to-be-calibrated light field camera; screening all the white images at the aperture scale value based on the micro-image centers and the median value of the micro-image centers to obtain screened white images; determining a micro-image radius according to micro-images in a center region of the screened white image and in combination with a covariance matrix; determining linear parameters of a linear relationship between the micro-image radius and an actual aperture value in a defocus theory by using a least square method according to the micro-image radius and the actual aperture value; determining the initial intrinsic parameters of the to-be-calibrated light field camera according to the linear parameters and factory parameters of the to-be-calibrated light field camera.

3. The light field camera calibration method of claim 2, wherein, screening all the white images at the aperture scale value based on the micro-image centers and the median value of the micro-image centers to obtain screened white images, specifically comprising: calculating a Euclidean distance between each micro-image center in each white image and the median value of the micro-image centers; generating an error map by degrading a micro-lens array into a grid image according to the Euclidean distance; determine similarity between the error map and a corresponding white image by using a structural similarity index; if the similarity is less than a similarity error, discard the corresponding white image of the error map to obtain a screened white image.

4. The light field camera calibration method of claim 1, wherein, The expression of the geometric observation mode based on the virtual depth domain is: Among them, M i M is the center of the i-th microlens; j Let M be the center of the j-th microlens; j -M i | represents the Euclidean distance between the centers of the i-th and j-th microlenses; v represents the virtual depth domain; C i C is the center of the i-th micro-image; j The center of the j-th micro-image; Let I be the unit direction vector from the j-th micro-image center to the i-th micro-image center; i I is the i-th corner point in the micro-image; j ' Let be the corner point after back-projection of the i-th corner point in the micro-image.

5. The light field camera calibration method of claim 1, wherein, The total cost function is: where T is a set of camera intrinsic parameters and camera extrinsic parameters to be optimized; is the coordinate of the detected corner point in the nth frame of the calibration board image; is the coordinate of the corner point in the real world; k,l is the coordinate of the corner point in the real world projected by the micro-lens numbered (k, l); is the normalized weight corresponding to the blur degree of the corner point after quantization; is the blur radius of the corner point in the nth frame of the calibration board image; and is the cost function of the center reprojection error of the micro-image.

6. An optical field camera calibration system, characterized by, The light field camera calibration system is used to implement the light field camera calibration method in any one of claims 1-5, and the light field camera calibration system comprises: an image acquisition module, configured to acquire a plurality of white images captured by a to-be-calibrated light field camera at a plurality of aperture scale values; an initial value determination module, configured to determine initial values of intrinsic parameters and high-confidence center points of the to-be-calibrated light field camera according to the plurality of white images; an angle point optimization module, configured to determine angle point blur radii and optimized angle point coordinates of a calibration board image according to the initial values of the intrinsic parameters and based on a geometric observation mode based on a virtual depth domain; the calibration board image is an image captured by the to-be-calibrated light field camera from multiple angles on a black-and-white chessboard. determining angle point blur radii and optimized angle point coordinates of a calibration board image according to initial values of intrinsic parameters and based on a geometric observation mode based on a virtual depth domain, specifically comprising: determining initial angle point coordinates of the calibration board image by using a template matching algorithm; determining a plurality of angle point clusters by using a density-based clustering algorithm according to the initial angle point coordinates; determining angle point blur radii according to the initial values of the intrinsic parameters and linear parameters; classifying initial angle points in each angle point cluster according to the angle point blur radii to obtain a clear angle point set, a sub-clear angle point set and a blur angle point set; optimizing initial angle points in the clear angle point set, the sub-clear angle point set and the blur angle point set by using the geometric observation mode based on the virtual depth domain to obtain optimized angle point coordinates; a calibration module, configured to perform iterative optimization on a total cost function by using a Levenberg-Marquardt algorithm according to the initial values of intrinsic parameters and high-confidence center points of the to-be-calibrated light field camera, the angle point blur radii and the optimized angle point coordinates of the calibration board image, to obtain optimized camera intrinsic parameters and optimized camera extrinsic parameters, and complete calibration of the light field camera; the total cost function is determined according to angle point re-projection errors, blur radius re-projection errors and micro-image center re-projection errors.

7. A computer device comprising: A memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that the processor executes the computer program to implement the light field camera calibration method in any one of claims 1-5.

8. A computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the light field camera calibration method in any one of claims 1-5.

9. A computer program product comprising a computer program, characterized in that, The computer program is executed by the processor to implement the light field camera calibration method in any one of claims 1-5.

Images