A light field camera and its calibration method, device, and medium

Abstract

The present application is applicable to the technical field of light field cameras, and particularly relates to a light field camera and its calibration method, device, and medium. The method performs frequency domain conversion on the direction plane coordinates in the acquired initial light field data to obtain frequency domain light field data, screens the frequency domain light field image with a frequency domain threshold to obtain light field component data, pairs all sub-views according to the light field component data to obtain sub-view pairs, generates matching point pairs according to the corner positions of the calibration board collected in the sub-view pairs, respectively forms a fundamental matrix with each matching point pair and the internal and external parameters, constructs an epipolar constraint equation, obtains the initial internal and external parameters with the minimum of the epipolar constraint equation as the objective, and calibrates the light field camera using the internal and external parameters obtained by optimizing the initial internal and external parameters. Through frequency domain analysis, the computational redundancy in light field data processing is effectively reduced. By using high-frequency feature extraction and the establishment of the epipolar constraint equation, the initial value estimation of camera calibration is accelerated, and the efficiency of camera calibration is improved.

Description

Technical Field

[0001] This application is applicable to the technical field of light field cameras, and particularly relates to a light field camera and its calibration method, device, and medium. Background Art

[0002] At present, the calibration technology of light field cameras has important value in modern industrial inspection. Light field cameras can record the spatial position and direction information of light, thereby realizing the three-dimensional reconstruction of scenes. This enables light field cameras to be widely used in industrial inspection. For example, in the inspection of printed circuit boards (PCBs), it can obtain the three-dimensional image of the PCB through one-time imaging, and then accurately measure the length of the PCB pins and detect surface defects. However, the high-precision application of light field cameras is inseparable from efficient and accurate calibration methods.

[0003] Existing research pays more attention to the improvement of calibration accuracy, while ignoring the impact of calibration speed on industrial inspection efficiency. In the actual industrial environment, the internal and external parameters of light field cameras inevitably change due to operations such as zooming, refocusing, and mechanical vibration, so frequent recalibration is required. The calibration process takes a long time, directly affecting the efficiency of industrial inspection. It can be seen that the current light field camera calibration methods have defects such as redundant high-dimensional data calculation and low efficiency, instability in solving high-dimensional non-linear equations, and complex lens distortion on calibration accuracy.

[0004] Therefore, how to optimize the high-dimensional data calculation during the calibration of light field cameras to improve the calibration efficiency has become an urgent problem to be solved. Summary of the Invention

[0005] In view of this, the embodiments of this application provide a light field camera and its calibration method, device, and medium to solve the problem of how to optimize the high-dimensional data calculation during the calibration of light field cameras to improve the calibration efficiency.

[0006] In a first aspect, the embodiments of this application provide a light field camera calibration method, and the light field camera calibration method includes:

[0007] Obtain the initial light field data in the calibration board scene collected by the light field camera, perform frequency domain conversion on the direction plane coordinates in the initial light field data to obtain frequency domain light field data, and screen the frequency domain light field image based on the condition that the horizontal frequency domain value or the vertical frequency domain value is greater than the frequency domain threshold to obtain light field component data;

[0008] Use the image collected by the acquisition unit of the light field camera as a sub-view, and pair all sub-views according to the light field component data to obtain at least one pair of matching sub-view pairs;

[0009] For any pair of sub-views, pair the corner positions of the calibration board collected in the pair of sub-views to generate matching point pairs, and traverse all pairs of sub-views to obtain all matching point pairs;

[0010] For each matching point pair, respectively construct the epipolar constraint equation corresponding to the matching point pair with the fundamental matrix formed by the internal and external parameters, and calculate the initial internal and external parameters with the goal of minimizing all epipolar constraint equations;

[0011] Optimize the initial internal and external parameters to obtain the optimized internal and external parameters, and calibrate the light field camera according to the optimized internal and external parameters.

[0012] In a second aspect, an embodiment of the present application provides a light field camera calibration device, which includes:

[0013] A frequency domain screening module, configured to obtain initial light field data based on the calibration board scene collected by the light field camera, perform frequency domain conversion on the direction plane coordinates in the initial light field data to obtain frequency domain light field data, and screen the frequency domain light field image with the condition that the horizontal frequency domain value or the vertical frequency domain value is greater than the frequency domain threshold to obtain light field component data;

[0014] A sub-view pair determination module, configured to use the images collected by the acquisition unit of the light field camera as sub-views, and pair all sub-views according to the light field component data to obtain at least one pair of matching sub-view pairs;

[0015] A matching point pair determination module, configured to, for any pair of sub-views, pair the corner positions of the calibration board collected in the pair of sub-views to generate matching point pairs, and traverse all pairs of sub-views to obtain all matching point pairs;

[0016] An internal and external parameter initialization module, configured to respectively construct the epipolar constraint equation corresponding to the matching point pair with the fundamental matrix formed by the internal and external parameters, and calculate the initial internal and external parameters with the goal of minimizing all epipolar constraint equations;

[0017] A light field camera calibration module, configured to optimize the initial internal and external parameters to obtain the optimized internal and external parameters, and calibrate the light field camera according to the optimized internal and external parameters.

[0018] In a third aspect, an embodiment of the present application provides a light field camera, which includes a processor, a memory, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the light field camera calibration method described in the first aspect is implemented.

[0019] Fourthly, an embodiment of the present application provides a computer-readable storage medium storing a computer program, which when executed by a processor, implements the light field camera calibration method as described in the first aspect.

[0020] The beneficial effects of the embodiments of the present application compared with the prior art are as follows: The present application acquires initial light field data in a calibration board scene collected by a light field camera, performs frequency domain conversion on the direction plane coordinates in the initial light field data to obtain frequency domain light field data, screens the frequency domain light field image on the condition that the horizontal frequency domain value or the vertical frequency domain value is greater than the frequency domain threshold to obtain light field component data, uses the images collected by the acquisition unit of the light field camera as sub-views, pairs all sub-views according to the light field component data to obtain at least one pair of matching sub-view pairs, for any sub-view pair, pairs according to the corner point positions of the calibration board collected in the sub-view pair to generate matching point pairs, traverses all sub-view pairs to obtain all matching point pairs, respectively combines each matching point pair with the fundamental matrix formed by the internal and external parameters to construct a polar line constraint equation corresponding to the matching point pair, takes the minimum of all polar line constraint equations as the target, calculates the initial internal and external parameters, optimizes the initial internal and external parameters to obtain the optimized internal and external parameters, and calibrates the light field camera according to the optimized internal and external parameters. Through frequency domain analysis, the calculation redundancy in light field data processing is effectively reduced, and by using high-frequency feature extraction and the establishment of polar line constraint equations, the initial value estimation of camera calibration is accelerated, and the efficiency of camera calibration is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings required for use in the embodiments or the description of the prior art. Obviously, the following drawings are only some embodiments of the present application, and those of ordinary skill in the art can also obtain other drawings without creative efforts based on these drawings.

[0022] Figure 1 is a schematic diagram of an application environment of a light field camera calibration method provided in Embodiment 1 of the present application;

[0023] Figure 2 is a schematic flowchart of a light field camera calibration method provided in Embodiment 2 of the present application;

[0024] Figure 3 is a schematic principle diagram of a light field camera calibration method provided in Embodiment 2 of the present application;

[0025] Figure 4 is a schematic flowchart of a light field camera calibration method provided in Embodiment 3 of the present application;

[0026] Figure 5It is a schematic flowchart of a light field camera calibration method provided in Embodiment 4 of this application;

[0027] Figure 6 It is a schematic flowchart of a light field camera calibration method provided in Embodiment 5 of this application;

[0028] Figure 7 It is a schematic structural diagram of a light field camera calibration device provided in Embodiment 6 of this application;

[0029] Figure 8 It is a schematic structural diagram of a light field camera provided in Embodiment 7 of this application. Detailed implementation manners

[0030] In the following description, for the purpose of illustration rather than limitation, specific details such as specific system architectures and technologies are presented in order to thoroughly understand the embodiments of this application. However, those skilled in the art should clearly understand that this application can also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to avoid unnecessary details from interfering with the description of this application.

[0031] It should be understood that when used in the specification of this application and the appended claims, the term "comprising" indicates the presence of the described features, wholes, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components, and / or their combinations.

[0032] It should also be understood that the term "and / or" as used in the specification of this application and the appended claims refers to any combination and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0033] As used in the specification of this application and the appended claims, the term "if" can be interpreted as "when", "once", "in response to determining", or "in response to detecting" according to the context. Similarly, the phrase "if determined" or "if [the described condition or event] is detected" can be interpreted as meaning "once determined", "in response to determining", "once [the described condition or event] is detected", or "in response to detecting [the described condition or event]" according to the context.

[0034] In addition, in the description of the specification of this application and the appended claims, the terms "first", "second", "third", etc. are only used for differentiating descriptions and cannot be understood as indicating or implying relative importance.

[0035] References to "one embodiment" or "some embodiments" etc. described in the specification of this application mean that specific features, structures, or characteristics described in connection with that embodiment are included in one or more embodiments of this application. Thus, statements such as "in one embodiment", "in some embodiments", "in other some embodiments", "in still other embodiments", etc. that appear in different places in this specification do not necessarily all refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized. The terms "comprising", "including", "having" and their variants all mean "including but not limited to", unless otherwise specifically emphasized.

[0036] The embodiments of this application can acquire and process relevant data based on artificial intelligence technology. Among them, artificial intelligence is the theory, method, technology, and application system that uses a digital computer or a machine controlled by a digital computer to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use knowledge to obtain the best results.

[0037] Artificial intelligence basic technologies generally include technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, mechatronics, etc. Artificial intelligence software technologies mainly include several major directions such as computer vision technology, robotics, biometric technology, speech processing technology, natural language processing technology, and machine learning / deep learning.

[0038] It should be understood that the magnitudes of the sequence numbers of the steps in the following embodiments do not mean the order of execution is prior or subsequent. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation to the implementation process of the embodiments of this application.

[0039] In order to illustrate the technical solution of this application, it will be described below through specific embodiments.

[0040] An optical field camera calibration method provided by Embodiment 1 of this application can be applied, for example, in Figure 1In the application environment, the light field camera is set in the environment for PCB detection accordingly, and can effectively collect images of the PCB on the platform. The computer is connected to the light field camera to obtain the data collected by the light field camera. Among them, the computer can include but is not limited to a palm computer, a desktop computer, a notebook computer, an ultra-mobile personal computer (UMPC), a netbook, a cloud light field camera, a personal digital assistant (PDA), etc., or can also be implemented by an independent server. In addition, the light field camera calibration method can be executed in the processor of the light field camera itself. Of course, camera calibration can also be performed through the computer connected to it.

[0041] See Figure 2 , which is a schematic flowchart of a light field camera calibration method provided in the second embodiment of the present application. The above light field camera calibration method is applied to the light field camera or the computer. As Figure 2 shown, the light field camera calibration method may include the following steps:

[0042] Step S201: Obtain the initial light field data in the calibration board scene collected by the light field camera, perform frequency domain conversion on the direction plane coordinates in the initial light field data to obtain the frequency domain light field data, and screen the frequency domain light field image based on the condition that the horizontal frequency domain value or the vertical frequency domain value is greater than the frequency domain threshold to obtain the light field component data.

[0043] In this embodiment, the light field camera includes an image sensor, a microlens array, and a main lens. The image sensor is placed at twice the focal length of the microlens array, and the microlens array is placed at the focal plane of the camera. After being refracted by the main lens and focused by the microlens, it is finally recorded by the acquisition unit on the image sensor.

[0044] When calibrating, the light field camera should be aligned with the calibration board for image acquisition, rather than the PCB, so as to obtain the initial light field data in the calibration board scene. The initial light field data can include 7 dimensions. Of course, the simplified light field data can be described using 4 dimensions, that is, 4D light field data, which is expressed as , where is the direction plane coordinate of the light field, representing the index of each microlens corresponding pixel, and the range is , R represents the boundary value of the sampling range of the microlens array on the direction plane, which determines the discrete sampling number of the light ray direction. is the spatial position plane coordinate of the light field, representing the position index of the microlens, and the range is , , and They respectively represent the resolution of the sensor in the horizontal and vertical directions.

[0045] The above-mentioned initial light field data includes the direction information and spatial position information of the light rays. By converting the direction dimension of the light field data from the spatio-temporal domain to the frequency domain through two-dimensional discrete Fourier transform, the information of the direction dimension can be transferred to the frequency domain for processing. The frequency domain transformation formula is as follows:

[0046] ;

[0047] Wherein, is the light field data in the frequency domain, and are the frequency coordinates, representing the coordinates of the direction dimension in the frequency domain. Through this transformation, the light field data is converted from the spatio-temporal domain to the frequency domain, reducing the computational complexity in the spatio-temporal domain.

[0048] In the frequency domain, the low-frequency part usually represents the overall information of the light field, while the high-frequency part contains more detailed geometric structures and parallax information. To more effectively extract the features with important geometric information, a frequency domain threshold is set, and only the high-frequency components (i.e., the light field component data) that satisfy or in the frequency domain are retained. The screening process of the high-frequency components is as follows:

[0049] .

[0050] As Figure 3 shown, it is a schematic diagram of the principle of a light field camera calibration method provided in the second embodiment of the present application. It can be seen that the light rays pass through the main lens and the microlens array to reach the plane where the image sensor is located (i.e., the image plane), and each acquisition unit in the sensor can record the complete image information.

[0051] Step S202: Using the images collected by the acquisition units of the light field camera as sub-views, and pairing all the sub-views according to the light field component data to obtain at least one pair of matching sub-view pairs.

[0052] In this embodiment, each acquisition unit of the image sensor of the light field camera acquires a corresponding sub-view. After the above-mentioned frequency domain screening, each sub-view corresponds to the screened light field component data. Using this light field component data as the image information of the sub-view, all the sub-views are paired to find at least one pair of matching sub-view pairs. Among them, the matching of the two sub-views in the sub-view pair indicates that the image information they record is relatively close, which can more effectively improve the probability of successful matching of the subsequent matching point pairs.

[0053] After screening the high-frequency components, adjacent sub-views are selected for pairing. The sub-view pairs corresponding to the high-frequency components usually contain more geometric details and parallax information, which can help improve the effectiveness of subsequent camera parameter estimation.

[0054] Optionally, according to the light field component data, all sub-views are paired to obtain at least one pair of matching sub-view pairs, including:

[0055] According to the light field component data, determine the image feature information corresponding to all sub-views;

[0056] For any sub-view, obtain the adjacent sub-views of the sub-view, perform feature point matching between the image feature information of the sub-view and the image feature information of each adjacent sub-view, and determine the adjacent sub-view that matches the sub-view as the matching sub-view;

[0057] Take the sub-view and the matching sub-view as a sub-view pair, traverse all sub-views, and obtain at least one pair of matching sub-view pairs.

[0058] Among them, when matching sub-view pairs, pair the sub-view with its adjacent sub-views, and find the two most matching sub-views from the adjacent sub-views as the sub-view pair. Using adjacent sub-view matching can reduce the amount of data for matching and improve the matching efficiency.

[0059] In addition, matching can be performed by extracting features from sub-views and calculating the similarity of features to obtain the two sub-views with the highest feature similarity.

[0060] Step S203, for any sub-view pair, pair according to the corner positions of the calibration board collected in the sub-view pair to generate matching point pairs, traverse all sub-view pairs, and obtain all matching point pairs.

[0061] In this embodiment, for any sub-view pair, the positions representing the same point in the two sub-views should be the same. Therefore, pair according to the corner positions of the collected calibration board to obtain matching point pairs.

[0062] Among them, each sub-view pair can obtain at least one matching point pair. If there are N intersection points in the calibration board, one sub-view should obtain N matching point pairs. All sub-view pairs need to calculate matching point pairs to form multiple matching point pairs for subsequent parameter calculation.

[0063] Detect and match the feature points in each pair of sub-views. Each pair of sub-views uses a corner detection algorithm to extract the corner positions of the calibration board (for example, a checkerboard board), and record them according to the horizontal and vertical coordinates of the image plane as and , then, through the feature matching method, ensure that the corner points in each pair of sub-views correspond one by one to form matching point pairs .

[0064] Optionally, pair the corner point positions of the calibration plates collected in the pair of sub-views to generate matching point pairs, including:

[0065] Detect the corner points of the calibration plate images included in the first sub-view and the second sub-view of the pair of sub-views, obtain the first set of corner points corresponding to the first sub-view and the first corner point positions corresponding to each corner point, as well as the second set of corner points corresponding to the second sub-view and the second corner point positions corresponding to each corner point;

[0066] Match each corner point in the first set of corner points with each corner point in the second set of corner points one by one, determine the matched corner points as a matching point pair, and the matching point pair includes the first corner point position of the corresponding corner point in the first set of corner points and the second corner point position of the corresponding corner point in the second set of corner points.

[0067] Among them, for a sub-view, it is necessary to use a corner point detection algorithm to detect the corner points in the figure, determine the corner point positions of each corner point, and then match one corner point in one sub-view with all the corner points in another sub-view one by one to obtain the paired corner points.

[0068] In the above matching, it can be determined according to the distance of the corner point positions of the corner points. For example, the distance between two corner points on the calibration plate is S, and it is required that the distance between the matched corner points in the two sub-views is less than S / 2, so that the corner point matching result can be obtained quickly and accurately.

[0069] Step S204, respectively combine each matching point pair with the fundamental matrix formed by the internal and external parameters to construct the epipolar constraint equation corresponding to the matching point pair, and calculate the initial internal and external parameters with the goal of minimizing all the epipolar constraint equations.

[0070] In this embodiment, after obtaining the matching point pairs, an epipolar constraint equation is constructed to solve the camera parameters. For each pair of sub-views, assume that their matching point pairs satisfy the epipolar constraint, as follows:

[0071] ;

[0072] Among them, F is the fundamental matrix, which describes the geometric relationship between the two sub-views. The estimation process of the fundamental matrix is carried out by minimizing the epipolar error of the matching point pairs. Therefore, the objective function is as follows:

[0073] ;

[0074] The fundamental matrix estimation of each pair of sub-views can be obtained by the least squares method, thus providing necessary constraints for the estimation of the internal and external parameters of the camera.

[0075] Furthermore, there is a clear mathematical relationship between the fundamental matrix F and the internal and external parameters of the camera. Let K be the internal parameter matrix of the camera, R be the rotation matrix of the camera, and T be the translation vector of the camera. Then the relationship between the fundamental matrix and the internal and external parameters is as follows:

[0076] ;

[0077] where, is the skew-symmetric matrix of the translation vector T, .

[0078] By jointly estimating multiple fundamental matrices, a joint optimization problem can be constructed to optimize the internal and external parameters of the camera. The objective function of the joint optimization is as follows:

[0079] ;

[0080] The above objective function can be solved by the Levenberg-Marquardt algorithm to obtain the initial parameter estimate as the initial internal and external parameters of the light field camera.

[0081] Step S205: Optimize the initial internal and external parameters to obtain the optimized internal and external parameters, and calibrate the light field camera according to the optimized internal and external parameters.

[0082] In this embodiment, after obtaining the initial internal and external parameters, through operations such as distortion optimization, the optimized internal and external parameters are obtained. These optimized internal and external parameters can be used to calibrate the light field camera. The calibrated light field camera can be used for subsequent PCB image acquisition work to obtain accurate images, which is helpful for subsequent PCB detection work.

[0083] An embodiment of the present application obtains initial light field data in a calibration board scene collected by a light field camera, performs frequency domain conversion on the direction plane coordinates in the initial light field data to obtain frequency domain light field data, screens the frequency domain light field image on the condition that the horizontal frequency domain value or the vertical frequency domain value is greater than the frequency domain threshold to obtain light field component data, uses the image collected by the acquisition unit of the light field camera as a sub-view, pairs all sub-views according to the light field component data to obtain at least one pair of matching sub-view pairs, for any sub-view pair, pairs according to the corner point positions of the calibration board collected in the sub-view pair to generate matching point pairs, traverses all sub-view pairs to obtain all matching point pairs, respectively combines each matching point pair with the fundamental matrix formed by the internal and external parameters to construct a polar line constraint equation corresponding to the matching point pair, takes the minimum of all polar line constraint equations as the target, calculates the initial internal and external parameters, optimizes the initial internal and external parameters to obtain optimized internal and external parameters, and calibrates the light field camera according to the optimized internal and external parameters. Through frequency domain analysis, the calculation redundancy in light field data processing is effectively reduced. By using high-frequency feature extraction and the establishment of polar line constraint equations, the initial value estimation of camera calibration is accelerated, and the efficiency of camera calibration is improved.

[0084] See Figure 4 , which is a schematic flowchart of a light field camera calibration method provided by Embodiment 3 of the present application. As Figure 4 shown, in the above step S205, optimizing the initial internal and external parameters to obtain optimized internal and external parameters may include the following steps:

[0085] Step S401: Obtain the three-dimensional world coordinates of any point on the calibration board in the world coordinate system, and use the initial internal and external parameters to convert the three-dimensional world coordinates into three-dimensional camera coordinates in the camera coordinate system.

[0086] In this embodiment, it is set that the th point of the calibration board is represented in the world coordinate system (i.e., three-dimensional world coordinates), where are respectively the three-dimensional coordinates of the th point of the calibration board, and 1 is the homogeneous coordinate. Through the external parameter (where R is the rotation matrix and T is the translation vector), the th point is converted from the world coordinate system to the camera coordinate system as follows:

[0087] ;

[0088] where is the coordinate in the camera coordinate system (i.e., three-dimensional camera coordinates).

[0089] The above initial internal and external parameters are the parameters obtained through the above step S204. Among them, the initial internal and external parameters include the internal parameter K and the external parameters R and T. The initial three-dimensional camera coordinates can be obtained by performing coordinate system transformation on the coordinates using the external parameters among them.

[0090] Step S402: Obtain the coefficient matrix of the light field camera.

[0091] Among them, the coefficient matrix characterizes the transformation relationship between the three-dimensional position of any point on the calibration plate in the world coordinate system and the measured value of the light field ray. The light field ray is the light between the main lens and the microlens array, and the above 4D light field data characterizes the light between the microlens array and the image sensor (i.e., the image plane). The coefficient matrix can be used to express the three-dimensional position in the form of a light field ray.

[0092] Step S403: Obtain the calibration matrix, and set the non-linear parameters in the calibration matrix to fixed values to obtain an approximate calibration matrix.

[0093] Among them, the calibration matrix characterizes the transformation relationship between the parameters of the light field ray in the light field camera and the light on the image plane.

[0094] Parameterization of the light field ray and the light on the image plane The relationship between them is described by the calibration matrix ;

[0095] Among them, the calibration matrix is as follows:

[0096] ;

[0097] Among them, are the parameters of the calibration matrix, and the non-linear parameter is . Approximate or fix the non-linear parameter in the calibration matrix to obtain an approximate calibration matrix.

[0098] Step S404: Under the first-order distortion coefficient, form a parameter vector to be solved with the approximate calibration matrix, the external parameter to be optimized, and the first-order distortion coefficient, and construct a linear equation with the result of multiplying the coefficient matrix by the parameter vector to be solved and the three-dimensional camera coordinates.

[0099] In this embodiment, considering that the camera lens may have radial distortion, the relationship between the undistorted image coordinates and the distorted image coordinates is defined as follows:

[0100] ;

[0101] Among them, the distortion coefficient describes the radial distortion, which is the sum of the squares of the image coordinates.

[0102] On this basis, a complete optimization objective function is constructed. Combining the light field ray mapping and the distortion model, the objective function is as follows:

[0103] ;

[0104] Among them, is the number of midpoints of the calibration board, P is the number of light field rays, and are the number of sampling points of the light field rays in the direction respectively, which determine the sampling density and angular resolution in the direction dimension, is the light field ray at the position, is the three-dimensional position of the th checkerboard point in the camera coordinate system, and the reprojection error measures the distance between the light field ray and the point on the calibration board.

[0105] The optimization process is divided into three levels, and more non-linear terms are gradually introduced at each level to improve the accuracy and efficiency of the optimization. In this embodiment, the primary level is involved. First, the high-order distortion terms and are ignored, and the optimization problem is simplified to a linear form. By simplifying the distortion model, only the first-order distortion coefficient is considered, and the high-order terms , are ignored. The approximate linear distortion formula is as follows:

[0106] ;

[0107] Then, a linear equation system is constructed as follows: ;

[0108] Among them, A is the coefficient matrix, X is the parameter vector to be solved, including the parameters of the approximate calibration matrix , the external parameters R, T and the distortion parameter , and B is the measurement value vector, which is represented by the ray parameters or pixel coordinates actually observed in the light field camera (i.e., the above three-dimensional camera coordinates).

[0109] Step S405, use the singular value decomposition method to solve the linear equation, and obtain the preliminary parameter estimation result of the parameter vector to be solved.

[0110] In this embodiment, the linear equation system is quickly solved by the singular value decomposition method to obtain the preliminary parameter estimation result .

[0111] Among them, the preliminary parameter estimation result is used to obtain the optimized internal and external parameters. The preliminary parameter estimation result includes the external parameters. Combining the external parameters with the internal parameters in the above initial internal and external parameters can obtain the optimized internal and external parameters.

[0112] In the embodiment of the present application, considering the influence of distortion, in order to optimize the distortion to obtain the optimized parameters, an approximate linear equation method is adopted to reduce the complexity of the optimization model and improve the efficiency of distortion optimization.

[0113] See Figure 5 , which is a schematic flowchart of a light field camera calibration method provided in the fourth embodiment of the present application. As Figure 5 shown, after solving the linear equation by using the singular value decomposition method in the above step S405 to obtain the preliminary parameter estimation result of the parameter vector to be solved, the following steps may be included:

[0114] Step S501: According to the calibration matrix and the first-order distortion coefficient in the preliminary parameter estimation result, and combining the second-order distortion coefficient, predict the measurement value of the light field ray reaching the three-dimensional camera coordinates to obtain the first coordinate prediction value.

[0115] Step S502: Taking the minimum reprojection error between the first coordinate prediction value and the three-dimensional camera coordinates as the target, use the Gauss-Newton iteration algorithm to iteratively update the preliminary parameter estimation result to obtain the updated parameter estimation result, and the updated parameter estimation result is used to obtain the optimized internal and external parameters.

[0116] In this embodiment, the optimization of the intermediate layer is involved, and the second-order distortion coefficient is introduced to further optimize the objective function as follows:

[0117] ;

[0118] Among them, is a low-order nonlinear model, .

[0119] The second-order distortion coefficient is introduced into the distortion model as follows:

[0120] ;

[0121] Then, substitute the distortion model into the ray reprojection error to construct the nonlinear objective function as follows:

[0122] ;

[0123] In the formula, is the first coordinate prediction value, Is the measured value of the light field ray.

[0124] The Gauss-Newton algorithm is used to iteratively update the parameters, and the update formula is as follows: ;

[0125] Among them, J is the Jacobian matrix, which represents the first-order partial derivative of the residual vector r with respect to the parameter vector X.

[0126] The elements of the Jacobian matrix are as follows:

[0127] ;

[0128] Among them, Is the error between the model prediction value and the actual observation value, and r is the residual vector, which represents the deviation between the light field ray and the three-dimensional position of the calibration board.

[0129] In the embodiment of the present application, after optimizing the parameters of the first-order distortion coefficient, the second-order distortion coefficient is introduced into the distortion model, and the light field ray of the point on the calibration board is transformed and predicted to obtain the predicted value, and the reprojection error is calculated with the three-dimensional camera coordinates obtained above. Therefore, based on the first-order and second-order distortions, the optimization of the parameters is completed, and a more accurate parameter optimization result can be obtained.

[0130] See Figure 6 , which is a schematic flowchart of a light field camera calibration method provided in the fifth embodiment of the present application. As Figure 6 Shown, in the above step S502, with the goal of minimizing the reprojection error between the coordinate prediction value and the three-dimensional camera coordinates, the preliminary parameter estimation result is iteratively updated. After obtaining the updated parameter estimation result, the following steps may be included:

[0131] Step S601: According to the calibration matrix, first-order distortion coefficient, and second-order distortion coefficient in the updated parameter estimation result, combined with the third-order distortion coefficient, predict the measured value of the light field ray reaching the three-dimensional camera coordinates to obtain the second coordinate prediction value.

[0132] Step S602: With the goal of minimizing the reprojection error between the second coordinate prediction value and the three-dimensional camera coordinates, and using the gradient of the objective function with respect to the parameter as the gradient, perform trust region optimization on the updated parameter estimation result to obtain the final parameter estimation result, and the final parameter estimation result is used to obtain the optimized internal and external parameters.

[0133] In this embodiment, in the optimization of the high-level layer, the third-order distortion coefficient And other high-order non-linear terms are introduced to optimize the complete light field camera model as follows:

[0134] ;

[0135] The optimization objective function at this time includes all high-order distortion terms, and the ultimate goal is to minimize all reprojection errors. Restore the complete distortion model, including the third-order distortion coefficients , as follows:

[0136] ;

[0137] Adopt the trust region method to optimize the algorithm, and use the results of the previous two-layer optimization as the initial values for further optimization , R, T and . In the trust region optimization, the gradient of the objective function represents the partial derivative vector of the objective function with respect to the parameters, and its calculation is as follows:

[0138] ;

[0139] Finally, obtain the accurate light field camera parameters .

[0140] Optionally, with the goal of minimizing the reprojection error between the predicted value of the second coordinate and the three-dimensional camera coordinates, and using the objective function and the partial derivative vector of the parameters as the gradient, after performing trust region optimization on the updated parameter estimation results to obtain the final parameter estimation results, it further includes:

[0141] Extract the external parameters in the final parameter estimation results, and extract the internal parameters in the initial internal and external parameters;

[0142] Combine the external parameters with the internal parameters to obtain the optimized internal and external parameters of the light field camera.

[0143] Among them, optimize several times to obtain the optimized external parameters, and combine them with the internal parameters in the initial internal and external parameters to obtain the optimized internal and external parameters.

[0144] The embodiment of the present application avoids the optimization process from being easily trapped in local minima by means of hierarchical multi-optimization, and improves the stability of solving complex high-dimensional non-linear equations.

[0145] Corresponding to the light field camera calibration method in the above embodiment, Figure 7 shows the structural block diagram of the light field camera calibration device provided in Embodiment 6 of the present application. For the sake of simplicity, only the parts related to the embodiment of the present application are shown.

[0146] See Figure 7 , the light field camera calibration device includes:

[0147] The frequency-domain filtering module 71 is configured to obtain initial light field data in the calibration board scene collected by the light field camera, perform frequency-domain conversion on the direction plane coordinates in the initial light field data to obtain frequency-domain light field data, and filter the frequency-domain light field image on the condition that the horizontal frequency-domain value or the vertical frequency-domain value is greater than the frequency-domain threshold to obtain light field component data;

[0148] The sub-view pair determination module 72 is configured to use the images collected by the acquisition unit of the light field camera as sub-views, and pair all the sub-views according to the light field component data to obtain at least one pair of matching sub-view pairs;

[0149] The matching point pair determination module 73 is configured to, for any sub-view pair, pair according to the corner positions of the calibration board collected in the sub-view pair to generate matching point pairs, and traverse all sub-view pairs to obtain all matching point pairs;

[0150] The internal and external parameter initialization module 74 is configured to respectively form a fundamental matrix with each matching point pair and the internal and external parameters, construct an epipolar constraint equation corresponding to the matching point pair, and calculate the initial internal and external parameters with the goal of minimizing all the epipolar constraint equations;

[0151] The light field camera calibration module 75 is configured to optimize the initial internal and external parameters to obtain optimized internal and external parameters, and calibrate the light field camera according to the optimized internal and external parameters.

[0152] Optionally, the sub-view pair determination module 72 includes:

[0153] The image feature extraction unit is configured to determine the image feature information corresponding to all sub-views according to the light field component data;

[0154] The adjacent sub-view matching unit is configured to, for any sub-view, obtain the adjacent sub-views of the sub-view, perform feature point matching on the image feature information of the sub-view and the image feature information of each adjacent sub-view, and determine the adjacent sub-view that matches the sub-view as the matching sub-view;

[0155] The sub-view pair determination unit is configured to use the sub-view and the matching sub-view as a sub-view pair, and traverse all sub-views to obtain at least one pair of matching sub-view pairs.

[0156] Optionally, the matching point pair determination module 73 includes:

[0157] The corner detection unit is configured to detect the corners of the calibration board image included in the first sub-view and the second sub-view of the sub-view pair, obtain the first set of corners corresponding to the first sub-view and the first corner positions corresponding to each corner, as well as the second set of corners corresponding to the second sub-view and the second corner positions corresponding to each corner;

[0158] A corner point matching unit, configured to match each corner point of the first set of corner points with each corner point of the second set of corner points one by one, and determine the matched corner points as a pair of matched points, where the pair of matched points includes the first corner point position of the corresponding corner point in the first set of corner points and the second corner point position of the corresponding corner point in the second set of corner points.

[0159] Optionally, the light field camera calibration module 75 includes:

[0160] An initial transformation unit, configured to obtain the three-dimensional world coordinates of any point on the calibration board in the world coordinate system, and use the initial internal and external parameters to convert the three-dimensional world coordinates into the three-dimensional camera coordinates in the camera coordinate system;

[0161] A coefficient matrix acquisition unit, configured to obtain the coefficient matrix of the light field camera, where the coefficient matrix represents the transformation relationship between the three-dimensional position of any point on the calibration board in the world coordinate system and the measured value of the light field ray;

[0162] A calibration matrix acquisition unit, configured to obtain the calibration matrix, set the non-linear parameters in the calibration matrix to fixed values to obtain an approximate calibration matrix, where the calibration matrix represents the transformation relationship between the parameters of the light field ray in the light field camera and the light ray on the image plane;

[0163] A linear equation construction unit, configured to form a parameter vector to be solved with the approximate calibration matrix, the external parameters to be optimized, and the first-order distortion coefficient under the first-order distortion coefficient, and construct a linear equation with the result of multiplying the coefficient matrix by the parameter vector to be solved and the three-dimensional camera coordinates;

[0164] A first-order optimization unit, configured to solve the linear equation using the singular value decomposition method to obtain a preliminary parameter estimation result of the parameter vector to be solved, and the preliminary parameter estimation result is used to obtain the optimized internal and external parameters.

[0165] Optionally, the light field camera calibration module 75 further includes:

[0166] A first prediction unit, configured to, after solving the linear equation using the singular value decomposition method to obtain a preliminary parameter estimation result of the parameter vector to be solved, predict the measured value of the light field ray reaching the three-dimensional camera coordinates according to the calibration matrix and the first-order distortion coefficient in the preliminary parameter estimation result, in combination with the second-order distortion coefficient, to obtain a first coordinate prediction value;

[0167] A second-order optimization unit, configured to use the Gauss-Newton iteration algorithm to iteratively update the preliminary parameter estimation result with the goal of minimizing the reprojection error between the first coordinate prediction value and the three-dimensional camera coordinates, to obtain an updated parameter estimation result, and the updated parameter estimation result is used to obtain the optimized internal and external parameters.

[0168] Optionally, the light field camera calibration module 75 further includes:

[0169] A second prediction unit, configured to iteratively update the preliminary parameter estimation result with the aim of minimizing the reprojection error between the coordinate prediction value and the three-dimensional camera coordinates. After obtaining the updated parameter estimation result, according to the calibration matrix, the first-order distortion coefficient, and the second-order distortion coefficient in the updated parameter estimation result, combined with the third-order distortion coefficient, predict the measurement value of the light field ray reaching the three-dimensional camera coordinates to obtain a second coordinate prediction value;

[0170] A third-order optimization unit, configured to perform trust-region optimization on the updated parameter estimation result with the aim of minimizing the reprojection error between the second coordinate prediction value and the three-dimensional camera coordinates, and using the partial derivative vector of the objective function with respect to the parameters as the gradient, to obtain the final parameter estimation result, and the final parameter estimation result is used to obtain the optimized internal and external parameters.

[0171] Optionally, the light field camera calibration module 75 further includes:

[0172] A parameter extraction unit, configured to perform trust-region optimization on the updated parameter estimation result with the aim of minimizing the reprojection error between the second coordinate prediction value and the three-dimensional camera coordinates, and using the partial derivative vector of the objective function with respect to the parameters as the gradient, to obtain the final parameter estimation result. Then, extract the external parameters in the final parameter estimation result and extract the internal parameters in the initial internal and external parameters;

[0173] An optimized parameter determination unit, configured to combine the external parameters and the internal parameters to obtain the optimized internal and external parameters of the light field camera.

[0174] It should be noted that the information interaction, execution process, etc. between the above modules, due to being based on the same concept as the method embodiment of the present application, for their specific functions and the technical effects brought, please refer to the method embodiment part specifically, and will not be elaborated here.

[0175] Figure 8 This is a schematic structural diagram of a light field camera provided in Embodiment VII of the present application. As Figure 8 shown, the light field camera of this embodiment includes: at least one processor ( Figure 8 only one is shown in the figure), a memory, and a computer program stored in the memory and executable on at least one processor. When the processor executes the computer program, it implements the steps in any of the above-mentioned method embodiments of the light field camera calibration method.

[0176] The light field camera may include, but is not limited to, a processor and a memory. Those skilled in the art can understand that Figure 8 this is only an example of a light field camera, and does not constitute a limitation on the light field camera. The light field camera may include more or fewer components than shown in the figure, or combine some components, or different components. For example, it may further include a network interface, a display screen, and an input device, etc.

[0177] The so-called processor may be a CPU, and the processor may also be other general-purpose processors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0178] The memory includes a readable storage medium, an internal memory, etc. Among them, the internal memory may be the memory of the light field camera, and the internal memory provides an environment for the operation of the operating system and computer-readable instructions in the readable storage medium. The readable storage medium may be the hard disk of the light field camera, and in some other embodiments, it may also be an external storage device of the light field camera. For example, a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the light field camera. Further, the memory may also include both the internal storage unit of the light field camera and the external storage device. The memory is used to store the operating system, application programs, a boot loader, data, and other programs, such as the program code of a computer program. The memory may also be used to temporarily store the data that has been output or will be output.

[0179] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the above division of each functional unit and module is used as an example. In actual applications, the above functions can be allocated to different functional units and modules according to needs, that is, the internal structure of the device is divided into different functional units or modules to complete all or part of the functions described above. Each functional unit and module in the embodiment can be integrated into a processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above integrated unit can be implemented in the form of hardware or in the form of a software functional unit. In addition, the specific names of each functional unit and module are only for the convenience of mutual distinction and do not limit the protection scope of this application. The specific working process of the units and modules in the above device can refer to the corresponding process in the foregoing method embodiment and will not be elaborated here. If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on such an understanding, to implement all or part of the processes in the above method embodiment of this application, a computer program can be used to instruct the relevant hardware to complete. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by a processor, the steps of the above method embodiment can be implemented. Among them, the computer program includes computer program code, and the computer program code can be in the form of source code, object code, executable file or some intermediate form, etc. The computer-readable medium can at least include: any entity or device capable of carrying the computer program code, recording medium, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signal, telecommunication signal, and software distribution medium. For example, a USB flash drive, a mobile hard disk, a magnetic disk or an optical disc, etc. In some jurisdictions, according to legislation and patent practice, the computer-readable medium cannot be an electrical carrier signal and a telecommunication signal.

[0180] To implement all or part of the processes in the above method embodiment of this application, it can also be completed by a computer program product. When the computer program product runs on the light field camera, the light field camera can be made to execute the steps in the above method embodiment when executed.

[0181] In the above embodiments, the descriptions of each embodiment have their own emphases. For the parts not detailed or recorded in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0182] Those of ordinary skill in the art will realize that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be implemented by electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are executed in hardware or software depends on the specific application and design constraints of the technical solution. A professional technician can use different methods for each specific application to implement the described functions, but such implementation should not be considered to exceed the scope of this application.

[0183] In the embodiments provided in this application, it should be understood that the disclosed device / light field camera and method can be implemented in other ways. For example, the device / light field camera embodiments described above are merely illustrative. For example, the division of modules or units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed couplings or direct couplings or communication connections to each other can be through some interfaces. The indirect couplings or communication connections of devices or units can be electrical, mechanical or other forms.

[0184] The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they can be located in one place, or they can be distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0185] The above embodiments are only used to illustrate the technical solutions of this application, rather than to limit them; although this application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements for some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included in the protection scope of this application.

Claims

1. A method for calibrating a light field camera, characterized in that, The light field camera calibration method includes: Obtain the initial light field data in the calibration board scene collected by the light field camera, perform frequency domain conversion on the direction plane coordinates in the initial light field data to obtain frequency domain light field data, and screen the frequency domain light field data based on the condition that the horizontal frequency domain value or the vertical frequency domain value is greater than the frequency domain threshold to obtain light field component data; Use the image collected by the acquisition unit of the light field camera as a sub-view, and pair all sub-views according to the light field component data to obtain at least one pair of matching sub-view pairs; For any sub-view pair, pair according to the corner positions of the calibration board collected in the sub-view pair to generate matching point pairs, and traverse all sub-view pairs to obtain all matching point pairs; Respectively form a fundamental matrix with each matching point pair and the internal and external parameters, construct a polar line constraint equation corresponding to the matching point pair, and calculate the initial internal and external parameters with the goal of minimizing all polar line constraint equations; Optimize the initial internal and external parameters to obtain optimized internal and external parameters, and calibrate the light field camera according to the optimized internal and external parameters; The step of pairing according to the corner positions of the calibration board collected in the sub-view pair to generate matching point pairs includes: Detect the corners of the calibration board image included in the first sub-view and the second sub-view of the sub-view pair to obtain the first set of corners corresponding to the first sub-view and the first corner position corresponding to each corner, as well as the second set of corners corresponding to the second sub-view and the second corner position corresponding to each corner; Match each corner of the first set of corners with each corner of the second set of corners one by one, and determine the matching corners as a matching point pair, where the matching point pair includes the first corner position corresponding to the corresponding corner in the first set of corners and the second corner position corresponding to the corresponding corner in the second set of corners.

2. The light field camera calibration method according to claim 1, wherein The step of pairing all sub-views according to the light field component data to obtain at least one pair of matching sub-view pairs includes: Determine the image feature information corresponding to all sub-views according to the light field component data; For any sub-view, obtain the adjacent sub-views of the sub-view, perform feature point matching between the image feature information of the sub-view and the image feature information of each adjacent sub-view, and determine the adjacent sub-view that matches the sub-view as a matching sub-view; Take the sub-view and the matching sub-view as a sub-view pair, and traverse all sub-views to obtain at least one pair of matching sub-view pairs.

3. The light field camera calibration method according to claim 1, wherein The step of optimizing the initial internal and external parameters to obtain optimized internal and external parameters includes: Obtain the three-dimensional world coordinates of any point on the calibration board in the world coordinate system, and use the initial internal and external parameters to convert the three-dimensional world coordinates into three-dimensional camera coordinates in the camera coordinate system; Obtain the coefficient matrix of the light field camera, where the coefficient matrix represents the transformation relationship between the three-dimensional position of any point on the calibration board in the world coordinate system and the measured value of the light field ray; Obtain a calibration matrix, set the non-linear parameters in the calibration matrix to fixed values to obtain an approximate calibration matrix, where the calibration matrix characterizes the transformation relationship between the parameters of light field rays in the light field camera and the light rays on the image plane; Under the first-order distortion coefficient, form a parameter vector to be solved with the approximate calibration matrix, the external parameters to be optimized, and the first-order distortion coefficient, and construct a linear equation with the result of multiplying the coefficient matrix by the parameter vector to be solved and the three-dimensional camera coordinates; Use the singular value decomposition method to solve the linear equation to obtain a preliminary parameter estimation result of the parameter vector to be solved, and the preliminary parameter estimation result is used to obtain optimized internal and external parameters.

4. The light field camera calibration method according to claim 3, wherein After using the singular value decomposition method to solve the linear equation to obtain a preliminary parameter estimation result of the parameter vector to be solved, it further includes: According to the calibration matrix and the first-order distortion coefficient in the preliminary parameter estimation result, combined with the second-order distortion coefficient, predict the measurement value of the light field ray reaching the three-dimensional camera coordinates to obtain a first coordinate prediction value; With the goal of minimizing the reprojection error between the first coordinate prediction value and the three-dimensional camera coordinates, use the Gauss-Newton iterative algorithm to iteratively update the preliminary parameter estimation result to obtain an updated parameter estimation result, and the updated parameter estimation result is used to obtain optimized internal and external parameters.

5. The light field camera calibration method according to claim 4, wherein, After iteratively updating the preliminary parameter estimation result with the goal of minimizing the reprojection error between the coordinate prediction value and the three-dimensional camera coordinates to obtain an updated parameter estimation result, it further includes: According to the calibration matrix, the first-order distortion coefficient, and the second-order distortion coefficient in the updated parameter estimation result, combined with the third-order distortion coefficient, predict the measurement value of the light field ray reaching the three-dimensional camera coordinates to obtain a second coordinate prediction value; With the goal of minimizing the reprojection error between the second coordinate prediction value and the three-dimensional camera coordinates, and using the partial derivative vector of the objective function and the parameter as the gradient, perform trust region optimization on the updated parameter estimation result to obtain a final parameter estimation result, and the final parameter estimation result is used to obtain optimized internal and external parameters.

6. The light field camera calibration method according to claim 5, wherein After performing trust region optimization on the updated parameter estimation result with the goal of minimizing the reprojection error between the second coordinate prediction value and the three-dimensional camera coordinates and using the partial derivative vector of the objective function and the parameter as the gradient to obtain a final parameter estimation result, it further includes: Extract the external parameters from the final parameter estimation result and extract the internal parameters from the initial internal and external parameters; Combine the external parameters with the internal parameters to obtain the optimized internal and external parameters of the light field camera.

7. A light field camera calibration device, characterized in that, The light field camera calibration device includes: A frequency domain screening module, configured to obtain initial light field data based on a calibration board scene collected by the light field camera, perform frequency domain conversion on the direction plane coordinates in the initial light field data to obtain frequency domain light field data, and screen the frequency domain light field data with the condition that the horizontal frequency domain value or the vertical frequency domain value is greater than the frequency domain threshold to obtain light field component data; A sub - view pair determination module, which uses the images collected by the acquisition unit of the light - field camera as sub - views, and pairs all sub - views according to the light - field component data to obtain at least one pair of matching sub - view pairs; A matching point pair determination module, which, for any sub - view pair, pairs according to the corner positions of the calibration board collected in the sub - view pair to generate matching point pairs, and traverses all sub - view pairs to obtain all matching point pairs; An internal and external parameter initialization module, which forms a fundamental matrix with each matching point pair and internal and external parameters respectively, constructs an epipolar constraint equation corresponding to the matching point pair, and calculates the initial internal and external parameters with the goal of minimizing all epipolar constraint equations; A light - field camera calibration module, which optimizes the initial internal and external parameters to obtain optimized internal and external parameters, and calibrates the light - field camera according to the optimized internal and external parameters; The pairing according to the corner positions of the calibration board collected in the sub - view pair to generate matching point pairs includes: A corner detection unit, which detects the corners of the calibration board images included in the first sub - view and the second sub - view of the sub - view pair, obtains a first set of corners corresponding to the first sub - view and the first corner position corresponding to each corner, and a second set of corners corresponding to the second sub - view and the second corner position corresponding to each corner; A corner matching unit, which matches each corner of the first set of corners with each corner of the second set of corners one by one, determines the matching corners as a matching point pair, and the matching point pair includes the first corner position of the corresponding corner in the first set of corners and the second corner position of the corresponding corner in the second set of corners.

8. A light field camera, characterized in that, The light - field camera includes a processor, a memory, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the light - field camera calibration method according to any one of claims 1 to 6.

9. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the light - field camera calibration method according to any one of claims 1 to 6.

Images