Method for calibrating a plenoptic camera

Abstract

The invention describes a method for calibrating a plenoptic camera, comprising a step S1 of acquiring a real object point (P_obj_r) of a calibration target by the plenoptic camera, a step S6 of determining a set of object light rays (60), from associated image light rays (50) and projection parameters (Par_Proj) of a projection model of the plenoptic camera, a step S7 of determining a theoretical object point (P_obj_th) associated with the real object point (P_obj_r), the theoretical object point (P_obj_th) corresponding to a barycenter of the set of intersections of the object light rays (60) with the calibration target, and a step S8 of calculating a calibration error (Δcal) as a function of a position of the theoretical object point (P_obj_th) and the position of the real object point (P_obj_r). Figure for the summary: Figure 6

Description

Title of the invention: Method for calibrating a plenoptic camera FIELD OF THE INVENTION

[0001] The present invention relates to the field of plenoptic cameras. The invention relates in particular to the field of methods for calibrating plenoptic cameras suitable for use in three-dimensional metrology. STATE OF THE ART

[0002] A plenoptic camera makes it possible to represent an object or a scene in three dimensions from a set of two-dimensional images of the object or the scene.

[0003] A plenoptic camera, as illustrated in [Fig.l], comprises a main optic 1, a photosensitive sensor 3, and a matrix (or network) of microlenses 2 arranged between the main optic 1 and the sensor 3.

[0004] The main optic 1, such as a lens, receives light rays coming from an object. The photosensitive sensor 3 is located near the microlens array 2, and comprises a detection surface formed of a plurality of pixels arranged in a pixel array.

[0005] Thus, the center of each microlens 21 receives all the light rays coming from the object point conjugate to this center, these rays having different directions. The parameters of the camera, in particular the focal length of the main optics and that of the optical elements, are chosen so that the images formed by the optical elements on the photosensitive sensor do not overlap.

[0006] The microlens 21 distributes the rays of different directions over several pixels of the photosensitive sensor 3. The pixels receiving all the light rays from the same microlens 21 form a group called a macropixel 31. A macropixel 31 comprises at least two pixels extending in the horizontal direction and / or in the vertical direction.

[0007] The image taken by the sensor therefore contains sufficient information to form images seen from different points of view. A plenoptic camera thus makes it possible, with a single compact system, to carry out spatial sampling, in position, and angular sampling, in orientation, of the distribution of light rays in space. A plenoptic camera thus captures the depth information of the light field.

[0008] Document WO 2017 / 042494 describes a plenoptic camera having a modified microlens arrangement to increase the resolution of the plenoptic camera, i.e. to obtain a denser sampling of the observed volume.

[0009] The classical approximation of geometric optics makes it possible to treat the large majority of study cases in non-coherent imaging and when the characteristic distances are large compared to the wavelength of the light.

[0010] The approximation of geometric optics replaces the problem of electromagnetic wave propagation with an equivalent formalism of light ray tracing. The wavelength is neglected in Maxwell's equations of light wave propagation. The path of light in a medium verifies the eikonal equation. Electromagnetic fields are replaced by scalar functions, and a light ray only keeps the information of the direction of energy propagation: the effects of phase shift in the amplitude of the wave are not taken into account and interference and diffraction are thus neglected.

[0011] The geometric wavefront coming from an object point corresponds to the surface of equal phase of the light wave coming from the object point, all the points of the geometric wavefront having taken the same travel time from the object point.

[0012] In geometric optics, the direction of optical rays is perpendicular to the gradient direction of the geometric wavefront surface. The direction of wave propagation, which is represented by the optical ray plot, is orthogonal to the wavefront surface at every point on that surface.

[0013] When the Gauss conditions are respected, that is to say close to the optical axis, therefore when the rays have a small angle of incidence with respect to the optical axis and the point of incidence is close to the optical axis, the Gauss approximation is valid and makes it possible to simplify the mathematical relationships of geometric optics.

[0014] In the Gaussian approximation, the curvatures and thicknesses of the optics are neglected, the optical system is considered to be completely stigmatic, and is only characterized by its focal length and the Descartes conjugation laws valid from plane to plane.

[0015] In the Gaussian approximation, a point source object P_obj emits a series of light rays propagating upstream of the optical system with an input wavefront S_obj. The input wavefront S_obj, after transformation by the optical system represented by a surface S_lens, results in an output wavefront S_id. All rays downstream of the optical system converge towards an image point P_im. In other words, a point object results in a point image. The output wavefront of a main optic of the converging lens type S_id is then spherical. The surface of the output wavefront S_id corresponds to an ideal reference sphere, centered on the image point P_hn, as illustrated in [Fig.2a].

[0016] However, an optical system is not actually stigmatic and leads to optical aberrations. Thus, an object point P_obj emitting a series of light rays, after transformation by the optical system, does not result in an image point P_im where all the rays converge, but in a more or less spread-out image zone of intersection of rays Z_im. In other words, the image of a point object is not a point, but a zone. This phenomenon is visible on the zooms on the zones of intersection of the light rays in [Fig.3].

[0017] The geometric wavefront S_th at the output of the optical system is then not perfectly spherical, but is on the contrary deformed with respect to the ideal reference sphere S_id of an optical system without aberrations. As illustrated in [Fig.2b], the output wavefront S_th can be decomposed into an ideal wavefront S_id of an optical system without aberrations, and into an aberration function W.

[0018] The aberration function W is characteristic of any surface which corresponds to the phase deviation of the wavefront at the output of the optical system S_th with respect to the ideal wavefront S_id. In other words, the deviation of the surface of the wavefront S_th from the ideal spherical surface S_id corresponds to the aberration of the optics. The image spread zone Z_hn resulting from the intersection of the rays coming from a point object point P_obj at the output of the optical system is all the wider as the aberration of the optics is large.

[0019] The aberration function W can be decomposed into primary aberrations and higher order aberrations. Primary aberrations include spherical aberration, coma aberration, astigmatism, Petzval field curvature, and distortion.

[0020] Higher order aberrations include for example optical depth aberration, called depth distortion. [Fig.3] illustrates the ray tracing of a set of object points P_obj belonging to the same object plane, through a lens 1 exhibiting optical depth aberration.

[0021] Due to depth distortion, the image of a plane through the main optic is not a plane, but a curved surface. The curved surface is the image surface of the object plane, resulting from the hypothesis of approximation of the intersection of rays. In other words, the curved surface constitutes the set of points in the image space P_hn corresponding to the images of the set of object points P_obj by lens 1.

[0022] When the Gaussian conditions are met, i.e. near the optical axis for angles of incidence of only a few degrees, higher order aberrations can be neglected compared to primary aberrations. On the other hand, when the rays deviate from the optical axis, for example for angles of incidence relative to the optical axis ranging from a few degrees up to about 25 degrees, higher order aberrations can no longer be neglected.

[0023] The aberration function W can be decomposed into a series of polynomials, such as the Seidel series of polynomials, or the Zernike.

[0024] The primary aberrations correspond to the third-order aberrations of the polynomial series, that is to say to an expansion of the aberration function W up to order 3. This expansion is valid when the Gauss conditions are respected.

[0025] Higher order aberrations correspond to fifth order aberrations of the polynomial series, i.e. to an expansion of the aberration function W beyond order 3, in particular up to order 5. This expansion represents the aberrations in the case where the Gauss conditions are not respected.

[0026] Plenoptic cameras can be used in three-dimensional reconstruction, for three-dimensional metrology. They make it possible, by a non-contact optical means, to guarantee measurements with a view to controlling the quality of products resulting from a manufacturing process, for example by identifying geometric tolerance defects in a product leaving the production line, etc.

[0027] A plenoptic camera used for three-dimensional metrology must be calibrated with great precision, in order to lead to a three-dimensional reconstruction of the object to be controlled as precise as possible.

[0028] Known methods for calibrating a plenoptic camera are based on ray tracing theory. This approach is based on geometric optics, and is relatively simple to implement.

[0029] Some known calibration methods neglect the optical aberrations of the optical system, in particular of the input lens, of the plenoptic camera. These methods are mainly aimed at calibrating the alignment between the microlens array and the camera sensor. These calibration methods can be used in applications for generating images of various viewpoints and focuses. But they do not allow a calibration that is precise enough for metrology applications.

[0030] Other known methods for calibrating a plenoptic camera model the distortion of the plenoptic camera's optical system, which is a primary aberration of the main lens. For example, the plenoptic camera model described in Ihrke, Ivo, John Restrepo, and Lois Mignard-Debise, "Principles of Light Field Imaging: Briefly revisiting 25 years of research." IEEE Signal Processing Magazine 33.5 (2016), takes radial distortion into account. The calibration accuracy is improved compared to a model that does not take any distortion into account, but remains far from the accuracy required for metrological applications. In particular, subview images have other non-radial aberrations. WO 2017 / 198945 A1 also proposes a method for three-dimensional reconstruction using a plenoptic camera, taking into account the radial distortion of the input optics in the plenoptic camera model.

[0031] Other models for calibrating a plenoptic camera take into account radial distortion and tangential distortion, which are also primary aberrations of the optical system, which result from misalignment between the sensor and the input optics. The resulting calibration remains too imprecise to be used in metrological applications.

[0032] The known calibration methods described above do not take into account higher-order optical aberrations, which have an impact on the calibration of a plenoptic camera. Consequently, the accuracy of the calibration and subsequent three-dimensional reconstruction by the plenoptic camera is insufficient for metrological applications. These known methods therefore do not allow 3D reconstructions of metrological quality to be obtained.

[0033] Diffraction theory allows for the modeling of higher-order optical aberrations. However, this diffraction theory is very complex to implement. Therefore, the calibration of a plenoptic camera is not advantageously carried out within the framework of diffraction theory.

[0034] Finally, some existing ray tracing-based calibration models take into account optical depth aberration, or depth distortion, which is a higher-order aberration.

[0035] A first category of depth distortion estimation models consists of parametric models. These parametric models are based on the assumption that the image of a point source via the main lens is a point in the intermediate space.

[0036] However, it is known that this hypothesis of intersection of rays is false in the presence of optical aberration. This is highlighted in the zooms of [Fig.3] at the level of the intersection of rays. Indeed, the rays coming from an object point and which are refracted by a lens do not intersect at a single image point, but in an image zone of intersection of rays. Consequently, this hypothesis is a source of error in these models.

[0037] For example, the paper “On the Calibration of Focused Plenoptic Cameras”, Johannsen Ole et al, Heidelberg Collaboratory for Image Processing, Dagstuhl 2012 Seminar on time-of-flight imaging an GCPR 2013 workshop on imaging new modalities, describes a calibration method. This paper assumes the use of a “thin lens” model to consider that all rays coming from the same object point intersect after being imaged by the input optics of the plenoptic camera. The use of this model leads to an error concerning the estimation of the virtual point from rays that do not intersect, and an error due to all the optical aberrations not taken into account in the “thin lens” model. The depth distortion model used in this document, which is a sphere whose radius varies linearly with depth, is based only on experimental observations and does not guarantee any generalization to other optics. This document therefore aims to describe a simplified camera model, which does not allow for precise calibration taking into account higher-order aberrations.

[0038] Similarly, the document "Metric calibration of a focused plenoptic camera based on a 3D calibration target", N. Zeller et al, Karlsruhe University of Applied Sciences, ISPRS annals of photogrammetry, remote sensing and spatial information sciences, is also based on a "thin lens" model which assumes that all rays coming from the same object point intersect after being imaged by the input optics of the plenoptic camera. The aberrations treated are also not higher order aberrations. This document therefore also proposes to describe a simplified camera model, which does not allow for precise calibration taking into account higher order aberrations.

[0039] Furthermore, parametric models for estimating depth distortion consider that the curvature of the surface that approximates the points of the image space varies linearly with depth. This approximation does not correspond to the reality of the conjugation law between the curvature of the image surface and the depth, and thus generates additional errors. Parametric models therefore do not lead to sufficient calibration accuracy for metrological applications.

[0040] A second category of depth distortion estimation models consists of non-parametric models, based on look-up tables. This is the case, for example, of the model described in Meng, Lingfei, et al. "Object space calibration of plenoptic imaging systems." US Patent No. 9,918,077. Mar. 13, 2018. A look-up table is constructed between each pixel of the sensor and the position and direction characteristics of the ray that corresponds to it in object space.

[0041] Nevertheless, these non-parametric models based on look-up tables depend heavily on the accuracy of the detection of the points of the calibration target on the sensor and on the various measurement noises, because the look-up table is constructed directly from the measurements. In addition, the resolution and accuracy of the 3D reconstruction depend on the density of data present in the look-up table. Finally, these methods do not offer the possibility of analyzing the performance of the system as a function of the various parameters resulting from the calibration. The non-parametric models therefore also do not lead to sufficient calibration accuracy for metrological applications. Statement of the invention

[0042] An aim of the invention is to propose a method for calibrating a ple- camera noptic leading to improved calibration accuracy.

[0043] Another object of the invention is to propose a method for calibrating a plenoptic camera taking into account primary and higher order optical aberrations of the main optics of the plenoptic camera.

[0044] Another aim of the invention is to propose a method of three-dimensional reconstruction by a plenoptic camera having improved reconstruction precision.

[0045] According to a first aspect, the invention relates to a method for calibrating a plenoptic camera comprising the following steps: SI: acquisition of a real object point from a calibration target by the plenoptic camera, a position of the real object point being known; S6: determining a set of object light rays, wherein each object light ray is determined from an associated image light ray and a set of input data comprising projection parameters of a projection model of the plenoptic camera; S7: determination of a theoretical object point associated with the real object point, the theoretical object point corresponding to a barycenter of the set of intersections of the object light rays with the calibration target; and S8: Calculation of a calibration error as a function of a position of the theoretical object point and the position of the real object point.

[0046] Some preferred but non-limiting features of the calibration method described above are the following, taken individually or in combination:

[0047] - the method further comprises the following steps, carried out prior to step S6 for determining the set of object light rays: S5: determination of a set of image light rays, in which each image light ray passes through a respective pixel of a set of pixels belonging to a macropixel of a photosensitive sensor of the plenoptic camera and through a single optical element of a matrix of optical elements of the plenoptic camera, said single optical element being assigned to said respective pixel; S0: determining several sets of input data from the determined image light rays, each set of input data comprising projection parameters of a projection model of the plenoptic camera, in which the projection parameters are representative of a primary aberration and a higher-order aberration of a main optic of the plenoptic camera;

[0048] - the method may further comprise the following step, carried out subsequently to step S8 of calculating the calibration error: S9: determination of the calibration set corresponding to the input data set minimizing the calibration error;

[0049] - the method further comprises the following steps, carried out prior to step Real object point acquisition IF: S31: determining a relative position of a matrix plane, on which the optical elements of the matrix of optical elements are arranged, relative to a sensor plane, on which the pixels of the sensor are arranged; S32: assignment to a macropixel of the sensor of a single optical element of the matrix of optical elements, from the relative position of the matrix plane with respect to the sensor plane; and S4: determination of a set of pixels of the sensor, each pixel of the set of pixels belonging to a macropixel of the sensor and being associated with the unique optical element assigned to said macropixel, each pixel of the set of pixels being further associated with the image position of the real object point;

[0050] - the calibration target has a predetermined position and a real geometry known defined by relative positions of several real object points with respect to each other, the positions of the real object points are expressed in a reference frame of the calibration target and the positions of the theoretical object points are expressed in a reference frame of the plenoptic camera, and for each set of input data determined in step S0, step S8 of calculating the calibration error comprises the following sub-steps: S81: calculation of a first calibration error corresponding to a sum, carried out on the several real object points of the calibration target, of the distances between each real object point and each associated theoretical object point; S82: calculation of a second calibration error corresponding to a sum, carried out on the several theoretical object points, of a sum of the distances between the theoretical object point and each object light ray corresponding to the theoretical object point; and S83: calculation of the calibration error for the predetermined position of the calibration target, corresponding to a sum of the first calibration error and the second calibration error;

[0051] - the method further comprises a step of moving the calibration target between several predetermined positions, the relative positions of the several predetermined positions of the calibration target with respect to each other being known, wherein some of the several object points are acquired in step S1 for several different predetermined positions of the calibration target;

[0052] - the method further comprises the following steps: S52: determining a sampling of a gradient of an output wavefront surface from the set of image light rays determined in step S5; S53: determination of several representative functions of the front surface output wave from the gradient sampling determined in step S52, each of said several representative functions of the output wavefront surface corresponding to a sum: - an ideal wavefront surface at an output of a main optic without optical aberration, the ideal wavefront surface being a spherical surface having a spherical surface radius; and - an aberration function adapted to take into account primary and higher order optical aberrations of the main optics, the aberration function being adapted to be decomposed into a series of polynomials; step S53 corresponding to a determination of the coefficients associated with degrees of polynomial of the series of polynomials of the aberration function up to a degree representative of a higher order aberration of the main optics, the projection parameters of each set of input data comprising the coefficients determined in step S53;

[0053] - the projection parameters of each input data set com take : - the radius of the spherical ideal wavefront surface (S_id); - the coefficients up to the fifth degree of the series of polynomials of the aberration function; and - a distance between the main optics and the plenoptic camera sensor;

[0054] - the positions of the optical elements on the matrix plane are known, the positions of the sensor pixels on the sensor plane are known, and the relative position of the array plane to the sensor plane is determined in step S3 from the following steps: S310: acquisition by the plenoptic camera of a reference object; S320: detection on the sensor plane of the image of the reference object by the plenoptic camera; S330: determining the positions of the optical elements on the sensor plane, from the image of the reference object detected in step S320; S340: determining a geometric transformation between the matrix plane and the sensor plane, from the known positions of the optical elements on the matrix plane and the positions of the optical elements on the sensor plane determined in step S330;

[0055] - step S340 of determining the geometric transformation between the plane of matrix and sensor plan includes the following steps: S341: determination of a homography between the matrix plane and the sensor plane; and S342: determination of a rotation and / or a translation between the matrix plane and the sensor plane, from the homography determined in step S35.

[0056] - the method may further comprise a step S41 of determining a position of each optical element assigned to a respective pixel of the set determined in step S4 on the sensor plane, from the relative position of the matrix plane with respect to the sensor plane determined in step S3, each image light ray determined in step S5 passing through said pixel and said respective optical element;

[0057] - the optical elements are microlenses or pinholes.

[0058] According to a second aspect, the invention relates to a method for three-dimensional reconstruction of an object using a plenoptic camera, said object comprising a plurality of object points, the method comprising the following steps: S100: calibrating the plenoptic camera by determining a calibration input data set minimizing a calibration error of the plenoptic camera using a calibration method according to the first aspect; for each object point of the object to be reconstructed three-dimensionally: S101: acquisition of the object point by the plenoptic camera; S102: determining an image position of the object point, said image position corresponding to a position detected on a sensor of the plenoptic camera of the image of the object point through the plenoptic camera; S104: determining a set of pixels of the sensor, each pixel of the set of pixels belonging to a macropixel of the sensor and being associated with a single optical element of a matrix of optical elements of the plenoptic camera, said single optical element being assigned to a macropixel of the sensor, each pixel of the set of pixels being further associated with the image position of the object point; S105: determining a set of image light rays, in which each image light ray passes through a respective pixel of the set of pixels and through the single optical element assigned to said respective pixel during step S104; S106: determining a set of object light rays, wherein each object light ray is determined from an associated image light ray determined in step S105 and the projection parameters of the calibration set determined in step S100; and S107: three-dimensional reconstruction of the object point, said object point corresponding to a barycenter of the set of intersections of the object light rays determined in step S106.

[0059] According to a third aspect, the invention relates to a three-dimensional reconstruction device comprising a plenoptic camera and a digital processing means, the plenoptic camera comprising a main optic, a photosensitive sensor and a matrix of optical elements arranged between the main optic and the sensor, the matrix of optical elements comprising a plurality of optical elements arranged on a matrix plane, the sensor comprising at least one macropixel comprising a plurality of pixels arranged on a sensor plane, the digital processing means comprising a plenoptic camera model adapted to be calibrated by a method of calibrating the plenoptic camera according to the first aspect, the digital processing means being adapted to implement a method of three-dimensional reconstruction of an object according to the second aspect. DESCRIPTION OF FIGURES

[0060] Other characteristics, aims and advantages of the present invention will appear on reading the detailed description which follows, given by way of non-limiting example, which will be illustrated by the following figures:

[0061] [Fig. 1], already commented on, is a diagram illustrating a plenoptic camera.

[0062] [Fig.2a], already commented on, is a diagram illustrating ray tracings from of a wavefront for an ideal optical system without optical aberrations.

[0063] [Fig.2b], already commented on, is a diagram illustrating ray tracings from of a wavefront for a real optical system with optical aberrations.

[0064] [Fig. 3], already commented on, is a diagram illustrating a classical approach to modeling a higher order optical aberration.

[0065] [Fig.4] is a diagram illustrating basic characteristics of a camera plenoptic of a three-dimensional reconstruction device according to one embodiment of the invention.

[0066] [Fig.5a] is a diagram illustrating a matrix of optical elements of a plenoptic camera of a three-dimensional reconstruction device according to one embodiment of the invention.

[0067] [Fig.5b] is a diagram illustrating a relative position of a matrix of optical elements with respect to a sensor of a plenoptic camera of a three-dimensional reconstruction device according to an embodiment of the invention.

[0068] [Fig.5c] is a diagram illustrating an image of a reference object acquired by a plenoptic camera of a three-dimensional reconstruction device according to one embodiment of the invention.

[0069] [Fig.6] is a functional diagram illustrating a determination of a theoretical object point in a calibration method according to an embodiment of the invention.

[0070] [Fig.7] is a functional diagram illustrating a determination of a set of input data minimizing a calibration error in a calibration method according to one embodiment of the invention.

[0071] [Fig.8a] is a functional diagram illustrating a determination of a relative position of a matrix plane with respect to a sensor plane in a calibration method according to an embodiment of the invention.

[0072] [Fig.8b] is a functional diagram illustrating a determination of a geometric transformation between the matrix plane and the sensor plane in a calibration method according to an embodiment of the invention.

[0073] [Fig.9] is a functional diagram illustrating steps of a method for three-dimensional reconstruction of an object according to one embodiment of the invention.

[0074] [Fig. 10] is a diagram illustrating a ray trace from an image point and an input and output wavefront surface, respectively for an ideal optical system without optical aberrations, and for a real optical system with optical aberrations. DETAILED DESCRIPTION OF THE INVENTION

[0075] A method for calibrating a plenoptic camera is partially illustrated by way of non-limiting example in [Fig. 6]. The plenoptic camera, as illustrated by way of example in [Fig. 1], may comprise a main optic 1, a photosensitive sensor 3 and a matrix of optical elements 2 arranged between the main optic 1 and the sensor 3, the matrix of optical elements 2 comprising a plurality of optical elements 21 arranged on a matrix plane Pmat, the sensor 3 comprising at least one macropixel 31 comprising a plurality of pixels arranged on a sensor plane Pcapt.

[0076] The calibration method comprises the following steps: SI: acquisition of a real object point P_obj_r from a calibration target by the plenoptic camera, a position of the real object point P_obj_r being known; S6: determination of a set of object light rays 60, in which each object light ray 60 is determined from an associated image light ray 50 and from a set of input data E comprising projection parameters Par_Proj of a projection model of the plenoptic camera; S7: determination of a theoretical object point P_obj_th associated with the real object point P_obj_r, the theoretical object point P_obj_th corresponding to a barycenter of the set of intersections of the object light rays 60 with the calibration target; and S8: calculation of a calibration error Acal as a function of a position of the theoretical object point P_obj_th and the position of the real object point P_obj_r.

[0077] Step S1 may comprise an acquisition of several real object points P_obj_r of the calibration target by the plenoptic camera, the relative positions of the real object points P_obj_r with respect to each other being known. Step S2 is then a step of determining several image positions P_im of the several real object points P_obj_r, each image position P_hn corresponding to a position detected on the sensor 3 of the image of a respective real object point P_obj_r through the plenoptic camera. The calibration error Acal is then calculated in step S8 by function of the positions of the several theoretical object points P_obj_th determined for each of the several real object points P_obj_r, and of the positions of the several associated real object points P_obj_r.

[0078] Each object light ray 60 is determined in step S6 from an associated image light ray 50 determined in step S5 and the projection parameters Par_Proj of the input data set E determined in step S0. The theoretical object point P_obj_th associated with the real object point P_obj_r can be determined by triangulation of the object light rays 60 determined in step S6.

[0079] The calibration method may further comprise the following steps, carried out prior to step S6 of determining the set of object light rays 60: S5: determination of a set of image light rays 50, in which each image light ray 50 passes through a respective pixel of a set of pixels belonging to a macropixel 31 of a photosensitive sensor 3 of the plenoptic camera and through a single optical element 21 of a matrix of optical elements 2 of the plenoptic camera, said single optical element 21 being assigned to said respective pixel; S0: determination of several sets of input data E from the determined image light rays 50, each set of input data E comprising projection parameters Par_Proj of a projection model of the plenoptic camera, in which the projection parameters Par_Proj are representative of a primary aberration and a higher order aberration of a main optic 1 of the plenoptic camera; and wherein the method further comprises the following step, performed subsequently to step S8 of calculating the calibration error Acal: S9: determination of the calibration set E_min corresponding to the input data set E minimizing the calibration error Acal.

[0080] The calibration method may further comprise the following steps, carried out prior to step SI of acquiring the real object point P_ob_r: S31: determination of a relative position of a matrix plane Pmat, on which the optical elements 21 of the matrix of optical elements 2 are arranged, relative to a sensor plane Pcapt, on which the pixels of the sensor 3 are arranged; S32: allocation to a macropixel 31 of the sensor 3 of a single optical element 21 of the matrix of optical elements 2, from the relative position of the matrix plane Pmat with respect to the sensor plane Pcapt; and S4: determination of a set of pixels 3 of the sensor 3, each pixel of the set of pixels belonging to a macropixel 31 of the sensor 3 and being associated with the single optical element 21 attributed to said macropixel 31, each pixel of the set of pixels being further associated with the image position P_im of the real object point P_obj_r.

[0081] The steps of the calibration method can be carried out in any conceivable order.

[0082] The position of the real object point P_obj_r, respectively of the theoretical object point P_obj_th, can correspond to three-dimensional Cartesian coordinates x_obj_r, y_ob_r, z_obj_r, respectively x_obj_th, y_ob_th, z_obj_th of the real object point P_obj_r, respectively of the theoretical object point P_obj_th.

[0083] The projection model of the plenoptic camera is adapted to determine the position of the theoretical object point P_obj_th, which corresponds to an object of the image point P_im determined on the sensor 3, from the defined input data set E. In other words, the projection model of the plenoptic camera is adapted to reconstruct, from the image point P_im, the associated theoretical object point P_obj_th, that is to say to carry out an inverse projection of the image point P_im observed through the plenoptic camera, to deduce the associated object point.

[0084] The inverse projection of the image point P_hn through the plenoptic camera is determined from a plenoptic camera model taking into account higher-order aberrations of the main optics 1, via the projection parameters Par_Proj. The resulting calibration therefore takes into account, in the projection model of the plenoptic camera, the primary geometric aberrations as well as the higher-order optical aberrations of the main optics 1 of the plenoptic camera.

[0085] Such a plenoptic camera calibration method makes it possible to better model the optical aberrations of the optical system of the main optics 1, including higher order optical aberrations.

[0086] By taking into account higher order aberrations, the calibration method makes it possible to obtain three-dimensional reconstructions having better accuracy, in particular three-dimensional reconstructions of metrological quality. The method thus makes it possible to improve the accuracy of the calibration and the three-dimensional reconstruction by the plenoptic camera, and to provide a calibration having an accuracy compatible with use of the plenoptic camera in three-dimensional metrology.

[0087] The method makes it possible to calculate the calibration error Acal associated with each set of input data E. Thus, the calibration set E_min comprising the projection parameters Par_Proj minimizing the calibration error Acal can be determined. This calibration set E_min minimizing the calibration error Acal corresponds to the best calibration of the projection model of the plenoptic camera.

[0088] Furthermore, the method described is generic and can be applied to any plenoptic camera.

[0089] Using the inverse light ray tracing, i.e. starting from the image point P_im to obtain the theoretical object point P_obj_th, makes it possible to further increase the accuracy of subsequent three-dimensional reconstructions by the plenoptic camera. The calibration error Acal is calculated at the calibration target and the projection parameters Par_Proj of the calibration model are therefore optimized to minimize the three-dimensional reconstruction errors directly in the object space. The reconstruction of the theoretical object point P_obj_th from the image point P_im observed on the sensor 3 carried out in the calibration process corresponds to the three-dimensional reconstruction of an object carried out by the plenoptic camera once calibrated. Thus, the projection model of the plenoptic camera, in particular the calibration set E_min, makes it possible to characterize the plenoptic camera and also to know the error during the subsequent three-dimensional reconstruction.

[0090] Using inverse light ray tracing also simplifies the calibration process. Indeed, the image light rays 50 are directly obtained without having to calculate an entire wavefront surface. The wavefront surface can be reconstructed from the image light rays 50 belonging to the set of image light rays 50.

[0091] The projection model of the plenoptic camera may be based on the tracing of inverse light rays through the plenoptic camera. The projection parameters Par_Proj may correspond to coefficients representative of a wavefront surface used for tracing inverse light rays. The inverse projection of the image point P_im through the plenoptic camera is thus determined from the tracing of inverse light rays from the set of projection parameters Par_Proj taking into account the higher-order aberrations of the main optics 1, and corresponds to the theoretical object point P_obj_th.

[0092] Several real object points P_obj_r of the calibration target can be acquired, in order to detect the several corresponding image positions on the sensor 3 of each of the several real object points P_obj_r of the calibration target. In parallel, several respective theoretical object point positions P_obj_th are determined by means of the projection model of the plenoptic camera, as a function of the input data set E determined in step S0. The calculation of the calibration error Acal is to be carried out as a function of the comparison of the several positions of the theoretical object points P_obj_th and the several positions of the respective associated real object points P_obj_r, for each input data set E.

[0093] This calibration method thus makes it possible to improve the robustness and precision of the calibration determination E_min minimizing the calibration error Acal of the projection model of the plenoptic camera, thus improving the robustness and precision of the calibration of the plenoptic camera.

[0094] The number of real object points P_obj_r of the calibration target can be adapted according to the number of projection parameters Par_Proj of the data set input to be determined. The greater the number of real object points P_obj_r of the calibration target, the more robust the corresponding determination of the input data set is, i.e. the more the convergence of the optimization algorithm is established, therefore the more accurate the calibration of the projection model of the plenoptic camera is. For example, at least 25 real object points P_obj_r of the calibration target can be acquired.

[0095] The calibration method is therefore simple to implement. The complexity of the calibration method is similar to the complexity of a calibration method based on ray tracing models without taking into account higher order aberrations. Preliminary notions

[0096] An optical axis z designates an axis along which the different optical elements 21 of the plenoptic camera are positioned, in particular an axis along which the main optics 1, the matrix of optical elements 2, and the photosensitive sensor 3 are positioned.

[0097] A light ray is represented by a straight line in a plane passing through the optical axis z. A direction of a light ray designates an incidence, that is to say an angle, of the light ray with respect to the optical axis z.

[0098] A center of an optical element 21 is a particular point of the optical element 21 such that a light ray incident at this particular point is not deflected, its incident and emerging parts being parallel to each other.

[0099] In the remainder of the application, the term image is used to designate the set of image point(s) P_hn of a real object point P_obj_r through the plenoptic camera. An object point acquired by the plenoptic camera is in fact likely to generate several image points. For example, the term image position P_im of the real object point P_obj_r is used to designate the set of image points P_im of the real object point P_obj_r through the plenoptic camera, the image position P_im being able to comprise one or more points, or even an area of ​​space. The hnage point(s) P_hn of a real object point P_obj_r can be imaged by one or more optical elements 21 of the matrix of optical elements 2, and the image P_im can be formed on one or more macropixels 31(s) of the sensor 3. In particular, the image P_im of the real object point P_obj_r can be formed on several pixels of several macropixels 31 of the sensor 3 of the plenoptic camera.The image position P_hn then includes the position of all the pixels of the macropixels 31 of the sensor 3 on which the image of the real object point P_obj_r is formed. Plenoptic camera

[0100] [Fig.4] illustrates an example of main characteristics of an optical system, in particular of the main optic 1 of the plenoptic camera.

[0101] The main optics 1 may be a complex optical system, comprising a plurality of optical elements. In particular, the main optics 1 may be formed from a lens or a combination of lenses.

[0102] The optical system of the main optics 1 comprises an object focal plane F and an image focal plane F'. The main optics 1 receives light rays coming from an object point.

[0103] The optical system of the main optics 1 comprises an entrance pupil 11, which corresponds to the diaphragm of the optical system that limits the light radiation coming from an object. The main optics 1 also comprises an exit pupil 12, which corresponds to the image of the entrance pupil 11 by the main optics 1.

[0104] The entrance pupil 11 is positioned at an entrance of the main optics 1. The exit pupil 12 is positioned at an exit of the main optics 1. The entrance pupil 11 can be located at a position z = 0 of the optical axis z. The entrance pupil 11 is located upstream of the exit pupil 12 on the optical axis z.

[0105] The matrix of optical elements 2, or network of optical elements, may comprise a plurality of optical elements 21, as shown in [Fig.l]. In particular, the centers of the optical elements 21 may be aligned so as to form rows and columns of optical elements 21, as illustrated by way of non-limiting example in [Fig.5a],

[0106] The plurality of optical elements 21 is arranged in a matrix plane Pmat, as illustrated in [Fig.5b]. The optical elements 21 are thus arranged in a coplanar manner. The positions of the centers of the optical elements 21 in the matrix plane Pmat are known.

[0107] In a first exemplary embodiment, the optical elements 21 are microlenses. The optical center of a microlens is the point of intersection between a plane of the microlens and the optical axis z. The microlenses can be spherical or have a more complex shape allowing a junction between two adjacent microlenses. The microlenses typically have a diameter of the order of a fraction of a millimeter. The “pinhole” type model is used for the microlenses, due to their small dimensions.

[0108] In a second exemplary embodiment, the optical elements 21 are pinholes. The optical center of a pinhole is the center of a pinhole hole. The matrix of optical elements 2 is an opaque flat surface in which holes of very small diameter are formed, typically spaced apart by a fraction of a millimeter.

[0109] The matrix of optical elements 2 can be arranged in the image focal plane F' of the main optics 1, or outside this image focal plane F'. The matrix of optical elements 2 can be arranged close to the photosensitive sensor 3.

[0110] The photosensitive sensor 3 comprises at least one macropixel 31 comprising a plurality of pixels arranged on a sensor plane Pcapt. Each pixel of the sensor 3 receives the light of a single light ray via the main optics 1. Each macropixel 31 of the sensor 3 receives all the light rays coming from the same optical element 21, the macropixel 31 corresponding to the group formed by all the pixels of the sensor 3 receiving the light rays coming from said same optical element 21. A macropixel 31 comprises at least two pixels extending in the horizontal direction and / or in the vertical direction. The macropixels 31 of the sensor 3 may be located in the same sensor plane Pcapt. The plurality of pixels of the different macropixels 31 of the sensor 3 thus forms the sensor plane Pcapt, or detection surface of the sensor 3. The positions of the pixels in the sensor plane Pcapt are known.The matrix of optical elements 2 can be arranged substantially parallel to the sensor 3.

[0111] The photosensitive sensor 3 captures the received light to produce an image. The photosensitive sensor 3 may be a matrix sensor 3, typically a CCD matrix. The photosensitive sensor 3 may be a photosensitive electronic component composed of photosites adapted to convert electromagnetic radiation (UV, visible or IR) into an analog electrical signal. This signal is then digitized by digital processing means 4, such as an analog-digital converter, to obtain a digital image composed of pixels, each pixel corresponding to a photosite of the photosensitive sensor 3. Real object points and calibration target

[0112] The real object point P_obj_r is a source point of interest intended to be acquired by the plenoptic camera and whose image is intended to be detected on the sensor 3.

[0113] The relative positions of the real object points P_obj_r with respect to each other are known prior to the calibration of the plenoptic camera. The real object points P_obj_r are located upstream of the entrance of the main optics 1, that is to say upstream of the entrance pupil 11, on the optical axis z.

[0114] The real object points P_obj_r are points of a calibration target. A calibration target is a flat surface comprising several real object points P_obj_r. The calibration target can be arranged in any plane of the main optics 1, including for example in an object focal plane F of the main optics 1. The calibration target can be located substantially at a distance where the objects to be reconstructed three-dimensionally will be located once the calibration of the plenoptic camera has been carried out.

[0115] The calibration target may have a predetermined position. Relative positions of several different predetermined positions of the calibration target with respect to each other are known.

[0116] A real geometry of the calibration target is defined by relative positions of several real object points P_obj_r with respect to each other. The real geometry of the calibration target is known.

[0117] For a predetermined position of the calibration target, a position of the image of the calibration target on the sensor 3, corresponding to all the image points of the calibration target through the plenoptic camera, can be detected.

[0118] The calibration target can be a checkerboard. The real object points P_obj_r are then the corners of the checkerboard. Known methods can be used to detect the corners of the checkerboard on the sensor 3 of the plenoptic camera.

[0119] Alternatively, the calibration target may be a grid of points. The real object points P_obj_r are then the centers of the points of the grid. The detection of the centers of the points on the sensor 3 may be carried out for example by thresholding the image then calculating the centroids of the connected components. The real object points P_obj_r of the grid of points must be sufficiently separated to be able to make the matching between the different image points detected on the sensor 3 and the different real object points P_obj_r of the calibration target.

[0120] Alternatively, the calibration pattern may be a flat LCD or LED screen comprising a plurality of pixels distributed in rows and columns. These pixels are therefore distributed on a flat surface. The distance between the rows and the distance between the columns of pixels are known data. Indeed, these are manufacturing data of the screen. The pixels of the screen constitute the real object points P_obj_r of the calibration pattern. During calibration, only one pixel of the screen is lit at a time, so that only one real object point P_obj_r is imaged at a time by the plenoptic camera, the steps of the calibration method being successively repeated for several successively lit pixels of the flat LCD or LED screen.

[0121] Alternatively, the calibration pattern may be a screen with coding. Each pixel on the screen sends a binary code sequence that contains the row and column number of the pixel in the screen, i.e. the position of the pixel in the screen. All pixels are lit at the same time, according to the code corresponding to each pixel. The code corresponding to each pixel corresponds to the lighting of a single pixel at a time, the pixel having a specific row and column number and forming the actual object point P_obj_r to be imaged by the plenoptic camera. For example, the code may be a binary code, sinusoidal fringes, etc. Such a calibration pattern makes it possible to automate and increase the speed of the calibration process of the plenoptic camera.

[0122] Repetition of the steps for several real object points and / or for several predetermined positions of the calibration target

[0123] For a predetermined position of the calibration target, several real object points P_obj_r of the calibration target can be acquired.

[0124] The method may comprise a step of moving the calibration target between several predetermined positions, the relative positions of the several predetermined positions of the calibration target with respect to each other being known. Some of the several object points P_obj_r are acquired in step SI for several different predetermined positions of the calibration target.

[0125] The plenoptic camera may acquire a series of images of the calibration target, each image corresponding to a specific predetermined position of the calibration target, the relative position of which with respect to the other predetermined positions of the calibration target is known.

[0126] The calibration target can be moved in rotation and / or translation. Motorized stages can be used to move the calibration target between the different predetermined positions. Then, only a first position of the calibration target must be determined in absolute terms. The other positions of the calibration target are determined in relative terms with respect to this first position via the motorized stages.

[0127] Calibrating the plenoptic camera from several predetermined positions of the calibration target makes it possible to improve the robustness and the precision of the determination of the calibration set E_min minimizing the calibration error Acal of the projection model of the plenoptic camera, thus improving the robustness and the precision of the calibration of the plenoptic camera.

[0128] The number of predetermined positions of the calibration target can be adapted according to the number of projection parameters Par_Proj of the input data set to be determined. The greater the number of predetermined positions of the calibration target, the more robust the corresponding determination of the input data set, and therefore the more precise the calibration of the projection model of the plenoptic camera.

[0129] In an exemplary embodiment, the calibration target is a screen and the calibration method takes place in the following manner.

[0130] The screen is placed in front of the plenoptic camera in an initial screen position. The initial screen position is not known, and will be determined when estimating the input data set E minimizing the calibration error Acal. A single pixel of the screen, forming the real object point P_obj_r and corresponding to a given pixel position of the screen, is lit. The relative position of the pixel with respect to the other pixels of the screen is known.

[0131] An image of the lit pixel is acquired from the plenoptic camera during step SL. The acquired image formed on the sensor 3 is black, except at the level of the macropixel(s) 31 corresponding to the projection onto the sensor 3 of the screen pixel which is lit. Thus, the detection of the image point P_im during step S2 is carried out immediately, without detection and correspondence processing.

[0132] The step of lighting a pixel and acquiring the lit pixel is repeated for several different screen pixels, or even for all the screen pixels. The screen pixels that are thus lit successively constitute the real object points P_obj_r, which are also called points of interest of the calibration target.

[0133] The calibration target is then moved. For each position of the calibration target, the steps of lighting the points of interest of the calibration target and acquiring the corresponding real object points P_obj_r are carried out.

[0134] The calibration target can be moved for example to 25 different screen positions, and include 100 different real object points P_obj_r corresponding to 100 points of interest of the calibration target. Alternatively, the calibration target can be moved to between 15 and 20 different screen positions, and include 50 different real object points P_obj_r.

[0135] S31: determination of a relative position of the matrix plane with respect to the sensor plan

[0136] The matrix of optical elements 2 comprises a plurality of optical elements 21 arranged on a matrix plane Pmat, the positions of centers of the optical elements 21 on the matrix plane Pmat being known. The geometric model of the matrix of optical elements 2 is known. In other words, the coordinates of the centers of the optical elements 21 in the coordinate system of the matrix plane Pmat, and the distances between the centers of the optical elements 21 of the matrix of optical elements 2, are known.

[0137] The sensor 3 comprises a plurality of pixels arranged on a sensor plane Pcapt, the positions of the pixels on the sensor plane Pcapt being known.

[0138] The matrix plane Pmat may be substantially parallel to the sensor plane Pcapt, or may have a non-zero relative position with respect to the sensor plane Pcapt, as illustrated by way of non-limiting example in [Fig.5b], i.e. may not be perfectly parallel to the sensor plane Pcapt.

[0139] The relative position of the matrix plane Pmat with respect to the sensor plane Pcapt can be determined in step S31 from the following steps, as illustrated by way of non-limiting example in [Fig.8a] and in [Fig.8b]: S310: acquisition by the plenoptic camera of a reference object; S320: detection on the sensor plane Pcapt of image 5 of the reference object by the plenoptic camera; S330: determination of the positions of the centers of the optical elements 21 on the sensor plane Pcapt, from the image 5 of the reference object detected in step S320; S340: determination of a geometric transformation Tmat / capt between the matrix plane Pmat and the sensor plane Pcapt, from the positions of the centers of optical elements 21 on the matrix plane Pmat known and the positions of the centers of optical elements 21 on the sensor plane Pcapt determined in step S330.

[0140] This model makes it possible to determine the position of the centers of the optical elements 21 in the reference frame linked to the plane of the sensor 3. In other words, it is a matter of determining the position of the matrix of optical elements 2 in the reference frame defined by the photosensitive sensor 3 of the plenoptic camera.

[0141] Step S31 of determining the relative position of the matrix plane Pmat with respect to the sensor plane Pcapt can be carried out only once, prior to the other steps of the calibration method. Step 31 requires the acquisition of a uniform image.

[0142] The position of each optical element center 21 in the matrix plane Pmat being known, the position of the matrix of optical elements 2 can be determined in step S330 by the projection of the centers of the optical elements 21 onto the sensor 3. In other words, the detection of the projection onto the sensor 3 of each optical element center 21 in the reference frame linked to the sensor 3 makes it possible to determine the relative position between the plane linked to the optical elements 21 Pmat and the plane linked to the sensor 3 Pcapt.

[0143] The reference object acquired during step S310 may be a flat surface located in the matrix plane Pmat, the images of the centers of the optical elements 21 being formed in the same plane which corresponds to the sensor plane Pcapt.

[0144] The reference object acquired during step S310 may be a flat surface of uniform color. [Fig.5c] illustrates by way of non-limiting example an image 5 of such a reference object acquired by the plenoptic camera. For example, the reference object may be a sheet of white paper, a backlight, an LCD screen with all the pixels lit with the same gray level value, etc.

[0145] The determination of the position of the centers of the optical elements 21 carried out in step S330 may correspond to a determination of the position of the centers of the optical elements 21 in the reference frame linked to the sensor 3, that is to say in the sensor plane Pcapt. The positions of the centers of the optical elements 21 of the matrix of optical elements 2 in the sensor plane Pcapt may be determined in step S330 by segmenting the acquired image and then determining the centers of each ellipse.

[0146] The geometric transformation Tmat / capt between the matrix plane Pmat and the sensor plane Pcapt determined in step S340 corresponds to the geometric transformation to be applied between the centers of the optical elements 21 and their projections on the sensor 3, to pass from the position of each of the centers of the optical elements 21 to the corresponding point in the image formed on the sensor 3, that is to say to pass from the matrix plane Pmat to the sensor plane Pcapt.

[0147] Step S340 of determining the geometric transformation Tmat / capt between the matrix plane Pmat and the sensor plane Pcapt may comprise the following steps, as illustrated by way of non-limiting example in [Fig.8b]: S341: determination of a homography between the matrix plane Pmat and the sensor plane Pcapt; S342: determination of a rotation and / or a translation between the matrix plane Pmat and the sensor plane Pcapt from the homography determined in step S341.

[0148] The transformation Tmat / capt is then a homography, from which it is possible to determine the position, in terms of rotation and translation, of one plane relative to the other. The homography between the positions of the centers determined in the sensor plane Pcapt thanks to the image acquisition of the plenoptic camera, and the positions of the centers of the optical elements 21 in the matrix plane Pmat, which are known, is then determined.

[0149] The homography can correspond to a rotation and / or a translation between the matrix plane Pmat and the sensor plane Pcapt. The translation along the optical axis z is determined to within a factor when the position of the main objective is not yet determined.

[0150] The relative position of the matrix plane Pmat with respect to the sensor plane Pcapt can be determined from the determined geometric transformation Tmat / capt.

[0151] S32: allocation of a single optical element to a macropixel

[0152] Step S32 of assigning to each macropixel 31 of the sensor 3 the single optical element center 21 is carried out from the relative position of the matrix plane Pmat with respect to the sensor plane Pcapt determined in step S31.

[0153] Step S32 of assigning to each macropixel 31 of the sensor 3 the single optical element center 21 can be carried out only once, after step S31 and prior to the other steps of the calibration method. Step 32 requires the acquisition of a uniform image.

[0154] Thus, the calibration method makes it possible to associate, with a given macropixel 31 of the sensor 3 corresponding to a group of pixels of the sensor 3, a single optical element 21 which corresponds to this macropixel 31.

[0155] Step S32 can be carried out for each macropixel 31 of the sensor 3, in order to assign to each macropixel 31 of the sensor 3 a single corresponding optical element 21, the different macropixels 31 of the sensor 3 being assigned to different optical elements 21.

[0156] Thus, each pixel of the macropixel 31 of the sensor 3 is associated with a single optical element 21, a ray passing through said pixel of the macropixel 31 of the sensor 3 also passing through said single optical element 21.

[0157] The method therefore makes it possible to carry out a mapping between the pixels of the sensor 3 and the optical elements 21 which correspond to it. In the case where the optical elements 21 are microlenses, the method makes it possible to determine the pixel mapping. microlens.

[0158] The calibration method may further comprise a step S33 of determining a position of the center of the optical element 21 assigned to the pixel of the sensor 3 on the sensor plane Pcapt.

[0159] The position of the center of the optical element 21 is determined by determining the transformation Tmat / capt between the matrix plane Pmat and the sensor plane Pcapt determined in step S330.

[0160] S4: determination of the set of pixels of the sensor

[0161] Step S4 makes it possible to associate, with each pixel of a macropixel 31 of the sensor 3 on which the image of the real object point P_obj_r is detected, a single optical element 21. S5: determination of image light rays

[0162] The propagation of the light rays is carried out starting from the pixels of the sensor 3 towards the imaged scene.

[0163] Each image light ray 50 passes through a respective pixel of the set of pixels of the sensor 3 determined in step S4 and through the single optical element 21 assigned to said respective pixel during step S32. Thus, for a single pixel of the sensor 3 associated with the image position P_im of the real object point P_obj_r, a single corresponding image light ray 50 is traced. The number of image light rays 50 determined in step S5 corresponds to the number of pixels of the sensor 3 forming the image position P_im of the real object point P_obj_r.

[0164] In other words, the pixel of the sensor 3 provides a first position of the light ray at the sensor 3. The position of the optical element 21 associated with the pixel of the sensor 3 provides a second position of the image light ray 50 at the matrix of optical elements 2. The image light ray 50 corresponds to the straight line passing through the first position and the second position of the image light ray 50.

[0165] The position and direction of each image light ray 50 of the set of image light rays 50 are thus deduced from the image position P_im of the real object point P_obj_r determined in step S2 and from the allocation of the macropixels 31 to the optical elements 21 carried out in step S32.

[0166] S0: determination of several sets of input data

[0167] Steps S4 to S8 of the calibration method are repeated for different sets of input data E of the projection model of the plenoptic camera.

[0168] An input data set E can be determined as a function of the set of image light rays 50 determined in step S5. The input data set E is determined so as to make it possible to accurately reconstruct, via the model of the plenoptic camera and from the several real object points P_obj_r acquired during step S1 and whose relative positions relative to the plenoptic camera are known, the several corresponding theoretical object points P_obj_th, i.e. so as to obtain theoretical object points P_obj_th, which are as close as possible to the corresponding real object points P_obj_r according to proximity criteria described below. Several sets of input data E are thus determined, and the calibration set E_min corresponds to the set of input data E which minimizes the calibration error Acal.

[0169] An input data set E(i) of a current iteration (i-th iteration) can be determined as a function of the calibration error Acal associated with the input data set E(il) determined for during a previous iteration (i-th iteration). More particularly, an optimization algorithm can be used so as to optimize a determination time of a calibration set E_min capable of minimizing the calibration error Acal, with a view to reducing the calibration time. Such optimization algorithms, such as the Levenberg Marquardt algorithm, are well known to those skilled in the art.

[0170] Alternatively, the different projection parameters Par_Proj of the input data set E can be determined by scans with given steps, over defined ranges of parameter values.

[0171] S52: Determining a sampling of a gradient of a front surface output wave

[0172] The calibration method may further comprise a step S52 of determining a sampling of a gradient of an output wavefront surface S_sor from the set of image light rays 50 determined in step S5.

[0173] A direction of each image light ray 50 is perpendicular to the gradient of the output wavefront surface S_sor at a given point on the output wavefront surface S_sor. Indeed, the image light rays 50 propagate perpendicular to the gradient of the output wavefront surface S_sor at the exit of the exit pupil 12 of the main optics 1.

[0174] A sampling of the gradient of the output wavefront surface S_sor can thus be determined from the direction of the image light rays 50 determined in step S5. The sampling of the gradient comprises a number of points corresponding to the number of image light rays 50 determined in step S5. The sampling of the gradient of the output wavefront surface S_sor takes into account the primary and higher-order aberrations of the main optics 1.

[0175] S53: Determination of an output wavefront surface

[0176] The calibration method may further comprise a step S53 of determining several functions representative of the output wavefront surface S_sor from the gradient sampling determined in step S52.

[0177] Each of said several functions representative of the output wavefront surface may be a three-dimensional surface determined by decomposition of a representative function of the output wavefront surface S_sor in a series of orthogonal polynomials.

[0178] Each of said several representative functions of the output wavefront surface S_sor corresponds to a sum: - an ideal wavefront surface S_id at an output of a main optic 1 without optical aberration, the ideal wavefront surface S_id being a spherical surface having a spherical surface radius; and - an aberration function W adapted to take into account primary and higher order optical aberrations of the main optic 1, the aberration function W being adapted to be decomposed into a series of polynomials.

[0179] In other words, the output wavefront surface S_sor from the real object point P_obj_r at the exit pupil 12 of the main optics 1 of the plenoptic camera can be written: S_sor = S_id + W.

[0180] Step S53 corresponds to a determination of the coefficients associated with degrees of polynomial of the series of polynomials of the aberration function W up to a degree representative of a higher order aberration of the main optics 1. The projection parameters Par_Proj of each set of input data E comprise the coefficients determined in step 53.

[0181] The several representative functions of the output wavefront surface S_sor are thus determined from the plot of all the image light rays 50, and take into account the primary and higher order aberrations of the main optics 1.

[0182] For example, several different output wavefront surface representative functions S_sor may be determined, each output wavefront surface representative function S_sor having a gradient substantially equal to the gradient determined in step S52 at each point of the gradient sampling determined in step S52. Alternatively, an output wavefront surface S_sor may correspond to a wavefront surface that minimizes a total distance with all the points of the gradient sampling determined in step S52 according to a given minimization criterion. The output wavefront surface representative functions S_sor may be determined according to the methods set forth in the document Janssen, AJEM "Zernike expansion of derivatives and Laplacians of the Zernike circle polynomials." JOSA A 31.7 (2014): 1604-1613.

[0183] The ideal wavefront surface S_id represents a geometric wavefront surface of a geometric wavefront that would be obtained by passing light rays from the real object point P_obj_r through a perfect main optic exhibiting no distortion.

[0184] The ideal wavefront surface S_id can correspond to a sum of the surface of input wavefront S_en and a surface representing the main optics S_lens. Thus, the output wavefront surface S_sor can be written: S_sor = S_id + W.

[0185] The surface representing the main optics S_lens and the aberration function W being identical for all the acquired real object points P_obj_r, the acquisition of a sufficient number of real object points P_obj_r by the plenoptic camera makes it possible to determine the coefficients of the series of polynomials of the aberration function W, therefore the projection parameters Par_Proj of the input data set E of the projection model of the camera, with sufficient precision, and thus to determine theoretical objects associated with each real object point P_obj_r with sufficient precision. The parameters of the several input data sets E are determined from the knowledge of the output wavefront surface S_sor, and the relation S_sor = S_id + W.The acquired real object points P_obj_r can correspond to several real object points P_obj_r of a calibration target, and / or to real object points P_obj_r acquired for several predetermined positions of the calibration target.

[0186] [Fig. 10] illustrates an example of ray tracing from the propagation of a wavefront in the case of absence of aberrations, i.e. in the case where optical aberrations are neglected. In this case, the wavefront surface at the exit of the exit pupil 12 of the main optics 1, which corresponds to the ideal exit surface S_id indicated in dotted lines in [Fig. 10], is a perfect sphere.

[0187] For a perfect optic of focal length f presenting no distortion, the ideal surface S_id corresponds to a spherical wavefront at the output of the perfect optic converging at the distance f, and is written ■ . , r~-> 2 S_ia = \[f -x -y2

[0188] [Fig. 10] further illustrates an example of ray tracing from the propagation of a wavefront in the case of the presence of aberrations, i.e. in the case where the optical aberrations are not neglected. In this case, the wavefront surface at the exit of the exit pupil 12 of the main optics 1, i.e. the exit wavefront surface S_sor, is a surface of arbitrary shape, which is the sum of the ideal exit surface S_id of a perfect optic, and the aberration function W.

[0189] The aberration function W can be decomposed into a series of Seidel polynomials. Seidel theory formalizes the aberration function W as a series of polynomials. Alternatively, the aberration function W can be decomposed into a series of Zernike polynomials.

[0190] The projection parameters Par_Proj of each input data set € may include: - the radius of the spherical ideal wavefront surface S_id; - the coefficients up to the fifth degree of the series of polynomials of the function of aberration W; and / or; - a distance between the main optic 1 and the sensor 3 of the plenoptic camera.

[0191] Thus, the projection parameters Par_Proj of the projection model of the plenoptic camera are representative not only of the primary order aberrations of the main optics 1, which correspond to a development of the aberration function W up to order 3, but also of the higher order aberrations, which correspond to a development of the aberration function W beyond order 3, in particular up to order 5. Furthermore, the distance between the main optics 1 and the sensor 3 is likely to differ from the distance announced in the manufacturing data. Several sets of input data E having several different distances between the main optics 1 and the sensor 3 can thus be determined in order to calculate the calibration error Acal.

[0192] S6: determination of object light rays

[0193] Each object light ray 60 is determined in step S6, from each corresponding image light ray 50 and from a set of input data determined in step S0.

[0194] Thus, for each input data set E of the several input data sets E determined in step S0, a corresponding set of object light rays 60 is determined in step S6.

[0195] Indeed, the projection parameters Par_Proj make it possible to reconstruct, from the tracing of the image light rays 50, an output wavefront surface S_sor calculated by an algorithmic solution, and which models the higher-order aberrations of the main optics 1. The object light rays 60 can then be reconstructed from the output wavefront surface S_sor calculated for each calibration set E among the several calibration sets E determined in step S0.

[0196] In particular, the calibration method may further comprise a step of propagating the determined output wavefront surface S_sor to the main optics 1 of the plenoptic camera, in particular to the surface S_lens representing the main optics 1. The determined and propagated output wavefront surface S_sor corresponds to a geometric wavefront surface originating from the real object point P_obj_r at an output of the main optics 1 of the plenoptic camera, i.e. at the exit pupil 12 of the main optics 1 on the optical axis z. The output wavefront surface S_sor takes into account the primary and higher-order aberrations of the main optics 1.

[0197] The ideal wavefront surface at the input of the main optics 1 of the plenoptic camera may be substantially spherical and converge substantially towards the theoretical object point P_obj_th. The method may further comprise a step of propagating the ideal wavefront surface at the input of the main optics 1 from a surface of the main optics S_lens of the plenoptic camera to the calibration target.

[0198] Each object light ray 60 can be traced from the ideal wavefront surface at the input of the determined main optics 1, the object light ray 60 passing through a predetermined point of the ideal wavefront surface at the input of the main optics 1 and being of direction perpendicular to the gradient of the ideal wavefront surface at the input of the main optics 1 at this predetermined point. S7: Determination of the theoretical object point

[0199] The position of the theoretical object point P_obj_th on the calibration target is calculated from the tracing of the object light rays 60.

[0200] In theory, the real object point P_obj_r corresponds to a point object. However, in practice, the object light rays 60, obtained from the image of the real object point P_obj_r on the sensor 3, do not converge into a single point object, but into an area of ​​space around the object point to be reconstructed.

[0201] The theoretical object point P_obj_th associated with the real object point P_obj_r can be determined by triangulation of the object light rays 60 determined in step S6. In other words, the position of the theoretical object point P_obj_th corresponds to the position of a point which minimizes a sum of the distances between this point and the set of straight lines corresponding to the object light rays 60 determined in step S6. The triangulation can be carried out in the least squares sense.

[0202] Thus, the set of several theoretical object points P_obj_th associated with the several real object points P_obj_r can be reconstructed, then compared to the set of several real object points P_obj_r associated, by similarity between two 3D point clouds.

[0203] S8: calculation of the calibration error, and S9: determination of the calibration set

[0204] The position of each theoretical object point P_obj_th is compared to the position of each associated real object point P_obj_r, in order to calculate a calibration error Acal.

[0205] The calibration error Acal is associated with the input data set E determined in step S0, i.e. the projection parameters Par_Proj of the projection model from which the object light rays 60 are calculated. The theoretical object point P_obj_th can be estimated by triangulation from the object light rays 60.

[0206] The calibration error Acal can in particular take into account a difference between the positions of the theoretical object points P_obj_th and the positions of the associated real object points P_obj_r, the method comprising a step of comparison between the positions of the theoretical object points P_obj_th and the positions of the associated real object points P_obj__r. Thus, all of the positions of the theoretical object points P_obj_th, obtained using the projection parameters Par_proj, estimated by intersection of rays 60, is compared to the set of real object points P_obj_r of the calibration target to deduce the calibration error Acal.

[0207] Furthermore, the calibration error Acal can take into account a criterion of quality of intersection of the object light rays 50.

[0208] As illustrated by way of non-limiting example in [Fig.7], the calibration error Acal is calculated for each of the input data sets E defined in step S0, a calibration error Acal being associated with each input data set E.

[0209] The calibration set E_min corresponds to the input data set leading to the minimum calibration error Acal, i.e. the input data set optimizing the projection model of the plenoptic camera. For this calibration set E_min, the calibration error Acal is minimized, and the calibration accuracy is therefore optimized. The projection parameters Par_Proj of the calibration set E_min thus allow an accurate reconstruction of the real object point P_obj_r, which takes into account a primary aberration and a higher-order aberration of the main optics 1.

[0210] This calibration criterion by minimizing the calibration error Acal corresponds to the minimization of the average of the distance between the reconstruction of a theoretical object point P_obj_th from an image point P_im by the model of the plenoptic camera, and the real object point P_obj_r of the calibration target.

[0211] The calibration target has a predetermined position and a known real geometry defined by relative positions of several real object points P_obj_r with respect to each other. The positions of the real object points P_obj_r are expressed in a reference frame of the calibration target and the positions of the theoretical object points P_obj_th are expressed in a reference frame of the plenoptic camera.

[0212] For each set of input data E determined in step S0, step S8 of calculating the calibration error Acal may comprise the following sub-steps: S81: calculation of a first calibration error ôcall corresponding to a sum, carried out on the several real object points P_obj_r of the calibration target, of the distances between each real object point P_obj_r and each associated theoretical object point P_obj_th; S82: calculation of a second calibration error ôcal2 corresponding to a sum, carried out on the several theoretical object points P_obj_th, of a sum of the distances between the theoretical object point P_obj_th and each object light ray 60 corresponding to the theoretical object point P_obj_th; and S83: calculation of the calibration error Acal for the predetermined position of the calibration target, corresponding to a sum of the first calibration error ôcall and the second calibration error ôcal2.

[0213] The first calibration error ôcall corresponds to a registration error between the reference frame of the calibration target and the reference frame of the plenoptic camera. Indeed, minimizing the first calibration error ôcall amounts to determining a geometric transformation between the reference frame of the calibration target and the reference frame of the plenoptic camera, for example a rotation and / or a translation between the reference frame of the calibration target and the reference frame of the plenoptic camera, which minimizes the distances between the real object points P_obj_r and the theoretical object points P_obj_th. The first calibration error ôcall is furthermore representative of a distance between a theoretical geometry of the calibration target defined by relative positions of the determined theoretical object points P_obj_th and the real geometry of the calibration target.Thus, the first calibration error ôcall is representative of the geometric constraint of the calibration target, i.e. of the reconstitution by the camera model comprising the input data set E determined in step S0 of the geometry of the calibration target. The first calibration error ôcall corresponds to an error between a geometry of the known target and a geometry of the target estimated by intersection of the object light rays 60, i.e. by comparison of two point clouds. The relative distances between the theoretical object points P_obj_th must be substantially identical to the relative distances between the real object points P_obj_r.

[0214] For the calibration set E_min, the positions of the theoretical object points P_obj_th obtained by the projection model of the plenoptic camera are generally the closest to the positions of the real object points P_obj_r acquired by the plenoptic camera, and the theoretical geometry of the calibration target generally reconstructs with the best precision the real geometry of the calibration target. In other words, the point cloud corresponding to the set of real object points P_obj_r used for the calibration must correspond substantially to the point cloud corresponding to the set of theoretical object points P_obj_th determined.

[0215] The second calibration error ôcal2 corresponds to an error in the quality of intersection of the object light rays 60, calculated for all the object light rays 60. Indeed, the object light rays 60 are assumed to intersect perfectly at a single point. Thus, the sum of the distances between the intersections of the object light rays 60 corresponding to the theoretical object points P_obj_th and the calibration target constitute the second calibration error ôcal2. The second calibration error is independent of the real object points P_obj_r. Thus, the second calibration error ôcal2 can be expressed by the following equation: 102161

[0217] where:

[0218] M represents the number of theoretical object points P_obj_th,

[0219] N{ represents the number of light rays objects 60 associated with the ith object point theoretical P_obj_th, and

[0220] represents a distance between the theoretical object point P_obj_th and the jth object light ray 60 associated with the ith theoretical object point P_obj_th.

[0221] When the input data set E further comprises a projection parameter Par_Proj corresponding to a distance between the main optics 1 and the sensor 3, the calibration distance, corresponding to the distance of the calibration set E_min, corresponds substantially to a real distance between the main optics 1 and the sensor 3.

[0222] The calibration set E_min can correspond to the set of parameters minimizing the following expression:

[0223] _. . yW yM n 1 d \2where: E_min = mm Par), Pobj_r) N represents the number of predetermined positions of the calibration chart, M represents the number of real object points P_obj_r of the calibration chart, Li.j represents the number of optical elements 21 which observe the j-th real object point P_obj_r of the i-th predetermined position of the calibration chart, d() represents a distance, in this case calculated between the detected real object points P_obj__r and the theoretical object points P_obj_th determined by the model of the plenoptic camera, represents the k-th position of the j-th pixel of the i-th predetermined position of the calibration chart, Par represents a vector containing the set of projection parameters Par_Proj of the input data set E to be determined in the calibration process; for example, Par may contain the radius of the spherical ideal wavefront surface S_id and / or the coefficients up to the fifth degree of the series of polynomials of the aberration function W and / or a distance between the main optics 1 and the sensor 3, represents the rotation of the calibration target in its i-th predetermined position, represents the translation of the calibration target in its i-th predetermined position, and Xij represents the intersection of the object light rays 60 at the level of the calibration target. This criterion makes it possible to measure the quality of intersection of the rays for all the points of the calibration target at the different predetermined positions of the calibration target. Three-dimensional reconstruction process

[0224] A method for three-dimensional reconstruction of an object using a plenoptic camera, said object comprising a plurality of object points, comprises the following steps, as illustrated by way of non-limiting example in [Fig.9]: S100: calibrating the plenoptic camera by determining a calibration input data set E_min minimizing a calibration error Acal of the plenoptic camera using a calibration method according to the first aspect; for each object point of the object to be reconstructed three-dimensionally: S101: acquisition of the object point by the plenoptic camera; S102: determining an image position of the object point, said image position corresponding to a position detected on a sensor 3 of the plenoptic camera of the image of the object point through the plenoptic camera; S104: determination of a set of pixels of the sensor 3, each pixel of the set of pixels belonging to a macropixel 31 of the sensor 3 and being associated with a single optical element 21 of a matrix of optical elements 2 of the plenoptic camera, said single optical element 21 being assigned to a macropixel 31 of the sensor 3, each pixel of the set of pixels being further associated with the image position of the real object point; S105: determination of a set of image light rays 50, in which each image light ray 50 passes through a respective pixel of the set of pixels and through the single optical element 21 assigned to said respective pixel during step S104; S106: determination of a set of object light rays 60, in which each object light ray 60 is determined from an associated image light ray 50 determined in step S105 and the projection parameters Par_Proj of the calibration set E_min determined in step S100; and S107: three-dimensional reconstruction of the object point, said object point corresponding to a barycenter of the set of intersections of the object light rays 60 determined in step S106.

[0225] Step S106 may comprise a calculation of an output wavefront surface S_sor and / or an input wavefront surface S_en and where appropriate an output wavefront surface gradient S_sor and / or an input wavefront surface gradient S_en, from the projection model of the plenoptic camera with the calibration set E_min determined in step S100. Each object light ray 60 may correspond to the object of the associated image light ray 50 through the main optics 1 of the plenoptic camera.

[0226] The steps of the method are repeated for each object point of the object to be reconstructed three-dimensionally, in order to carry out the three-dimensional reconstruction of the entire object.

[0227] The ray leaving the pixel of the sensor 3 passes through the center of the optical element 21 associated with the pixel during step S32, the position of the center of the optical element 21 being determined, if necessary, in step S33.

[0228] The three-dimensional reconstruction can be carried out in step S107 by triangulation, i.e. by intersection of the rays. The intersection of these rays can be calculated in the least squares sense.

[0229] As with the calibration method, the propagation of the rays for the three-dimensional reconstruction is carried out starting from the pixels of the sensor 3 towards the imaged scene. The three-dimensional reconstruction using the method described above has advantages of reconstruction precision with a quality allowing it to be used for three-dimensional metrology, due to the precision obtained via the calibration method described above. Three-dimensional reconstruction device

[0230] A three-dimensional reconstruction device may comprise a plenoptic camera and a digital processing means 4.

[0231] The plenoptic camera comprises a main optic 1, a photosensitive sensor 3 and a matrix of optical elements 2 arranged between the main optic 1 and the sensor 3, the matrix of optical elements 2 comprising a plurality of optical elements 21 arranged on a matrix plane Pmat, the sensor 3 comprising at least one macropixel 31 comprising a plurality of pixels arranged on a sensor plane Pcapt. The plenoptic camera may comprise one or a combination of the elements described above.

[0232] The digital processing means 4 or digital processing calculator comprises a plenoptic camera model adapted to be calibrated by a plenoptic camera calibration method as described above. The digital processing means 4 is adapted to implement a method of three-dimensional reconstruction of an object as described above.

[0233] The digital processing means 4 is thus adapted to provide a three-dimensional reconstruction of an imaged object scene from the data, i.e. the pixel values, of an image or a series of images of the scene provided as output by the camera, as a function of the calibration set E_min Param of the plenoptic camera.

[0234] In other words, the digital processing means 4 is configured to determine, from a set of images provided by the photosensitive sensor 3, a distance between an object point and the plenoptic camera.

[0235] In particular, the digital processing means 4 is adapted to associate with each other the different pixels which correspond to the same object point.

[0236] Other embodiments may be envisaged and a person skilled in the art may easily modify the embodiments or examples set out above or consider others while remaining within the scope of the invention.

Claims

Claims

1. A method of calibrating a plenoptic camera comprising the following steps: SI: acquisition of a real object point (P_obj_r) of a calibration target by the plenoptic camera, a position of the real object point (P_obj_r) being known; S6: determination of a set of object light rays (60), in which each object light ray (60) is determined from an associated image light ray (50) and from a set of input data (E) comprising projection parameters (Par_Proj) of a projection model of the plenoptic camera; S7: determination of a theoretical object point (P_obj_th) associated with the real object point (P_obj_r), the theoretical object point (P_obj_th) corresponding to a barycenter of the set of intersections of the object light rays (60) with the calibration target; and S8: calculation of a calibration error (Acal) as a function of a position of the theoretical object point (P_obj_th) and the position of the real object point (P_obj_r), the method further comprising the following steps, carried out prior to step S6 of determining the set of object light rays (60): S5: determining a set of image light rays (50), in which each image light ray (50) passes through a respective pixel of a set of pixels belonging to a macropixel (31) of a photosensitive sensor (3) of the plenoptic camera and through a single optical element (21) of a matrix of optical elements (2) of the plenoptic camera, said single optical element (21) being assigned to said respective pixel; S0: determination of several sets of input data (E) from the determined image light rays (50), each set of input data (E) comprising projection parameters (Par_Proj) of a projection model of the plenoptic camera, in which the projection parameters (Par_Proj) are representative of a primary aberration and a higher order aberration of a main optic (1) of the plenoptic camera; and wherein the method further comprises the following step, performed subsequently to step S8 of calculating the calibration error (Acal): S9: determination of the calibration set (E_min) corresponding to the input data set (E) minimizing the calibration error (Acal), the calibration target having a predetermined position and a known real geometry defined by relative positions of several real object points (P_obj_r) with respect to each other, the positions of the real object points (P_obj_r) being expressed in a frame of reference of the calibration target and the positions of the theoretical object points (P_obj_th) being expressed in a frame of reference of the plenoptic camera, in which for each input data set (E) determined in step SO, step S8 of calculating the calibration error (Acal) comprises the following substeps: S81: calculation of a first calibration error (ôcall) corresponding to a sum, carried out on the several real object points (P_obj_r) of the calibration target,distances between each real object point (P_obj_r) and each associated theoretical object point (P_obj_th); S82: calculation of a second calibration error (ôcal2) corresponding to a sum, carried out on the several theoretical object points (P_obj_th), of a sum of the distances between the theoretical object point (P_obj_th) and each object light ray (60) corresponding to the theoretical object point (P_obj_th); and S83: calculation of the calibration error for the predetermined position of the calibration target, corresponding to a sum of the first calibration error (ôcall) and the second calibration error (ôcal2)..,

2. Method according to claim 1, further comprising the following steps, carried out prior to step S1 of acquiring the real object point (P_ob_r): S31: determining a relative position of a matrix plane (Pmat), on which the optical elements (21) of the matrix of optical elements (2) are arranged, relative to a sensor plane (Pcapt), on which the pixels of the sensor (3) are arranged; S32: assigning to a macropixel (31) of the sensor (3) a single optical element (21) of the matrix of optical elements (2), from the relative position of the matrix plane (Pmat) relative to the sensor plane (Pcapt); and S4: determining a set of pixels (3) of the sensor (3), each pixel of the set of pixels belonging to a macropixel (31) of the sensor (3) and being associated with the single optical element (21) assigned to said macropixel (31), each pixel of the set of pixels being further associated with the image position (P_im) of the real object point (P_obj_r).

3. A method of calibrating a plenoptic camera according to claim 1 or 2, comprising a step of moving the calibration target between several predetermined positions, the relative positions of the several predetermined positions of the calibration target with respect to each other being known, wherein some of the several object points (P_obj_r) are acquired in step S1 for several different predetermined positions of the calibration target.

4. A method for calibrating a plenoptic camera according to any one of the preceding claims, further comprising the following steps: S52: determining a sampling of a gradient of an output wavefront surface (S_sor) from the set of image light rays (50) determined in step S5; S53: determining several representative functions of the output wavefront surface (S_sor) from the gradient sampling determined in step S52, each of said several representative functions of the output wavefront surface (S_sor) corresponding to a sum: - of an ideal wavefront surface (S_id) at an output of a main optic (1) without optical aberration, the ideal wavefront surface (S_id) being a spherical surface having a spherical surface radius;and - an aberration function (W) adapted to take into account primary and higher order optical aberrations of the main optics (1), the aberration function (W) being adapted to be decomposed into a series of polynomials; step S53 corresponding to a determination of the coefficients associated with degrees of polynomial of the series of polynomials of the aberration function (W) up to a degree representative of a higher order aberration of the main optics (1), the projection parameters (Par_Proj) of each set of input data (E) comprising the coefficients determined in step S53.;

5. A method of calibrating a plenoptic camera according to claim 4, wherein the projection parameters (Par_Proj) of each input data set (E) comprise: - the radius of the spherical ideal wavefront surface (S_id);

6.

7. - the coefficients up to the fifth degree of the series of polynomials of the aberration function (W); and - a distance between the main optics (1) and the sensor (3) of the plenoptic camera. A method of calibrating a plenoptic camera according to claim 2 taken in combination with any one of claims 3 to 5, wherein the positions of the optical elements (21) on the matrix plane (Pmat) are known, the positions of the pixels of the sensor (3) on the sensor plane (Pcapt) are known, and wherein the relative position of the matrix plane (Pmat) with respect to the sensor plane (Pcapt) is determined in step S3 from the following steps: S310: acquisition by the plenoptic camera of a reference object; S320: detection on the sensor plane (Pcapt) of the image (5) of the reference object by the plenoptic camera; S330: determination of the positions of the optical elements (21) on the sensor plane (Pcapt), from the image (5) of the reference object detected in step S320; S340: determining a geometric transformation (Tmat / capt) between the matrix plane (Pmat) and the sensor plane (Pcapt), from the known positions of the optical elements (21) on the matrix plane (Pmat) and the positions of the optical elements (21) on the sensor plane (Pcapt) determined in step S330; and wherein the method may further comprise a step S41 of determining a position of each optical element (21) assigned to a respective pixel of the set determined in step S4 on the sensor plane (Pcapt), from the relative position of the matrix plane (Pmat) with respect to the sensor plane (Pcapt) determined in step S3, each image light ray (50) determined in step S5 passing through said pixel and said respective optical element (21). Method for three-dimensional reconstruction of an object using a plenoptic camera, said object comprising a plurality of object points, the method comprising the following steps: S100: calibrating the plenoptic camera by determining a set of calibration input data (E_min) minimizing a calibration error (Acal) of the plenoptic camera using a calibration method according to any one of the preceding claims; for each object point of the object to be reconstructed three-dimensionally:

8. S101: acquisition of the object point by the plenoptic camera; S102: determining an image position of the object point, said image position corresponding to a position detected on a sensor (3) of the plenoptic camera of the image of the object point through the plenoptic camera; S104: determining a set of pixels of the sensor (3), each pixel of the set of pixels belonging to a macropixel (31) of the sensor (3) and being associated with a single optical element (21) of a matrix of optical elements (2) of the plenoptic camera, said single optical element (21) being assigned to a macropixel (31) of the sensor (3), each pixel of the set of pixels being further associated with the image position of the object point; S105: determining a set of image light rays (50), in which each image light ray (50) passes through a respective pixel of the set of pixels and through the single optical element (21) assigned to said respective pixel during step S104; S106: determination of a set of object light rays (60), in which each object light ray (60) is determined from an associated image light ray (50) determined in step S105 and the projection parameters (Par_Proj) of the calibration set (E_min) determined in step S100; and S107: three-dimensional reconstruction of the object point, said object point corresponding to a barycenter of the set of intersections of the object light rays (60) determined in step S106. Three-dimensional reconstruction device comprising a plenoptic camera and a digital processing means (4), the plenoptic camera comprising a main optic (1), a photosensitive sensor (3) and a matrix of optical elements (2) arranged between the main optic (1) and the sensor (3), the matrix of optical elements (2) comprising a plurality of optical elements (21) arranged on a matrix plane (Pmat), the sensor (3) comprising at least one macropixel (31) comprising a plurality of pixels arranged on a sensor plane (Pcapt), the digital processing means comprising a plenoptic camera model adapted to be calibrated by a method for calibrating the plenoptic camera according to any one of claims 1 to 6, the digital processing means (4) being adapted to implement a method for three-dimensional reconstruction of an object according to claim 7. 41

Images