Method for simulating an ophthalmic lens on the eye of a subject viewing a virtual three-dimensional scene using a light field display

Abstract

A method for simulating an ophthalmic lens on the eye of a subject viewing a virtual scene using a light field display, the light field display comprising a light field window, the method comprising: - calculating by ray tracing virtual rays between points of the virtual scene and points of a pupil plane, the virtual rays passing through a virtual lens and being determined based on a model providing a deflection angle of the virtual rays through the virtual lens, the deflection angle depending on at least one angle of incidence and a position of the virtual rays on the virtual lens, repeating the calculating step by performing ray tracing on a plurality of virtual rays passing through the virtual lens and combining pairs of one point of the virtual scene and another point of the pupil plane; - determining a light field representing the modified virtual scene; and - generating the light field representing the modified virtual scene on the light field window towards the eye of the subject.

Description

[Technical field]

[0001] The present disclosure relates to a method for simulating ocular correction for the eyes of a subject viewing a virtual three-dimensional scene using a light field display.

[0002] More precisely, the present disclosure relates to a method for simulating a corrective eye lens on one or both eyes of a subject viewing a three-dimensional scene. [Background technology]

[0003] There are many documents that describe how to simulate corrective ocular lenses, such as simulators that use Virtual Reality (VR) or Augmented Reality (AR). In most cases, these solutions use renderings for each eye to reconstruct the third dimension thanks to convergence properties, thus decoupling accommodation and convergence.

[0004] In addition, different methods are known to simulate accommodation. For example, it is also known to use active optical elements in combination with virtual or augmented reality techniques to modulate the focusing plane. These methods allow to calculate two or three accommodation states to partially correct the refraction of the eye. However, these methods do not adequately render the multiple possible positions of the focal plane in a real three-dimensional environment, and therefore cannot accurately simulate synthetic type lenses, such as progressive addition lenses (PAL), which are referred to as progressive lenses below.

[0005] For example, the simulation of progressive addition lenses using virtual reality devices is often consistent in simulating distortion, but not in simulating lens power distribution. Specifically, methods for simulating power lens distribution often use two-dimensional multiple blurred images, which break the connection between accommodation and convergence. Therefore, these methods cannot obtain a simulation of what the retina would perceive with a real lens.

[0006] There is therefore a need for a method that allows for the simulation and evaluation of complex corrective ophthalmic lenses, and in particular a method that can provide proper binocular rendering with correct accommodation and convergence properties. Summary of the Invention [Means for solving the problem]

[0007] Accordingly, one object of the present disclosure is a method for simulating an ophthalmic lens on an eye of a subject viewing a virtual three-dimensional scene using a light field display, the light field display comprising a light field window, the method comprising: - receiving a set of input parameters including scene parameters of a virtual three-dimensional scene; - calculating by ray tracing virtual rays between points of the virtual 3D scene and points of the pupil plane, the virtual rays passing through a virtual lens and defined based on a model providing a deflection angle of the virtual rays through the virtual lens, the deflection angle depending on at least one angle of incidence and a position of the virtual rays on the virtual lens, repeating the calculating step by performing ray tracing on a plurality of virtual rays passing through the virtual lens and combining pairs of one point of the virtual 3D scene and another point of the pupil plane so as to scan the light field window in spatial and angular directions; - determining a light field representing the modified virtual three-dimensional scene based on the scene parameters and a deflection angle associated with each virtual ray of the plurality of virtual rays; - generating a light field representing the modified virtual three-dimensional scene on a light field window toward a pupil plane of the subject.

[0008] In the method according to the present disclosure, the properties of the light field display allow to define the positioning and direction of each light ray on its surface. These properties are used in the steps of the present disclosure to allow the subject to correctly view the virtual three-dimensional scene without wearing glasses. In particular, the present disclosure proposes to simulate the refractive optical correction of eye correction lenses using a light field display.

[0009] Thus, according to the method of the present disclosure, it is possible to calculate virtual rays between the light field display and the scene. These virtual rays are calculated over the entire surface of the light field display. By taking into account all the refractive optical effects of the eye correction lens, the method allows to simulate the proximity of objects through the virtual lens. Furthermore, by taking into account all the prismatic effects of the eye correction lens, the method allows to simulate the distortion effects when looking through the virtual lens. The simulation is therefore more accurate. Such a feature allows to maintain the association between convergence and accommodation, which can then be taken into account in the same step, which is not the case in the prior art. For example, it is possible to associate the proximity of objects in the virtual three-dimensional scene with each ray of the light field display, which is very useful in the simulation of synthetic lenses. Thus, the method according to the present disclosure allows to improve the simulation of eye correction lenses by providing a simulation closer to the real life experience for the user.

[0010] The light field window may correspond to a field of view of at least 40 degrees in the horizontal plane and 40 degrees in the vertical plane, and / or an angular resolution of less than 5 degrees.

[0011] In one embodiment, the light field window may extend spatially over a surface area of ​​at least 3 mm in the horizontal plane and 3 mm in the vertical plane, and / or may exhibit a spatial resolution of at least 0.5 dots / mm.

[0012] According to certain aspects, a method according to the present disclosure includes acquiring a three-dimensional image of a scene using a three-dimensional capture device to extract scene parameters, the scene parameters including geometric and / or photometric characteristics of the scene.

[0013] According to another particular aspect, determining the light field includes applying geometric and / or photometric properties of the scene.

[0014] Advantageously, the calculating and / or generating steps according to the method of the present disclosure include selecting a subset of the plurality of virtual rays passing through the pupil of the subject's eye.

[0015] Advantageously, the model is a simulation model determined on the basis of geometrical optics calculations and / or wave optics calculations and / or photometric optics calculations.

[0016] In an embodiment of the present disclosure, the model includes the shape of the lens and the positioning of the lens relative to the eye.

[0017] Advantageously, the model comprises a distortion model of the virtual lens, and / or a spherical power model of the virtual lens comprising at least spherical parameters, and / or an astigmatic power model of the virtual lens comprising at least cylindrical and / or axial parameters, or J0 and J45 parameters, and / or a filter lens.

[0018] According to certain aspects, the determining step of the method according to the present disclosure includes a step of representing the modified virtual three-dimensional scene further including applying a convolution or image transformation adapted to simulate blur, distortion, diffraction, chromatic aberration, scattering, or transmission attenuation.

[0019] According to certain advantageous embodiments, the method according to the present disclosure is adapted to simulate an other ophthalmic lens on the other eye of the subject, the method including calculating, determining, and generating steps to simulate an other ophthalmic lens for the other eye of the subject, and the subject parameters in the receiving step include other parameters of the pupil diameter of the other eye and / or the pupil position of the other eye relative to the light field display.

[0020] Advantageously, the virtual lens comprises a progressive or multifocal lens, a single vision lens, a spectacle lens, a contact lens or a filter.

[0021] In one embodiment, the virtual ray and the plurality of virtual rays in the calculating step are configured to propagate from a pupil plane to at least one object in the scene.

[0022] Alternatively, the virtual ray and the plurality of virtual rays in the calculating step are configured to propagate from at least one object in the scene to a pupil plane.

[0023] It is a further object of the present disclosure to provide a system comprising a computer and a light field display, the system being adapted and configured to perform the method according to the present disclosure.

[0024] Advantageously, a light field display according to the present disclosure exhibits spatial resolution, which is defined as the number of pixels per unit distance in one direction of alignment.

[0025] According to a particular embodiment, the light field window has a size that corresponds to the emission surface of the light field window.

[0026] Field of view per angular resolution means the angular step of emission. The angular resolution can be defined as the smallest angular step between two emission directions of a given (macro)pixel of the light field display.

[0027] The ratio of field of view to angular resolution is equal to the number of emission directions along which the field of view is defined (eg horizontal or vertical).

[0028] Preferably, the methods according to the present disclosure are configured to be computer-implemented.

[0029] As used herein, a computer may be a processor, a computing module, or a calculator or computing unit. The receiving, calculating, and determining steps may be implemented by a single computing module, or they may each be implemented by separate computing modules.

[0030] It is a further object of the present disclosure to provide a device configured to perform all the steps of the method according to the present disclosure.

[0031] It is a further object of the present disclosure to provide a computer program product comprising one or more stored sequences of instructions accessible to a processor and which, when executed by the processor, causes the processor to perform a method according to the present disclosure.

[0032] It is a further object of the present disclosure to provide a computer readable medium having one or more sequences of instructions for the computer program product of the present disclosure.

[0033] The following description, referring to the attached drawings, will make clear what constitutes the present disclosure and how it can be achieved. The present disclosure is not limited to the embodiments shown in the drawings. Therefore, when a feature mentioned in a claim is followed by a reference sign, it should be understood that such sign is included only for the purpose of enhancing the comprehension of the claim, and does not limit the scope of the claim.

[0034] Reference is now made to the following brief description taken in conjunction with the accompanying drawings and detailed description, where like reference numbers represent like parts. [Brief description of the drawings]

[0035] [Figure 1] 1 shows a side view of a first type of light field display device according to the prior art; [Diagram 2] 1 shows a side view of a second type of light field display device according to the prior art; [Diagram 3] 4 shows an example of the preliminary steps of the computation used to construct the distortion model. [Figure 4] 4 shows an example of the preliminary steps of the computation used to construct the distortion model. [Diagram 5] 4 shows an example of the preliminary steps of the computation used to construct the distortion model. [Figure 6] 13 shows an example of a preliminary step in the calculations used to apply spherical and cylindrical power models to a light field display. [Figure 7] 13 shows an example of a preliminary step in the calculations used to apply spherical and cylindrical power models to a light field display. [Figure 8] 13 shows an example of a preliminary step in the calculations used to apply spherical and cylindrical power models to a light field display. [Figure 9] 1 shows a first embodiment of the method. [Figure 10] 2 illustrates a second embodiment of the method. [Figure 11] 1 shows a schematic side view of a light field display for use in the method; [Figure 12] 2 illustrates a schematic representation of an example embodiment of the calculating and determining steps of the method; [Figure 13] 2 illustrates a schematic representation of an example embodiment of the calculating and determining steps of the method; [Figure 14]2 illustrates a schematic representation of an example embodiment of the calculating and determining steps of the method; [Figure 15] 2 illustrates a schematic representation of an exemplary embodiment of the generating step of the method; [Figure 16] 4 shows a schematic diagram of the selection steps applied in the calculating and determining steps of the method; [Figure 17] 2 shows diagrammatically a selection step applied in the generating step of the method; [Figure 18] 1 illustrates a system according to the present disclosure that allows a subject to view a scene. [Figure 19] 4 illustrates a schematic representation of another exemplary embodiment of the generating step of the method; [Figure 20] 4 illustrates a schematic representation of another exemplary embodiment of the generating step of the method; DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0036] In the following description, the drawings are not necessarily drawn to scale, and certain features may be shown in generalized or schematic form for clarity and conciseness or for informational purposes. In addition, while the making and using of various embodiments are discussed in detail below, it should be understood that as described herein, many inventive concepts are provided that may be embodied in a variety of contexts. The embodiments discussed herein are merely representative and do not limit the scope of the invention. It will also be apparent to one skilled in the art that all technical features defined in relation to a process may be replaced by a device, either individually or in combination, and conversely, all technical features defined in relation to a device may be replaced by a process, either individually or in combination.

[0037] definition As used herein, a Light Field (LF) is a vector function that describes the rays of light flowing in every direction through every point in space. This vector function can also embed information that represents the spectrum of light.

[0038] Conventional display devices generate 2D images, where the light characteristics of the image depend on the position (x,y) of each point in the image. In contrast, light field display devices (i.e., LFDs) are configured to generate light fields, allowing the amount of light and its color or spectral characteristics to be controlled not only as a function of position (x,y) but also as a function of at least one direction. Some light field display devices generate 3D light fields, where the amount of light and its color or spectral characteristics depend on the position (x,y) and one direction (u) in the image. Generating a 3D light field (x,y,u) is also referred to as 1D integral imaging.

[0039] Some light field displays generate 4D light fields, where the amount of light depends on the position (x,y) in the image and on two transverse directions, for example, denoted (u,v). In other words, (x,y) corresponds to some coordinates in one plane, (u,v) corresponds to some coordinates in another plane, and the direction of the light ray is defined by a set (x,y) and a set (u,v). In general, the direction of vector u is in a horizontal plane, and the direction of vector v is in a vertical plane. Vectors u and v can also be angular coordinates. Generating a 4D light field (x,y,u,v) is also called integral imaging. A light field display device can generally control the position and direction of the light ray for each color (for example, RGB, i.e., red, blue, green, in a three-color-based display device).

[0040] For example, the light field display device according to the present disclosure is similar to the light field display disclosed in WO 2021 / 122640 A. All the characteristics of the light field display disclosed in this document are used in the present disclosure.

[0041] Ideally, a light field display device would be like a window, allowing light rays to be transmitted in different directions from each point in the image.

[0042] Different types of light field display devices are available and can be used in the methods and systems according to the present disclosure.

[0043] For example, FIG. 1 shows a side view of a known light field display 60. In this case, the light field display includes a display panel 20 and a parallax barrier 30. In addition, the simulated display panel 20 includes individually addressable pixels 21, 22, etc. The display panel 20 may include a column of pixels, or more generally a 2D array of pixels. In the example shown in FIG. 1, the pixels are positioned in an orthogonal reference system (X,Y,Z) with a constant pitch, denoted p, along the axis X. The pixels are grouped according to macropixels 222. The macropixels 222 are formed by adjacent pixels that are configured to generate light rays at different angles from the same position of the light field display. The light rays of each macropixel 222 propagate according to multiple directions after intersecting at the same point 46 in the light field display 60. As shown in FIG. 1, the parallax barrier 30 includes a plate that is arranged parallel to the surface of the display panel and at a distance, denoted f, from the surface of the display panel 20. The plate contains a number of pinholes 31 or microholes arranged with a pitch denoted Δx along an axis X. In general, the pixel pitch p is much smaller than the pinhole array pitch Δx. In addition, the light field display 60 exhibits a pitch Δy along an axis Y (not shown in FIG. 1). The LFD 60 generates a 3D light field 69 everywhere in the half-space above the microholes 31. FIG. 1 shows an image plane 50 parallel to the light field display device 60, at a distance l from the parallax barrier 30. The image plane 50 is used for a 4D representation of the light field beams. Each light field beam can be represented by a 4-tuple of coordinates (x, y, u, v), where (x, y) represent the coordinates in the plane of the microholes and (u, v) represent the coordinates in the plane 50 at a distance l. The minimum angle Δθ between two successive light rays generated by adjacent pixels passing through the same pinhole gives the angular resolution of the macropixel 222. The angle θ at which the maximum aperture of the light field 69 depends on the pinhole pitch Δx and the distance fMAX has.

[0044] However, other types of parallax barriers 30 can be used instead of pinhole arrays and thus simulated according to the methods of the present disclosure. For example, the LF can be projected directly onto the pupil of the eye by creating virtual pinholes. Each virtual pinhole is associated with a specific light source that is turned on individually. For each virtual pinhole, all directions are driven by a spatial light modulator (SLM).

[0045] 2 shows a side view of another known light field display device 60 comprising a high resolution display panel 20 combined with a lenslet array 40. The lenslet array 40 comprises an array of microlenses 41, 42, 43, etc. arranged in close proximity to the display panel 20. For example, the lenslet array 40 is arranged above the display panel 20. The pitch between the microlenses is noted as Δx along the axis X. As a function of the addressed pixel 21, the light field passing through the microlenses 44 has a specific vector u. As a result, the LFD 60 generates a 3D light field in an image plane 50 parallel to the LFD at a distance l from the lenslet array 40. The generated light field depends on the position of the activated pixel.

[0046] Of course, the light field display devices described above can operate in a similar manner along direction Y, using a 2D array of pixels and a 2D pinhole array, or a 2D grid of reflective surfaces or a 2D lenslet array, respectively, to generate a 4D light field.

[0047] The light field display device 60 allows to control the direction of light rays coming from a point of the image. Thus, the light field display device 60 allows to control the position and direction of each light ray forming the light field without using additional variable focal length lenses. As a result, images can be virtually displayed at multiple distances using only the light field display device at a fixed distance from the subject's eyes. More precisely, the light field display device 60 allows to form images at controlled distances from the light field display device.

[0048] However, parallax barrier LFDs or lenslet array LFDs generally impose a trade-off between spatial and angular resolution.

[0049] The properties of the light field can be used to correct for refractive error in a subject's eye.

[0050] Consider a light field of parallel beams generated using any type of light field display device as described above. The accommodation of the subject's eye is in a relaxed state. A visual stimulus is displayed at infinity. If the subject's eye is emmetropic, the image (e.g., a point) formed by the emmetropic eye through the cornea, pupil, and lens is focused onto the subject's retina. Thus, the retinal image of the light source point is focused at a point on the retina and is perceived as sharp. In contrast, if the subject's eye is in a relaxed state but the eye is affected by myopia, the image (e.g., a point) formed by the myopic eye (or myopic eye) is focused at a point in front of the subject's retina. Thus, for a myopic eye receiving a light field of parallel beams, the retinal image of the point extends over an area and is perceived as blur. For example, a light field display device can be used to generate a light field of deflected light rays to compensate for the aberrations of the myopic eye. Then, if the myopic eye is in a relaxed state, the image (e.g., a point) formed by the myopic eye is focused onto the subject's retina 19. Therefore, the retinal image of a point is perceived sharply by a myopic subject.

[0051] Similarly, for a subject with a hyperopic eye in a relaxed state, the image of a point formed using a light field of a parallel beam is focused behind the subject's retina. By knowing the value of the hyperopia, it is possible to generate a light field of a convergent beam to compensate for the aberrations of the hyperopic eye. Then, when the hyperopic eye is in a relaxed state, the image (e.g., of a point) of the LF of the convergent beam formed by the hyperopic eye is focused on the subject's retina. Thus, the retinal image of the point is perceived sharply by the hyperopic subject.

[0052] More generally, by knowing the refractive error of the eye, it is also possible to generate a light field that the eye would see in focus. A light field display device can be used to generate light fields along different directions, for example along the X and Y directions, that correspond to different ocular aberrations. Thus, by knowing the direction of the eye's axis and the amount of astigmatic error (or cylinder), it is possible to compensate for astigmatism as well as spherical error.

[0053] This may also include higher order aberrations (HOA), so the light field produced is optimized to form a sharp image on the retina.

[0054] Thus, light field display properties allow for the positioning and direction of light rays on its surface, and these properties are used in the method according to the present disclosure.

[0055] In order to use a real light field display to correctly show the user the scene as seen through the corrective lens, we used the properties of the light field to simulate a corrective eye lens using a light field display.

[0056] process FIG. 9 illustrates a block diagram of a method 1000 for simulating an ophthalmic lens on a subject's eye 70 viewing a virtual three-dimensional scene 80 using a light field display 60, according to an example embodiment.

[0057] Virtual means simulated and / or calculated.

[0058] By virtual three-dimensional (3D) scene is meant a representation of at least one object or objects whose 3D coordinates and colors in space are known. For example, the virtual 3D scene 80 represents a single object 81 or multiple objects 81. Additionally, the virtual 3D scene may be an image containing at least one object or multiple objects. In one embodiment, the virtual 3D scene 80 is divided into multiple points, and each point of the scene 80 has a three-dimensional position in space, preferably relative to the eye 70 of the subject 71. In another embodiment, the virtual 3D scene 80 is a virtual point.

[0059] The light field display 60 according to the present disclosure comprises a light field window 65 .

[0060] The light field window 65 is an emission surface of the light field display 60. The light field window is similar to a standard window through which a 3D scene is viewed. In the following examples, the light field window 65 corresponds to an active area of ​​the light field display 60 used in the method according to the present disclosure. In general, the light field window 65 comprises an emission surface configured to generate a light field 69 towards the subject's eye 70. The light field display 60 comprises or is associated with a computation unit configured to perform the steps of the method 1000 according to the present disclosure.

[0061] By computing unit is meant a computer or processor configured to execute instructions for performing steps of the methods according to the present disclosure.

[0062] The method 1000 includes receiving 1002 a set of input parameters including scene parameters of the virtual three-dimensional scene 80. The scene parameters are therefore generally known. For example, the scene parameters include an image as input parameters, or a set of 3D coordinates of points representing one or more 3D objects.

[0063] Optionally, the set of input parameters includes other parameters, such as light field display parameters and object parameters. For example, the object parameters include at least a pupil diameter of the eye 70 and, preferably, a pupil position of the eye 70 relative to the light field display 60. The light field display parameters are typically determined as a function of technical characteristics of the light field display 60. For example, the light field display parameters include the dimensions of the light field display 60 and / or the dimensions of the light field window 65. In addition, the light field display parameters include the number of pixels 64 included in the light field display 60, the spatial density of the pixels, and / or the pixel size of these pixels.

[0064] In addition, the method 1000 comprises a step 1004 of calculating, by ray tracing, a virtual ray 100 between a point of the virtual three-dimensional scene 80 and a point of the pupil plane 72 of the eye 70, the virtual ray 100 passing through the virtual lens 90. According to the method 1000, the virtual ray 100 is determined based on a model providing a deflection angle of the virtual ray 100 passing through the virtual lens 90, the deflection angle depending on at least one incidence angle and the position of the virtual ray 100 on the virtual lens 90. The method 1000 is then configured to repeat the calculation step 1004 by performing ray tracing on a plurality of virtual rays 101 passing through the virtual lens 90, so as to scan the light field window 65 in spatial and angular directions, and by combining pairs of one point of the virtual three-dimensional scene 80 and another point of the pupil plane 72 of the eye 70.

[0065] Preferably, the calculating step 1004 is further based on a set of input parameters, such as light field display parameters, object parameters, and scene parameters.

[0066] The method 1000 then includes a step 1006 of determining a light field 69 representing the modified virtual three-dimensional scene 140 based on the scene parameters and a deflection angle associated with each virtual ray of the plurality of virtual rays. The determination made by step 1006 of the light field 69 representing the modified virtual three-dimensional scene 140, 83 is also based on the angular and spatial sampling of the light field display 60.

[0067] As explained in the definition section, light field 69 means the amount of light flowing in every direction through every point in space. According to the method 1000, the light field 69 determined in the determining step 1006 is preferably calculated to reach the subject's eye 70, specifically the pupil of the eye under consideration.

[0068] The method 1000 includes a step 1008 of generating a light field 60 representing the modified virtual three-dimensional scene 140, 83 on a light field window 65 aimed at the eye 70 of the subject 71. In the generating step 1008, the generated light field 69 is real, meaning that the generated light field is configured to really reach the eye 70 of the subject 71.

[0069] By modified virtual three-dimensional scene 140 , 83 is meant the image of the virtual three-dimensional scene 80 that is displayed to the subject 71 .

[0070] Although the method 1000 is disclosed to simulate an ophthalmic lens on the eye 70 of the subject 71, the method 1000 may be adapted to simulate the other ophthalmic lens 90 on the other eye 70 of the subject 71. Thus, the method 1000 in this embodiment includes a calculating step 1004, a determining step 1006, and a generating step 1008 for simulating the other ophthalmic lens 90 for the other eye 70 of the subject 71. In addition, the subject parameters in the receiving step 1002 include other parameters of the pupil diameter of the other eye 70, and / or the pupil position of the other eye 70 relative to the light field display 60 allowing, and / or the interpupillary distance between the two eyes 70 for generating the optimized light field 69.

[0071] Fig. 10 shows a block diagram of a method 2000 for simulating an ophthalmic lens on an eye 70 of a subject 71 viewing a virtual three-dimensional scene 80 using a light field display 60. The method 2000 disclosed in Fig. 10 includes all steps disclosed in the method 1000 shown in Fig. 9. Therefore, only the differences between Fig. 9 and Fig. 10 will be described.

[0072] If the scene parameters are not known, the method 2000 includes, prior to the receiving step 1002, a step 2002 of acquiring a three-dimensional image of the virtual three-dimensional scene using a three-dimensional capture device for extracting scene parameters, the scene parameters including geometric and / or photometric properties of the virtual three-dimensional scene 80. In addition, even if the scene parameters are known, the acquiring step 2002 may allow the scene parameters to be verified or perfected to improve the realism of the simulation of the method according to the present disclosure. In another embodiment, with respect to the method 1000, the scene may be artificial. By artificial, it is meant a scene extracted from an image. The image may be an acquired image, or a hand-drawn image such as a sketch, or an image obtained using software by artificial intelligence.

[0073] The method 2000 then optionally comprises a preliminary step 2012 configured to determine the distance between the light field display 60 and the eye 70 of the subject 71. For example, the preliminary step 2012 comprises acquiring an image of one or both eyes 70 of the subject 71 using an image capture device, the image capture device being placed on the light field display 60 or on the frame of the light field display 60. The preliminary step 2012 then comprises inferring from the acquired image the distance between one or both eyes 70 of the subject 71 and the light field display 60. This inference can be achieved, for example, by knowing the size of a particular feature of the image, for example the interpupillary distance, or by knowing the focus used to acquire the image. This distance is set as the distance measured between the light field display 60 and the eye 70 of the subject 71. This preliminary step 2012 optionally determines the interpupillary distance between the eyes 70 of the subject 71. In this manner, the method 2000 of the present disclosure obtains actual data for the subject 70, which may be used to validate or complete the subject parameters.

[0074] In addition, the determining step 1006 allows other characteristics of the virtual three-dimensional scene 80 to be taken into account, including step 2004 of applying geometric parameters of the virtual three-dimensional scene 80, in particular the photometric characteristics of the virtual three-dimensional scene 80, to improve the rendering of the virtual three-dimensional scene 80 relative to the object 71.

[0075] The geometric parameters include a 3D cartography of the virtual 3D scene 80 in order to obtain the actual 3D position of at least one object of the virtual 3D scene 80 or to obtain all the actual 3D positions of the objects 81 and elements contained in the virtual 3D scene 80. By position is meant the coordinates of at least one object or the coordinates of all the points of the object. The 3D position is preferably expressed relative to the eye 70 of the subject 71, making it possible to improve the robustness and accuracy of the steps of the method 2000 according to the present disclosure.

[0076] For example, the image capture device may comprise or be a plenoptic camera.

[0077] Photometric characteristics means radiant energy and / or light reflection model. Thus, photometric characteristics include the radiant energy of at least one object 81 of the virtual three-dimensional scene 80, preferably of all objects 81 and / or elements contained in the virtual three-dimensional scene 80. In addition, photometric characteristics include a model of the light reflection model of at least one object 81 of the virtual three-dimensional scene 80, preferably of all objects 81 or elements contained in the virtual three-dimensional scene 80. This radiant energy may also include a spectrum.

[0078] Optionally, the calculating step 1004 and / or the generating step 1008 of the method 2000 includes a step 2006 or 2008 of selecting a subset of a plurality of virtual or real rays passing through the pupil of the eye 70 of the subject 71 .

[0079] In this example, the calculating step 1004 includes a step 2006 of selecting a subset of the virtual rays passing through the pupil of the eye 70 of the object 71. The calculating step 1004 is therefore intended to be configured to calculate only the virtual rays that reach the eye 70 (i.e. the virtual eye used in the ray tracing of the calculating step 1004). The virtual rays calculated by ray tracing that do not reach the object's eye (virtual eye) are not calculated or taken into account. The calculating step 1004 is therefore optimized, since only the virtual rays of interest are calculated in the calculating step 1004. The calculating step 1004 requires less time, especially when scattering effects are taken into account. The method according to the present disclosure is thus optimized.

[0080] In this example, the generating step 1008 includes a step 2008 of selecting a subset of the plurality of light rays that pass through the pupil of the eye 70 of the subject 71. The generating step 1008 is therefore arranged and / or configured to only generate the light field 69 based on the subset of light rays that reach the subject's eye (i.e. the virtual eye) in the calculating step 1004.

[0081] The generating step 1008 is optimized because the generated light field 69 is based only on light rays that can reach the eye 70 (the real eye) of the subject 71. Thus, the generating step 1008 is faster, more robust and accurate. The method 2000 according to the present disclosure is thus optimized.

[0082] Optionally, the determining step 1006 includes a step 2010 of rendering a modified virtual three-dimensional scene 140, 83, further including applying a convolution or image transformation adapted to simulate blur, distortion, diffraction, or chromatic aberration. EP 2020905 B1 and PCT Publication WO 2009077617 A1 describe such image transformations or distortions.

[0083] In another embodiment, this representing step may be used to test the recognition of the subject 71 after performing the representing step 2010. Refractive parameters of the eye 70 of the subject 71 may be used to improve the refractive correction of the subject 71.

[0084] Optionally, the method 2000 includes a step 2014 of simulating a light field display 69 for the eye 70 of the subject 71 and for the virtual three-dimensional scene 80. For example, the simulating step 2014 corresponds to displaying a virtual light field display 60 between the virtual eye 70 of the subject 71 and the virtual three-dimensional scene 80. Thus, the calculating step 1004 and the determining step 1006 are performed in real time. Thus, the method 2000 according to the present disclosure is faster since only light rays entering the eye 70 are selected.

[0085] Figure 11 shows a light field display 60 placed in front of an eye 70 of a subject 71 to display a virtual three-dimensional scene 80. The light field display 60 depicted in Figure 11 is generic. Figures 1 and 2 show a specific implementation of the light field display 60, in which the virtual three-dimensional scene 80 is also referred to as the scene 80.

[0086] Typically, the light field display 60 has dimensions between 3 and 147 millimeters (mm). In this embodiment, the surface of the light field display 60 is configured to extend spatially over a surface area of ​​at least 3 mm in the horizontal plane and 3 mm in the vertical plane. In this embodiment, the light field display 60 is configured to extend along a first direction 61 and along a second direction 62. Optionally, the first direction 61 and the second direction 62 are vertical. The light field display 60 is, for example, at least 3 millimeters (mm) in the first direction 61 and at least 3 mm in the second direction 62. In another embodiment, the light field display 60 can be 7 mm x 7 mm or 8 mm x 8 mm. A light field display 60 of this size can be used for a head mounted display (wearable headset) that is more immersive than a standalone screen. In addition, the light field display 60 exhibits a spatial resolution of at least 0.5 dots / mm along the first direction 61 and / or the second direction 62. For example, for an 8mm x 8mm light field display 60, the spatial resolution is 1 dot / mm.

[0087] The light field display 60 shown in Fig. 11 is composed of a pixel array 63, which is configured to project and generate at least one light field to one eye 70 of a subject. In this example, the pixel array 63 is a matrix of pixels 64 (in Figs. 1 and 2 the pixels 64 are numbered 21, 22), which matrix of pixels 64 comprises a number of lines and columns. The matrix of pixels 64 extends spatially according to a first direction 61 and a second direction 62. The pixel array 63 defines an emission surface of the light field display 60.

[0088] The light field display 60 has a light field window 65. In general, the light field window 65 corresponds to the emission surface of the light field display 60, and therefore the light field window 65 and the light field display 60 have the same size.

[0089] The light field window 65 can be seen on a first side 66 and a second side 67. It should be understood that in accordance with the present disclosure, the light field window 65 is an active surface, which is activated and used in the steps of the methods 1000, 2000 in accordance with the present disclosure.

[0090] In this example, the light field 69 corresponds to a field of view of at least 40 degrees in the horizontal plane and 40 degrees in the vertical plane, and / or an angular resolution of less than 5 degrees, relative to the gaze axis of the eye 70, allowing fine details to be displayed with high resolution, e.g., the angular resolution is a few minutes of arc, e.g., 5 minutes of arc.

[0091] By angular resolution is meant the angular pitch of the emission that can be implemented by the light field display 60, which is indicated in Figures 1 and 2 by the angle Δθ.

[0092] The light field display 60 is positioned relative to the eye 70 of the subject 71. In particular, a reference coordinate system is positioned on the subject's eye 70. These features allow the steps of the methods 1000, 2000 to be implemented accurately yet easily.

[0093] A distance d separates the light field display 60 from the subject's eye 70. Preferably, the distance d is known. The distance d is typically part of the light field parameters for positioning the light field display 60 relative to the eye 70 of the subject 71.

[0094] The distance d typically has a value between 5 and 85 millimeters for virtual reality or augmented reality headset applications designed for short distances, or the distance d may have a value between 100 millimeters and 1000 millimeters for smartphone or tablet applications designed for between short and medium distances, or the distance d may have a value between 2 meters and 5 meters for applications such as TVs designed for long distances. In another example, the distance d is at most a few millimeters and as close to zero as possible. In fact, the emission surface in this case is a notebook bounded by a mechanical frame. In another embodiment, the distance d is obtained in a preliminary step 2012, for example with an image capture device arranged on the light field display 60. The distance d corresponds to a distance measured between the light field display 60 and the eyes 70 of the subject 71, making it possible to provide the subject 71 with the actual conditions of the field of view. This preliminary step 2012 may optionally determine the interpupillary distance between the eyes 70 of the subject 71 and the pupil diameter of each eye 70 of the subject 71. The image capture device may be a camera or a time-of-flight sensor.

[0095] In this example, the horizontal plane of the light field display 60 (including the second direction 62) also includes the axis z of a reference coordinate system located relative to the eye 70 of the subject 71 and centered on the pupil center of the eye 70, marked 0, or the center of rotation, marked CRO. The center of rotation (CRO) is preferred because it does not rely on the convergence of the eye 70 when viewing close objects. The first direction 61 and the second direction 62 are then perpendicular to the axis z.

[0096] The scene 80 shown in Fig. 11 is virtual. In this example, the scene 80 includes one object 81 (i.e., a virtual object) that has a three-dimensional position represented relative to the eye 70 of the subject 71. For example, the object 81 has three-dimensional coordinates noted as (x, y, z). This means that every point of the object has known coordinates.

[0097] As shown in Fig. 11, a (0,x,y,z) reference coordinate system is located on the eye 70 of a subject 71. The reference coordinate system includes three axes, marked x, y, and z, respectively. The z-axis passes through the pupil center of the eye 70. In Fig. 11, the center of rotation of the eye is indicated by the origin O of the reference coordinate system.

[0098] Although only one eye 70 is shown in FIG. 11, the light field display 60, preferably the pixel array 63 of the light field display 60, is configured to project or generate another light field 69 to the other eye 70 of the subject 71 (not shown in FIG. 11). The light field display 60 is thus configured to generate a first light field towards the first eye 70 of the subject 71 and a second light field 69 towards the second eye 70 of the subject 71. The first light field 69 and the second light field 69 can be generated simultaneously or alternately. In this way, the method 1000 or 2000 is configured to simulate the other light field 69 in front of the other eye 70 of the subject 71. This method allows the subject 71's vision to converge. This convergence is automatically applied by ray tracing when the position of each eye 70 relative to the object 71 is taken into account. Thus, the pupil distance and the difference in viewpoint between the two eyes are taken into account. The subject 71 will then converge and combine the two images that the subject has created on the real object. For demonstration and evaluation of eye correction in far vision, regardless of the physical position of the light field display 60, the scene 80 or at least one object 81 in the scene 80 is preferably placed near infinity to reduce accommodation. In another embodiment, FIG. 11 represents a simulating step 2014 (optional step) used in the calculating step 1004, where the light field display 60, the eye 70, and the scene 80 are all virtual. These parameters also include the angular resolution Δθ and the maximum angle θ of emission of light rays passing through the same point 46 of the light field window 65. max The maximum angle θ max Includes.

[0099] For example, the light field display parameters include at least the number of pixels, the size of the pixels, the spatial density of the pixels, and / or the dimensions of the light field display 60 and / or the light field window 65, as well as the distance d separating the light field display 60 from the subject's eye 70.

[0100] The light field viewing parameters are used as input for at least the calculating step 1004 and the determining step 1006. For example, the light field window 65 is a function of the light field viewing parameters. In addition, the scene parameters include the three-dimensional position of the object 81 in the scene 80, e.g., the coordinates (x, y, z) of the object 81.

[0101] 12, 13 and 14 diagrammatically represent example embodiments of the calculating 1004 and determining 1006 steps.

[0102] For clarity, Figures 12, 13 and 14 show virtual rays for one field of view.

[0103] 12 illustrates a scene 80, a light field window 65 positioned in front of an eye 70, and a virtual lens 90 positioned between the scene 80 and the light field window 65. In this example, the scene 80, the light field window 65, the eye 70, and the virtual lens 90 are simulated, for example, by lens design software, and therefore are not real.

[0104] It should be noted that the light field window 65 shown in Figures 11-14 is a schematic diagram of a side view of a light field window 65 belonging to the light field display 60 shown in, for example, Figures 1 and 11. However, other types of light field displays 60 can be used to appreciate different embodiments.

[0105] The virtual lens 90 is represented by a model, which is a model of a real eye corrective lens typically worn by a subject. The virtual lens 90 comprises a first diopter 91 directed towards the eye 70 and a second diopter 92 directed towards the scene 80. Each diopter 91, 92 allows to deflect a light ray, i.e. a virtual ray 100, 101, according to a deflection angle. Thus, each virtual ray 100, 101 passing through the virtual lens 90 is deflected by the first and second diopters 91, 92.

[0106] A model of a virtual lens 90 refers to the lens design and optical properties of the virtual lens 90. Thus, it should be understood that while the virtual lens 90 refers to a simulated physical implementation, the model relates to the optical properties of the associated virtual lens.

[0107] For example, the model of the virtual lens 90 shown in Fig. 12 includes the shape of the ophthalmic corrective lens (i.e., real lens) and the positioning of the ophthalmic corrective lens relative to the eye 70 of the subject 71, as well as the optical properties of the material (e.g., refractive index and chromatic dispersion properties). Thus, the positioning of the virtual lens 90 is relative to the eye 70. Preferably, the shape of the ophthalmic corrective lens includes the shapes of the first and second diopters 91, 92. For example, the shapes of the diopters 91, 92 may be convex, concave, flat, spherical, aspheric, cylindrical, toric, progressive, etc.

[0108] A model of the virtual lens 90 is determined before the calculating step 1004. This model is known and corresponds, for example, to a lens design of an ophthalmic corrective lens for the eye 70 of the subject 71. Typically, this lens design model includes at least the shape of the front and back surfaces of the lens, the refractive index and the position of this lens in space. Alternatively, the model 90 is determined computationally. In that case, the model is determined on the basis of geometrical optics calculations and / or wave optics calculations.

[0109] The model of the virtual lens 90 illustrated in Figures 12 to 14 includes a distortion model of the virtual lens 90. Typically, this distortion model includes at least the spherical parameters of the virtual lens, including at least the cylinder and axis parameters or the J0 and J45 parameters, and an astigmatism power model. This model makes it possible to take into account all the refractive parameters and refractive optical effects of the real lens (i.e. the ophthalmic corrective lens) that will be worn by the subject 71.

[0110] 3 and 4 show calculations for determining a distortion model, a spherical power model, and an astigmatism power model, which are calculated in an optional calculating step 2016 performed before the calculating step 1004.

[0111] 3, 4 show an exemplary distortion model constructed using a simulation method. In this example, a virtual lens 90 is illustrated for the eye 70 of the subject 71. To obtain the distortion model of the virtual lens 90, a computer generates a set of rays (virtual rays) emerging from the pupil of the eye 70 and calculates the deflection angle at the exit of the virtual lens 90. In this calculation, each ray propagates along a different direction and each ray is deflected by a first diopter 91 and a second diopter 92 of the virtual lens 90. The virtual lens 90 presents an optical axis, marked P, which is collinear with the z-axis of the reference coordinate system. In this example, only one ray is shown. An initial ray 93 starts from the eye 70 of the subject 71 at a first end and reaches the first diopter 91 at a second end. The initial ray contacts the first diopter 91 at a point of the virtual lens 90, marked 95. An initial ray 93 comes from the center of the eye 70 and, passing through the virtual lens 90, is deflected by refraction to form a deflected ray 94 at the exit of the virtual lens 90 on the second diopter 92. The deflected ray 94 intersects the second diopter 92 at a point marked 96. The initial ray 93 coming from the eye 70 assumes, at a position 95 of the first diopter 91, an angular position defined by initial angles marked alpha1 and beta1. The initial angles are expressed relative to the reference coordinate system (0,x,y,z) of the eye 70. The angle alpha is defined relative to the axis z in the plane (0,x,z). The angle beta is defined relative to the axis z in the plane (0,y,z). At the point 96 of the second diopter 92, the deflected ray 94 is in an angular position expressed depending on the deflection angles, called alpha2, beta2. The deflection between the initial ray 93 and the deflected ray 94 is calculated using a typical law of refraction, such as the Snell-Cartesian law. Thus, the deflection angle is obtained using the lens geometry and ray tracing methods. For example, the distortion model of the virtual lens 90 for an incident ray i can be expressed by the following equation: P alpha (alpha 1,i ,beta 1,i )=alpha 2,i’ -alpha 1,i and P beta (alpha 1,i,beta 1,i )=beta 2,i’ -beta 1,i where i corresponds to the index of the initial ray 93, i' corresponds to the index of the deflected ray 94, and P alpha and P beta represents the angular deviation along the gaze direction (α, β) according to alpha and beta introduced by the virtual lens 90, and i represents the index of the initial ray.

[0112] In this example, the virtual lens 90 has a thickness that is not constant but varies depending on the position of the virtual lens 90. The virtual lens 90 shown therefore presents an optical power that varies across the field of view of the virtual lens 90. To obtain a deflection model of the virtual lens 90, the virtual lens 90 is scanned spatially and angularly across its surface area in order to determine the deflection angle for each point of the virtual lens 90, and thus for each point 95 of the first diopter 91 and for each point of the second diopter 92. A scan of the deflection angles expressed according to alpha2, beta2 is thus performed.

[0113] If the calculating step 1004 is performed without any virtual lens 90, the light field display 60 provides a modified virtual three-dimensional scene 80 with the object 81 in three-dimensional space without any deformation. Therefore, to take into account the deflection model, the position of the object 81 in the scene 80 (virtual scene) is calculated by the distance between the eye 70 and the object 81, i.e., the alpha 2,object , beta 2,object Move your view according to the viewing angle marked: alpha 2,object =alpha 1,object -P alpha (alpha 1,object ,beta 1,object ) and beta 2,object =beta 1,object -P beta (alpha 1,object ,beta 1,object ).

[0114] 5 shows an example where an initial object point M1 is at a distorted position M2 calculated by ray tracing through the pupil of the eye 70 of the subject 71 when viewed through a virtual lens 90. The light field calculated for point M1 is then applied to point M2 to simulate the distortion model of the virtual lens 90. The same process is applied for different colors to simulate chromatic aberration effects. The distortion calculation may be realized in the same way as the distortion calculation provided in the disclosure of EP2020905.

[0115] Additionally, in one embodiment, the virtual lens 90 exhibits an attenuation factor. This means that the intensity or radiant energy of a virtual ray passing through the virtual lens 90 is attenuated by the virtual lens 90. The attenuation depends on the attenuation factor of the virtual lens 90. Thus, the luminosity of the object M1 in the virtual scene 80 is attenuated by the attenuation factor of the virtual lens 90. The attenuation depends on the transmission of the virtual lens 90 at the location where the virtual ray emitted from the object M1 passes through the virtual lens 90, for example at the le level of the point numbered 96 on the virtual lens 90.

[0116] In another embodiment, the virtual lens 90 is configured such that the virtual ray passes through the virtual lens at a position alpha 1,i , beta 1,i , and is further characterized by a transmittance and scattering function that depends on the wavelength (or its spectral characteristics) of the virtual ray.

[0117] The scattering function of the virtual lens 90 is the angle alpha 1,i , beta 1,i , the transmission of the virtual lens 90 is uniform. Transmission can be due to absorption or reflection, with wavelength dependence.

[0118] The scattering function in this disclosure can be modeled by a bidirectional transmittance distribution function, denoted BTDF, also known as the bidirectional scattering function and numbered 150. The scattering function can be implemented in the calculating step 1004. The BTDF to be simulated can include: - A scattering model known as the geometric scattering model, which can be used in the case of rough surfaces with defects larger than the wavelength; - Mie scattering due to particles of approximately the same size as the wavelength; Rayleigh scattering due to particles smaller than the wavelength. The scattering functions of the present disclosure are also taken into account in the generating step 1008 (see FIG. 19).

[0119] The simulation of the BTDF in the calculating step 1004 can take a lot of time. To limit the calculation time, the calculating method 1004 can be more rapid if the simple object 81 from the virtual scene 80 is close to a flat image. The simulation of the scattering can be obtained, for example, by angular convolution. It is efficient to consider only the virtual rays passing through the pupil of the eye and not the iris, sclera, or other non-transparent media that cross in front of the retina.

[0120] To improve the computation time of the computing step 1004, the method of the present disclosure may implement a BTDF such as the method disclosed in the publication “Image-based Rendering of Spatially-Varying BSDF”, 2010, by Hongsong Li and Fengxia Li.

[0121] One advantage of the light field display 60 is that it is possible to simulate the change in refractive power of the lens as perceived by the user without using a blur function depending on the position defined between the virtual object and the subject's eye. This position is represented by z iThe object 81 is represented by an angular position given by the deflection angles alpha2, beta2. For example, the coordinates of the object, after distortion correction, can be written as object (alpha2, beta2, z1). The distance z1 between the object and the eye then depends on the refractive parameters of the eye 70 of the subject 71.

[0122] For example, if only the spherical power model, denoted Sphere, of the virtual lens 90 is considered, then the distance z1 between the object and the subject's eye may vary as a function of the distance, denoted z2, according to the following formula:

[0123]

number

[0124] Then, when the spherical and astigmatic power models of the virtual lens 90 are taken into account, the distance between the object of the scene 80 and the eye 70 of the subject 71 depends on the astigmatism parameters axis and cylinder, which are denoted as Axe(alpha2, beta2 and Asti(alpha2, beta2). Then, the distance z1 between the object of the scene 81 and the eye 70 of the subject 71 varies depending on the distance z2 according to the following formula:

[0125]

number

[0126] (A is the direction of the meridian).

[0127] 6, 7 and 8 show an example of an embodiment for determining the distance between the virtual object 81 and the eye 70 of the subject 71, taking into account the astigmatic power model of the virtual lens 90. The characteristics of the light field display are used. In this example, the light field generated towards the eye 70 of the subject takes into account the spherical and astigmatic power model of the virtual lens 90, using the lenslets of the light field display 60 shown in FIG. 2. In FIGS. 6, 7 and 8, an astigmatic axis A of 90 degrees is considered (see FIG. 6). The ray 107 of the light field 69, reaching the eye 70 of the subject 71, has an angular position expressed according to the deflection angles alpha2, beta2. The distance between the virtual object 81 and the eye 70 of the subject 71 depends on the respective focal plane. For example, according to the focal plane z, x, the distance between the virtual object 81 and the eye 70 of the subject 71 is indicated as z1 (FIG. 7), while according to the field of view plane z, y, the distance between the virtual object 81 and the eye 70 of the subject 71 is noted as z2 (see FIG. 8). This means that the lens of the light field display 60 simulates a point at distance z1 (i.e., maximum distance) according to the plane z, x, and the lens of the light field display 60 simulates a point at distance z2 (i.e., maximum distance) according to the plane z, y. Thus, when the axis A of 45 degrees is considered, the distance between the virtual object 81 and the light field display 60 is expressed as z3=(z1+z2) / 2, which varies according to the cos(2A) law.

[0128] For each object 81 or point of an object in the scene 80, this point has a proximity P obj Having:

[0129]

number

[0130] P obj is independent of the distance d between the eye 70 and the light field display 60, 1 / z is the inverse of the distance from the eye to the object, and P corris the resulting eye correction add power at the point where the object is seen, P eye is the corrective power required for eye 70 to see a perfectly sharp image at infinity or any desired distance.

[0131] For example, the light field generated for object 81 is P obj (P obj =1 / z), the eye 70 adjusts this distance, and z is independent of d, even if the distance d is changing. Thus, the distance z of the displayed virtual object 81 is independent of the position (i.e., d) of the light field display 60. If the distance d is changing, for example when the light field display 60 is used on a smartphone, the characteristics of the light field display 60 may be recalculated in real time to keep the distance z between the modified virtual object 81 and the subject's eye 70 constant, independent of the distance d.

[0132] The model may be modified based on the parameters of the real lens worn by the subject. Thus, the light field may be generated for a specific correction, for example, for the spherical parameter (Sph) and the astigmatism parameters including cylinder and axis (Cyl and Axis) corresponding to the prescription part of the eye. In addition, the light field may be generated for other features such as a specific pupil diameter, a specific distance between the light field display 60 and the subject's eye 70, and the characteristics of the screen and the three-dimensional scene 80, including the characteristics of the eye correction lens that exhibits specific optical properties and final high-order aberrations, and the distance between the eye 70 and the object 81.

[0133] In the example of FIG. 12 (and FIGS. 11-15), the simulated virtual lens 90 comprises a progressive power lens. Thus, the properties of a synthetic lens can be simulated and reproduced. In another embodiment, the virtual lens comprises a multifocal lens, a single vision lens, a spectacle lens, or a contact lens. Contact lenses as well as Intra Ocular Lenses (IOLs) can be represented by a power profile model in the pupil plane.

[0134] Additionally, the scene 80 shown in FIG. 12 includes three objects, referred to as first, second and third objects 81a, 81b, 81c, positioned at different positions in the scene 80 with different proximity.

[0135] In this example, a virtual ray 100 associated with a real ray 200 is shown in Fig. 12. The virtual ray 100 propagates from the virtual scene 80 to the light field window 65, and the real ray propagates from the light field window 65 to the eye 70 of the object 71. To determine a value, for example a gray value or a luminance level, of the real ray 200 to be generated in the generating step 1008, the calculating step 1004 is configured to calculate a virtual ray 100 between the pupil plane numbered 72 and the virtual scene 80. The virtual ray reaching the scene 80 is associated with a color and / or a luminance intensity. The value of the real ray generated by the light field display 60 in the generating step 1008 is due to the color and / or luminance values ​​of the points of the scene 80.

[0136] In this example, a virtual ray numbered 100 is calculated. The virtual ray 100 reaches a scene point numbered 82. The scene point has (x,y,z) coordinates and is associated with a color and / or luminosity. The virtual ray 100 is deflected by a virtual lens 90. The real ray 200 generated in the generating step will then have the color and / or luminosity of the point 82.

[0137] After the calculation of the virtual ray 100 shown in FIG. 12, the calculating step 1004 is configured to repeat the calculation by calculating a number of virtual rays 101 .

[0138] In the example of FIG. 13, different virtual rays 100, 101a, 101b, 101c, 101d are illustrated, forming a plurality of virtual rays 101. In this example, the plurality of virtual rays 101 are configured to propagate from the pupil plane 72 to all objects 81 of the scene 80. The plurality of rays 101 pass through the same point 46 of the light field window 65, but have different emission directions (see FIG. 1 and FIG. 2 for the location of the point 46). The different emission directions are associated with points of the light field window 65, which correspond to pixels 64, respectively. The different emission directions passing through the points 46 of the light field window 65 are associated with the number of neighboring pixels 64 belonging to the same line and / or the same column of the matrix of pixels 63. In this example, the number of neighboring pixels is given by a 2D pitch, which in this example is a pitch ΔyΔx, and the light field window 65 is a two-dimensional light field window 65. The number of neighboring pixels given by the 2D pitch is associated with a macro-pixel 222.

[0139] In this example, the plurality of virtual rays 101 includes virtual rays labeled 101a, 101b, 101c, and 101d, respectively. The plurality of virtual rays 101 reach the scene 80 at a point on the scene 80 that is different from the point on the scene 80 reached by virtual ray 100. For example, virtual ray 101a of the plurality of virtual rays 101 reaches a point on the second object 81b, virtual ray 101b of the plurality of virtual rays 101 reaches a point on the third object 81c, virtual ray 101c of the plurality of virtual rays 101 reaches another point on the third object 81c, and virtual ray 101d of the plurality of virtual rays 101 reaches another point on the second third object 81c.

[0140] Preferably, the number of virtual rays 101 starting from the same point 46 is calibrated in the light field display parameters. The value of the number of virtual rays 100, 101 depends on the spatial and angular pitch and on the angle θ maxdefines the maximum aperture of the Light Field 69.

[0141] For the sake of clarity in FIGS. 12-15, the virtual ray 100 is included in a number of rays marked 101.

[0142] In the example shown in Figure 14, multiple other virtual rays are represented and labeled 102, 103, and 104, respectively. These multiple other virtual rays 102, 103, and 104 originate from different points in pupil plane 72 and reach points in scene 80. These multiple other virtual rays 102, 103, and 104 reach points in scene 80 that are different from the points in scene 80 reached by the multiple virtual rays 101 shown in Figures 12 and 11.

[0143] The examples represented in figures 12 to 14 allow scanning a model of the virtual lens 90 spatially according to the first and second directions 61, 62 over the entire surface of the light field window 65 and angularly over the aperture angle of the light field window 65. In the model, each point and each angle of the light field display 60 is associated with one point of the scene 80 by a virtual ray. The corresponding point is calculated by taking into account the deflection angle of the virtual ray as it passes through the virtual lens 90 at the interface with each diopter 91, 92. It should be noted that the examples are divided for clarity in order to illustrate the spatial and angular scanning of the model of the virtual lens 90 performed in the calculating step 1004.

[0144] Thus, to reconstruct a virtual object 81 in a scene 80, the virtual object 81 is divided into a number of virtual object points, and each point in the scene 80 is optionally associated with a color and / or a luminosity.

[0145] In another embodiment, the plurality of virtual rays 101 in the calculating step 1004 are configured to propagate from an object 81 in the scene 80 to the pupil plane 72. This is particularly useful when the propagation of the virtual rays is calculated based on wave optics.

[0146] FIG. 15 illustrates a schematic example of an embodiment of the generating step 1008 .

[0147] After calculating the virtual ray 100 and all the virtual rays 101, 102, 103, 104 and determining the real rays associated with these virtual rays in the determining step 1006, the method 1000 or 2000 performs the generating step 1008. The generating step 1008 is based on a typical 3D reconstruction method. In this example, a number of light fields 69 are generated toward the eye 70 of the object 71. The generated light fields 69 are associated with all the virtual rays, i.e. the rays of the virtual rays 101, 102, 103, 104, which were calculated in the calculating step 1004 and the determining step 1006. In this example, each light field 69 includes a number of rays (real rays) with different propagation directions. The virtual rays on the back of the light field 69 are extended to become real rays and emerge from the front of the light field window 65.

[0148] The real rays of the light field 69 are in the longitudinal extension of the virtual rays 101, 102, 103, 104 relative to the light field window 65. The real rays allow the user to view the modified virtual three-dimensional scene 140, 83. The rays of the light field 69 include color and / or luminosity corresponding to values ​​determined in the calculating 1004 and generating 1006 steps.

[0149] Fig. 19 shows another example of the generating step 1008. Reference number 81a in Fig. 19 is associated with a virtual object 81 of a virtual scene 80, while reference number 83 corresponds to an image (i.e., a displayed image) of the virtual object 81a through the light field display 60. Although a virtual lens 90 is shown for clarity, in reality, the optical effect of the virtual lens 90 is simulated by the light field display 60 (i.e., the use of the light field display 60 according to the present disclosure does not require the wearer to wear his / her own glasses). In this example, light rays starting from a point of the virtual lens 90 are scattered towards the subject's eye 70 according to the BTDF 150.

[0150] In a generating step 1008, the light field display 60 generates, for each virtual ray coming from the virtual object 81a, a scattered beam according to the BTDF 150. Thus, each real ray of the light field 69 propagates towards the eye 70 according to a scattering lobe corresponding to the BTDF 150. For example, a virtual ray 190a coming from a virtual object 81a in the virtual scene 80 is refracted according to the virtual lens 90 and scattered according to the BTDF 150a of the virtual lens 90 at a point numbered 193a, where the ray emitted from the object 81a passes through the virtual lens 90 (where the light is scattered). Another virtual ray 190b coming from the virtual object 81a is refracted according to the virtual lens 90 and scattered according to the BTDF 150b at a point 193b located on the virtual lens 90.

[0151] In the example shown above, each BTDF 150 is represented as a circle and includes a main light source 692, which is numbered 692a for the main light ray 692 associated with the virtual ray 190a of the virtual object 81a, and numbered 692b for the main light ray 692 associated with the virtual ray 190b of the virtual object 81a. The main light ray 692 has the maximum probability. By main light ray 692, we mean a real light ray 692 that is not scattered, but only refracted. The BTDF 150 also includes a second light ray 693, which is numbered 693a for the second light ray associated with the virtual ray 190a, and numbered 693b for the virtual ray 190b. The second light ray 693 has a reduced intensity or a reduced probability compared to the main light ray 692. Each ray of the BTDF 150 reaches the pupil plane 72. The BTDF 150 varies in spatial and angular directions along the surface of the virtual lens 90. This therefore means that the secondary light ray 693b associated with the virtual light ray 190b does not necessarily have the same probability or intensity as the secondary light ray 693a associated with the virtual light ray 190a. For example, at the point 193b of the first diopter 91, several emerging secondary light rays 693b are generated by the light field display 60 in different directions with a probability that depends on the BTDF 150b. The light rays 194 linked to the image 83 are a schematic representation of the light rays emitted by the light field display 60 to simulate the effect of a corrective lens on the object 71.

[0152] According to the method of the present disclosure, the BTDF is preferably taken into account in the calculating step 1004 and the determining step 1006 in order to be generated by the light field display 60 in accordance with the BTDF 150 in the generating step 1008 .

[0153] 20 shows another example of the generating step 1008. In this example, a virtual lens 90 is placed between the light field window 65 and the eye 70 of a subject 71.

[0154] To determine the value of the real light ray 700, generated from a point of the light field window 65 with a given angle between the real light ray 700 and the plane of the light field display 60, and reaching a point of the pupil plane 72, the calculating step 1004 is configured to determine a virtual light ray 701 associated with the real light ray 700, starting from a point of the pupil plane 72. The virtual light ray 701 is then extended towards the virtual scene 80. To take into account the model of the virtual lens 90, the calculating step 1004 calculates the virtual light ray 701 deflected by the virtual lens 90 and then extends it to the virtual scene 80. The virtual light ray 701 reaches the virtual object 81a at a point 82 having coordinates (x, y, z). Each point of the object 81a of the virtual scene 80 is associated with a color and / or a luminosity. Thus, the point 82 is associated with a color and / or a luminosity. Thus, in the generating step 1008, the value of the real light ray 700 generated by the light field display 60 associated with the virtual light ray 701 is then the color and / or luminosity value of the point 82. This operation is then repeated for other real rays, for example the real light ray numbered 703 associated with the virtual light ray 704, and the point object numbered 84, in order to spatially and angularly scan the light field display 60.

[0155] FIG. 16 or FIG. 17 show schematic examples of embodiments of the selecting step applied in the calculating step 1004 and / or generating step 1008, respectively.

[0156] FIG. 16 shows a schematic diagram of the selecting step 2006 applied in the calculating step 1004 .

[0157] In this example, the calculating step 1004 calculates only the virtual rays that reach the eye 70 of the object 71. For example, the virtual rays marked 103 in Fig. 16 include fewer virtual rays than the virtual rays marked 103 shown in Fig. 14. In addition, the virtual ray marked 100 has been removed from Fig. 16 because the ray (real ray) associated with this virtual ray 100 does not reach the eye 70 of the object 71. Therefore, the calculating step 1004 takes less time.

[0158] This step can be realized, for example, by calculating a ray (virtual ray) that reaches, at a first end, the eye 70 of the subject 71 and, at a second end, a point of the light field window 65 facing the first side 66 of the light field display 60. This ray can also be associated with a virtual ray located at the second side 67 of the light field display 60. If this ray does not reach the eye 70 of the subject 71, the associated virtual ray (starting from the second side 67 of the light field display) is not taken into account in the calculation step 1004.

[0159] Following the selecting step 2006 applied to the calculating step 1004, the method according to the present disclosure advantageously performs a selecting step 2008 applied to the generating step 1008.

[0160] In that case, in the determining step 1006, only the real rays of the light field 69 that reach the eye 70 of the object 71 are calculated. This selection makes it possible to generate in the generating step 1008 only the light field 69 that includes the real rays that reach the eye 70 of the object 71. For example, in FIG. 17, the generated light field 69 includes only the real rays that reach the eye 70 of the object 71. These real rays correspond to the virtual rays selected in the selecting step 2006 that are applied in the calculating step 1004. The method according to the present disclosure is therefore optimized since the calculating step 1004, the determining step 1006 and the generating step 1008 require less time and consume less energy. In addition, it provides better visual comfort.

[0161] device 18 shows a subject 71 viewing a modified virtual 3D scene 140 according to method 1200 or 2000. The modified virtual 3D scene 140 corresponds to the virtual 3D scene 80 as it is actually displayed to the subject 71. The modified virtual 3D scene 140 includes all elements included in the virtual 3D scene 80.

[0162] In this example, the subject 71 views the modified virtual three-dimensional scene 140 through the system 130. The system 130 is a virtual reality (VR) device or an augmented reality (AR) device 130 worn by the subject 71. In this example, the light field display 60 is included in the system 130. Specifically, the light field display 60 is configured to be placed in front of the eyes of the subject 71.

[0163] The modified virtual three-dimensional scene 140 shown in Fig. 18 is a representation of an image of a scene in the current life of the subject 71. The modified virtual three-dimensional scene 140 is clearly seen by the subject 71. The field of view of the subject 71 is thus corrected.

[0164] The system 130 is connected to a computer 120, or a computing module 120, or a calculator 120 configured to control the light field display 60. In this example, the calculator 120 is integrated into the system 130. In another embodiment, the calculator 120 is remotely connected to the system 130.

[0165] The computer 120 is configured to perform at least the receiving step 1002, the calculating step 1004, and the determining step 1006 of the method 1200 or 2000 according to the present disclosure. The light field display 60 and the computer 120 are configured to perform the generating step 1008.

[0166] The system 130 is adapted and configured to operate the methods 1000 and 2000 .

[0167] In one embodiment, the light field display 60 is used for two eyes 70 of a subject 71 .

[0168] In another embodiment, the system 130 includes two light field displays 60, one associated with each of the two eyes 70 of the subject 71. The system 130 is configured to generate and display two virtual rectified three-dimensional scenes 140, allowing for binocular vision.

[0169] In another embodiment, the system 130 may be an augmented reality device 130. Thus, the light field display 60 is placed on the real lens of the object to project the modified virtual three-dimensional 140 onto the real lens of the object. The system 130 may be, for example, an augmented reality pair of glasses.

Claims

1. 1. A method for simulating an ophthalmic lens on an eye of a subject viewing a virtual three-dimensional scene using a light field display, comprising: the light field display comprises a light field window; The method comprises: receiving a set of input parameters including scene parameters of the virtual three-dimensional scene; Calculating virtual rays between points of the virtual 3D scene and points of a pupil plane by ray tracing, the virtual ray passes through a virtual lens and is defined based on a model that provides a deflection angle of the virtual ray through the virtual lens; the deflection angle depends on at least one of an angle of incidence and a position of the virtual ray on the virtual lens; repeating the computing step by ray tracing a plurality of virtual rays passing through the virtual lens to spatially and angularly scan the light field window, and combining pairs of one point in the virtual 3D scene and another point in the pupil plane; determining a light field representing a modified virtual three-dimensional scene based on the scene parameters and the deflection angle associated with each virtual ray of the plurality of virtual rays; and generating the light field representing the modified virtual three-dimensional scene onto the light field window toward the pupil plane of the subject.

2. The method of claim 1 , wherein the light field window corresponds to a field of view of at least 40 degrees in the horizontal plane and 40 degrees in the vertical plane, and / or an angular resolution of less than 5 degrees.

3. 3. The method of claim 1 or 2, wherein the light field window extends spatially over a surface area of ​​at least 3 mm in the horizontal plane and 3 mm in the vertical plane, and / or with a spatial resolution of at least 0.5 dots / mm.

4. acquiring a three-dimensional image of a scene using a three-dimensional capture device to extract said scene parameters; A method according to any one of claims 1 to 3, comprising obtaining said scene parameters, said scene parameters including geometric and / or photometric properties of said scene.

5. The method of claim 4 , wherein determining the light field comprises applying the geometric and / or photometric properties of the scene.

6. The method according to any one of claims 1 to 5, wherein the calculating and / or generating steps comprise selecting a subset of a plurality of virtual rays passing through a pupil of the eye of the subject.

7. The method according to any one of claims 1 to 6, wherein the model is determined on the basis of geometrical optics calculations and / or wave optics calculations and / or photometric optics calculations.

8. The method of any one of claims 1 to 7, wherein the model includes the shape of the lens and the positioning of the lens relative to the eye.

9. 9. The method according to claim 1, wherein the model comprises a distortion model of the virtual lens, and / or a spherical power model of the virtual lens comprising at least spherical parameters, and / or an astigmatic power model of the virtual lens comprising at least cylindrical and / or axial parameters, or J0 and J45 parameters, and / or a filter lens.

10. 10. The method of claim 1, wherein the determining step comprises the step of representing the modified virtual three-dimensional scene, further comprising applying a convolution or image transformation adapted to simulate blurring, distortion, diffraction, chromatic aberration, scattering, or transmission attenuation.

11. the method is adapted to simulate another ophthalmic lens on the other eye of the subject; The method includes the steps of calculating, determining, and generating to simulate the other ophthalmic lens for the other eye of the subject; The method according to any one of claims 1 to 10, wherein the target parameters in the receiving step include other parameters of the pupil diameter of the other eye and / or the pupil position of the other eye relative to the light field display.

12. The method of any one of claims 1 to 11, wherein the virtual lens comprises a progressive or multifocal lens, a single vision lens, a spectacle lens, a contact lens, or a filter.

13. The method according to any one of claims 1 to 12, wherein the virtual ray and the plurality of virtual rays in the calculating step are configured to propagate from the pupil plane to at least one object in a scene.

14. The method according to any one of claims 1 to 12, wherein the virtual ray and the plurality of virtual rays in the calculating step are configured to propagate from at least one object in a scene to the pupil plane.

15. 1. A system comprising a computer and a light field display, The system is adapted and configured to carry out the method according to any one of claims 1 to 14.

Images