METHOD FOR SIMULATION OF OPTICAL PRODUCTS

Abstract

The present invention relates to a method (100) for simulating optical lenses in augmented reality using a mobile apparatus (11) comprising at least an input peripheral (11a) and a display peripheral (11b), the method (100) being characterized in that it comprises at least, a capture phase (P1) a data acquisition phase (P2) while the user is looking at an object in the environment, and a simulation phase (P3) performed by at least a processing unit (11c) allowing a virtual lens to be produced and superposed on at least an image of a user's environment.

Description

Optical product simulation process Technical field of the invention

[0001] The present invention relates to the field of optics and more particularly to the field of trying on optical products such as glasses or lenses, using augmented reality. State of the art

[0002] In the field of optical product fitting, it is known to use information about an individual's vision contained in their prescription to select corrective glasses or lenses appropriate for their vision and manufacture them for fitting. The disadvantage of this type of fitting is that if the selected optical product is not suitable for the patient, it is discarded and another optical product must be manufactured again.

[0003] To overcome this drawback, a method of simulating lenses in augmented reality is well known which, using a tablet, allows a user to simulate the effect of a lens or glass on an image of the user's environment captured in real time. However, this type of method does not allow for realistic simulation of the effect of a power-variable lens or glass, i.e. a lens or glass characterized by the fact that it offers, over its entire surface, different powers dedicated to different viewing distances. Indeed, this simulation system does not take into account the distance of the different elements of the environment with the tablet and therefore does not allow for the most faithful reproduction possible of the 3D perception of the visual field perceived according to the design of the lens represented.Thus, this type of system provides a simulation experience that is not very representative of reality for a wearer, particularly in the case of the simulation of progressive lenses intended to correct so-called "near" vision and so-called "distance" vision.

[0004] The aim of the present invention is therefore to overcome the drawbacks mentioned above and to propose a method for simulating glasses or optical lenses based on augmented reality allowing a realistic restitution of the effect of glasses or optical lenses on the visual perception of an environment composed of elements located at different distances from the user.

[0005] According to the invention, there is therefore proposed a method for simulating lenses or optical glasses in augmented reality using a mobile device comprising at least one input device and one display device, said method being remarkable in that it comprises at least one capture phase (P1) comprising at least the following step: a1) Capturing at least one image of a user's environment using at least one image capture element; a data acquisition phase (P2), comprising at least the following step: b1) Determining the distance between the mobile device and at least one object in the environment using at least one measurement unit;and a simulation phase (P3) implemented by said at least one processing unit, comprising at least the following step: c1) Application of a virtual glass or lens reproducing the optical effect of a real glass or lens of determined optical correction on said at least one image of the environment as a function of the data acquired during the data acquisition phase (P2) and display of the simulation on a display device.;

[0006] A "mobile device" is a portable computing device that can be used independently. This includes, for example, a tablet, a smartphone or a laptop.

[0007] The term "user" refers to the individual using the mobile device for the purpose of obtaining a simulation of corrective lenses or glasses; this could be, for example, a patient for whom a corrective product has been designed or an optician.

[0008] A lens is a transparent, very thin, concave optical device that is placed on the cornea of ​​the eye to correct vision defects. A lens type refers to lenses that offer different types of optical correction and have different types of coatings or tints.

[0009] An "input device" is a piece of peripheral computer equipment that provides data to an information processing system such as a computer. Examples include a keyboard, a mouse, or a touchscreen (a touchscreen can be considered both an input and an output device). This device may be integrated into the mobile device, i.e., an integral part of the device, or it may transmit its data to it via a wired or wireless connection.

[0010] A display device is an output device that displays information, for example, the screen of the mobile device or a screen that receives data from the mobile device via a wired or wireless connection.

[0011] An image capture element is a device such as a camera or camcorder. This device may be integrated into the mobile device (e.g., a smartphone or tablet camera) or be independent and connected to the mobile device.

[0012] A "measuring unit" is an instrument or application that allows distance measurement. This instrument can be integrated into the mobile device or transmit its data to it via a wired or wireless connection.

[0013] The term "processing unit" refers to all the elements of the mobile device (processor, memory, etc.) used to store and process information about the environment and the view of a future wearer.

[0014] A "virtual lens" is an image filter created in augmented reality and allowing optical effects to be created on an image (creation of blurred areas for example, changing the color of the image, etc.)

[0015] Preferably, during step c1, said glass or said virtual lens is chosen from a pre-existing database of glasses or lenses with different modifiable maps or is completely created according to needs.

[0016] The way in which the different optical correction power zones are arranged on a glass or lens surface is called the "mapping".

[0017] Advantageously, during the simulation phase (P3), the user can choose to simulate a glass or lens: multifocal, power variation, progressive, degressive, specific positioning, suppression, myopia progression, fixed or degraded or bi-degraded tint, photochromic, polarizing, coating, or anti-fatigue.

[0018] Preferably, the data acquisition phase (P2) comprises a step b2 which takes place during observation of the environment by a wearer through the display device of the mobile device and which consists of determining the movements of the wearer's head, and the distance between the user and the mobile device using a motion sensor.

[0019] A patient for whom the lenses or glasses are intended is called a "wearer".

[0020] Even more preferably, the determination of movements at the level of the wearer's head consists of detecting the movements of the wearer's face and eyes relative to the mobile device using a motion sensor.

[0021] Advantageously, step a1 is a step of capturing in real time at least one image of a user's environment using at least one image capture element.

[0022] Preferably, in step b1, the unit of measurement for determining the distance between the mobile device and the object is a telemetric device or application.

[0023] A "telemetric device or application" is a device or application that allows the distance of an object to be measured by computer means (applications based on calculating distance following the capture of an image of the object, for example), optical means (laser telemetry, for example), acoustic means (sonar, for example) or radioelectric means (radar, for example).

[0024] Advantageously, the measuring unit is a laser, ultrasonic, acoustic rangefinder or a distance measuring application.

[0025] Preferably, at least the image capture element and the measurement unit are integrated into the mobile device.

[0026] Advantageously, the mobile device is a tablet or a smartphone and the display device is the screen of said tablet or said smartphone.

[0027] Preferably, in step c1, the simulation is displayed on the display device of the mobile device. Brief description of the figures

[0028] Other advantages and characteristics will emerge more clearly from the following description of a method of carrying out a method according to the invention, with reference to the appended figures in which:

[0029] is a flowchart of the steps of the method according to the invention,

[0030] is a schematic view of an embodiment of a system for implementing the method according to the invention.

[0031] In accordance with Figures 1 and 2, a method 100 is therefore described for simulating lenses or optical glasses in augmented reality using a mobile device 11 comprising at least one input device 11a and one display device 11b, said method 100 being remarkable in that it comprises at least one capture phase (P1) comprising at least the following step:

[0032] a1) Capturing at least one image of a user's environment using at least one image capturing element 12, 110;

[0033] a data acquisition phase (P2), comprising at least the following step:

[0034] b1) Determining the distance between the mobile device 11 and at least one object in the environment using at least one measuring unit 14, 210;

[0035] and a simulation phase (P3) implemented by said at least one processing unit 11c of the mobile device, comprising at least the following step:

[0036] c1) Application of a virtual glass or lens reproducing the optical effect of a real glass or lens of determined optical correction, on said at least one image of the environment, according to the data acquired during the data acquisition phase (P2) and display of the simulation on a display device, 310;

[0037] The mobile device 11 may be a tablet, a smartphone or a laptop. Regardless of the mobile device 11, the display device 11b will be the screen of the mobile device 11 or an independent screen connected to the mobile device 11 by a wired or wireless connection (for example, USB cable connection, Bluetooth connection, etc.). If the mobile device 11 is a tablet or a smartphone, the input device 11a will preferably be a touchscreen. If the mobile device 11 is a laptop, the input device 11a will be a keyboard and / or a mouse and / or a touchscreen. This input device 11a will be used to select the different types of glasses or lenses to be simulated or to create glasses or lenses with new maps. It may also be used to enter the optical correction of the product for the simulation.

[0038] The image capture element 12 is a camera or camcorder type device that can be integrated into the mobile device (camera of a tablet or smartphone for example) or connected to it by a wired or wireless connection (for example, USB cable connection, Bluetooth connection, etc.). The purpose of this image capture element 12 is to capture an image of the environment and to transmit this image to the display device 11b of the mobile device 11. The captured image may be an image taken in real time or in delayed mode.

[0039] The processing unit 11 then creates a custom glass or lens image by processing all the data acquired during the data acquisition phase P2 and taking into account a certain amount of information relating to the general manufacturing of optical glasses or lenses previously stored on the mobile device 11. Taking into account the distance between the environment and the mobile device 11 is a determining factor in the accuracy of the simulation of the lens or glass. Indeed, this makes it possible to reproduce as faithfully as possible what can be perceived by an ametropic or emmetropic eye equipped with a device for compensating for its ametropia and / or presbyopia. Any lens or glass with power variation offers, over its entire surface, a map of different powers dedicated to viewing distances that can be between infinity and 30 cm.This implies that in the user's field of vision, any object perceived through the optical zone that does not correspond to its distance position will be perceived as blurred. The user is therefore obliged to modify his direction of gaze in order to use the optical zone that corresponds to the distance of the targeted object. For example, in the case of a progressive lens, if the subject lowers his eyes to go down a staircase, he uses the optical zone reserved for near vision and the targeted steps will be seen as blurred. Another example: If a subject looks into the distance with a progressive lens, the objects located on the periphery of his horizontal visual field will be perceived as blurred due to the isoastigmatisms present laterally on the surface of said progressive lens.

[0040] The advantage of this distance measurement (step b1) is therefore to restore as faithfully as possible the 3D perception of the visual field perceived according to the mapping of the lens or glass selected and thus demonstrate the differences existing between different types of lenses or glasses.

[0041] This lens or glass developed in augmented reality is superimposed on the image of the environment captured by the image capture element 12.

[0042] The user looking at the display device then sees exactly what they would see if they were wearing real lenses or corrective lenses. Following this simulation, if the eyewear product is suitable, the manufacturing of the product can begin.

[0043] Preferably, during step c1, said glass or said virtual lens is chosen from a pre-existing database of glasses or lenses with different modifiable mappings or is completely created according to the needs. The user can select from a database a glass or a lens with a “typical” mapping, the processing unit 11c will then generate a virtual lens of particular optical correction according to this mapping: for example, progressive lens. The user can, if necessary, modify the mapping of a lens or a glass from the database or choose to create a new glass or a new virtual lens with a tailor-made mapping so as to meet the needs of the future wearer.

[0044] Advantageously, during the simulation phase (P3), the user can choose to simulate a glass or lens: multifocal, power variation, progressive, degressive, specific positioning, suppression, myopia progression, fixed or degraded or bi-degraded tint, photochromic, polarizing, coating, or anti-fatigue.

[0045] Preferably, the data acquisition phase (P2) comprises a step b2 which takes place during observation of the environment by a wearer through the display peripheral 11b of the mobile device 11 and which consists of determining the movements of the wearer's head, and the distance between the wearer and the mobile device 11 using a motion sensor 13.

[0046] During the data acquisition phase (P2), the visual behavior of the wearer in relation to the image of the environment that he observes on the display device 11b is determined and recorded, the distances between him and the mobile device 11 and between the mobile device 11 and an object in the environment are measured. These data will be transmitted to the processing unit 11c which will record and interpret them in order to determine the optical correction necessary for the wearer and in order to simulate a lens or glass corresponding to this correction.

[0047] Even more preferably, the determination of movements at the wearer's head consists of detecting the movements of the wearer's face and eyes relative to the mobile device 11 using a motion sensor 13. The motion sensor 13 can take the form of any type of sensor that can detect movement. Preferably, it will be an optical sensor that can be part of the mobile device 11 or connected to it by a wired or wireless connection. The sensor 13 can detect the movements of a wearer who is observing the environment while holding the mobile device 11 in his hands. It can detect, for example, that the wearer is approaching or moving back the mobile device 11, that the wearer is moving the device 11 up, down, left or right, or even determine the distance between the wearer and the mobile device 11 using an acceleration sensor.

[0048] Advantageously, step a1 is a step of capturing in real time at least one image of a user's environment using at least one image capture element 12.

[0049] Preferably, in step b1, the measurement unit 14 for determining the distance between the mobile device 11 and the object is a telemetric device or application. A “telemetric device or application” is a device or application for measuring the distance of an object by computer means (applications based on calculating distance following the capture of an image of the object, for example), optical means (laser telemetry, for example), acoustic means (sonar, for example) or radioelectric means (radar, for example).

[0050] If the mobile device 11 is a tablet or a smartphone, the measurement unit 14 may be a remote measurement application such as, for example, “Mapulator” or “EasyMeasure” on Android and iOS. “Easy measure” makes it possible to measure, using an image capture device 12 in real time and in augmented reality, several distances, including the distance between the mobile device 11 and an object, in a precise manner. In a particular embodiment, the measurement unit 14 will be a laser rangefinder connected to the mobile device 11 by a wired or wireless connection and whose operation will be as follows: It will project a laser beam onto an object in the environment which will in turn return the light beam, the rangefinder will then calculate the phase shift between transmission and reception.

[0051] Preferably, the mobile device 11 is equipped with LiDAR (Light Detection and Ranging) technology. LiDAR technology allows detection and estimation of distance by light or laser. This distance measurement technology is possible by analyzing the properties of a beam of light returned to its transmitter (mobile device 11). LiDAR uses light to measure distance using invisible pulsed green spectrum lasers. These pulses (which occur thousands of times per minute) measure the time it takes for the light to return to the sensor. In doing so, it creates a “picture” of the environment in front of the scanner.

[0052] Advantageously, the measuring unit 14 is a laser, ultrasonic, acoustic rangefinder or a distance measuring application.

[0053] Preferably, the image capture element 12 and the measurement unit 14 are integrated into the mobile device 11. The term "integrated" here means that the mobile device 11 has control of these different elements, either these elements are an integral part of the body / housing of the mobile device 11, or they are external to it (they are not part of the housing of the mobile device 11) but can only operate under the control of the mobile device 11. In this way, the simulation is easier to set up. The motion sensor 13 can also be integrated into the mobile device 11. Of course, if motion sensors 13, image capture elements 12 and / or measurement units 14 different from those integrated into the mobile device 11 are required, it is entirely possible to use the appropriate device(s) by connecting them via a wired or wireless connection to the mobile device 11.

[0054] Advantageously, the mobile device 11 is a tablet or a smartphone and the display device 11 is the screen of said tablet or said smartphone. A device powerful enough to support this type of simulation will be chosen from among the existing mobile devices 11. For example, an iPad-type tablet will be chosen. On the tablet or smartphone, the image capture element 12 and the measurement unit 14 will preferably be at the rear of the device 11, while the image sensor will be at the front on the screen side.

[0055] Even more advantageously, the simulation is displayed on the display device 11b of the mobile device 11.

[0056] Finally, it goes without saying that the examples of methods 100 in accordance with the invention which have just been described are only particular illustrations, in no way limiting the invention.

Claims

Method (100) for simulating lenses or optical glasses in augmented reality using a mobile device (11) comprising at least one input device (11a) and one display device (11b), said method (100) being characterized in that it comprises at least, a capture phase (P1) comprising at least the following step: a1) Capturing at least one image of a user's environment using at least one image capture element (12) (110); a data acquisition phase (P2) comprising at least the following step: b1) Determining the distance between the mobile device (11) and at least one object in the environment using at least one unit of measurement (14) (210);and a simulation phase (P3) implemented by at least one processing unit (11c) of the mobile device (11), comprising at least the following step: c1) Application of a virtual glass or lens reproducing the optical effect of a real glass or lens of determined optical correction, on said at least one image of the environment, according to the data acquired during the data acquisition phase (P2) and display of the simulation on a display device (310); Method (100) according to claim 1 characterized in that during step c1, said virtual glass or lens is selected from a pre-existing database of glasses or lenses with different modifiable mappings or is created with a custom mapping. Method (100) according to any one of the preceding claims in that during the simulation phase (P3), the user chooses to simulate a lens or a lens: multifocal, variable power, progressive, degressive, specific positioning, suppression, myopia progression, fixed tint or degraded or bidegraded, photochromic, polarizing, coated, or anti-fatigue. Method (100) according to any one of the preceding claims characterized in that the data acquisition phase (P2) comprises a step b2 which takes place during the observation of the environment by a wearer through the display device (11b) of the mobile device and which consists of determining the movements of the wearer's head, and the distance between the wearer and the mobile device (11) using a motion sensor (13). Method (100) according to claim 4 characterized in that the determination of movements at the level of the wearer's head consists of detecting the movements of the wearer's face and eyes relative to the mobile device (11) using a motion sensor (13). Method (100) according to any one of the preceding claims characterized in that step a1 is a real-time capture step of at least one image of a user's environment using at least one image capture element (12). Method (100) according to any one of the preceding claims characterized in that, in step b1, the unit of measurement (14) enabling the determination of the distance between the mobile device (11) and the object is a telemetry device or application. Method (100) according to claim 7, characterized in that the unit of measurement (14) is a laser, ultrasonic, acoustic or remote measurement rangefinder application. Method (100) according to any one of the preceding claims characterized in that at least the image capture element (12), and the unit of measurement (14) are integrated into the mobile device (11). Method (100) according to any one of the preceding claims characterized in that the mobile device (11) is a tablet or a smartphone and in that the display device (11b) is the screen of said tablet or said smartphone.