A method for determining the orientation of a person's head in a natural visual posture in 3D.

Abstract

The present invention relates to a method to be implemented by computer means for determining an oriented 3D representation of a human head in a natural viewing pose, said method comprising the steps of: a) receiving a first data set corresponding to a first 3D representation of a head of a person in a first frame and a position of a center of rotation of at least one eye of the person in the first frame; b) determining a second data set corresponding to a head orientation of the person in a neutral viewing pose in a second frame; c) determining an oriented 3D representation of the person's head by orienting a first 3D representation of the person's head based on the orientation of the person's head in step b) in the common frame.

Description

Technical Field

[0002]

[0001] The present invention relates to a method for determining a 3D representation of a person's head oriented in a natural visual pose. The present invention further relates to a method for determining fitting parameters for a pair of spectacle lenses suitable for a person using the 3D representation of the person's head obtained by the method according to the present invention.

Background Art

[0002] The design and manufacture of ophthalmic lenses suitable for a wearer and a selected spectacle frame currently require one or more measurements to be taken at an optician's. <​​​​​​​​​​​​​​​​​​​​ Therefore, measurements performed in an optician's shop are inaccurate due to the unnatural conditions under which the measurements are taken.

[0009] To correct the aforementioned shortcomings of the prior art, the present invention proposes a method for determining an oriented 3D representation of a person's head that allows for taking into account the posture of the individual's body. [Overview of the project] [Means for solving the problem]

[0010] To achieve this objective, the present invention proposes a method for being carried out by computer means for determining an oriented 3D representation of a person's head in a natural visual posture, and this method is a) A step of receiving a first dataset corresponding to a first 3D representation of a person's head in a first frame and the position of the rotation center of at least one of the person's eyes in the first frame, b) A step of determining a second dataset corresponding to the orientation of the head of a person in a natural visual posture in the second frame, c) In a common frame, determine the oriented 3D representation of the person's head by oriented the first 3D representation of the person's head based on the orientation of the person's head in step b), and Includes.

[0011] Advantageously, this method makes it possible to separate morphological measurements from posture in order to determine an oriented 3D head scan in a predefined posture using the position of the rotation center of at least one of the person's eyes.

[0012] Thanks to the present invention, morphological measurements can be performed with accurate measurements without taking posture into account, and it is possible to determine the position of the rotation center of at least one eye of a person in a three-dimensional head scan, which will be used as a measurement frame for the next step.

[0013] Then, the posture is guided by the eye care practitioner (ECP) in a "unrestrained" situation during the second session.

[0014] The common frame is the association of steps a) and b) by orienting morphological measurements thanks to posture measurements. This common frame is an oriented 3D head scan in a defined posture and can be used and reused to fit actual or virtual frames for designing ophthalmic lenses for eyeglass frames.

[0015] According to further embodiments that can be considered individually or in combination, - During step b), the direction of the person's gaze is determined. - The base point of the common frame is based on the rotation center of the person's at least one eye, - In step b), the person is placed in a realistic situation where the position of the person's head is not constrained. - In step b), the person performs one or more habitual activities, - A person's habitual activities include the following activities, namely: Reading from various media (books, magazines, smartphones, laptops, desktop computers), ○ Use of a smartphone or tablet (to watch videos, view photos, take photos), Working at a desk, 〇 Driving, 〇 Practicing sports, Cooking, Playing games on a game console, ○ To sit and rest, Watching TV, Playing a musical instrument, It is one of them, - In step b), the position of the center of rotation of the person's eyeballs is determined from one or more acquisitions of images of the person's head. - To determine the direction of the person's gaze, an image of the person's pupil or corneal reflection is identified within the image captured in step b), and the direction of the gaze is inferred from there according to the position of the center of rotation in the second reference frame, the position of which is determined by a first set of data in the first reference frame. - In order to determine the direction of the person's gaze, the position of the element that is targeted by the gaze and belongs to the person's environment is determined in a second reference frame. - In step b), the second set of data includes a series of second images of the person's head taken while the person fixes their gaze on a visual target, the position of this visual target relative to the second image capture device is determined between each capture of the second image. - The method further includes a calibration step prior to step b), during which a second device for capturing two-dimensional or three-dimensional images is calibrated, in which an image of a dedicated standard object is captured by the second image capture device. - The spatial position and / or orientation of the second image capture device is determined during step b), and information relating to the position and / or orientation of the person's head in space in a natural visual posture is determined therefrom. - The first dataset includes the positions of the rotation centers of the person's binoculars in the first frame and the base point of the common frame corresponding to the person's monoculars. - The first dataset is acquired by a first device for capturing three-dimensional images, which then constitute the first representation of a person's head. - When obtaining the first dataset, the position of the rotation center in the first reference frame is: Step 1: At least two images of a person's head are captured using an image capture device, wherein the images of the person's head posture with respect to the image capture device are different, and in the images the person has fixed their gaze on a targeting point at a predetermined position. 〇 The step of determining the gaze direction of the person corresponding to each of the two images, 〇 A step in which the position of the rotation center of at least one eye of a person is inferred therefrom, determined by - The step of obtaining step b) and the first data set are each performed using different image capture devices. - In the step prior to step c), each of the first and second data sets is stored corresponding to an identifier of a person.

[0016] Another object of the present invention is a method for determining fitting parameters of a pair of spectacle lenses suitable for a person using a 3D representation of the person's head oriented in the direction obtained by the method according to the present invention.

[0017] The present invention also relates to a system for executing a method for determining a 3D representation of a person's head oriented in a natural visual posture as described above according to the present invention. More specifically, the system a memory, executing program instructions stored in the memory to at least ]>a) receiving a first data set corresponding to a first 3D representation of a person's head in a first frame and the position of the rotation center of at least one eye of the person in the first frame; b) determining a second data set corresponding to the orientation of the head of a person in a natural visual posture in a second frame; c) determining a 3D representation of the person's head oriented in the direction by orienting the first 3D representation of the person's head based on the orientation of the person's head in step b) in a common frame and a processor configured to perform including.

[0018] The invention further provides that the processor is accessible and, when executed by the processor, causes the processor to - A method for determining a 3D representation of a person's head oriented in a natural visual posture according to the present invention, and / or - A method for determining fitting parameters for a pair of eyeglass lenses suitable for a person using an oriented 3D representation of the person's head obtained by the method according to the present invention. This relates to a computer program product that includes a sequence of one or more stored instructions that cause the steps of the following to be carried out.

[0019] The invention also relates to computer-readable storage media having programs recorded therein, wherein the programs cause a computer to perform at least one of the methods of the invention.

[0020] The invention further relates to a device including a processor that stores a sequence of one or more instructions and is adapted to perform at least one of the steps of the method according to the invention.

[0021] As will be evident from the following considerations, unless otherwise specified, any use of terms such as “computing” and “calculating” throughout this specification is correctly understood to refer to the operation and / or processing of a computer or computing system, or similar electronic computing device, that manipulates and / or converts data, which is represented as electronic or other physical quantities within the registers and / or memory of a computing system, to other data, which is similarly represented as physical quantities within the memory, registers, or other such information storage, transfer, or display devices of a computing system.

[0022] Embodiments of the present invention may include an apparatus for performing the operations described herein. This apparatus may be specifically constructed for a desired purpose, or may include a general-purpose computer or digital signal processor ("DSP") that is selectively invoked or reconfigured by a computer program stored within the computer. Such computer programs may be stored in computer-readable storage media, including, but not limited to, floppy disks, optical disks, CD-ROMs, magneto-optical disks, read-only memory (ROM), random access memory (RAM), electrically programmable read-only memory (EPROM), electrically erasable and programmable read-only memory (EEPROM), magnetic or optical cards, or other types of media suitable for storing electronic instructions and capable of being coupled to a computer system bus.

[0023] The processes and representations presented herein are not inherently related to any particular computer or other device. Various general-purpose systems may be used with programs that follow the teachings herein, or it may be convenient to construct a more specialized device to perform the desired method.

[0024] Desired structures for various of these systems will emerge from the following description. Furthermore, embodiments of the present invention are not described with reference to any particular programming language. It will be understood that various programming languages ​​may be used to carry out the teachings of the invention as described herein.

[0025] Herein, several embodiments of the present invention will be described with reference to the following drawings, as just one example. [Brief explanation of the drawing]

[0026] [Figure 1] This is a diagram illustrating the flowchart of a method for determining an oriented 3D representation of a person's head in a natural visual posture, according to the present invention. [Figure 2] This is a diagram illustrating a chart flow of a method for determining an oriented 3D representation of a person's head in a natural visual posture, according to another embodiment of the present invention. [Figure 3] This is an explanatory diagram of a method for measuring a person's posture. [Modes for carrying out the invention]

[0027] The elements in the figures are illustrative for conciseness and clarity, and are not necessarily drawn to actual size. For example, the dimensions of some elements in the figures may be exaggerated compared to others to help improve understanding of embodiments of the present invention.

[0028] Referring to Figure 1, the present invention relates to a method for determining an oriented 3D representation of the head of a person in a natural visual posture. In practice, this method is carried out by computer means.

[0029] In the sense of this invention, the wearer's natural visual posture is the wearer's posture in a natural environment, that is, a posture without any constraints. For example, the wearer's field of vision must be clear, in other words, it must not be facing a measuring instrument or a mirror.

[0030] In the context of this invention, "frame" means a reference frame unless it specifically refers to an "eyeglass frame."

[0031] This method, at least a) Step S2 of receiving the first dataset, b) Step S4 for determining the second dataset, c) Directional 3D representation determination step S6 and Includes.

[0032] During the first dataset receiving step S2, a first dataset is received, corresponding to a first 3D representation of the person's head in the first frame and the position of the rotation center of at least one of the person's eyes in the first frame.

[0033] For example, the first dataset is in the form of cloud points or a mesh.

[0034] During the acquisition of the three-dimensional image, the person is positioned in front of the measuring instrument in an undefined pose, allowing for the acquisition of the correct head metric in order to determine the position of the rotation center of at least one eye and to be within the measurement frame for step b).

[0035] The first dataset is represented by an undefined frame, which may be either an instrument frame or a head frame.

[0036] Then, a second dataset corresponding to the orientation of the person's head in a natural visual posture in the second frame is determined during the second dataset determination step S4.

[0037] The second dataset may further include features of reference points in the 3D representation of a person's head, such as the bridge of the nose (also called the root of the nose) and / or the tip of the nose, and / or the outer corner or outer corner of the eye and / or the inner corner or inner corner of the eye.

[0038] During this step, the eye care practitioner measures the wearer's posture in a different environment than the one used to collect the first set of data, for example. This posture is a natural visual posture, i.e., unconstrained. For example, the wearer's field of vision must be clear (not in front of a mirror) to adopt a more representative posture. In other words, this natural visual posture is the one verified by the eye care practitioner in an environment closer to the wearer's daily life.

[0039] To achieve this objective, head position (3D head scan, on-image markers, etc.) and line-of-sight target are measured in the same second frame. Various methods described below may be employed to measure a person's posture.

[0040] For example, preferably, in step S4, the person is placed in a real-world situation in which the posture of the person's head is not constrained.

[0041] Advantageously, during step b) related to the determination of the second dataset, the person engages in one or more habitual activities, for example, the following activities, i.e., - Reading from various media (books, magazines, smartphones, laptops, desktop computers) - Using a smartphone or tablet (watching videos, viewing photos, taking photos), - Working at a desk, - Driving, - Practicing sports, - Cooking, - Playing games on a game console, - To sit and rest, - Watching TV, - Playing a musical instrument, To carry out one of them.

[0042] Preferably, during step b), the direction of the person's gaze is determined.

[0043] According to one embodiment, during step b) related to determining the second dataset, the person may be wearing eyeglass frames, for example, eyeglass frames selected by the wearer. According to such an embodiment, the second set of data may further include features of the eyeglass frames, such as the mid-bridge point of the frame and / or boxed data and / or bottom point.

[0044] During the oriented 3D representation determination step S6, the oriented 3D representation of the person's head is determined by oriented the first 3D representation of the person's head, based on the orientation of the person's head determined during step S4 in a common frame linked to the head.

[0045] Preferably, the base point of the common frame is based on the rotation center of the person's at least one eye.

[0046] Preferably, the first dataset includes the positions of the rotation centers of the person's eyes in the first frame and the base point of the common frame corresponding to the person's monocular eye.

[0047] Furthermore, each of the first and second datasets is advantageously stored in a step prior to step S6, corresponding to the person's identifier.

[0048] According to another embodiment compatible with the previous embodiment, the position of the rotation center of the person's eyes is further determined from the acquisition of one or more images of the person's head in step b).

[0049] Preferably, to determine the direction of the person's gaze, an image of the person's pupil or corneal reflection is identified in the image captured in step b). The direction of the gaze is then inferred from there, depending on the position of the center of rotation in a second reference frame, which is determined by a first set of data in the first reference frame.

[0050] Advantageously, in order to determine the direction of the person's gaze, the position of the elements targeted by the gaze and belonging to the person's environment in a second reference frame is determined.

[0051] According to another embodiment compatible with the previous embodiment, in step S4, the second set of data preferably includes a series of second images of the person's head taken while the person fixes his gaze on a visual target. The position of this visual target with respect to the second image capture device is determined between each capture of the second image.

[0052] According to another embodiment compatible with the previous embodiment, the first dataset is preferably acquired by a first device for capturing a three-dimensional image. This three-dimensional image then constitutes the first representation of a person's head.

[0053] Preferably, the three-dimensional image is created ear to ear to fit the frame.

[0054] During the acquisition of the three-dimensional image, the person is positioned in front of the measuring instrument in an undefined pose, allowing for the acquisition of the correct head metric in order to determine the position of the rotation center of at least one eye and to be within the measurement frame for step b).

[0055] Such three-dimensional images can be obtained by photogrammetry, pattern / flange projection, stereoscopic viewing, etc.

[0056] For example, according to the embodiment illustrated in Figure 2, when acquiring the first dataset, the position of the rotation center in the first reference frame is advantageously... - Step S22 wherein at least two images of a person's head are captured using an image capture device, the images of the person's head posture relative to the image capture device are different, and in those images the person has fixed their gaze on a targeting point at a predetermined position, - Step S24 for determining the direction of the person's gaze for each of the two images, - The position of the center of rotation of at least one of the person's eyes is inferred from there, step S26, It is determined by [the following].

[0057] Therefore, the position of the center of rotation in the first frame is determined from multiple images of the person's head captured with different head poses while maintaining the same line of sight direction, i.e., with various head tilts, rotations, and caps, but the line of sight is always on the same target object (e.g., the central camera lens).

[0058] Of course, the position of the center of rotation in the first frame may instead be determined from multiple images of the person's head captured with their head fixed while the target object is moving.

[0059] Preferably, step S4 and the step of acquiring the first dataset are performed individually using different image capture devices.

[0060] According to another embodiment compatible with the previous embodiment, the method further includes a calibration step prior to step b), during which a second device for capturing two-dimensional or three-dimensional images is calibrated. An image of a dedicated standard object is captured by the second image capture device during this calibration step.

[0061] Advantageously, the spatial position and / or orientation of the second image capture device is determined during step b). Furthermore, information relating to the spatial position and / or orientation of the person's head in a natural visual posture is determined therefrom.

[0062] According to another embodiment compatible with the previous embodiment, the same or at least the same type of image capture device may be used when determining the first and second sets of data.

[0063] For example, the image capture device is positioned on a table on the eye care practitioner's side so that the person has a wide, clear field of view when looking forward in a natural posture. The image capture device is tilted to view the person's face. The height of the image capture device is lower than the position of the person's head.

[0064] During step a), the person lowers their head and looks at the image capture device, moving their head over at least two shots or over a time series.

[0065] The advantage is that by lowering the head, the angle of the head and the field of view of the camera are close together, allowing for more accurate measurements.

[0066] This step makes it possible to measure and determine accurate 3D measurements of the face, the position of the center of rotation of the eyeballs on the head, and ultimately the position and / or shape of the eyeglass frames the person may be wearing.

[0067] In the second phase, the person is placed in a position with no postural constraints. The eye care practitioner performs a posture measurement in the optimal position for lens fitting measurement. This posture measurement can be performed using the same image capture device in a natural head position. Such a measurement may be performed in a different location at an extended period after the measurement in step a). The head posture measurement is a simple and rapid measurement.

[0068] Another object of the present invention relates to a method for determining fitting parameters for a pair of eyeglass lenses suitable for a person using an oriented 3D representation of the person's head obtained as described above by the method according to the present invention.

[0069] The fitting parameters can be determined using all the collected data.

[0070] For example, fitting parameters may include the center of ocular rotation distance, which is the Euclidean distance between the 3D positions of the center of ocular rotation of the right eye and the center of ocular rotation of the left eye.

[0071] The fitting parameters may also include the interpupillary distance, which can be determined using the nasal tip or nasal bridge point. The orthographic projection of the nasal tip or nasal bridge point is calculated according to the orientation of the head. The intersection of the line connecting the two centers of rotation of the eyeballs and the projection of the nasal tip or nasal bridge point provides a section for determining the interpupillary distance. The interpupillary distance is the Euclidean distance between this point and each center of rotation of the eyeballs.

[0072] Such fitting parameters can then be used to manufacture and provide a pair of eyeglass lenses suitable for a person.

[0073] Here, we will explain various methods for measuring a person's posture. [Examples]

[0074] Example 1 According to the first method, the eye care practitioner (ECP) administers "natural" posture measurements to determine the orientation of the rotation center of at least one of the person's eyes. The eye care practitioner, thanks to their expertise, can determine for themselves when the wearer is in good posture.

[0075] For example, in a static situation, a person can see a distant object directly in front of them, the ECP is in front of the person, and they fixate on the ECP's right and left eyes one after the other. In a dynamic situation, the person can walk or engage in social interaction.

[0076] A dedicated device, located away from the person's field of vision, measures head pose in relation to a previously made 3D representation of the head and determines the orientation of the ERC of at least one eye's center of rotation (ERC) in the global environment.

[0077] Posture can be recorded in a single shot or over a time series. Measurements may be made online or offline, and the eye care practitioner selects the best part or moment of the posture measurement to determine the orientation of the ERC.

[0078] Example 2 According to the second method, an eye care practitioner (ECP) marks, for example, an old pair of comfortable eyeglasses or a new pair of eyeglasses that have been manually fitted by the ECP, with a cross or line on the lens where the fitting provides a good posture.

[0079] Then, the natural posture is measured from the lens fitting.

[0080] For example, a person may wear glasses and stand in front of a mirror containing a device for capturing three-dimensional images, such as a 3D scanner. The person aligns their eyes with marks on the lenses, and the 3D scanner measures the head with the glasses on again to calculate the natural head posture from an initial 3D scan of the wearer's head.

[0081] Here, the head angle that provides the correct posture is recorded for that person and can be used to fit other eyeglasses.

[0082] Further fitting measurements may be taken by controlling the head posture in front of a 3D scanner or other device and asking the person to position their head correctly for fitting measurements.

[0083] Thanks to 3D scanning that includes the wearer's cyclops ERC, when the newly digitized glasses are remotely fitted to the wearer's head, their lens fitting measurements can be calculated based on pre-recorded good fitting postures.

[0084] Example 3 According to the third method illustrated in Figure 3, the person is wearing a virtual reality headset. The position and / or orientation of the virtual reality headset on the wearer's head can be determined within the room (real world) thanks to the 3D representation of the person's head.

[0085] The person immerses themselves in a virtual reality environment and performs one or more activities that can be personalized based on the wearer's lifestyle.

[0086] Head posture is measured by the placement of the helmet in the real world, thanks to the headset tracking system and the application of the headset / head.

[0087] The eye care practitioner can select a fitting position based on the results of the previous step.

[0088] As mentioned above, the head angle that provides the correct posture is recorded for that person and can be used to fit other eyeglasses.

[0089] Example 4 According to the fourth method, a person wears a head posture measurement device, such as a headband equipped with an inertial motion unit (IMU) sensor or a sensor directly attached to the head. The orientation of the wearer's head can be determined.

[0090] The eye care practitioner measures the lens fitting posture in one or more different tasks / activities.

[0091] IMU sensor data can be saved whenever desired by the eye care practitioner.

[0092] Several tasks / activities, such as reading, direct vision at distance, primary eye position, using a smartphone, using a computer, and office desk setup, may be measured.

[0093] The eye care practitioner can select a fitting position based on the results of the previous step.

[0094] Here, the head angle that provides the correct posture is recorded for that person and can be used to fit other eyeglasses.

[0095] The present invention, disclosed above, enables the determination of an oriented 3D representation of a person's head, taking into account the posture of the person's body and the posture of the person's head in a natural visual posture.

[0096] The present invention has been described above using multiple embodiments without limiting the overall concept of the invention.

[0097] Those skilled in the art will likely come up with many further modifications and variations if they refer to the above-described exemplary embodiments, which are merely illustrative and not intended to limit the scope of the invention as defined solely by the appended claims.

[0098] In the claims, the word “contains” does not exclude other elements or steps, and the indefinite article “a” or “an” does not exclude plurality. The mere fact that different dependent claims refer to different features does not mean that combinations of these features cannot be used advantageously. No reference numeral in the claims should be construed as limiting the scope of the invention.

Claims

1. A method implemented by computer means for determining an oriented 3D representation of a person's head in a natural visual posture, a) The steps of receiving a first dataset corresponding to a first 3D representation of the head of the person in a first frame and the position of the rotation center of at least one eye of the person in the first frame, b) The step of determining a second dataset corresponding to the orientation of the person's head in a natural visual posture in the second frame, c) A step of determining an oriented 3D representation of the person's head by orienting the first 3D representation of the person's head based on the orientation of the person's head in step b) within a common frame, Includes, The first dataset and the second dataset are performed individually and acquired by the same device for capturing three-dimensional images. The first three-dimensional image constitutes a first representation of the head of the person, The method wherein the second dataset includes features of the eyeglass frame, such as the mid-bridge point of the frame and / or boxed data and / or the lowest point.

2. The method according to claim 1, wherein the direction of the person's gaze is determined during step b).

3. The method according to claim 1 or 2, wherein the base point of the common frame is based on the rotation center of the at least one eye of the person.

4. The method according to any one of claims 1 to 3, wherein in step b), the person is placed in a real situation in which the posture of the person's head is not constrained.

5. The method according to any one of claims 1 to 4, wherein in step b), the person performs one or more habitual activities.

6. The method according to any one of claims 1 to 5, wherein in step b), the position of the rotation center of the person's eye is determined from the acquisition of one or more images of the person's head.

7. The method according to claim 6, wherein, in order to determine the direction of the person's gaze, the image of the person's pupil or corneal reflection is identified in the image captured in step b), the direction of the gaze is inferred from therewith, depending on the position of the rotation center in the reference second frame, the position is determined in the first dataset in the reference first frame.

8. The method according to claim 7, wherein the position in the second frame of a reference element that is targeted by the line of sight and belongs to the environment of the person is determined in order to determine the direction of the line of sight of the person.

9. The method according to any one of claims 1 to 8, step b) wherein the second dataset includes a series of second images of the person's head taken while the person fixes his gaze on a visual target, the position of the visual target with respect to a second image capture device is determined between each capture of the second image.

10. The method according to any one of claims 1 to 9, wherein the first dataset includes the positions of the rotation centers of both of the person's eyes in the first frame and the base point of the common frame corresponding to the person's monocular eye.

11. The method according to any one of claims 1 to 10, wherein the first dataset is acquired by a first device for capturing three-dimensional images, and the three-dimensional images then constitute the first 3D representation of the head of the person.

12. In obtaining the first dataset, the position of the rotation center in the reference first frame is, - A step in which at least two images of the person's head are captured using an image capture device, wherein the images of the person's head's visual posture with respect to the image capture device are different, and in the images the person has fixed his gaze on a targeting point at a predetermined position, - A step of determining the gaze direction of the person corresponding to each of the two images; - A step of inferring the position of the rotation center of at least one eye of the person from the gaze direction, The method according to claim 11, as determined by...

13. A method for determining fitting parameters for a pair of eyeglass lenses suitable for a person using the oriented 3D representation of the person's head obtained by the method according to any one of claims 1 to 12.

14. A computer program product comprising one or more stored instruction sequences that are accessible to a processor and, when executed by the processor, cause the processor to perform the steps in the method according to any one of claims 1 to 13.

Images