Simultaneous execution of visual line tracking calibration and visual acuity test
Patent Information
- Application Number
- JP2024116632
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2023-08-25
- Filing Date
- 2024-07-22
- Publication Date
- 2025-03-07
Smart Images
Figure 2025031571000001_ABST
Abstract
Description
[Technical field]
[0001] The present disclosure relates to an optical instrument incorporating both eye-tracking calibration and vision testing functions. The present disclosure also relates to a system incorporating both eye-tracking calibration and vision testing functions. The present disclosure also relates to a method of incorporating both eye-tracking calibration and vision testing functions.
[0002] Traditionally, eye exams have been used to help maintain eye health and overall health. Traditionally, eye exams involve displaying a visual target on a standardized eye chart placed at a predetermined distance from the subject to determine the size of the visual target that the subject can read. Regular eye exams can be used to identify refractive errors in visual acuity, determine if the patient's eye is suffering from any eye disease, assist in the development of vision in patients suffering from unidentified eye diseases, monitor the eye health of patients suffering from chronic diseases, and update eye prescriptions when the optical properties of the eye change due to any eye disease. Refractive errors occur when a user suffers from myopia, hyperopia, astigmatism, etc. Examples of eye conditions include glaucoma, cataracts, diabetic retinopathy, etc. Examples of chronic diseases that may affect eye health include diabetes and high blood pressure.
[0003] However, existing techniques and devices for performing eye tests have several limitations associated with them. Traditionally, eye tests to measure a user's visual acuity are performed by displaying optotypes of various sizes on a screen (e.g., a standardized eye chart) and having the user read the optotypes with one eye. This test is then repeated by switching to the other eye, which is time consuming and tedious for the user. Eye tests are also used to screen the eye motility of a user's eye. Traditionally, this has been done by tracing moving calibration dots on a screen using an eye tracking means to determine the health of the ocular muscles of the user's eye. However, the screening of eye motility is performed separately for the user's eye, i.e., separately from the screening of the user's visual acuity. Furthermore, existing techniques and devices do not take into account how well the user can see the calibration dots, which results in erroneous rendering of the user's eye when using conventional rendering algorithms, thereby generating erroneous calibration data for the eye tracking means.
[0004] In light of these considerations, a need exists to overcome the aforementioned deficiencies.
[0005] It is an object of the present disclosure to provide an optical instrument, system and method for incorporating eye tracking calibration and vision testing together to facilitate an accurate assessment of a user's eyes. By nature, performing an accurate assessment of a user's eyes requires repetitive operations. The object of the present disclosure is achieved by an optical instrument, system and method for vision testing, which incorporates simultaneous eye tracking calibration as defined in the attached independent claims. Advantageous features are set out in the attached dependent claims. Throughout the description and claims of this specification, the words "comprise," "include," "have," and the like mean including an element, but not only the element. They do not exclude the presence of other components, items, numbers, or steps not expressly disclosed. Furthermore, unless the context dictates otherwise, the singular encompasses the plural. In particular, where the original text uses an indefinite article, the specification contemplates the plural as well as the singular, unless the context requires otherwise. [Brief description of the drawings]
[0006] [Figure 1] FIG. 1 is a block diagram illustrating the architecture of an optical instrument and a system including the optical instrument according to one embodiment of the present disclosure. [Diagram 2] 1 is a schematic diagram of an optical instrument according to one embodiment of the present disclosure. [Diagram 3] 1 illustrates an environment in which a system incorporating simultaneous eye tracking calibration and vision testing capabilities is used, according to one embodiment of the present disclosure. [Figure 4A] 1 illustrates an exemplary visual target and an exemplary input means, according to one embodiment of the present disclosure. [Figure 4B] 1 shows the flow of an exemplary visual acuity test of a user's eyes, performed using an exemplary optotype and an exemplary input means. [Diagram 5] 1 is a flowchart illustrating steps of a method for incorporating eye tracking calibration and vision testing together, according to one embodiment of the present disclosure. Detailed Description of the Embodiments
[0007] The following detailed description illustrates embodiments of the present disclosure and how they may be practiced. Although several forms for carrying out the present disclosure have been disclosed, those skilled in the art will recognize that other forms for carrying out the present disclosure are possible.
[0008] According to a first aspect, the present disclosure provides an optical apparatus comprising: Eye-tracking measures and; At least one processor; wherein the processor: (a) displaying on a screen an image for a user representing at least one visual indicia; (b) obtaining a predefined user input indicating whether a user was able to clearly see a user indicia depicted in the user image while using the optical instrument; (c) determining whether the user was able to clearly see the user indicia depicted in the user image based on the predetermined user input; (d) if it is determined that the user was able to clearly see the user visual indicator, collecting information indicative of at least one of: a size of the user visual indicator that was clearly seen by the user, a location of the user visual indicator and / or visual indicator features in the user image, and an appearance of the user's eye features when the user was able to clearly see the user visual indicator displayed at the location; (e) if it is determined that the user was unable to clearly see the user visual target, collecting information indicative of at least one of a size of the user visual target that the user was unable to clearly see, a position of the user visual target and / or visual target feature in the user image, and an appearance of the user's eye features when the user was unable to clearly see the user visual target displayed at the position, and repeating steps (a) to (e) using a next image representing at least one next visual target that is larger in size than the user visual target represented in the user image; (f) processing the collected information to determine one or more optical powers specific to the user and simultaneously calibrate an eye tracking means; The present invention is configured to perform the following steps:
[0009] The optical instrument described above incorporates simultaneous calibration of the eye tracking means and visual acuity testing. Here, visual targets of various sizes at various positions are displayed on a screen, information is collected about whether the user can clearly see the user visual target, and the user's eye movements when looking at the user visual target at various positions are determined. This information is used to dynamically determine one or more refractive powers specific to the user, while dynamically calibrating the eye tracking means and establishing an accurate mapping between the user's eye movements and the corresponding positions of the visual target. This makes the calibration process simple and efficient. The one or more refractive powers can then be used to improve the user's visual experience by generating at least one of the one or more refractive powers with an active optical element of the optical instrument (or another optical instrument). Advantageously, the active optical element can be easily implemented in various types of optical instruments, such as, for example, glasses, sunglasses, smart glasses, head-mounted displays, etc. As a synergistic effect of the above-mentioned features, visual acuity testing and calibration of the eye tracking means are performed simultaneously, thereby reducing the processing resources, time, and cost required. When the user can see the user's optotype clearly or when the user cannot see the user's optotype clearly, information about the user's eyes when seeing the user's optotype and the user's eyes when seeing the user's optotype are collected, and the performance of the eyes is analyzed. In addition, by changing the size of the optotype when repeating the visual acuity test, the degree of the user's visual acuity can be known. This allows one or more refractive powers to be determined accurately and precisely.
[0010] According to a second aspect, the present disclosure provides a system, comprising: an optical instrument relating to the first way of perceiving; A screen used to display images; At least one input means used to obtain user input; Equipped with.
[0011] The system incorporates the functionality of simultaneously calibrating the eye tracking means and testing the visual acuity, where an image representing a visual target is displayed on a screen and the user provides input using said input means based on whether the user visual target can be clearly seen. As a synergy of the above features, the data can be used to simultaneously test the visual acuity and calibrate the eye tracking means while using the optical instrument. The screen and at least one input means are well known in the art and do not require additional time or cost to construct.
[0012] According to a third aspect, the present disclosure provides a method implemented by an optical instrument including an eye-tracking means and at least one processor, the method comprising: (a) displaying on a screen an image for a user representing at least one visual indicia; (b) obtaining a predefined user input indicating whether a user was able to clearly see a user target depicted in the user image while using an optical instrument according to a first perspective; (c) determining whether the user was able to clearly see the user indicia depicted in the user image based on the predetermined user input; (d) if it is determined that the user was able to clearly see the user visual indicator, collecting information indicative of at least one of: a size of the user visual indicator that was clearly seen by the user, a location of the user visual indicator and / or visual indicator features in the user image, and an appearance of the user's eye features when the user was able to clearly see the user visual indicator displayed at the location; (e) if it is determined that the user was unable to clearly see the user's visual target, collect information indicative of at least one of a size of the user visual indicator that the user could not clearly see, a location of the user visual indicator and / or visual indicator feature in the user image, and an appearance of the user's eye features when the user could not clearly see the user visual indicator displayed at the location; repeating steps (a)-(e) using a next image representing at least one next indicia larger in size than the indicia represented in the user image; (f) processing the collected information to determine one or more optical powers specific to said user and simultaneously calibrate an eye tracking means; Includes.
[0013] The method described above simultaneously calibrates the eye tracking means and tests the visual acuity. Displaying different sizes of visual targets at different positions on the screen allows information to be collected about the size of the visual target that the user can see and also allows determining how the user's eyes flexibly move when viewing the visual target at different positions. This information can be used to calibrate the eye tracking means for each of the different visual targets displayed to the user. The synergistic effect of the above features is that the visual acuity test and the calibration of the eye tracking means are performed simultaneously, reducing the processing resources, time, and cost required. Information about the user's eye when viewing the user's visual target and the user's eye when the user can clearly view the user's visual target or when the user cannot clearly view the user's visual target is collected and the performance of the eye is analyzed. By displaying different sizes of visual targets to the user and repeating the steps of the method, the degree of the user's visual acuity can be determined. This allows one or more refractive powers to be accurately and precisely determined. Thus, the method described herein is simple, stable, fast, reliable, and easy to implement.
[0014] Throughout this disclosure, the term "optical instrument" refers to equipment used by a user to perform the vision test together with the aforementioned calibration. In some implementations, the optical instrument may be a wearable instrument that is worn over the user's eye. Examples of such optical instruments include, but are not limited to, eyeglasses, sunglasses, smart glasses, head-mounted displays, etc. In other implementations, the optical instrument may be a non-wearable instrument. As an example, in such an implementation, the optical instrument may be placed on a table that is placed in front of the user.
[0015] In implementations in which the optical device is wearable, the optical device may comprise an active optical element for each square. Throughout this disclosure, the term "active optical element" refers to an optical element that has one or more controllable optical parameters. These optical parameters may be, for example, actively (i.e., dynamically) adjustable.
[0016] Throughout this disclosure, the term "eye tracking means" refers to special equipment employed to at least one of detect the direction of gaze of a user of the optical instrument, track the direction of gaze of a user of the optical instrument, and monitor the characteristics of the user's eye when the user uses the optical instrument. The characteristics of the user's eye may include at least one of the following: pupil shape of the user's eye, pupil size, corneal reflection of light emanating from the real environment onto the surface of the user's eye, relative position of the pupil to the corneal reflection, and relative position of the pupil to the corner of the user's eye. Eye tracking is performed when an operational optical instrument is being used by the user.
[0017] In some embodiments, the eye tracking means is implemented by at least one of a contact lens with a sensor, a camera monitoring the user's eye function. Such eye tracking means are well known in the art. The eye tracking means is configured to collect eye tracking data and transmit the eye tracking data to at least one processor. It will be appreciated that the eye tracking data is collected repeatedly during operation of the optical instrument by the eye tracking means, since the user's gaze and eye characteristics keep changing while using the optical instrument. The updated eye tracking data (data indicative of the gaze direction and the user's eye characteristics) allows adaptively controlling the active optical elements of the optical instrument in a precise manner to efficiently check the user's vision. In some embodiments, if the eye tracking means is implemented as a camera, the eye tracking data is in the form of an image of the user's eye. In some embodiments, if the eye tracking means is implemented by a contact lens with a sensor, the eye tracking data is sensor data collected from the sensor.
[0018] It will be appreciated that the at least one processor is communicatively coupled to the eye tracking means and, in some embodiments, the active optical element. The at least one processor may be implemented as any one of a microprocessor, a microcontroller, or a controller. As an example, the at least one processor may be implemented as an Application Specific Integrated Circuit (ASIC) chip or a Reduced Instruction Set Computer (RISC) chip.
[0019] The user image is a visual representation of at least one visual target at a particular location. In other words, the user image includes shape information and / or color information of at least one visual target, and in some embodiments includes other attributes related to the at least one visual target (e.g., depth information, luminance information, transparency information, etc.). In some embodiments, the user image is a visible light image, where the visible light image is constructed using a red-green-blue (RGB) spectrum. In some embodiments, the user image is a high quality (i.e., high resolution) image. The user image may be in JPEG format, JPG format, PNG format, etc. The user image is displayed to a user to assess the ability to accurately recognize and identify at least one visual target.
[0020] Throughout this disclosure, the term "optotype" refers to a standardized number, a standardized alphabetic character, and / or a standardized symbol, having a specific size and a specific location in the user image used to test the user's visual acuity, allowing for consistency even when the visual acuity test is performed by different medical professionals, hospitals, and / or optical instruments. Here, a specific size of at least one optotype is used to determine the user's visual acuity, and a specific location of at least one optotype is used to evaluate the ability of the user's eyes to track the at least one optotype and to calibrate the eye tracking means. Here, when the screen is at a predefined distance from the user (as described below), the specific size of the at least one optotype corresponds to a specific level of visual detail of the at least one optotype that the user can accurately identify. This specific level of visual detail can be expressed in the form of a ratio between a predefined distance and a distance at which another user with normal visual acuity (i.e., no eye condition problems) can clearly see the at least one optotype displayed on the screen. Furthermore, the specific location of the at least one visual symbol or the specific location of the visual symbol feature corresponds to a relative location with respect to at least one corner of the screen on which the at least one visual symbol is displayed and / or a relative location of the visual symbol feature with respect to the entirety of the at least one visual symbol. Here, the term "visual symbol feature" refers to a discernible gap and / or a discernible mark present in the visual symbol provided to the user that the user must clearly see in order to determine the user's visual acuity. It will be understood that the at least one visual symbol has a high contrast with the background to enhance visibility. Examples of the at least one visual symbol that is a standardized number include, but are not limited to, at least one digit number, a decimal notation, and a sequence of numbers. Examples of the at least one visual symbol that is a standardized alphabetic character include, but are not limited to, E, C, H, O, N, D, A, and T. Examples of the at least one visual symbol that is a standardized symbol include, but are not limited to, a ring, a circle, a square, a cross, a triangle, a diamond, and a star.The at least one visual indicia may be dynamically selected by a medical professional and / or a user.
[0021] In some embodiments, the at least one optotype comprises at least one of the following: Landolt C optotype, Snellen optotype, Sloan optotype, Early Treatment Diabetic Retinopathy Study (ETDRS) optotype, HOTV letter, Allen figure, alphabetic character (e.g., standardized alphabetic character), number (e.g., standardized number), symbol (e.g., standardized symbol), design, freeform. The Landolt C optotype has a ring with optotype features (i.e., predefined gaps) at specific positions (e.g., top position, bottom position, left position, right position, etc.). Advantageously, the Landolt C optotype can be used when a user is unfamiliar with numbers and / or alphabetic characters, when a user cannot distinguish numbers and / or alphabetic characters, and when an individual has a cognitive or visual impairment. The Snellen optotype, ETDRS optotype, and HOTV optotype have standardized numbers and / or standardized alphabetic characters in uppercase. These standardized numbers and / or standardized alphabetic characters are arranged in rows and columns, decreasing in size from top to bottom. The ETDRS optotype consists of a specific number of strokes and standardized spacing between standardized numbers and / or standardized alphabetic letters. The ETDRS optotype is displayed in individuals with problematic eye conditions, such as diabetic retinopathy. The HOTV letters consist only of the standardized alphabetic letters H, O, T, and V. The Allen figures consist of standardized shapes designed to be easily recognized by users who cannot read or recognize standardized numbers and / or standardized alphabetic letters. The designs have at least one of standardized alphabetic letters, standardized numbers, standardized symbols, and decorative patterns, and can be created by a medical professional based on the user's medical history or can be pre-designed. The free shapes are unique and distinct patterns. Beneficially, the free shapes are used when the user is unable to recognize the standardized alphabetic letters, standardized numbers, and standardized symbols.
[0022] The at least one processor is communicatively coupled to a screen. Here, the term "screen" refers to a physical surface (e.g., a wall, a projection screen, etc.) or a monitor (e.g., a television monitor, a computer monitor, a smartphone display, etc.) on which an image for a user is displayed. In some embodiments, the screen is integrated with a display device, and the at least one processor is communicatively coupled to the display device. In such embodiments, the screen is directly coupled to the hardware and / or software elements of the display device. Examples of such display devices include, but are not limited to, mobile phones, tablets, phablets, computers, laptops, personal digital assistants, smart watches, and televisions. In other implementations, the screen is separate from the display device, and the at least one processor is communicatively coupled to the display device in a wired and / or wireless manner. In such implementations, the screen is indirectly coupled to the hardware and software components of the display device. Examples of such display devices include, but are not limited to, projectors, displays of external monitors, etc.
[0023] In some embodiments, the user views the screen from a predetermined distance (D1) ranging from 25 cm to 750 cm. The predetermined distance (D1) ranges from 25, 50, 65, 85, 160, 320, 480, 640, or 700 cm to 40, 55, 75, 95, 190, 380, 570, 690, or 750 cm. The technical advantage of placing the screen at a predetermined distance from the user is that the measurements from performing a visual acuity test on the user can be standardized between different users and different sessions of visual acuity checks. In one embodiment, to determine the visual acuity of the user for near vision, the predetermined distance (D1) ranges from 35 cm to 40 cm. In another example, to determine the visual acuity of intermediate vision, the predetermined distance (D1) ranges from 50 cm to 75 cm.
[0024] Throughout this disclosure, the term "predetermined user input" refers to a response and / or feedback provided by a user when the user views a user image and clearly and accurately perceives a user visual target during use of the optical instrument. The "predetermined user input" is provided via at least one input means and received by at least one processor, as described below. After being received by at least one processor, in some embodiments, the predetermined user input is provided by the user in the form of a voice message, a text message, haptic feedback, touch feedback, a gesture, a movement, and the like.
[0025] In some embodiments, the predetermined user input is obtained via at least one input means communicatively coupled to the at least one processor. In some embodiments, the at least one input means is communicatively coupled to the at least one processor in a wired and / or wireless manner. The at least one input means may comprise at least one of a touch sensor, a button, a keyboard, a mouse, an XR controller, an audio sensor, a remote controller, a physical slider, a microphone, and a recorder. As an example, the input means may be implemented as a touch-sensitive surface of a smartphone. In this case, the processor of the smartphone is configured to execute a software application for providing a user with a user interface in which the user can provide the predetermined user input. The at least one input means provides flexibility to a user using the optical instrument by allowing the user to provide the predetermined input when the user can clearly see the user target. A technical effect of communicatively coupling the at least one input means to the at least one processor is that the predetermined user input is accurately transmitted to the at least one processor for analysis, interpretation, and / or execution of an action.
[0026] In a first example, when the optical instrument is used, a user image may be displayed on a screen. The user image may represent a Landolt C optotype. The user may view the screen from a predetermined distance of 300 cm. The user may provide input using at least one input means, where the at least one input means may be a remote controller including four buttons, namely an up direction button, a down direction button, a left direction button, and a right direction button, which may correspond to optotype features located at the top, bottom, left, and right positions, respectively, in the Landolt C optotype. Suppose in the user image that the optotype is located at the left position in the Landolt C optotype. In one example, the user cannot clearly see the optotype at the right position in the Landolt C optotype, and a predetermined user input may be provided by pressing the up direction button. Alternatively, in another example, the user cannot clearly see the optotype at the left position in the Landolt C optotype, and the user may provide a predetermined user input by pressing the left direction button on the remote controller.
[0027]
[0004] In some embodiments, the optical instrument further comprises: · An active optical element provided per eye; a frame holding the active optical element for each eye; at least one input means attached to a temple of said frame for use by a user to provide a predetermined user input; Equipped with.
[0028] The frame is designed such that the active optical element is fixedly positioned relative to the frame. In an example where the optical device is implemented as glasses, the frame may hold two active optical elements, a first active optical element for a first eye of a user and a second active optical element for a second eye of a user. The material of the frame may be plastic, metal, polymer, etc. The frame may be lightweight, ergonomically designed, and easy to use.
[0029] The at least one input means attached to the temple of the frame provides flexibility to the user of the optical instrument by providing a predetermined user input by a physical slider or button attached to the temple of the frame. The technical advantage of attaching the at least one input means to the temple of the frame is that it frees the user's hands and allows the user to conveniently interact with the surrounding environment. For example, the at least one input means may be two physical sliders attached to the temple of the frame. Here, a first physical slider may be attached to the first temple of the frame and a second physical slider may be attached to the second temple of the frame. The first physical slider may be used when the user can clearly see the user's visual target, and the second physical slider may be used when the user cannot clearly see the user's visual target.
[0030] The at least one processor is configured to process the predefined user input using at least one processing algorithm based on a form of the predefined user input obtained from the user. The technical effect is that the predefined user input is validated, i.e., it is determined whether the user correctly identified the user indicia. Examples of the at least one processing algorithm may include, but are not limited to, a tokenization algorithm, a named entity recognition, a text classification algorithm, a signal processing algorithm, a natural language processing, a support vector machine, and a deep neural network.
[0031] If the at least one processor is configured to determine that the user can clearly see the user image, information is collected about the size of the user's visual target depicted in the user image at that time. The technical advantage of collecting information is that it helps to evaluate the level of the user's visual acuity and also provides a quantitative measure of the user's visual acuity. Here, the visual acuity is quantitatively measured based on the size of the user's visual target, and is measured in the form of a ratio between a predetermined distance at which the user's visual target is presented and the size of the user's visual target that the user can clearly see. In some embodiments, the size of the user's visual target includes at least one of the length, height, width, and spacing of the user's visual target. The size of the user's visual target corresponds to the acuity of the user's eyes and allows for the evaluation of visual acuity at various visual acuity levels. Various levels of visual acuity testing are achieved by gradually increasing or decreasing the size of the visual target, allowing for an accurate evaluation of visual acuity. This accurate evaluation is useful in distinguishing small changes in the user's visual acuity at various visual acuity levels.
[0032] When the user can clearly see the user visual target depicted in the user image, the appearance of the user's eye features changes relative to the user visual target. Such features may include at least one of the following: a position of the user's eye pupil, a shape of the user's eye pupil, tension of the user's eyelid, an openness of the user's eyes, an uplifted eyebrow, and a narrowing of the user's eyes. Information indicative of the user's eye features is collected using eye-tracking means when the optical instrument is used by the user. Such eye-tracking means have been described in detail above. The eye-tracking means is configured to collect information indicative of the appearance of the user's eye features and to transmit the information to the at least one processor. It will be appreciated that, since the appearance of the user's eye features keeps changing while the user uses the optical instrument, said information is repeatedly collected by the eye-tracking means during operation of the optical instrument.
[0033] The information collected when the user cannot see the user image clearly is obviously different from the information collected when the user can see the user image clearly. The technical advantage of collecting said information is that the optical instrument, with the help of the information collected by the eye tracking means, can then recognize when the user cannot see the user target clearly and can remind the user to repeat the visual acuity test accordingly. Here, said information is collected in a manner similar to that described above.
[0034] If the user cannot clearly see the user image, the vision test continues and the at least one processor is configured to display a next image to the user, where the at least one next object may be at least one of a different size, different location on the screen, and different object characteristics from the previously provided object. The next image is a visual representation of the at least one next object at a particular location. Like the initially provided image, the next image is also a visible light image. The next image is a high quality (i.e., high resolution) image. The next image may be in JPEG format, JPG format, PNG format, etc.
[0035] wherein the technical effect of displaying at least one next visual indicator larger than a previously provided indicator is to determine a maximum size of the visual indicator that can be clearly viewed by a user, where the size of the at least one next visual indicator is increased from the size of the initially provided visual indicator by one of: dynamically increasing the size of the visual indicator using an adaptive algorithm; manually increasing the visual indicator size by a person (e.g., a medical professional) who determines the user's visual acuity; or incrementally increasing the visual indicator size by at least one processor using a function or rule, where the adaptive algorithm is used to determine an appropriate size of the at least one next visual indicator in a next image based on a given user input.
[0036] In some embodiments, the at least one processor is configured to repeat steps (a) to (e) using another subsequent image if the user determines that the user indicia is clearly visible. The another subsequent image represents at least one another subsequent indicia smaller in size than the indicia represented in the previous image. The technical effect of displaying at least one subsequent indicia smaller than the previously provided indication is to determine the minimum size of the indicia that can be clearly visible by the user. The size of the at least one another subsequent indicia is made smaller than the size of the previously provided indicia in a manner similar to the above-described technique for increasing the size of the provided indicia.
[0037] In some implementations, the one or more determined refractive powers consist of a single refractive power. In other implementations, the one or more determined refractive powers consist of multiple refractive powers. Multiple refractive powers can be generated simultaneously in different regions of the active optical element, for example to enable focusing (using negative or zero refractive power, depending on the user's needs) on at least one other subsequent target smaller in size than the provided target, and to enable focusing (using zero or positive refractive power, depending on the user's needs) on at least one subsequent target larger in size than the provided target. In some embodiments, the one or more refractive powers in this respect are at least one of positive, negative, and zero refractive powers. Here, the positive refractive power is used to read or focus on at least one subsequent target larger than the provided target. The negative refractive power is used to read or focus on at least one other subsequent target smaller than the provided target. Zero refractive power is used when the user is not myopic and needs to focus on distant objects, or when the user is not hyperopic or farsighted and needs to focus on close objects.
[0038] At the same time, the at least one processor is configured to calibrate the eye tracking means by employing a calibration technique based on the processed collected information. In particular, calibration parameters of the eye tracking means are estimated using the collected information. These calibration parameters are then used to correct distortions of the eye tracking means, where the calibration parameters describe a mathematical relationship between the collected information and the eye tracking means. In some embodiments, when determining the calibration parameters, the at least one processor uses various functions, rules, and / or formulas. Such functions, rules, and formulas are well known in the art.
[0039]
[0005] In some embodiments, the optical instrument comprises an active optical element for each eye, and the at least one processor further comprises: Controlling an active optical element to generate at least one of the one or more optical powers; Repeat steps (a) to (e), Processing the newly collected information to further calibrate the eye-tracking means; It is configured as follows.
[0040] The technical effect of further calibrating the eye tracking means in this way is that it is ensured that the eye tracking means takes into account variations in the optical properties of the user's eye based on at least one of the generated refractive powers. Furthermore, by repeating steps (a) to (e), the accuracy of the calibration of the eye tracking means can be verified. In this regard, the at least one processor is configured to generate a control signal for controlling the active optical element based on how clearly the user can see the user image representing the user target. The control signal can be a voltage signal and / or a current signal. It will be understood that the driving signal of each active optical element can be different. The at least one processor generates at least one of the determined one or more refractive powers when processing the collected information. In order to verify whether the generation of a given refractive power (i.e. at least one of the one or more refractive powers) is compatible with the calibration of the eye tracking means and vice versa, the visual acuity test is repeated by displaying the user image to the user. Such a match is determined to be a match if the user can clearly see the user target in the user image and does not experience difficulty or blurring when viewing the user target. Continuing with reference to the first embodiment, if the user cannot clearly see the user target in the user image, the at least one processor may be configured to control the active optical element to generate a refractive power of +1.5 diopters. The visual acuity test may be repeated, which may generate new information that may be collected. This new information may be used to calibrate the eye tracking means. The visual acuity test may be further repeated to ensure that the generated refractive power of +1.5 diopters is compatible with the calibration of the eye tracking means.
[0041] In some embodiments, the at least one processor is configured to process newly collected information, where if the user was able to clearly see the given visual indicator, the newly collected information is indicative of at least one of: a size of the visual indicator that the user was able to clearly see, a location of the visual indicator and / or visual indicator features in an image provided to the user, and an appearance of the user's eye features when the user was able to clearly see the visual indicator displayed at the location, or, if the user was not able to clearly see the given visual indicator, the newly collected information is indicative of at least one of: a size of the visual indicator that the user was not able to clearly see, a location of the visual indicator and / or visual indicator features in an image provided to the user, and an appearance of the user's eye features when the user was not able to clearly see the visual indicator displayed at the location.
[0042] The at least one processor may then be configured to further calibrate the eye tracking means by using a calibration technique with the processed newly collected information, in particular further calibration parameters of the eye tracking means are estimated using the newly collected information, and these further calibration parameters are then used to further correct for distortions of the eye tracking means.
[0043] In some embodiments, the at least one processor is configured to adjust at least one of the one or more optical powers when it is determined that the user cannot see the given visual target clearly. The technical effect of adjusting at least one of the one or more optical powers is to ensure that the user's vision is corrected in a precise manner, so that the user can experience clearer and sharper vision and / or experience reduced eye strain. Even after generating the optical power as described above, the user may have difficulty seeing the given visual target. However, the given refractive power may be close to the refractive power (at least one of the one or more refractive powers) required for the user to see the given visual target clearly. Therefore, if more refractive power than the given refractive power is required, the absolute value of the refractive power is increased. Alternatively, if less refractive power than the given refractive power is required, the absolute value of the refractive power is decreased. Continuing with reference to the first embodiment, when the user uses an optical device with a refractive power of +1.5 diopters, steps (a) through (e) are repeated and the at least one processor may determine that the user still cannot see the next target clearly, and may adjust the refractive power by + / -0.5 diopters, for example by increasing the refractive power of the active optical element to +2.0 diopters.
[0044]
[0006] In some embodiments, the optical instrument comprises an active optical element for each eye, and the at least one processor: processing the eye tracking data collected by said eye tracking means to determine at least one of a gaze direction of a user's eyes, a point of gaze of said user; determining an optical depth at which the user is gazing based on at least one of a convergence angle of gaze directions of the user's eyes, an interpupillary distance of the user's eyes, and a point of gaze of the user; selecting at least one of the one or more optical powers produced by the active optical element based on an optical depth at which the user is gazing; It is configured as follows.
[0045] In this regard, the term "gaze direction" refers to the direction in which the user's eyes are gazing. The "gaze direction" may be represented by a gaze vector. A "gaze point" is a point in the real-world environment where the user's gaze is focused. A "gaze point" is a user's point of interest in the real-world environment. In some embodiments, the gaze point is determined by mapping the gaze directions of the user's two eyes to a corresponding point in the real-world environment where these gaze directions converge. In some embodiments, when processing the gaze tracking data, the at least one processor is configured to use at least one of an image processing algorithm, a feature extraction algorithm, and a data processing algorithm. Other suitable algorithms may also be employed.
[0046] The term "optical depth" refers to the optical distance between the point of interest and the optical instrument used by the user. In some embodiments, the at least one processor is configured to determine the optical depth when the gaze directions of the two eyes of the user converge on a particular part of the screen. Thus, the "convergence angle" is the angle subtracted by the gaze direction of the user's eyes when gazing at any part of the screen. In some embodiments, the at least one processor is configured to use triangulation to calculate the interpupillary distance of the user, where the interpupillary distance of the user's eyes may be the average interpupillary distance. It will be appreciated that if the convergence angle of the gaze direction of the user's eyes, the interpupillary distance of the user's eyes, and the user's point of gaze are already known to the at least one processor, the optical depth at which the user is gazing may be easily determined by the at least one processor, for example, using at least one mathematical technique. The at least one mathematical technique may be at least one of a triangulation technique, a geometry-based technique, and a trigonometry-based technique. Determining line-of-sight convergence distance based on line-of-sight direction convergence is well known in the art.
[0047] Here, at least one of the one or more refractive powers can be negative, zero or positive, as already mentioned above. Moreover, in this respect, the one or more refractive powers are different refractive powers, each corresponding to a different optical depth. At least one of the one or more refractive powers is selected from among the one or more refractive powers based on the optical depth at which the user is looking. The one or more refractive powers can be defined for the user for each of the different optical depths. At least one of the one or more refractive powers selected for one eye can be the same or different from at least one of the one or more refractive powers selected for the other eye of the user.
[0048] The present disclosure also relates to the second aspect as described above. The various embodiments and variants disclosed above with respect to the first aspect as described above also apply mutatis mutandis to the second aspect.
[0049] In some embodiments, the screen is either a display of a user's or medical professional's device or a projection surface of a projector. Here, the term "device" refers to a communication device operable to display a user-use image representing a user-use visual target when the optical instrument is in use. The technical effect of the screen is to enable a user-use image representing a user-use visual target to be dynamically presented to the user. Such dynamic presentation allows the user or medical professional to resize, reposition and / or reorient the user-use visual target during a visual acuity test. If the screen is a display of the user's device, it means that the device is owned and used by the user. In this regard, the user optionally interacts with the display to view the user-use image and provides a predefined user input to communicate whether the user was able to clearly see the user-use visual target depicted in the image. If the screen is a display of a medical professional, it means that the device belongs to the medical professional. Examples of such medical professionals include, but are not limited to, ophthalmologists, ophthalmologists, optometrists, etc. The device is optionally used to display a visual acuity test of the user, a diagnostic test, or medical information related to the user's visual acuity. Examples of devices include, but are not limited to, laptops, computers, tablets, phablets, mobile phones, smartphones, smart watches, etc. Examples of displays include, but are not limited to, television displays, monitors, laptop screens, smart watch displays, etc.
[0050] The projector is used to project a user image representing a user target onto a projection surface using light rays emitted from a light source. Here, the projection surface is used to display the user image representing the user target when the optical device is in use. The material of the projection surface must be such that it can reflect or diffuse the light rays to accurately form a predetermined image.
[0051] In a second example, there may be an exemplary environment in which a system for incorporating eye tracking calibration and vision testing functions together is used. The environment may include a system and a user. In this environment, the system is used to perform a user's vision check and simultaneously perform eye tracking calibration. The system may include an optical instrument (e.g., glasses, etc.), a screen (e.g., a television display, etc.), and at least one input means (e.g., a remote controller, etc.). The user may view the television display from a predetermined distance D1, e.g., 300 cm, when performing the test. During the vision test, a user image may be displayed on the television display, and the user image may represent a user optotype (e.g., a symbol, etc.). A predetermined user input may be provided by the user using the remote controller. The predetermined user input may indicate whether the user can clearly see a symbol that may be represented in an image provided to the user while using the glasses. Then, based on the provided user input, it may be determined whether the user can clearly see the symbol that may be represented in the user image.
[0052] The present disclosure also relates to a third aspect as described above. The various embodiments and variants disclosed above with respect to the first aspect also apply mutatis mutandis to the third aspect.
[0053] In some embodiments, if it is determined that the user can clearly see the user indicia, steps (a) through (e) are repeated using another subsequent image, where the another subsequent image depicts at least one another subsequent indicia that is smaller in size than the indicia depicted in the previous image.
[0054] In some embodiments, the method further comprises: controlling an active optical element of the optical instrument to generate at least one of the one or more optical powers; repeating steps (a) to (e); · Processing the newly collected information to further calibrate the eye-tracking means; Includes.
[0055]
[0004] In some embodiments, the method includes adjusting at least one of the one or more optical powers if it is determined that the user cannot clearly see the given indicia.
[0056] In some embodiments, the method further comprises: processing the eye tracking data collected by the eye tracking means to determine at least one of a gaze direction of the user's eyes, a point of gaze of the user; determining an optical depth at which the user is gazing based on at least one of a convergence angle of gaze directions of the user's eyes, an interpupillary distance of the user's eyes, and a point of gaze of the user; selecting at least one of one or more optical powers produced by the active optical element of the optical instrument based on an optical depth at which the user is gazing; Includes. [Detailed description of the drawing]
[0057] Referring to Fig. 1, a block diagram of an architecture of an optical instrument 102 and a system 100 including the optical instrument 102 is shown, according to one embodiment of the present disclosure. The optical instrument 102 comprises an eye tracking means 108 and at least one processor, depicted as processor 110. In some embodiments, the optical instrument 102 also comprises an active optical element 104 for each eye 106. The processor 110 is communicatively coupled to the active optical element 104 and the eye tracking means 108. The processor 110 is configured to perform various processes as described with respect to the first perspective above.
[0058] The system 100 also comprises a screen 112 separate from the optical instrument 102. In some embodiments, when the optical instrument 102 is in use, the screen 112 is viewed by a user from a predetermined distance D1. The system 100 further comprises at least one input means, depicted as input means 114. The processor 110 is communicatively coupled to the input means 114.
[0059] 1 is merely an example and should not unduly limit the scope of the claims of this application. Those skilled in the art will recognize many variations, alternatives, and modifications of the embodiments of the present disclosure. For example, the eye tracking means 108 can be implemented as a separate eye tracking means for each eye.
[0060] Referring to FIG. 2, a schematic diagram of an optical device 200 according to an embodiment of the present disclosure is illustrated. As illustrated, the optical device 200 is implemented as glasses. The optical device 200 comprises an eye tracking means 204, at least one processor (depicted as processor 206), a frame 208 adapted to hold an active optical element 202A for a right eye and an active optical element 202B for a left eye, and at least one input means (depicted as input means 210) attached to the frame 208. In some embodiments, the optical device 200 also comprises an active optical element for each eye (depicted as active optical element 202A for a right eye and active optical element 202B for a left eye). The processor 206 is communicatively coupled to the active optical elements 202A, 202B and the eye tracking means 204. The input means 210 can be implemented as, for example, a physical slider or button. The input means 210 can be used by a user to provide a predetermined user input. In an exemplary scenario, an image may be displayed on a screen to be provided to a user for checking the user's eye vision. The image represents a visual target to be provided to the user. The input means 210 may be used by the user while using the optical instrument 200 to indicate whether or not the user can clearly see the visual target represented in the image.
[0061] 2 is merely illustrative and should not unduly limit the scope of the claims herein. It should be understood that the specific implementation of the optical device 200 is provided as an example and should not be construed as limiting the number or type of active optical elements, the number or type of components of the eye tracking means, or the number or type of input means. Those skilled in the art will recognize many variations, alternatives, and modifications of the embodiments of the present disclosure.
[0062] Referring to FIG. 3, an environment 300 is illustrated in which a system incorporating simultaneous eye tracking calibration and visual acuity testing is used according to an embodiment of the present disclosure. The environment 300 includes the system and a user 302. In the environment 300, a test is performed to assess the visual acuity of the user's eyes. The system includes an optical instrument (used by the user 302 in performing simultaneous eye tracking calibration and visual acuity testing), a screen 304, and at least one input means (depicted as input means 306). The screen 304 is depicted as a display of a device 308 of the user 302. The user 302 views the screen 304 from a predefined distance D1 (e.g., 300 centimeters, etc.) during the visual acuity testing. During the visual acuity testing, an image 310 provided to the user is displayed on the screen 304. The image 310 represents at least one visual target (depicted as visual target 312 provided to the user, the visual target 312 being depicted as a symbol). A predetermined user input is provided by the user 302 using the input means 306. Here, the predetermined user input indicates whether the user 302 can clearly see the object 312 depicted in the image 310 while using the optical instrument. In determining whether the user 302 can clearly see the object 312, information is collected that indicates the size of the object 312 that the user can or cannot clearly see, the location of the object 312 in the image 310, and the appearance of the user's eye features when the user can or cannot clearly see the object 312 displayed at said location. Based on this information obtained when performing a vision test using the system, the eye tracking means is calibrated while determining one or refractive powers specific to the user.
[0063] 3 is merely an example, which should not unduly limit the scope of the claims herein. One of ordinary skill in the art could recognize many variations, substitutions, and modifications of the disclosed embodiments.
[0064] 4A, an exemplary visual target (depicted as visual target 402A in various orientations) and an exemplary input means (depicted as a remote controller 404) are shown. Referring to FIG. 4B, a flow diagram is shown in which an exemplary visual acuity test of a user's eye is performed using visual target 402A provided to the user and remote controller 404 according to an embodiment of the present disclosure.
[0065] In both Fig. 4A and Fig. 4B, the visual target 402A is depicted as a Landolt C visual target with visual target features (depicted as predetermined gaps 406). The input means 404 has four buttons, namely an up button A (indicated by an up arrow), a down button B (indicated by a down arrow), a left button C (indicated by a left arrow), and a right button D (indicated by a right arrow), which correspond to the respective positions of the visual target features 406 on the visual target 402A. Thus, the up button A corresponds to the upper position of the visual target feature 406 on the visual target 402A, the down button B corresponds to the lower position of the visual target feature 406, the left button C corresponds to the left position of the visual target feature 406, and the right button D corresponds to the right position of the visual target feature 406. When performing a visual acuity test, a user looks at the visual target 402A while using an optical instrument.
[0066] In FIG. 4B, a user-provided image 408A is displayed to the user. The image 408A represents a visual target 402A to be provided to the user. The user provides an input using the input means 404. The input corresponds to the location of the visual target feature 406 seen by the user. In one example, the user provides a predetermined user input by pressing the up button A. Based on the user input, it is determined (as indicated using the decision box 410) that the user cannot clearly see the visual target 402A depicted in the image 408A. When it is determined that the user cannot clearly see the visual target 402A, information is collected indicative of at least one of the following: the size of the visual target 402A that the user could not clearly see, the location of the visual target 402A and / or the visual target feature 406 in the image 408A, and the appearance of the user's eye features when the user could not clearly see the visual target 402A displayed at the location. At the same time, the visual acuity test is repeated by displaying the next image 408B on the screen. Next image 408B depicts at least one next indicia (depicted as next indicia 402B) that is larger in size than the size of indicia 402A depicted in image 408A.
[0067] In another example, the user provides a predetermined user input by pressing the right directional button D. Based on this user input, it is determined (as indicated by the decision box 410) that the user can clearly see the visual object 402A depicted in the image 408A. Once it is determined that the user can clearly see the visual object 402A, information is collected indicative of at least one of the following: the size of the visual object 402A that the user can clearly see, the location of the visual object 402A and / or the visual object feature 406 in the image 408A, and the appearance of the user's eye features when the user can clearly see the visual object 402A displayed at the location. The visual acuity test is then repeated by displaying another next image 408C on the screen. The next image 408C represents at least one other next visual object (depicted as next visual object 402C) that is smaller in size than the size of the visual object 402A depicted in the image 408A.
[0068] 4A-B are merely examples, and should not unduly limit the scope of the claims of this application. One of ordinary skill in the art would recognize many variations, alternatives, and modifications of the embodiments of the present disclosure.
[0069] Referring to FIG. 5, a flow chart showing steps of a method implemented by an optical instrument comprising an eye tracking means and at least one processor is shown. According to an embodiment of the present disclosure, the method comprises: In step 502, an image provided to a user is displayed on a screen. The image represents at least one visual target. In step 504, a user input is obtained indicating whether the user can clearly see the visual target provided to the user represented in the image while using the optical instrument. In step 506, based on the obtained user input, it is determined whether the user was able to clearly see the visual target represented in the image. If it is determined that the user was able to clearly see a given visual target, in step 508, information is collected indicative of at least one of the following: a size of the visual target that the user was able to clearly see, a location of the visual target and / or visual target features in the image, and an appearance of the user's eye features when the user was able to clearly see the visual target displayed at the location. If it is determined that the user was unable to clearly see a given visual target, then in step 510, information is collected indicative of at least one of the size of the visual target that the user was unable to clearly see, the location of the visual target and / or visual target feature in the image, and the appearance of the user's eye features when the visual target displayed at the location was not clearly seen. Also, if the user was unable to clearly see a given visual target, steps 502, 504 and 506 are repeated with at least one next visual target that is larger in size than the indicia depicted in the image. In step 512, the collected information is processed to determine one or more refractive powers specific to the user and simultaneously calibrate the eye tracking means.
[0070] The steps described above are merely exemplary and may include alternative steps, i.e., one or more steps may be added, one or more steps may be removed, or one or more steps may be performed in a different order, without departing from the scope of the appended claims.
Claims
1. An eye-tracking instrument; at least one processor; 11. An optical instrument comprising: (a) displaying on a screen an image for a user representing at least one visual indicia; (b) obtaining a predefined user input indicating whether a user was able to clearly see a user indicia depicted in the user image while using the optical instrument; (c) determining whether the user was able to clearly see the user indicia depicted in the user image based on the predetermined user input; (d) if it is determined that the user was able to clearly see the user visual indicator, collecting information indicative of at least one of a size of the user visual indicator that was clearly seen by the user, a position of the user visual indicator and / or visual indicator feature in the user image, and an appearance of the user's eye features when the user was able to clearly see the user visual indicator displayed at the position; (e) if it is determined that the user was unable to clearly see the user's visual target, collect information indicative of at least one of a size of the user visual indicator that the user could not clearly see, a location of the user visual indicator and / or visual indicator feature in the user image, and an appearance of the user's eye features when the user could not clearly see the user visual indicator displayed at the location; repeating steps (a)-(e) with a next image representing at least one next indicia larger in size than the indicia represented in the user image; (f) processing the collected information to determine one or more optical powers specific to the user and simultaneously calibrate an eye tracking means; 4. An optical instrument configured to perform the steps of:
2. 2. The optical apparatus of claim 1, wherein the at least one processor is configured to repeat (a) through (e) using another subsequent image if the user determines that the user can clearly see the user indicia, the another subsequent image depicting at least one another subsequent indicia that is smaller in size than the indicia depicted in the user image.
3. 13. An optical instrument according to any preceding claim, comprising an active optical element for each eye, the at least one processor comprising: Controlling the active optical element to generate at least one of the one or more optical powers; Repeating steps (a) to (e), - processing the newly collected information to further calibrate the eye-tracking means; An optical instrument configured to:
4. 4. The optical instrument of claim 3, wherein the at least one processor is configured to adjust at least one of the one or more optical powers if the processor determines that the user cannot clearly see a given visual target.
5. 10. The optical apparatus of any preceding claim, wherein the at least one optotype comprises at least one of a Landolt C optotype, a Snellen optotype, a Sloan optotype, an Early Treatment Diabetic Retinopathy Study (ETDRS) optotype, a HOTV letter, an Allen figure, an alphabetic character, a number, a symbol, a design, or a freeform.
6. 10. An optical apparatus according to any preceding claim, wherein the user views the screen from a predetermined distance (D1) lying in the range of 25 centimetres to 750 centimetres.
7. 10. An optical instrument according to any preceding claim, wherein predetermined user input is obtained via at least one input means communicatively coupled to the at least one processor.
8. 13. An optical instrument according to any preceding claim, comprising: - an active optical element provided for each eye; a frame holding said active optical element; at least one input means attached to a temple of said frame for use by a user to provide predetermined user input; An optical instrument comprising:
9. 13. An optical instrument according to any preceding claim, comprising an active optical element for each eye, the at least one processor comprising: processing eye tracking data collected by said eye tracking means to determine at least one of a gaze direction of a user's eyes, a point of gaze of said user; determining an optical depth at which the user is gazing based on at least one of a convergence angle of gaze directions of the user's eyes, an interpupillary distance of the user's eyes, and a point of gaze of the user; Selecting at least one of the one or more optical powers produced by the active optical element based on the optical depth at which the user is gazing; An optical instrument configured to:
10. An optical device according to any preceding claim; A screen used to display the image; At least one input means used to obtain user input; A system comprising:
11. The system of claim 10 , wherein the screen is either a display of a user's or medical professional's equipment or a projection surface of a projector.
12. 1. A method implemented by an optical instrument comprising an eye-tracking means and at least one processor, comprising: (a) displaying on a screen an image for a user representing at least one visual indicia; (b) obtaining a predetermined user input indicating whether a user was able to clearly see a user indicia depicted in the user image while using the optical instrument according to any one of claims 1 to 9; (c) determining whether the user was able to clearly see the user indicia depicted in the user image based on the predetermined user input; (d) if it is determined that the user was able to clearly see the user visual indicator, collecting information indicative of at least one of a size of the user visual indicator that was clearly seen by the user, a position of the user visual indicator and / or visual indicator feature in the user image, and an appearance of the user's eye features when the user was able to clearly see the user visual indicator displayed at the position; (e) if it is determined that the user was unable to clearly see the user's visual target, collect information indicative of at least one of a size of the user visual indicator that the user could not clearly see, a location of the user visual indicator and / or visual indicator feature in the user image, and an appearance of the user's eye features when the user could not clearly see the user visual indicator displayed at the location; repeating steps (a)-(e) with a next image representing at least one next indicia larger in size than the indicia represented in the user image; (f) processing the collected information to determine one or more optical powers specific to said user and simultaneously calibrating an eye tracking means; A method comprising:
13. 13. The method of claim 12, further comprising repeating steps (a) through (e) using another subsequent image if it is determined that the user can clearly see the user indicia, the another subsequent image depicting at least one another subsequent indicia that is smaller in size than the indicia depicted in the user image.
14. - controlling an active optical element of the optical instrument to generate at least one of the one or more optical powers; repeating (a) to (e); - processing the newly collected information to further calibrate the eye-tracking means; The method of claim 12 or 13, comprising:
15. 15. The method of claim 14, comprising adjusting at least one of the one or more optical powers if it is determined that the user cannot clearly see a given visual indicia.
16. - processing the eye tracking data collected by the eye tracking means to determine at least one of a gaze direction of the user's eyes, a point of gaze of the user; determining an optical depth at which the user is gazing based on at least one of a convergence angle of gaze directions of the user's eyes, an interpupillary distance of the user's eyes, and a point of gaze of the user; selecting at least one of one or more optical powers produced by the active optical element of the optical instrument based on an optical depth at which the user is gazing; The method of any of claims 12 to 15, comprising: