Pupil characteristic detection and spectacle correction / adjustment based on the detection

The system addresses eye-related issues in wearable devices by detecting and adjusting display positions based on pupil characteristics, enhancing user comfort and efficiency in manufacturing.

JP2024544621APending Publication Date: 2024-12-03HERU INC

Patent Information

Application Number
JP2024531636
Authority / Receiving Office
JP · JP
Patent Type
Applications
Current Assignee / Owner
Priority Date
2021-11-30
Filing Date
2022-11-08
Publication Date
2024-12-03

AI Technical Summary

Technical Problem

Conventional wearable technologies for vision assistance, such as smart glasses, often cause eye strain, fatigue, blurred vision, and astigmatism due to inadequate detection and customization of pupil-related characteristics, leading to inefficient manufacturing and high costs.

Method used

A system that detects pupil-related characteristics, such as pupillary distance, to adjust and correct display positions in wearable devices, using machine learning models to personalize the display based on individual user profiles, reducing eye-related issues like strain and astigmatism.

Benefits of technology

The system effectively reduces eye strain, fatigue, and astigmatism by accurately detecting and adjusting display positions based on user-specific pupil characteristics, improving the overall user experience and reducing manufacturing delays and costs.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 2024544621000001_ABST
    Figure 2024544621000001_ABST
Patent Text Reader

Abstract

In certain embodiments, pupil-related characteristic detection or spectacle correction / adjustment based thereon can be facilitated. In some embodiments, a stimulus can be presented at a location on a first display of a user device at a first time point to induce a first eye of the user to fixate the location. A stimulus can be presented at a location on a second display of the user device to induce a second eye of the user to fixate the location at a time point subsequent to the first time point. Movement of the first or second eye to fixate the location on the respective first or second display can be determined in association with the presentation of the stimulus at the second time point. In response to determining that the movement of the first or second eye exceeds a threshold amount of movement, a user profile can be generated that includes the user's pupil distance.
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application claims the benefit of priority to U.S. Patent Application No. 17 / 538,811, filed November 30, 2021, the contents of which are incorporated herein by reference in their entirety. [Background technology]

[0002] background

[0002] Although "smart glasses" and other wearable technologies exist to assist individuals with reduced vision, typical wearable technologies do not adequately address some of the challenges associated with traditional eyeglasses. For example, typical wearable technologies fail to address characteristics related to a user's pupils, which can cause eye strain, fatigue, blurred vision, astigmatism, headaches, or other eye / vision-related challenges for users looking through the wrong part of the lens. These and other shortcomings exist. Summary of the Invention [Problem to be solved by the invention]

[0003] Summary of the Invention

[0003] Aspects of the present invention relate to a method, device, or system that facilitates pupil-related characteristic detection and spectacle correction / adjustment based thereon. By appropriately correcting or adjusting the lens or display of a spectacle device (e.g., moving the lens or display closer or farther away from each other, adjusting the position of the presentation of one or more objects on the lens or display, etc.) based on the pupil-related characteristics of a user, the user may experience reduced eye-related issues (e.g., eye strain, fatigue, blurred vision, astigmatism, headaches, etc.).

[0004]

[0004] Conventional systems may exist to assist with reduced vision, but these conventional systems are usually unable to accurately and seamlessly detect pupil distance or other pupil-related characteristics. Moreover, in the context of digital spectacles, even if such pupil-related characteristics are known, customization by manufacturers of digital spectacles to take into account differences in individual patient profiles generally results in greater manufacturing costs and delays. [Means for solving the problem]

[0005]

[0005] To solve one or more of the above-mentioned technical problems, the spectacle device can detect characteristics related to one or more pupils of a user or adjust / correct one or more display devices of the spectacle device based on characteristics related to the user's pupils.

[0006]

[0006] In some embodiments, one or more positions of a display of the wearable device can be adjusted based on characteristics related to one or more pupils of the user. In some embodiments, in association with the presentation of the first and second stimuli at first and second time points on the first and second displays, respectively, the wearable device can determine that a predetermined amount of movement of the user's eyes occurs (e.g., in response to the presentation of the second stimulus after the presentation of the first stimulus). In response to determining that the eye movement exceeds a threshold amount of movement, the amount of movement can be used to automatically adjust the position of the display, and subsequent presentations of the respective stimuli on the device can be performed to determine whether the adjustment eliminates or reduces the subsequent corresponding eye movement. As an example, if there is no eye movement resulting from the presentation of a stimulus at the center of the second display immediately following the presentation of a stimulus at the center of the first display (or if there is only eye movement below the threshold amount of movement), the wearable device can determine that the display is appropriately adjusted for the user.

[0007]

[0007] In some embodiments, the wearable device can determine a pupillary distance between the pupils of the user's eyes based on an amount of eye movement (e.g., occurring in response to the presentation of a second stimulus after the presentation of a first stimulus) and can perform an adjustment of the display based on the determined pupillary distance. As an example, the wearable device can determine the pupillary distance based on the amount of movement, the individual distances between the center of each display and the edge of the display, the distance between the displays, or other information. The pupillary distance can be used to generate a user profile associated with the user, which can be used to perform future adjustments of the display of the wearable device (e.g., in cases where the wearable device can be worn by multiple users) or can be provided to one or more service providers to customize lenses or frames for the user.

[0008]

[0008] Various other aspects, features, and advantages of the present invention will become apparent through the detailed description of the present invention and the accompanying drawings. It should also be understood that both the general description above and the detailed description below are for illustrative purposes only and are not intended to limit the scope of the present invention. As used in this specification and the claims, the singular forms "a", "an" and "the" include plural references unless the context clearly indicates otherwise. In addition, the term "or" as used in this specification and the claims means "and / or" unless the context clearly indicates otherwise. [Brief description of the drawings]

[0009] BRIEF DESCRIPTION OF THE DRAWINGS [Figure 1A] A system is presented that facilitates spectacle correction / adjustment based on pupil-related characteristic detection in accordance with one or more embodiments. [Figure 1B]

[0010] 1 illustrates a system that implements a machine learning model to facilitate pupil-related characteristic detection or spectacle correction / adjustment based thereon in accordance with one or more embodiments. [Figure 1C]

[0011] 1 shows a diagram of an example spectacle apparatus in accordance with one or more embodiments. [Figure 1D]

[0011] FIG. 1 shows a diagram of an example spectacle apparatus in accordance with one or more embodiments. [Diagram 2]

[0012] 1 illustrates an exemplary vision system in accordance with one or more embodiments. [Diagram 3]

[0013] 1 illustrates an apparatus having an image processing device and a vision correction framework implemented on a wearable spectacles device according to one or more embodiments. [Figure 4A]

[0014] 1 illustrates a use case in which visual test presentations are displayed to a patient who does not have esotropia in accordance with one or more embodiments. [Figure 4B]

[0015] 1 illustrates a use case in which visual test presentations are displayed to a patient with esotropia in accordance with one or more embodiments. [Figure 4C]

[0016] 1 illustrates automated pupil distance detection or spectacle correction / adjustment based thereon in accordance with one or more embodiments. [Figure 4D]

[0016] Figure 1 illustrates automated pupil distance detection or spectacle correction / adjustment based thereon according to one or more embodiments. [Figure 4E]

[0016] Figure 1 illustrates automated pupil distance detection or spectacle correction / adjustment based thereon according to one or more embodiments. [Figure 4F]

[0016] Figure 1 illustrates automated pupil distance detection or spectacle correction / adjustment based thereon according to one or more embodiments. [Figure 4G]

[0016] Figure 1 illustrates automated pupil distance detection or spectacle correction / adjustment based thereon according to one or more embodiments. [Figure 4H]

[0016] Figure 1 illustrates automated pupil distance detection or spectacle correction / adjustment based thereon according to one or more embodiments. [Figure 4I]

[0016] Figure 1 illustrates automated pupil distance detection or spectacle correction / adjustment based thereon according to one or more embodiments. [Figure 4J]

[0016] Figure 1 illustrates automated pupil distance detection or spectacle correction / adjustment based thereon according to one or more embodiments. [Figure 4K]

[0016] Figure 1 illustrates automated pupil distance detection or spectacle correction / adjustment based thereon according to one or more embodiments. [Diagram 5]

[0017] 1 illustrates a flowchart of a method for facilitating pupil distance detection and spectacle correction / adjustment based thereon in accordance with one or more embodiments. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] Detailed Description of the Invention

[0018] In the following description, for purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of the embodiments of the present invention. However, those skilled in the art will understand that the embodiments of the present invention may be practiced without these specific details or with equivalent configurations. In other cases, well-known structures and devices are shown in block diagram form in order to avoid unnecessarily obscuring the embodiments of the present invention.

[0011]

[0019] FIG. 1A illustrates a system 100 for facilitating pupil-related characteristic detection or spectacle correction / adjustment based thereon, according to one or more embodiments. As illustrated in FIG. 1A, the system 100 may include one or more servers 102, client devices 104 (or client devices 104a-104n), or other components. The server 102 may include a configuration subsystem 112, a model manager subsystem 114, or other components. The client devices 104 may include a test subsystem 122, a visioning subsystem 124, or other components. Each client device 104 may include any type of mobile terminal, fixed terminal, or other device. By way of example, the client devices 104 may include user devices, desktop computers, notebook computers, tablet computers, smartphones, wearable devices, or other client devices. Users may utilize one or more client devices 104, for example, to interact with each other, with one or more servers, or with other components of the system 100.

[0012]

[0020] It should be noted that although one or more operations are described herein as being performed by a particular component of the client device 104, these operations may, in some embodiments, be performed by other components of the client device 104 or other components of the system 100. As an example, although one or more operations are described herein as being performed by a component of the client device 104, these operations may, in some embodiments, be performed by a component of the server 102. It should also be noted that although one or more operations are described herein as being performed by a particular component of the server 102, these operations may, in some embodiments, be performed by other components of the server 102 or other components of the system 100. As an example, although one or more operations are described herein as being performed by a component of the server 102, these operations may, in some embodiments, be performed by a component of the client device 104. Further, while some embodiments are described herein with respect to machine learning models, it should be noted that in other embodiments, other predictive models (e.g., statistical models or other analytical models) may be used instead of or in addition to the machine learning models (e.g., in one or more embodiments, a statistical model replaces the machine learning model and a non-statistical model replaces the non-machine learning model).

[0013]

[0021] In some embodiments, the system 100 may provide a visual test presentation to a user, the presentation including a set of stimuli (e.g., light stimuli, text, or images displayed to the user). During (or after) the presentation, the system 100 may obtain feedback related to the set of stimuli (e.g., feedback informing the user of whether or how to observe one or more stimuli of the set, feedback related to the subject's eye movement, gaze direction of the user's eye or eyes, or other feedback). As an example, the feedback may include an indication of a user's response to one or more stimuli (of the set of stimuli) or a lack of a user response to such stimuli. The response (or lack thereof) may relate to eye movement, a change in eye movement, gaze direction, a change in gaze direction, pupil size, a change in pupil size, or other response (or lack thereof). As another example, the feedback may include an eye image (or a set of eye images) captured during the visual test presentation. The eye image may be an image of the retina of the eye (e.g., the entire retina or a portion thereof), an image of the cornea of ​​the eye (e.g., the entire cornea or a portion thereof), or other eye image. As a further example, the feedback may include feedback related to movement of one or more eyes of the subject. For example, by using one or more eye images obtained during presentation of the stimuli, the system may obtain feedback related to the manner in which one or more eyes of the subject move in relation to the presentation of the stimuli. Additionally or alternatively, the system 100 may obtain feedback related to eye movement of one or more eyes of the subject via one or more eye tracking sensors during presentation of the stimuli.

[0014]

[0022] In some embodiments, one or more predictive models can be used to facilitate the determination or other operation of one or more pupil-related characteristics (e.g., pupillary distance between the user's pupils, one or more pupillary sizes of the user's pupils, one or more radii of curvature of the user's pupils, or other pupillary characteristics of the user). In some embodiments, the predictive models can include one or more neural networks or other machine learning models. As an example, a neural network can be based on a large collection of neural units (or artificial neurons). A neural network can loosely mimic the way a biological brain functions (e.g., via large clusters of biological neurons connected by axons). Each neural unit of a neural network can be connected to many other neural units of the neural network. Such connections can enable or inhibit its influence on the activation state of the connected neural units. In some embodiments, each individual neural unit can have a summing function that combines the values ​​of all its inputs into one. In some embodiments, each connection (or the neural unit itself) may have a threshold function, and thus a signal must exceed the threshold before it propagates to other neural units. These neural network systems may be self-learning, trained rather than explicitly programmed, and may perform much better in certain areas of problem solving compared to traditional computer programs. In some embodiments, the neural network may include multiple layers (e.g., when a signal path traverses from a front layer to a back layer). In some embodiments, backpropagation techniques may be utilized by the neural network, where forward simulation is used to reset weights on the "front" neural units.In some embodiments, the stimuli and inhibitions for a neural network may be more free-flowing, with connections that interact in more chaotic and complex ways.

[0015]

[0023] 1B, as an example, machine learning model 162 can obtain input 164 and provide output 166. In one use case, output 166 can be fed back to machine learning model 162 as an input to train machine learning model 162 (e.g., alone or in conjunction with a user notification of the accuracy of output 166, in conjunction with a label associated with the input, or in conjunction with other baseline feedback information). In another use case, machine learning model 162 can update its configuration (e.g., weights, biases, or other parameters) based on its evaluation of its prediction (e.g., output 166) and baseline feedback information (e.g., characteristics related to the user's pupil, the user's pupil distance, or other information). In another use case, if machine learning model 162 is a neural network, connection weights can be adjusted to compromise the difference between the neural network's prediction and the baseline feedback. In a further use case, one or more neurons (or nodes) of a neural network may require their individual errors to be sent backwards through the neural network to facilitate the update process (e.g., backpropagation of errors). Updates to connection weights may reflect, for example, the magnitude of the error propagated backwards after a forward pass is completed. This may result in, for example, training a predictive model to generate better predictions.

[0016]

[0024] In some embodiments, upon obtaining feedback related to the set of stimuli (displayed to the user), feedback related to one or more eyes of the user, feedback related to one or more eye movements, feedback related to one or more eye gaze directions, feedback related to one or more locations of the set of stimuli, or other feedback, the system 100 can provide feedback to the predictive model and can configure the predictive model based on the feedback. As an example, the predictive model can be automatically configured for the user (e.g., the predictive model can be personalized for the user based on feedback from the visual test presentation) based on (i) notification of the user's response to one or more stimuli (of the set of stimuli), (ii) notification of the user's lack of response to such stimuli, (iii) one or more eye images captured during the visual test presentation, (iv) one or more eye movements of the subject, (v) the subject's eye gaze direction, (vi) one or more locations of the stimuli, or (vii) other feedback. As another example, the predictive model can be trained based on such feedback and other feedback from other users to improve the accuracy of the results provided by the predictive model. In some embodiments, when a predictive model is constructed (e.g., for a user), system 100 can determine characteristics related to the user's pupils (e.g., pupillary distance between the user's pupils, one or more pupillary sizes of one or more pupils of the user, one or more radii of curvature of the user's pupils, or other pupillary characteristics of the user's pupils) to be used in one or more scenarios. In some embodiments, based on the characteristics related to the user's pupils, the wearable device of system 100 can adjust one or more displays or lenses or other components of the wearable device. Additionally or alternatively, the characteristics related to the user's pupils can be stored in a user profile, so that the characteristics related to the user's pupils can be used by an optometrist to generate eyeglasses for the subject.Also, in some embodiments, the user profile may be used to perform future adjustments of the display of a wearable device, where the wearable device is worn by multiple users, such that different users can wear the same wearable device and the wearable device can automatically adjust the display based on the pupil characteristics of each individual user.

[0017]

[0025] In some embodiments, the system 100 can facilitate adjustment / correction of the wearable device via one or more displays of the wearable device (e.g., adjustable wearable device lenses or displays, movable wearable device lenses or displays, variably positioned wearable device lenses or displays, etc.). As an example, with respect to a wearable device, the wearable device can include one or more displays or lenses. In some embodiments, the displays, lenses, or other components of the wearable device can be self-adjusting based on characteristics related to one or more pupils of the user. For example, in some embodiments, the characteristics related to the user's pupils can include a pupillary distance between the user's pupils, one or more pupillary sizes of the user's pupils, one or more radii of curvature of the user's pupils, or other pupillary characteristics of the user's pupils. In response to the system determining the characteristics related to one or more pupils of the user, the display of the wearable device can adjust / correct to "fit" the user (e.g., to the left and right, up and down, closer to one, further apart, etc.). As a result, for example, the wearable device may automatically improve the user experience by alleviating eye strain, headaches, blurred vision, astigmatism, or other eye-related issues, such as those related to display positioning or shape that is not appropriate for the user. Note that although one or more operations are described herein as being performed with respect to one or more displays of the wearable device, in some embodiments, these operations may be performed with respect to one or more lenses of the wearable device (e.g., dynamically adjusting lenses of the wearable device based on characteristics related to the user's pupils).

[0018]

[0026] In some embodiments, with respect to FIG. 1C, the client device 104 may include a spectacle device 170 forming a wearable device for the subject. In some embodiments, the spectacle device 170 may be part of a visioning system as described herein. The spectacle device 170 includes a left eyepiece 172 and a right eyepiece 174. Each eyepiece 172 and 174 may include or be associated with a digital monitor configured to display (e.g., provide on a screen or project onto the top of the eye) an image (e.g., a stimulus) to a respective eye of the subject. In various embodiments, the digital monitor may include a display, a display screen, a projector, or other components to generate an image display on a display screen or to project an image onto the eye (e.g., the retina of the eye). It should be understood that a digital monitor with a projector may be positioned elsewhere to project an image onto the subject's eye or onto the top of an eyepiece that has a screen, glass, or other surface onto which the image may be projected. In one embodiment, the left eyepiece 172 and the right eyepiece 174 may be positioned with respect to the housing 176 to fit into the subject's eye socket such that each eyepiece 172, 174 is capable of collecting data and displaying / projecting image data, which in further examples includes displaying / projecting image data to different eyes.

[0019]

[0027] The housing 176 may further include one or more outwardly facing sensors, which may be outwardly oriented image sensors 178 having a vision camera. In other embodiments, a fewer or greater number of outwardly oriented image sensors 178 may be provided. The outwardly oriented image sensors 178 may be configured to capture sequential images (e.g., of a user's field of view or other images).

[0020]

[0028] The housing 176 may further include a power unit 180 and a processing unit 182. The power unit 180 may include one or more batteries, a charging port, or other power source. The power unit may be configured to power one or more components of the spectacle apparatus 170. The processing unit 182 may include one or more computer processors, one or more electronic storage media components (e.g., computer memory, random access memory (RAM), etc.), one or more wired or wireless communication components (e.g., physical communication ports, cables, antennas, wireless transceivers, etc.), other control circuitry components, or other components.

[0021]

[0029] In some embodiments, with reference to FIG. 1D, the housing 176 further includes one or more inwardly directed sensors 184, which may be inwardly directed image sensors. As an example, the inwardly directed sensors 184 may include an infrared camera, photodetector, or other infrared sensor configured to track pupil movement, determine and track the subject's visual axis, track and determine the eye's gaze direction, and track and determine other eye-related movements. The inwardly directed sensor 184 (e.g., with an infrared camera) may be positioned at an upper portion with respect to the eyepieces 172, 174 so as not to obstruct the subject's field of view, its actual field of view, or the field of view displayed or projected to the subject. The inwardly directed sensor 184 may be aligned in direction to point to a hypothesized pupil area for better pupil or collimation tracking. In some examples, the inwardly directed sensor 184 may be embedded within the eyepieces 172, 174 to provide a continuous interior surface.

[0022]

[0030] The spectacle device 170 may further include one or more dynamic eyepieces (e.g., displays). For example, the left eyepiece 172 and the right eyepiece 174 may be dynamic, allowing each eyepiece to be adjustable. For example, the left and right eyepieces 172, 174 may be adjusted (e.g., left and right, up and down, from one side to the other, closer to each other, farther away, etc.) to "fit" the user. For example, based on characteristics related to the user's pupil or pupils, the spectacle device 170 or associated visioning system may facilitate adjustment of one or more eyepieces (e.g., eyepieces 172, 174) to allow for an improved user experience (e.g., by mitigating challenges related to the user's eye or eyes).

[0023]

[0031] In some embodiments, referring to FIG. 2, the system 100 may include a vision system 200, which includes a spectacle device 202 communicatively coupled to a network 204 for communicating with a server 206, a mobile cellular telephone 208, or a personal computer 210, any of which may include a vision correction framework 212 for implementing the processing techniques herein, such as image processing techniques, which may include these for a test mode or a visioning mode. In the illustrated example, the vision correction framework 212 includes a transceiver for communicating with the spectacle device 202 in the network 204, as well as a processor and memory for storing an operating system and applications therein that implement the techniques. The framework 212 includes a test module 214, which in this example includes a machine learning framework. The machine learning framework can be used in conjunction with the test protocol executed by the test module to adaptively adjust the test mode to more accurately evaluate ophthalmic pathology in a supervised or unsupervised manner. The results of the test module operation may include the development of a customized vision correction model 216 for the subject 218.

[0024]

[0032] The visioning module 220, in some embodiments, may also include a machine learning framework that accesses the customized vision correction model 216 to generate corrected visual images (e.g., stimuli) for display by the spectacle device 202. The vision correction framework 212 may also include a scene processing module that may process images used in test mode or visioning mode operation, and may also include operations described above or elsewhere herein with respect to the processing module. As described above and elsewhere herein, in some embodiments, the spectacle device 202 may include all or a portion of the vision correction framework 212.

[0025]

[0033] In the test mode, the spectacle device 170 or 202 and in particular an inwardly oriented image sensor having a tracking camera which may be positioned along the interior of the spectacle device 170 or 202 can be used to capture pupil and visual axis tracking data, eye gaze direction, or other eye characteristics which are used to accurately measure the subject's pupillary distance and visual axis.

[0026]

[0034] In some embodiments, with reference to FIG. 3, the system 100 may include a vision system 300, which includes a vision correction framework 302. The vision correction framework 302 may be implemented on top of an image processing device 304 and a spectacle device 306 that is placed on the subject. The image processing device 304 may be entirely housed within an external image processing device or other computer, although in other examples, all or a portion of the image processing device 304 may be implemented within the spectacle device 306.

[0027]

[0035] The image processor 304 may include a memory 308 storing instructions 310 for performing the testing or visioning modes described herein, which may include instructions for collecting high resolution images of the subject from the spectacle device 306. In the visioning mode, the spectacle device 306 may capture real-time visual field image data as raw data, processed data, or pre-processed data. In the test mode, the spectacle device may project test images (such as written "text," images of vehicles or other objects, or other stimuli) to test aspects of the subject's visual field.

[0028]

[0036] The spectacle device 306 may be communicatively connected to the image processing device 304 through a wired or wireless link. The link may be through Universal Serial Bus (USB), IEEE 1394 (Firewire), Ethernet, or other wired communication protocol devices. The wireless connection may be through any suitable wireless communication protocol such as WiFi, NFC, iBeacon, Bluetooth, Bluetooth low energy, etc.

[0029]

[0037] In various embodiments, the image processing device 304 may have a controller operatively connected to a database via a link connected to an input / output (I / O) circuit. Further databases may be linked to the controller in known manner. The controller includes a program memory, a processor (which may be referred to as a microcontroller or microprocessor), a random access memory (RAM), and an input / output (I / O) circuit, all of which may be interconnected via an address data bus. Although only one microprocessor is described, it should be understood that the controller may include multiple microprocessors. Similarly, the memory of the controller may include multiple RAMs and multiple program memories. The RAM or multiple RAMs and program memories may be implemented as semiconductor memories, magnetically readable memories, or optically readable memories. A link may operatively connect the controller to the capture device through the I / O circuit.

[0030]

[0038] The program memory or RAM can store various applications (i.e., machine-readable instructions) for execution by the microprocessor. For example, an operating system can generally control the operation of the vision system 300, such as the operation of the spectacle device 306 or the image processor 304, and in some embodiments can provide a user interface to the device for implementing the processes described herein. The program memory or RAM can also store various subroutines for accessing specific functions of the image processor 304 described herein. By way of example, and without limitation, the subroutines can include, among others, providing stimuli to be presented for display on one or more displays or lenses of the spectacle device to a subject, and obtaining one or more eye-related feedback of the user from the spectacle device (e.g., eye movement, eye gaze, changes in eye movement, changes in eye gaze, pupil size, changes in pupil size, or other eye-related characteristics).

[0031]

[0039] In addition to the above, the image processor 304 may include other hardware resources. The device may also include various types of input / output hardware, such as a visual display and one or more input devices (e.g., keypad, keyboard, etc.). In one embodiment, the display is touch-sensitive and may cooperate with a software keyboard routine as one of the software routines to receive user input. It may be advantageous for the image processor 304 to communicate with a wider network (not shown) through any of a number of known networking devices and techniques (e.g., through a computer network such as an intranet, the Internet, etc.). For example, the device may be connected to a database of aberration data.

[0032]

[0040] In some embodiments, system 100 may store the predictive models, visual defect information (e.g., informing the user of the detected visual defect), feedback information (e.g., feedback related to the stimuli displayed to the user or other feedback), or other information in one or more remote databases (e.g., in the cloud). In some embodiments, the feedback information, visual defect information, or other information associated with multiple users (e.g., two or more users, ten or more users, one hundred or more users, one thousand or more users, one million or more users, or other number of users) may be used to train one or more predictive models. In some embodiments, one or more predictive models may be trained or configured for a given user or type of device (e.g., a particular brand of device, a particular brand and model of device, a device having a particular set of features, etc.) and stored in association with the user or device type. As an example, instances of predictive models associated with a given user or device type may be stored locally (e.g., on the user's wearable device or other user device) and remotely (e.g., in the cloud), and such predictive model instances may be automatically or manually synchronized across one or more user devices and clouds to enable a user to have access to the latest configurations of predictive models across any of the user devices or clouds.

[0033]

[0041] In some embodiments, a user profile can be associated with a user or a device type. For example, by having a user profile associated with a user, a device (e.g., a spectacle device) can be configured to automatically adjust the display of the device based on the user's user profile. As a result, multiple users can wear the same device (or different devices), and the device can automatically adjust the display of the device based on the user's user profile. As another example, due to different devices (e.g., different spectacle devices) having different characteristics in the way they are constructed (e.g., display size, distance between the displays of the devices, mechanism that effects the adjustment of the displays, etc.), different devices may require different parameters to adjust such display for a user. In one use case, a first device may have a smaller display that requires the display to be adjusted at a greater distance apart (based on a user profile) than a second device (e.g., a device with a larger display). To mitigate these differences, having a user profile associated with a device type may allow these differences to be compromised, and the display of a particular device may be adjusted based on the user profile or type of device.

[0034]

[0042] Subsystems 112-124

[0043] In some embodiments, with reference to FIG. 1A , the test subsystem 122 can provide a visual test presentation to the user. As an example, the presentation can include a set of stimuli. During (or after) the presentation, the test subsystem 122 can obtain feedback related to the set of stimuli (e.g., feedback informing eye movement of one or more eyes of the user, feedback informing gaze direction of one or more eyes of the user, feedback informing whether or how the user observes one or more stimuli of the set). As an example, the feedback can include informing of a user's response to one or more stimuli (of the set of stimuli) or a lack of user response to such stimuli. The response (or lack thereof) can relate to eye movement, a change in eye movement, gaze direction, a change in gaze direction, pupil size, a change in pupil size, or a user modification of one or more stimuli or other user input (e.g., a user's reaction or other response to the stimuli). As another example, the feedback may include eye images or other eye movement / gaze direction information captured during visual test presentation (e.g., via one or more eye tracking sensors). The eye images may be images of the eye's pupil (e.g., the entire pupil or a portion thereof), the eye's retina (e.g., the entire retina or a portion thereof), the eye's cornea (e.g., the entire cornea or other portion), or other eye images, and the one or more eye images may be used to determine eye movement of the user's one or more eyes.

[0035]

[0044] In some embodiments, the test subsystem 122 can determine whether the user's eye or eyes satisfy a momentum threshold, and in response to the user's eye satisfying the momentum threshold, the test subsystem 122 can generate a user profile (e.g., based on the determination or feedback) that includes one or more pupil-related characteristics associated with the user. In some embodiments, the momentum threshold can be satisfied based on the user's eye exceeding the momentum threshold. For example, the user's eye movement may exceed the momentum threshold. In some embodiments, the momentum threshold can be satisfied based on the user's eye satisfying the momentum threshold. For example, the user's eye movement may be equal to (or exceed) the momentum threshold. In some embodiments, the momentum threshold can be a range, and the momentum threshold can be satisfied if the user's eye or eyes are within such range. Alternatively, the momentum threshold can be satisfied if the user's eye or eyes are outside of the momentum threshold range (when the momentum threshold is a range). Based on the user's eye movement satisfying the momentum threshold, the test subsystem 122 can generate a user profile that includes one or more pupil-related characteristics associated with the user. For example, the pupil-related characteristics can include a pupillary distance between the user's pupils, one or more pupillary sizes of the user's pupils, one or more radii of curvature of the user's pupils, or other pupil-related characteristics of the user. In some embodiments, the test subsystem 122 can facilitate adjustment / correction of a lens or display of a spectacle device based on the user's pupil-related characteristics.

[0036]

[0045] The eye tracking functions described herein may include the physical state or location of the pupil (e.g., visual axis, pupil size, or limbus), alignment, dilation, or line of collimation. The line of collimation, also referred to as the visual axis, is a goal that may be achieved by one or more of the pupil, the limbus (which is the edge between the cornea and the sclera), or possibly tracking of blood vessels on the surface of the eye or inside the eye. Thus, eye tracking may also include limbus or blood vessel tracking. Eye tracking may be performed by utilizing one or more inwardly facing image sensors as described herein. In various embodiments, the eye tracking functions may be used for determining characteristics related to one or more pupils. For example, the pupil tracking functions may be used to track the movement of one or more eyes of a user, the gaze direction of one or more eyes of a user, changes in pupil size, changes in gaze direction, or other changes. The test subsystem 122 may determine characteristics related to one or more pupils of a user based on the movement of one or more eyes of a user. For example, the pupil-related characteristics may include a pupillary distance between the user's pupils, one or more pupillary sizes of the user's pupils, one or more radii of curvature of the user's pupils, or other pupillary characteristics of the user's pupils. In some embodiments, the test subsystem 122 may determine the user's pupillary distance based on movement of (i) only one eye of the user, (ii) both eyes of the user, (iii) the gaze direction of only one eye of the user, (iv) the gaze direction of both eyes of the user, or (v) other feedback (e.g., other eye characteristics) obtained during the test.

[0037]

[0046] In some embodiments, the stimuli can be presented to the user during one or more visual acuity tests. For example, the test subsystem 122 can generate one or more stimuli for display for each eye of the user to be presented on one or more display portions of the spectacle device 170. The set of stimuli displayed to the user may include stimuli of different brightness, contrast, saturation, sharpness levels, or locations, and a response or lack of response to a stimulus having a particular brightness, contrast, saturation, or sharpness level, or location, can provide an indication of whether a portion of the user's visual field (corresponding to the location of the displayed stimulus) has a challenge related to eye movement, gaze direction, brightness, contrast, saturation, or sharpness. As an example, if the user's eye responds to the displayed stimulus (e.g., the eye moves to fixate the stimulus), the eye response can be used as an indication that the eye can observe the displayed stimulus. On the other hand, if the user's eye does not respond to the stimulus (e.g., the eye does not move to fixate the stimulus), the lack of eye response can be used as an indication that the eye cannot observe the displayed stimulus.

[0038]

[0047] In some embodiments, the first stimulus can be presented on the first display for the user's first eye at a location on the first display. This can cause the user's eye (e.g., the eye corresponding to the presentation of the first stimulus on the first display) to gaze at the location of the first stimulus on the first display. The second stimulus can be presented on the second display for the user's second eye (e.g., the user's other eye) at a location on the second display at a time subsequent to the presentation of the first stimulus on the first display. In some embodiments, the second stimulus can be presented when the first stimulus is no longer being presented. This can cause the user's other eye (e.g., the eye corresponding to the presentation of the second stimulus on the second display) to gaze at the location of the second stimulus on the second display. During (or subsequent to) the vision test, one or more eye tracking sensors (e.g., the inwardly directed sensor 184) can track the user's eye movement (e.g., response or other feedback) to determine whether the user's eyes move with respect to the presentation of the individual stimuli for each eye of the user. For example, when a second stimulus is presented to the user's other eye (e.g., the second eye), the eye tracking sensor can track the movement of the user's other eye, and the test subsystem 122 can determine whether the movement of the other eye satisfies a threshold amount of movement (e.g., moves by more than a threshold amount) to fixate on a location of the second stimulus. For example, as described above, the user's eye (e.g., the first eye) may already fixate on a location on the first display, so that when a second stimulus is presented for the user's other eye (subsequent to the presentation of the first stimulus), this may cause the user's other eye to move to fixate on a location of the second stimulus on the second display. Based on the other eye satisfying the threshold amount of movement (e.g., moves by more than a threshold amount), the test subsystem 122 can determine a characteristic related to one or more pupils of the user.For example, the user's pupil-related characteristic may include a pupillary distance between the user's pupils, which may be determined based on a determination (e.g., that the user's other eye exceeds a threshold amount of movement) and the movement of the other eye (e.g., the amount of movement of the other eye). Additionally or alternatively, the test subsystem 122 may generate a user profile (e.g., a pupillary characteristic) that includes the user's pupillary distance. The user's determined pupillary distance may be used to modify, adjust, or correct the distance between one or more displays of the spectacle device 170 or to generate / update the user's user profile. As a result, for example, the wearable device may automatically improve the user experience by alleviating eye strain, headaches, blurred vision, astigmatism, or other eye-related challenges, such as those related to display positioning or shape that is not appropriate for the user.

[0039]

[0048] In some embodiments, the location of the fixation point and the location of the stimuli presented to the user can be dynamically determined based on the user's eye response (or lack of response), the movement of one or more of the user's eyes, the gaze direction of one or more of the user's eyes, or other aspects of the user's eyes. As an example, during a visual test presentation, both fixation points (e.g., the locations of the stimuli) can be presented at center locations of respective display portions (e.g., the right and left displays of a wearable device). However, based on one or more defective visual field portions of the user (e.g., as determined by the user's eye movement during one or more visual test presentations, the user's eye response in relation to the presentation (or lack thereof) of the stimuli, or other feedback), the stimuli can be located at a location of the display where the user's eyes may not be able to observe at the current time. Thus, the test subsystem 122 can adjust / select a fixation point that is outside the defective visual field portion of the user to allow the user to observe or respond to the presented stimuli.

[0040]

[0049] As described, in some embodiments, the test subsystem 122 can adjust / select a fixation point (e.g., for one or more stimulus locations during a visual presentation test) based on eye movements of one or more eyes of the user (e.g., movements of only one eye of the user, movements of one or more eyes of the user, gaze direction of only one eye of the user, gaze direction of one or more eyes of the user, or other eye movements such as those occurring during a visual test presentation). In one use case, the test subsystem 122 can cause a first stimulus to be displayed at a first location on a user interface (e.g., of a wearable device or other device of the user) based on the fixation point. The test subsystem 122 can adjust / select a fixation point based on the eye movement information and cause a second stimulus to be displayed at a second location on the user interface during a visual test presentation based on the adjusted fixation point. The test subsystem 122 can obtain feedback information during a visual test presentation and generate visual defect information based on such feedback information. As an example, the feedback information may inform feedback related to the first stimulus, feedback related to the second stimulus, or feedback related to one or more other stimuli. Such feedback may inform (i) the user's response to the stimuli, (ii) the user's lack of response to the stimuli, (iii) whether or to what extent the user detects one or more stimuli, the degree of light sensitivity, distortion, or other aberrations, or (iv) other feedback. The generated visual defect information may be used (i) to train one or more predictive models, (ii) to determine one or more user profiles for the user, (iii) to facilitate live image processing to correct or modify an image for the user, (iv) to select one or more locations for the stimuli, or (v) to perform other operations described herein.

[0041]

[0050] In some embodiments, the eye tracking system can continuously acquire gaze coordinates (e.g., periodically, according to a schedule, or according to other automated triggers). A coordinate transformation can be performed to convert the spherical coordinates (θ,φ) of the eye movements into Gaussian coordinates (x,y) of the display. Thus, the device controller can determine the center location of the displayed stimulus. In some cases, low-pass filtering can be performed on the gaze data to remove subtle eye movements (e.g., subtle eye movements generated by constant movement and drafting that occur even during gaze because the eyes are never completely still) that could otherwise cause a shaking stimulus to be displayed to the user.

[0042]

[0051] In some embodiments, the test subsystem 122 can monitor one or more eye-related characteristics associated with the user's eyes during visual test presentation via two or more user interfaces (e.g., on two or more displays) and can determine one or more pupil-related characteristics based on the eye-related characteristics occurring during visual test presentation. For example, the user's pupil-related characteristics can include a pupillary distance between the user's pupils, one or more pupillary sizes of the user's one or more pupils, one or more radii of curvature of the user's pupils, or other pupillary characteristics of the user's pupils. With respect to pupillary distance, the distance between the centers of the user's pupils is measured to determine the user's pupillary distance. However, in cases where the user has one or more esotropias, determining the user's pupillary distance is not as straightforward. When the user's eyes have esotropia, it can be difficult to obtain an accurate measurement of the user's pupillary distance because esotropia results in different gaze directions for each eye. To overcome this technical challenge, the test subsystem 122 can determine deviation measurements (e.g., differences in eye movement for each of the user's eyes, differences in eye gaze for each of the user's eyes, etc.) to correct for pupillary distance measurement errors. For example, in some embodiments, eye deviation measurements can be determined based on eye-related characteristics (informed by monitoring occurring during stimulus presentation) and can be used to determine characteristics related to one or more of the user's pupils and to provide an improved user experience by adjusting the display of the wearable device (e.g., by mitigating eye / vision related challenges). As an example, the deviation measurements can inform deviations of an eye relative to the other eye, and the deviation measurements can be used to determine and correct errors in determining the user's pupillary distance.

[0043]

[0052] Also, as another example, the deviation measurement can inform one or more gaze directions of one or more eyes of a user. For example, by determining the amount of deviation, the test subsystem 122 can determine that the gaze directions of one or more eyes of a user are consistent with one another. For example, the test subsystem 122 can determine based on characteristics related to the monitored eyes that the amount of deviation measured within one eye with respect to another eye satisfies a deviation measurement threshold. For example, in some embodiments, the deviation measurement threshold can be satisfied when the amount of deviation measured in one eye with respect to another eye is within a deviation measurement threshold range (e.g., 1 degree, 2 degrees, 3 degrees, 1 millimeter, 2 millimeters, 3 millimeters, etc.). For example, if one eye of a user deviates from another eye of the user within a range of 2-3 degrees, the test subsystem 122 can determine that the gaze directions of the user's eyes are consistent with one another. In some embodiments, the deviation measurement threshold can be satisfied when the amount of deviation measured within one eye with respect to another eye is less than the deviation measurement threshold. For example, if one of the user's eyes deviates from the other of the user's eyes by less than 2 degrees, the test subsystem 122 may determine that the gaze directions of the user's eyes are consistent with each other. However, in some embodiments, if the test subsystem 122 determines that the gaze directions of the user's eyes are not consistent with each other, the test subsystem 122 may be configured to verify the deviation measurement (e.g., by performing one or more additional visual test presentations as described herein). In some embodiments, in response to determining that the gaze directions of the eyes are consistent with each other, the user's pupillary distance may be determined. For example, the user's pupillary distance may be determined based on the consistency determination and the locations of the first and second stimuli (e.g., stimuli presented during the visual test presentation). In other embodiments, the user's pupillary distance may be determined based on the distance between the locations of the first and second stimuli and the display of the wearable device.However, in some embodiments, in response to determining that the eye gaze directions are inconsistent with one another, the user's pupillary distance may also be determined based on (i) the individual locations of the stimuli on the displays, (ii) the distance between the displays of the wearable device, and (iii) a deviation measurement of the eye gaze.

[0044]

[0053] In some embodiments, the test subsystem 122 can determine deviation measurements or other visual deficiency information of a user's first eye by (i) causing a stimulus to be presented at a corresponding location on a first user interface for the first eye and at a corresponding location on a second user interface for the second eye at a given time, and (ii) determining the visual deficiency information based on one or more eye-related characteristics of the first eye that occur during the stimulus presentation. As an example, the stimulus can be presented at a center location on both user interfaces or at another corresponding location on both user interfaces (e.g., a location corresponding to another location with respect to both user interfaces, a location on one user interface that is the same as the second user interface, etc.). In one use case, upon presentation of a stimulus in front of both eyes (e.g., FIG. 4B), the dominant eye (e.g., the left eye in FIG. 4B) will instinctively move (e.g., within less than a second) to the corresponding position and fixate on the stimulus. The other eye (e.g., the right eye in FIG. 4B) will also move, but will not instinctively fixate the stimulus because the other eye is crossed, which will cause the user to see double. For example, the other eye will move instinctively, but this instinctive movement will result in the gaze direction of the other eye being toward a different location. However, when the user focuses the observation of the stimulus with the user's other eye, the other eye will move and fixate on the stimulus presented at the corresponding location on the user interface of the other eye. Since the stimuli are presented at corresponding locations on both user interfaces, the dominant eye will remain in a dominant state and continue to fixate on the stimulus presented at the corresponding location on the user interface of the dominant eye. The test subsystem 122 can measure the corrective movement of the other eye (as well as other changes in eye-related characteristics of the other eye) to determine a deviation measurement for the other eye (e.g., the amount of movement of the other eye can correspond to the amount of crossing of the other eye).

[0045]

[0054] In some embodiments, after obtaining a deviation measurement or other visual deficiency information of the user's first eye by measuring a change in an eye-related characteristic of the first eye (e.g., a movement of the first eye occurring upon presentation of a stimulus at a corresponding location on the first user interface for the first eye), the test subsystem 122 can cause the stimulus to be presented at the changed location on the first user interface for the first eye display. As an example, presentation of the stimulus at the changed location occurs in a state where the stimulus is not presented in the second user interface for the second eye (or at least in a state where the stimulus intensity of the second user interface does not meet a stimulus intensity threshold to prevent the second eye from responding to any stimulus in the second user interface). Based on one or more eye-related characteristics of the first eye or second eye not changing beyond a change threshold upon presentation at the changed location, the test subsystem 122 can confirm the deviation measurement or other visual deficiency information of the first eye. As an example, the deviation measurement of the first eye can be confirmed based on the first eye not moving beyond a movement threshold (e.g., no movement or other movement threshold) upon presentation of the stimulus at the changed location. Additionally or alternatively, the deviation measurement of the first eye may be confirmed based on the second eye not moving beyond its motor threshold, such that the deviation measurement of the user's eye may be confirmed, which may inform the display of the wearable device to be appropriately adjusted for the user.

[0046]

[0055] In some embodiments, the testing subsystem 122 can generate one or more user profiles associated with the user based on one or more deviation measurements or other visual deficiency information of one or more eyes of the user (e.g., obtained via one or more visual test presentations). As an example, each of the user profiles can include parameters or functions used to determine the user's pupillary distance or to adjust / correct the distance between one or more lenses or displays of the wearable device.

[0047]

[0056] In some embodiments, one or more visual tests may be performed to determine whether a deviation of the user's eye exists, to measure the deviation of the user's eye, to obtain feedback related to one or more eye characteristics, to generate one or more user profiles based on the eye-related characteristics, to measure the user's pupil distance, or to adjust the lens / display of the wearable device. In some embodiments, the same visual test may be repeated to confirm the results obtained by the system. In other embodiments, different visual tests may be performed based on one or more previous test results (e.g., based on feedback obtained from previous tests). The terms "location" and "place" referred to herein may be used interchangeably. In addition, the following paragraphs may refer to objects, distances, stimuli, or other items such as "first" and "second," but it should be noted that the terms "first" and "second" may inform different items, the same items, order of items, or other notices, and that one of ordinary skill in the art would understand that the context in which these terms are used should be interpreted as non-limiting.

[0048]

[0057] In one use case, with reference to FIG. 4A, when a stimulus 402 is presented to a patient (e.g., a patient without esotropia) at a central position on the right and left displays 403a and 403b of a wearable device and the distance between the displays is correct (e.g., based on the characteristics related to the user's pupils), both eyes (e.g., right and left eyes 404a and 404b) will instinctively move to and fixate the stimulus 402 at the central position on the respective wearable displays, and thus the patient will only observe one stimulus 402. In addition, as shown in FIG. 4A, the user's gaze directions are also consistent with each other (e.g., the collimation lines of the user's respective eyes 404a and 404b do not deviate with respect to each other). Thus, based on the above-mentioned eye responses, the test subsystem 122 can determine that the user does not have diplopia.

[0049]

[0058] In another use case, because the wearable device has two displays (e.g., right and left displays 403a and 403b), the distance between the displays (e.g., display distance 406) may be a general distance that is not currently set for the current user (e.g., not set for the user's pupillary distance). This may result in the user viewing two stimuli (e.g., stimulus 402), which may cause the user to experience eye strain, blurred vision, astigmatism, headaches, or other eye / vision-related challenges. Thus, based on the eye responses described above, the testing subsystem 122 may determine that the user has double vision and may experience one of the eye / vision-related challenges described above. However, the eye / vision-related challenges may be mitigated by determining the user's correct pupillary distance or by adjusting the display distance 406 based on the user's pupillary distance. Additionally or alternatively, the visioning subsystem 124 may generate a user profile that includes the user's correct pupillary distance. The user profile may then be used to perform future adjustments of the wearable device's display (e.g., in cases where the wearable device may be worn by multiple users) or may be provided to one or more service providers to customize lenses or frames for the user.

[0050]

[0059] 4A, if only one stimulus for one eye of a user (e.g., for the left eye 404b) is presented to the user at a first time point, the left eye 404b will fixate the stimulus 402 on the left display 403b. If another stimulus 402 is presented only for the right eye 404a at a time point after the first time point, and the stimulus presented for the left eye 404b is no longer visible to the user (e.g., the stimulus presented for the left eye 404b is not presented, is presented at a stimulus intensity threshold such that the left eye 404b cannot observe it, etc.), then the right eye 404a will not move (because the user does not have esotropia) and the testing subsystem 122 can determine that there is no eye movement that exceeds the eye movement threshold amount. Based on the user's eyes not exceeding the threshold amount, the testing subsystem 122 can determine the user's pupillary distance and the distance between the left and right display portions (e.g., display distance 406) based on the location of the stimulus 402 in relation to each of the left and right display portions 403b, 403a. In some embodiments, a user profile can be generated by the visioning subsystem 124 that includes the user's determined pupillary distance.

[0051]

[0060] In another use case, with reference to FIG. 4B, when the stimuli 402 are presented to a patient with esotropia at a central position on the right and left displays of the wearable device, one of the eyes (e.g., the dominant eye) will instinctively move to the central position and fixate on the stimulus 402 (e.g., the left eye 404b will instinctively fixate on the stimulus 402). The other eye (e.g., the right eye 404a) will also move but will not fixate on the stimulus 402 because the other eye is crossed, which will cause the user to see double (e.g., the user sees two stimuli instead of one). For example, the other eye will instinctively move, but this instinctive movement will result in the gaze direction of the other eye being towards a different position. Based on the above eye responses, the test subsystem 122 can determine that the user has diplopia. However, in a further use case, when a user focuses the observation of the stimulus 402 with the user's other eye (e.g., right eye 404a having esotropia), the other eye will move to and fixate the stimulus 402 presented at a central location on the user interface of the other eye. Because the stimulus 402 is presented at a central location on both displays 403a and 403b, the dominant eye will remain in a dominant state and will continue to fixate the stimulus 402 presented at a central location on the display of the dominant eye. To determine a deviation measure of the other eye, the corrective movement of the other eye (as well as other changes in eye-related characteristics of the other eye) can be measured (e.g., the amount of movement of the other eye can correspond to the amount of crossing of the other eye). Based on the deviation measure of the other eye exceeding a threshold amount of movement, the test subsystem 122 can determine the user's pupillary distance. In some embodiments, the wearable device can adjust / update the display distance 406 based on the determined pupillary distance. Additionally or alternatively, based on the deviation measurement of the other eye and the display distance 406, the visioning subsystem 124 can determine the user's pupillary distance in order to adjust the display distance 406.As a result, adjustment of the wearable device based on the user's pupillary distance can improve the user's experience, for example by alleviating eye strain, headaches, blurred vision, double vision, astigmatism, or other eye-related issues such as those related to improper display positioning for the user.

[0052]

[0061] In some cases, the test subsystem 122 may generate, retrieve from storage, or be configured to present one or more vision tests to the user to measure the user's pupillary distance. To accurately determine the user's pupillary distance or to verify whether the user may have double vision, the stimuli presented to the user on the respective displays of the wearable device may be presented at different time points (e.g., t1, t2, t3, t4, etc.). As a result, the one or more eye tracking sensors may be able to more accurately determine one or more eye characteristics of the user (e.g., eye movement, gaze direction, change in pupil size, etc.) in association with the stimulus presentation at such presentation time points.

[0053]

[0062] For example, with reference to FIG. 4C, at time t1, a stimulus (e.g., stimulus 402) can be presented at a location to only the left eye 404b by presenting a stimulus on the left display 403b and not on the right display 403a. For example, if a stimulus is presented at a location to both eyes 404a and 404b as shown in FIG. 4B immediately prior to the presentation of the stimulus to only the left eye 404b (e.g., at time t0 immediately prior to the presentation of the stimulus at time t1), the left eye 404b will not move because it is already fixating on the central location. However, if the left eye 404b is not yet fixating on the central location, the presentation of the stimulus to only the left eye 404b will cause the left eye 404b to instinctively move to the location of the stimulus and fixate on the stimulus 402. In some embodiments, this location may be the central location of the display. In other embodiments, based on a field of view having one or more defects detected by the system, the test subsystem 122 can adjust / select the location of the stimulus 402 to be located at a different position so that the user can observe the stimulus 402 (e.g., moved to the right or left, up or down).

[0054]

[0063] Additionally or alternatively, when a stimulus (e.g., stimulus 402) is presented to the left eye 404b on the left display at time t1, the stimulus can also be presented to the right eye 404a to the extent that the stimulus intensity of the stimulus presented to the right eye 404a does not meet the stimulus intensity threshold. For example, as shown in FIG. 4C, the stimulus 402 presented to the left eye 404b can meet the stimulus narrowing threshold to the extent that the user can observe the stimulus. However, the stimulus can also be simultaneously presented to the right eye 404a, but the stimulus presented to the right eye 404a may not meet the stimulus intensity threshold. For example, the stimulus intensity threshold can be a threshold level of luminance (e.g., measured in units of nits or other metric). In some embodiments, the stimulus intensity threshold can be met if the stimulus intensity meets or exceeds the stimulus intensity threshold. In other embodiments, the stimulus intensity threshold can be met if the stimulus intensity does not meet or exceed the stimulus intensity threshold. In some embodiments, if the stimulation intensity threshold is a range (e.g., 1000 nits to 1100 nits), the stimulation intensity threshold may be satisfied if the presented stimulus is within the stimulation intensity threshold range. In other embodiments, if the stimulation intensity threshold is a range, the stimulation intensity threshold may be satisfied if the presented stimulus is not within (e.g., falls outside) the stimulation intensity threshold range.

[0055]

[0064] As shown in FIG. 4D, a stimulus (e.g., stimulus 402) can be presented only to the right eye 404a at a location (e.g., at time t2) by presenting a stimulus on the right display 403a and not presenting a stimulus on the left display 403b. In some embodiments, the stimulus presented to the right eye 404a can occur subsequent to the presentation of the stimulus 402 on the left display 403b. In some embodiments, when the stimulus is presented to the right eye 404a on the right display at time t2, the stimulus can also be presented to the left eye 404b to the extent that the stimulus intensity of the stimulus presented to the left eye 404b does not meet the stimulus intensity threshold (e.g., does not stimulate the left eye 404b). Since the left eye 404b is not stimulated (e.g., has nothing to look at), the left eye 404b will lose its dominant state and will therefore move outward as a result of the right eye 404a taking over. When the stimulus 402 is presented only to the right eye 404a, the right eye 404a will instinctively take over and move to fixate a location. The test subsystem 122 can measure the movement of the right eye 404a to determine a deviation metric for the right eye 404a (e.g., the amount of movement can correspond to the amount of crossing of the right eye 404a). For example, the deviation metric can be determined by obtaining feedback from one or more eye tracking sensors as described above, and the deviation metric can be measured for each eye (e.g., right and left eyes 404a, 404b) for one or more vision tests. The feedback obtained from the eye tracking sensors can include one or more eye characteristics of the user's eyes (e.g., eye movement, eye gaze, pupil size, or other eye-related characteristics).

[0056]

[0065] In some embodiments, the visioning subsystem 124 can compare the deviation measure to a deviation threshold (e.g., a threshold amount of movement). The deviation threshold can have an integer, floating point value, binary value, percentage, or other value that informs a metric of deviation of one or more of the user's eyes. For example, the deviation measure and deviation threshold can be measured in degrees, millimeters, inches, polar coordinate pairs, Cartesian coordinate pairs, and the like. The deviation threshold can be a pre-set value based on a set of user or patient data that informs an acceptable amount of eye deviation (e.g., amount of eye movement, amount of gaze direction) for determining double vision, blurred vision, or other eye-related issues. The deviation threshold can be stored for retrieval in one or more storage devices on the server 102, the client device 104, or other database.

[0057]

[0066] In response to determining that the deviation measure of the user's left or right eye 404b, 404a exceeds the deviation threshold amount, the visioning subsystem 124 can generate a user profile for the user that includes the user's pupillary distance. In some cases, the deviation measure of the left or right eye 404b, 404a can be associated with (e.g., in association with) the stimulus presentation at time t2, and the user's pupillary distance can be determined based on the deviation measure of the left or right eye 404b, 404a.

[0058]

[0067] In various embodiments, the user's pupillary distance can be determined. For example, the visioning subsystem 124 can determine the user's pupillary distance based on the movement of one or more eyes of the user. In some embodiments, the visioning subsystem 124 can determine the user's pupillary distance based on deviation measurements of the user's left or right eye 404b, 404a. In some embodiments, the visioning subsystem 124 can determine the user's pupillary distance based on deviation measurements of only one eye of the user (e.g., only the left eye 404b, only the right eye 404a). In other embodiments, the visioning subsystem 124 can determine the user's pupillary distance based on deviation measurements of both eyes of the user (e.g., the left and right eyes 404b, 404a). In some embodiments, the visioning subsystem 124 can determine the user's pupillary distance based on the location of the respective stimuli presented on the right or left display 403a, 403b in relation to the distance between the right and left displays 403a, 403b (e.g., display distance 406). Additionally, in some embodiments, the user's pupillary distance may be determined by feeding the eye movement of the user's eyes (e.g., deviation measurements) and the location of the stimulus presented to the user to a predictive model, which can generate an output including characteristics related to one or more pupils of the user (e.g., the user's pupillary distance).

[0059]

[0068] In some embodiments, in response to the deviation measurement of at least one eye (e.g., right eye 404a, left eye 404b) of the user exceeding a deviation threshold, the user's pupillary distance can be determined by comparing the deviation measurement of one eye (e.g., right eye 404a) of the user to the other eye (e.g., left eye 404b). For example, the visioning subsystem 124 can compare the deviation measurement of each eye and can average the deviation measurements into one. Based on the average of the deviation measurements, the pupillary distance can be calculated by adding or subtracting the average deviation measurement to a pre-set distance (e.g., display distance 406). The user's pupillary distance can be determined because the distance between the displays (e.g., left and right displays 403b, 403a) of the wearable device is already known (e.g., as stored within one or more storage devices, servers, databases, on-board memory, etc. of the wearable device). Based on the determination of the user's pupillary distance, a user profile can be generated or updated to reflect the user's measured pupillary distance. Additionally or alternatively, the user's measured pupillary distance can also be used to adjust / correct the distance between the displays of the wearable device, so that, for example, the wearable device can automatically improve the user experience by alleviating eye strain, headaches, blurred vision, double vision, astigmatism, or other eye-related issues such as those related to display positioning or shape that is not suitable for the user.

[0060]

[0069] In some embodiments, the user's pupil distance can be determined based on (i) the distance between a stimulus (e.g., stimulus 402) and an edge of a respective display (e.g., left or right display 403b, 403a) and (ii) the distance between the respective displays of the wearable device (e.g., display distance 406). As shown in FIG. 4E, the stimulus 402 is presented at a location on the right display 403a and on the left display 403b. Such presentation of the stimulus can be associated with a distance between a center of the stimulus (e.g., stimulus 402) and one or more edges of the display. For example, the stimulus distances 408a-408d can represent the distance between the stimulus and an edge of the respective display. It should be noted that while only stimulus distances 408a-408d are shown, any distance (horizontal, vertical, diagonal, or the like) may be present and the horizontal stimulus distances 408a-408d are for illustrative purposes only. Because the testing subsystem 122 knows the location of the stimulus 402 relative to each display and the size of each display, the stimulus distances 408a-408d can be readily determined. Similarly, because the wearable device may already be set for the distance between the displays (e.g., display distance 406), the display distance 406 can also be readily determined, as it has already been set. In one use case, if the user's eyes have not moved beyond a deviation threshold amount (e.g., the user does not have diplopia or has esotropia), the user's pupillary distance can be determined based on (i) the stimulus distance 408a or 408b, (ii) the stimulus distance 408c or 408d, and (iii) the display distance 406.

[0061]

[0070] In some embodiments, a user profile may be generated. For example, based on feedback (e.g., eye movement, gaze direction, etc.) obtained in one or more vision tests, the visioning subsystem 124 may generate a user profile for the user. The user profile may include information related to one or more pupil-related characteristics of the user (e.g., pupillary distance between one or more pupils of the user, one or more pupillary sizes of the user's pupils, one or more radii of curvature of the user's pupils, or other pupillary characteristics of the user, etc.). In some embodiments, multiple user profiles may be generated in a manner such that each user profile is associated with one or more pupil-related characteristics. For example, a first user profile may be associated with pupillary distance and a second user profile may be associated with eye movement of one or more eyes of the user. As a result, a large amount of information related to the user may be stored in a manner such that one or more user profiles may be shared among other wearable devices, medical professionals, other users, or databases. For example, the user's optometrist may be interested in such information obtained during a vision test to generate customized eyeglasses for the user, to update one or more medical records of the user, or for other uses. As a result, by generating a user profile, the user profile may be shared and used by various sources, and detailed pupil-related characteristic information may be obtained that may be useful to medical professionals treating the user. Furthermore, one or more user profiles may be shared prior to an optometrist appointment with the user, so that the optometrist may have good notice of the user's ocular health. As a result, time consumed during such appointments may be saved, and more accurate eye examination results may be generated, which may help benefit not only the user, but also the optometrist.

[0062]

[0071] As shown in FIG. 4F, the position of one or more of the left or right displays 403b, 403a can be adjusted based on a user profile (e.g., a characteristic related to the user's pupils). As an example, the display distance 406 can be updated, adjusted, or otherwise corrected to an updated display distance 410 based on the user's determined pupillary distance. As a result, the position of the displays (e.g., the left display 403b, the right display 403a, etc.) of the wearable device is adjusted. For example, as previously shown in FIG. 4E, the display distance 406 is set to a first display distance. Upon determining the user's pupillary distance, the display distance 406 can be set, updated, adjusted, or otherwise corrected to a second display distance (e.g., the updated display distance 410) to provide relief from eye-related challenges (e.g., eye strain, headaches, blurred vision, astigmatism, or other eye-related challenges). Referring again to FIG. 4F, although the updated display distance 410 appears to be a larger distance, it should be noted that the updated display distance 410 is for illustrative purposes only and may actually be a smaller distance based on the determined pupil distance.

[0063]

[0072] In some cases, the crossing of the eyes alternates. This is because the user's eye is not dominant. Therefore, further tests can be performed to confirm the deviation measurement of the user's eyes. As shown in FIG. 4G, a stimulus (e.g., stimulus 402) can be presented to both eyes 404a and 404b at a predetermined position (e.g., at time t3) by presenting a stimulus on the left display 403b and on the right display 403a. In some embodiments, the stimulus can be presented at a center location (e.g., middle, center, etc.) of each display. If the crossing alternates (neither eye is dominant), the right eye 404a will remain in a fixated position, and the left eye 404b will remain in a crossed position. However, if the left eye 404b is dominant (as shown in FIG. 4G), the left eye 404b will instinctively move to and fixate on the predetermined position. Movement of the left eye 404b will cause the right eye 404a to cross and the gaze direction of the right eye 404a to be toward a different location. The testing subsystem 122 can measure the movement of the left eye 404b (e.g., via one or more eye tracking sensors) to determine or confirm the deviation measurement of the right eye 404a (e.g., the amount of movement of the left eye 404b can correspond to the amount of deviation of the right eye 404a).

[0064]

[0073] In further use cases, further tests can be performed to confirm the deviation metric of the non-dominant eye. For example, as shown in FIG. 4H, following one or more of the above steps described with respect to FIG. 4B-4E, a stimulus (e.g., stimulus 402) can be presented only to the left eye 404b at a predefined location (e.g., at time t4) by presenting a stimulus on the left display 403b and not on the right display 403a. To the extent that the left eye 404b loses gaze (e.g., due to the presentation in FIG. 4G), the presentation in FIG. 4H will cause the left eye 404b to instinctively move to obtain gaze at the predefined location. The movement of the left eye 404b will cause the right eye 404a to cross and the gaze direction of the right eye 404a to be toward a different location. The test subsystem 122 can determine the deviation metric of the right eye 404a (e.g., via one or more eye tracking sensors). In some embodiments, the testing subsystem 122 can validate the deviation measurement by comparing the new deviation measurement with the old one (e.g., a previous deviation measurement for the right eye 404a). By validating the deviation measurement for the user's eye or eyes, the system can use such deviation measurement information to determine (or select) a modified position of the stimulus in relation to the user's eye or eyes.

[0065]

[0074] For example, the test subsystem 122 can select a location for presentation of a stimulus based on one or more eye movements of the user (e.g., deviation, eye gaze, other eye movements, etc.). For example, if the right eye 404a is identified to have a deviation metric (e.g., not a straight line of sight), the deviation metric of the right eye can be used to determine a modified position for stimulus presentation on the right display 403a. The test subsystem 122 can select the determined modified position and present the stimulus (e.g., stimulus 402) at the modified position. As another example, if the left eye 404b is identified to have a deviation metric, the deviation metric of the left eye can be used to determine a modified position for stimulus presentation on the left display 403b. In some cases, if both eyes of the user have a deviation metric, the modified positions of the stimuli for each of the right and left displays 403a, 403b, respectively, can be based on the deviation metric of each individual eye of the user. 4I, based on the deviation measurement of the right eye 404a, a modified position for presenting the stimulus to the right eye 404a can be determined. Thus, the stimulus 402 can be presented in a center position on the left display 403b, while the stimulus 402 can be presented in a modified position on the right display 403b (e.g., at time t5).

[0066]

[0075] Thereafter, with respect to FIG. 4J, the stimulus 402 can be presented only to the right eye 404a (e.g., at time t6) by presenting the stimulus 402 at the altered position on the right display 403a and not on the left display 403b. Specifically, for example, the stimulus 402 deviates to the right by the same amount as the deviation measured in one or more of the above steps described with respect to FIGS. 4B-4E. If the deviation measurement is accurate, the right eye 404a will not move. If the deviation measurement is not accurate, the right eye 404a will move slightly, and the amount of movement can be measured by the wearable device (e.g., a wearable device pupil tracker), and the measurement of the slight movement can be used to fine-tune the deviation. As an example, the measurement and altered position can be used to determine an updated altered position for presenting the stimulus to the right eye 404a, and the updated altered position can be used to repeat one or more of the steps described with respect to FIGS. 4H-4J. Additionally or alternatively, one or more of the steps of Figures 4B-4E (e.g., re-determining the deviation measurement for the right eye 404a) may be repeated to re-determine the deviation measurement for one or more eyes of the user. Then, with reference to Figure 4K, the stimulus 402 may be presented (e.g., at time t7) to both eyes 404a and 404b by presenting the stimulus 402 in the changed position on the right display 403a and in a central position on the left display 403b. The user is no longer seeing double, since the stimulus 402 in front of the right eye 404a deviates to the right according to the deviation measurement (e.g., as determined or confirmed in one or more of the steps described above). As a result, the user's pupillary distance may be more accurately determined (e.g., because the stimulus 402 for the deviated eye is now aligned with the user's eye alignment / eye gaze).

[0067]

[0076] In some embodiments, during one or more vision tests, the test subsystem 122 may monitor the gaze direction of one or more eyes of the user to allow the test subsystem 122 to determine the user's pupillary distance (e.g., a characteristic related to one or more pupils of the user) and thereby cause one or more displays of the wearable device to be adjusted (e.g., based on the measured pupillary distance of the user). For example, the wearable device may include a pupil and collimation tracker to detect the gaze direction or other eye-related characteristic of the user's eye or eyes. The test subsystem 122 may determine that the user's gaze directions of the first and second eyes are consistent with each other when presented with respect to the presentation of the stimuli. In some embodiments, the location of the stimuli may be at the same location with respect to each display (e.g., the stimulus 402 is located at a central location with respect to each of the right display 403a and the left display 403b, respectively, the stimulus is located at the same non-central location with respect to each of the right display 403a and the left display 403b, respectively, etc.). In some embodiments, the test subsystem can measure deviations of one or more of the user's eyes during a vision test to determine a user's gaze direction with respect to each of the user's eyes fixating on a respective location of the stimuli 402, and can further determine that the gaze directions are consistent with each other. For example, in some embodiments, if the gaze directions of each eye meet a gaze direction threshold (e.g., within 1 degree, 2 degrees, 3 degrees, 1 millimeter, 2 millimeters, 3 millimeters, or other threshold amount) with respect to each other during presentation of the respective stimuli, the test subsystem 122 can determine that the gaze directions of each eye are consistent with each other. Based on the gaze directions being consistent with each other, (i) the respective locations of the stimuli on each display, (ii) the distance between the displays (e.g., the right display 403ba and the left display 403b), or (iii) one or more pupil-related characteristics of the user (e.g., the user's pupillary distance) can be determined based on the consistency determination.

[0068]

[0077] Based on feedback or other information obtained during one or more visual test presentations, the test subsystem 122 can determine that the gaze directions of the user's eyes that occur when the stimuli are presented at a first time point and a second time point (as described above with reference to FIGS. 4C-4E) are consistent with each other. For example, the test subsystem 122 can determine a first gaze location of a first eye at a first time point and a second gaze location of a second eye at a second time point via one or more eye tracking sensors. In some embodiments, the second time point may be a time point subsequent to the first time point. The test subsystem 122 can determine a first difference based on the location of the presented stimuli and the first gaze location for one eye of the user. For example, the user's right eye 404a may be fixating a location near, but not exactly at, the location of the stimuli presented on the right display 403a, and there may be a difference (e.g., in millimeters, degrees, or other metric) between the gaze location of the corresponding eye and the location of the stimuli. Similarly, the test subsystem 122 can determine a second difference based on the location of the presented stimuli and a second gaze location for the user's other eye (e.g., left eye 404b). The test subsystem 122 can compare the first difference to the second difference. For example, if the first difference and the second difference meet a predefined threshold difference (e.g., within 1 millimeter, 2 millimeters, 3 millimeters, 1 degree, 2 degrees, 3 degrees, etc.), the test subsystem can determine that the gaze directions of the user's eyes are consistent with one another. Based on the consistency determination (e.g., the gaze directions of one or more of the user's eyes are consistent with one another with respect to the presentation of the stimuli), one or more pupil-related characteristics (e.g., the user's pupillary distance) can be determined by the test subsystem 122, and one or more positions of one or more display portions of the wearable device can be adjusted based on the pupil-related characteristic (e.g., the user's pupillary distance).

[0069]

[0078] Additionally or alternatively, the test subsystem 122 can further determine that the gaze direction of the user's eyes is substantially perpendicular to the display of the wearable device. For example, by using one or more collimation trackers of the wearable device, the test subsystem can determine that the gaze direction is substantially perpendicular to the display of the wearable device (e.g., right display 403a, left display 403b, etc.). Based on the gaze direction being substantially perpendicular to the display of the wearable device, the test subsystem 122 can determine a more accurate pupil distance of the user, which can provide a more accurate adjustment of one or more displays of the wearable device.

[0070]

[0079] In some embodiments, the visioning subsystem 124 can provide (i) the gaze direction of the user's eye, (ii) the location of the presented stimulus, or (iii) other information to the predictive model. The predictive model can generate an output that includes one or more pupil-related characteristics of the user (e.g., the user's interpupillary distance or other pupil characteristics of the user). Based on the user's pupil-related characteristics, the wearable device can adjust one or more positions of a display portion of the wearable device. As a result, for example, the wearable device can automatically improve the user experience by alleviating eye strain, headaches, blurred vision, astigmatism, or other eye-related challenges, such as those related to a display positioning or shape that is not appropriate for the user.

[0071]

[0080] In some embodiments, the configuration subsystem 112 can store the predictive models, visual defect information (e.g., informing the user of a detected visual defect), feedback information (e.g., feedback related to stimuli displayed to the user or other feedback), or other information in one or more remote databases (e.g., in the cloud). In some embodiments, the feedback information, visual defect information, or other information related to multiple users (e.g., two or more users, ten or more users, one hundred or more users, one thousand or more users, one million or more users, or other number of users) can be used to train one or more predictive models. In one use case, if the predictive model being trained is a neural network or other machine learning model, the model manager subsystem 114 can provide (i) stimulus information (e.g., informing the set of stimuli and their associated characteristics, such as intensity levels, where the stimuli are displayed, etc.), and (ii) feedback information (e.g., informing feedback related to the set of stimuli) as inputs to the machine learning model to cause the machine learning model to predict the visual defect information or other information. The model manager subsystem 114 can provide baseline information (e.g., visual defect information) to the machine learning model. The machine learning model can evaluate its predicted output (e.g., predicted visual defect information, etc.) against the baseline information and can update its configuration (e.g., weights, biases, or other parameters) based on its evaluation of its predicted output. The above operations can be performed with further stimulus information (e.g., displayed to other users), further feedback information (e.g., feedback from these users related to the stimuli displayed to other users), and further baseline information to further train the machine learning model (e.g., by providing such information as input and baseline feedback to train the machine learning model and thereby enable the machine learning model to further update its configuration).

[0072]

[0081] In another use case, if the machine learning model is a neural network, the connection weights may be adjusted to compromise the difference between the neural network's predictions and the reference information. In a further use case, one or more neurons (or nodes) of the neural network may require their individual errors to be sent backwards through the neural network to facilitate the update process (e.g., backpropagating errors). The updates to the connection weights may, for example, reflect the error magnitudes propagated backwards after a forward pass is completed.

[0073]

[0082] In some embodiments, one or more predictive models can be trained or configured for a user or type of device (e.g., a particular brand of device, a particular brand and model of device, a device with a particular set of features, etc.) and stored in association with the user or device type. As an example, an instance of a predictive model associated with a user or device type can be stored locally (e.g., on the user's wearable device or other user device) or remotely (e.g., in the cloud), and such instance of a predictive model can be automatically or manually synchronized across one or more user devices and clouds to allow the user to have access to the latest configuration of the predictive model across any of the user devices or clouds. In one use case, upon detecting that a first user is using the wearable device (e.g., when the first user logs into the user's account or is identified via one or more other techniques), the configuration subsystem 112 can communicate with the wearable device to send the latest instance of the predictive model associated with the first user to the wearable device to allow the wearable device to have access to a local copy of the predictive model associated with the first user. In another use case, if a second user is detected to be later using the same wearable device, the configuration subsystem 112 can communicate with the wearable device to send the latest instance of the predictive model associated with the second user to the wearable device to allow the wearable device to have access to a local copy of the predictive model associated with the second user.

[0074]

[0083] 5 is an exemplary flow chart of method operations enabling various features and functions of the system detailed above. Each method operation presented below is for illustrative purposes and not for limitation. In some embodiments, for example, a method can be implemented with one or more additional operations not described or without one or more of the described operations. Additionally, the order in which the method operations are presented (described below) is not intended to be limiting.

[0075]

[0084] In some embodiments, the methods may be implemented in one or more processing devices (e.g., digital processors, analog processors, digital circuits designed to process information, analog circuits designed to process information, state machines, or other mechanisms for electronically processing information). A processing device may include one or more devices that perform some or all of the operations of the methods in response to instructions stored electronically in an electronic storage medium. A processing device may include one or more devices configured through hardware, firmware, or software specifically designed for performing one or more of the operations of the methods.

[0076]

[0085] FIG. 5 illustrates a flowchart of a method 500 for facilitating pupil distance detection and spectacle correction / adjustment based thereon in accordance with one or more embodiments.

[0077]

[0086] In operation 502, a first stimulus can be presented at a location on a first display of the head-mounted device at a first time point. For example, the stimulus can be presented for the first eye at a location on the first display of the head-mounted device at a first time point, such that the first eye of the user gazes at a location on the first display of the head-mounted device. In some embodiments, the stimulus can include a light stimulus, text, or an image that is displayed to the user. In some embodiments, the location can be a central location (e.g., a center, middle, or other central location) with respect to the display. In other embodiments, the location can be a non-central location (e.g., non-central) with respect to the display. Operation 502 can be performed by a subsystem that is the same as or similar to the testing subsystem 122 according to one or more embodiments.

[0078]

[0087] In operation 504, a second stimulus can be presented at a location on a second display of the head-mounted device at a second time point. For example, the stimulus can be presented for a second eye of the user at a location on the second display of the head-mounted device at a second time point subsequent to the first time point. In some embodiments, the stimulus can include a light stimulus, text, or an image that is displayed to the user. In some embodiments, the location can be a central location (e.g., a center, middle, or other central location) with respect to the display. In other embodiments, the location can be a non-central location (e.g., non-central) with respect to the display. Operation 504 can be performed by a subsystem that is the same as or similar to the testing subsystem 122, according to one or more embodiments.

[0079]

[0088] In operation 506, it can be determined whether the movement of the user's eye exceeds a threshold amount of movement. For example, a determination can be made as to whether the movement of the first or second eye (e.g., the user's eye) to gaze at a location on the respective first or second display in association with the stimulus presentation at the second time point exceeds a threshold amount of movement. In some embodiments, it can be determined that the movement of the first or second eye to gaze at a location on the respective first or second display in association with the stimulus presentation at the second time point exceeds a threshold amount of movement. Operation 506 can be performed by a subsystem the same as or similar to the testing subsystem 122.

[0080]

[0089] In operation 508, a user profile may be generated. For example, the user profile may have a pupillary distance for the user based on the user's eye movements. For example, in response to determining that the first or second eye movements exceed a threshold amount of movement, a user profile may be generated having a pupillary distance associated with the user, the pupillary distance being determined based on the first and second eye movements and defining a distance between the pupils of the first and second eyes. Operation 508 may be performed by a subsystem the same as or similar to the testing subsystem 122.

[0081]

[0090] In some embodiments, the various computers and subsystems shown in FIG. 1A may include one or more computing devices programmed to perform the functions described herein. A computing device may include one or more electronic storage (e.g., one or more prediction databases 132, which may include one or more training databases 134, one or more model databases 136, etc., or other electronic storage), one or more physical processors programmed with one or more computer program instructions, or other components. A computing device may include communication lines or ports to enable the exchange of information with a network (e.g., network 150) or other computing platforms via wired or wireless techniques (e.g., Ethernet, fiber optic, coaxial cable, WiFi, Bluetooth, near field communication, or other techniques). A computing device may include multiple hardware, software, or firmware components working together. For example, a computing device may be implemented by a cloud of computing platforms working together as a computing device.

[0082]

[0091] Electronic storage may include non-transitory storage media that electronically stores information. The electronic storage media of electronic storage may include one or both of: (i) system storage that is provided integrally with the server or client device (e.g., substantially non-removable), or (ii) removable storage that is removably connectable to the server or client device, for example, via a port (e.g., USB port, Firewire port, etc.) or device (e.g., disk drive, etc.). Electronic storage may include one or more of an optically readable storage medium (e.g., optical disk, etc.), a magnetically readable storage medium (e.g., magnetic tape, magnetic hard drive, floppy drive, etc.), a charge-based storage medium (e.g., EEPROM, RAM, etc.), a solid-state storage medium (e.g., flash drive, etc.), or other electronically readable storage medium. Electronic storage may include one or more virtual storage resources (e.g., cloud storage, virtual private network, or other virtual storage resources). The electronic storage may store information determined by a software algorithm, a processor, obtained from a server, obtained from a client device, or other information that enables the functions described herein.

[0083]

[0092] A processor may be programmed to provide information processing capabilities within a computing device. Thus, a processor may include one or more of a digital processor, an analog processor, a digital circuit designed to process information, an analog circuit designed to process information, a state machine, or other mechanism for electronically processing information. In some embodiments, a processor may include multiple processing units. These processing units may be located within the same physical device, or a processor may represent the processing functions of multiple devices working together. A processor may be programmed to execute computer program instructions to perform the functions described herein of subsystems 112-124 or other subsystems. A processor may be programmed to execute computer program instructions by software, hardware, firmware, any combination of software, hardware, or firmware, or other mechanisms for configuring processing capabilities in a processor.

[0084]

[0093] It should be understood that the descriptions of the functionality provided by the various subsystems 112-124 described herein are for illustrative purposes and are not intended to be limiting, as any of the subsystems 112-124 may provide more or less functionality than that described. For example, one or more of the subsystems 112-124 may be eliminated and some or all of its functionality may be provided by other of the subsystems 112-124. As another example, additional subsystems may be programmed to perform some or all of the functionality attributed to one of the subsystems 112-124 herein.

[0085]

[0094] Although the present invention has been described in detail for illustrative purposes based on what are presently considered to be the most practical or preferred embodiments, it should be understood that such details are for illustrative purposes only, and that the present invention is not limited to the disclosed embodiments, but rather is intended to cover modifications and equivalent arrangements within the scope of the appended claims. For example, it should be understood that the present invention contemplates that, to the extent possible, one or more features of any embodiment can be combined with one or more features of any other embodiment.

[0086]

[0095] The present technique may be more fully understood with reference to the enumerated embodiments below.

[0087] A1. A method comprising: causing a first stimulus to be presented at a first display location at a first time point; causing a second stimulus to be presented at a second display location at a second time point; determining whether a user's eye movement exceeds a threshold amount of movement in response to the presentation of the first or second stimulus; and generating a user profile having one or more pupil-related characteristics associated with the user based on the determination and the eye movement.

[0088] A2. The method of embodiment A1, further comprising: causing a first eye of a user to gaze at a location of a first stimulus on a first display based on presentation of a first stimulus on the user device; causing a second eye of the user to gaze at a location of a second stimulus based on presentation of a second stimulus on the user device; determining via one or more eye tracking sensors whether movement of the first or second eye to gaze at a location on the respective first or second display in association with the presentation of the stimulus at a second time exceeds a threshold amount of movement; and in response to a determination that movement of the first or second eye exceeds the movement threshold amount, generating a user profile having one or more pupil-related characteristics associated with the user.

[0089] A3. The method of any one of embodiments A1-A2, wherein the location on the first display is a central location on the first display and the location on the second display is a central location on the second display.

[0090] A4. A method of any one of embodiments A1-A2, wherein generating a user profile includes generating a user profile in response to a determination that movement of the second eye exceeds a threshold amount of movement in association with presentation of a stimulus at a second time, and one or more pupil-related characteristics are determined based on the movement of the second eye.

[0091] A5. The method of any one of embodiments A1 to A4, further comprising adjusting one or more positions of the first or second display based on one or more pupil-related characteristics of the user profile in response to a determination that the movement of the first or second eye exceeds a threshold amount of movement, such that the adjustment changes the distance between the first and second displays from a first distance to a second distance that is greater at the user device.

[0092] A6. The method of embodiment A5, wherein adjusting one or more positions of the first or second display includes adjusting one or more positions of the first or second display in response to a determination that second eye movement exceeds a threshold amount of movement associated with the presentation of the stimulus at a second time based on one or more pupil-related characteristics of the user profile, wherein the one or more pupil-related characteristics are determined based on the second eye movement.

[0093] A7. A method of any one of embodiments A5 to A6, wherein adjusting one or more positions of the first or second display includes adjusting a position of the first display and a position of the second display based on one or more pupil-related characteristics of a user profile, and the one or more pupil-related characteristics are determined based on movement of the second eye.

[0094] A8. The method of any one of embodiments A1-A7, further comprising: causing a first stimulus to be presented on a first display of a user device and a second stimulus to be presented on a second display of the user device; determining via one or more eye tracking sensors whether gaze directions of the first and second eyes, occurring individually in association with the presentation of the first and second stimuli, are consistent with each other; and in response to a determination that the gaze directions of the first and second eyes are consistent with each other, determining one or more pupil-related characteristics associated with the user based on (i) a location of the first stimulus on the first display, (ii) a location of the second stimulus on the second display, and (iii) a distance between the first and second displays.

[0095] A9. The method of any one of embodiments A1-A8, wherein determining one or more pupil-related characteristics includes, in response to determining that the gaze directions of the first and second eyes are both substantially perpendicular to the first and second displays, respectively, determining one or more pupil-related characteristics associated with the user based on (i) a location of the first stimulus on the first display, (ii) a location of the second stimulus on the second display, and (iii) a distance between the first and second displays.

[0096] A10. A method of any one of embodiments A1 to A9, further comprising selecting a location of the first or second stimulus based on movement of the first or second eye, and wherein presentation of the first and second stimuli is based on the selection of the location of the first or second stimulus.

[0097] A11. A method of any one of embodiments A1 to A10, further comprising selecting locations of the first and second stimuli based on movement of the first or second eye, and wherein presentation of the first and second stimuli is based on the selection of locations of the first and second stimuli.

[0098] A12. The method of any one of A1-A11, wherein the presentation of the stimulus at the second time point occurs in a state where the stimulus is not presented on the first display.

[0099] A13. A method according to any one of embodiments A1 to A12, wherein the stimulus presentation at the first time point occurs in a state where the stimulus is not presented on the second display, and the stimulus presentation at the second time point occurs in a state where the stimulus is not presented on the first display.

[0100] A14. A method of any one of embodiments A1 to A13, wherein the stimulation presentation at the first time point occurs in a state where the stimulation intensity of the second display does not satisfy the stimulation intensity threshold, and the stimulation presentation at the second time point occurs in a state where the stimulation intensity of the first display does not satisfy the stimulation intensity threshold.

[0101] A15. The method of any one of embodiments A1-A14, wherein one or more of the above operations are performed by a user device.

[0102] A16. The method of any one of embodiments A1-A15, wherein the user device includes a wearable device.

[0103] A17. The method of embodiment A16, wherein the wearable device includes one or more display portions configured to display one or more images (e.g., stimuli).

[0104] A18. The method of any one of embodiments A16-A17, wherein the one or more display portions include first and second display portions of a wearable device.

[0105] A19. The method of any one of embodiments A16-A18, wherein the wearable device includes a first monitor including a first display portion, and a second monitor including a second display portion.

[0106] A20. The method of any one of embodiments A16-A19, wherein the one or more display portions include one or more dynamic display portions on one or more transparent displays of the wearable device.

[0107] A21. The method of any one of embodiments A16-A20, wherein the wearable device includes a wearable spectacle device.

[0108] A22. The method of any one of embodiments A1-A21, wherein the characteristic relating to the one or more pupils includes a pupillary distance between the user's pupils, one or more pupillary sizes of the user's one or more pupils, one or more radii of curvature of the user's one or more pupils, or other pupillary characteristics of the user's one or more pupils.

[0109] B1. A method comprising: causing a stimulus to be presented for a first eye of a user on a first display at a first time point; causing a second stimulus to be presented for a second eye of the user on a second display at a second time point; determining whether gaze directions of the first and second eyes include consistency with respect to the presentation of the stimulus at the first and second time points; and determining one or more pupil-related characteristics of the user based on the consistency determination and the locations of the first and second stimuli.

[0110] B2. The method of embodiment B1, further comprising: causing a first eye of a user to gaze at a location of a first stimulus on a first display of a user device based on a stimulus being presented for the first eye at a first time point; causing a second eye of the user to gaze at a location of a second stimulus on a second display of the user device based on a second stimulus being presented for the second eye at a second time point subsequent to the first time point; determining via one or more eye tracking sensors whether gaze directions of the first and second eyes occurring individually in association with the presentation of the stimulus at the first and second time points include consistency with each other; and in response to a determination that the gaze directions of the first and second eyes include consistency with each other, determining one or more pupil-related characteristics of the user based on the consistency determination and the locations of the first and second stimuli.

[0111] B3. The method of any one of embodiments B1-B2, further including adjusting one or more positions of the first or second display based on the one or more pupil-related characteristics in response to determining one or more pupil-related characteristics of a user, such that the adjustment changes a distance between the first and second displays from a first distance to a second distance that is spaced apart on the user device.

[0112] B4. The method of any one of embodiments B2-B3, wherein determining whether the gaze directions of the first and second eyes occurring individually in association with the presentation of stimuli at the first and second time points include consistency with each other further includes: determining a first gaze location of the first eye at the first time point and a second gaze location of the second eye at the second time point via one or more eye tracking sensors; determining a first difference, the first difference being based on the location of the stimulus presented for the first eye and the first gaze location; determining a second difference, the second difference being based on the location of the stimulus presented for the second eye and the second gaze location; comparing the first difference to the second difference; and determining that the gaze directions of the first and second eyes include consistency with each other in response to the first difference and the second difference satisfying a predetermined threshold difference.

[0113] B5. The method of any one of embodiments B1-B4, wherein determining characteristics related to one or more pupils of the user is further based on a location of the first stimulus on the first display and a location of the second stimulus on the second display.

[0114] B6. The method of any one of embodiments B1-B5, wherein the presentation of the stimulus at the second time occurs in a state where the stimulus is not presented on the first display.

[0115] B7. A method according to any one of embodiments B1 to B5, wherein the presentation of the stimulus at the first time occurs in a state where the stimulus is not presented on the second display, and the presentation of the stimulus at the second time occurs in a state where the stimulus is not presented on the first display.

[0116] B8. The method of any one of embodiments B1-B7, wherein determining characteristics related to one or more pupils of the user is further based on (i) individual distances between the location and respective edges of the first and second displays and (ii) a first distance between the first and second displays.

[0117] B9. The method of any one of embodiments B1-B8, wherein one or more of the above operations are performed by a user device.

[0118] B10. The method of embodiment B9, wherein the user device includes a wearable device.

[0119] B11. The method of embodiment B10, wherein the wearable device includes one or more display portions configured to display one or more images (e.g., stimuli).

[0120] B12. The method of any one of embodiments B10-B11, wherein the one or more display portions include first and second display portions of a wearable device.

[0121] B13. The method of any one of embodiments B10-B12, wherein the wearable device includes a first monitor including a first display portion and a second monitor including a second display portion.

[0122] B14. The method of any one of embodiments B10-B13, wherein the one or more display portions include one or more dynamic display portions on one or more transparent displays of the wearable device.

[0123] B15. The method of any one of embodiments B10-B14, wherein the wearable device includes a wearable spectacle device.

[0124] B16. The method of any one of embodiments B1-B15, wherein the characteristic relating to the one or more pupils includes a pupillary distance between the user's pupils, one or more pupillary sizes of the user's one or more pupils, one or more radii of curvature of the user's one or more pupils, or other pupillary characteristics of the user's one or more pupils.

[0125] C1. A tangible, non-transitory, machine-readable medium storing instructions that, when executed by a data processing apparatus, cause the data processing apparatus to perform operations including those of any one of embodiments A1-A22 or B1-B16.

[0126] C2. A system including one or more processors and a memory storing instructions that, when executed by the processor, cause the processor to perform operations including those of any one of embodiments A1-A22 or B1-B16.

Claims

1. 1. A spectacle device that customizes the position of its display based on eye tracking sensor data, comprising: first and second displays configured to be spaced a first distance apart from one another in the spectacle apparatus; one or more eye tracking sensors; one or more processors executing computer program instructions; which instructions, when executed, causing the first eye of the user to gaze at a central location on the first display for the first eye by causing a stimulus to be presented for the first eye at a first time point at a central location on the first display such that stimulus presentation at the first time point occurs in the absence of a stimulus being presented on a second display for the second eye of the user; presenting a stimulus for the second eye at the central location on the second display at a second time point subsequent to the first time point, such that stimulus presentation at the second time point occurs without a stimulus being presented on the first display for the first eye; determining, via the one or more pupil tracking sensors, whether the second eye has moved beyond a threshold amount of movement to fixate the central location on the second display in association with the presentation of the stimulus at the second time; in response to determining that the second eye is not moving beyond the threshold amount of movement, determining a pupillary distance between pupils of the first and second eyes based on (i) the respective distances between the center location and edges of each of the first and second displays, and (ii) the first distance between the first and second displays; in response to determining that the second eye is moving beyond the threshold amount of movement, adjusting a position of one or more of the first or second displays based on an amount of second eye movement occurring during presentation of the stimuli at the second time, the adjustments causing the first and second displays to be spaced a second distance apart on the spectacle apparatus; An apparatus for generating an action including:

2. causing a stimulus to be presented for a first eye at a first time at a location on a first display of a head-mounted device such that the first eye of a user gazes at the location on the first display of the head-mounted device; At a second time point subsequent to the first time point, causing a stimulus to be presented for a second eye of the user at a location on a second display of the head-mounted device; determining, via one or more eye tracking sensors, whether movement of the first or second eye to fixate the location on the respective first or second display in association with the presentation of the stimulus at the second time exceeds a threshold amount of movement; in response to determining that the movement of the first or second eye exceeds the threshold amount of movement, generating a user profile associated with the user including a pupillary distance, the pupillary distance determined based on the movement of the first or second eye and defining a distance between the pupils of the first and second eyes; A method comprising:

3. The method of claim 2 , wherein the location on the first display is a central location on the first display and the location on the second display is a central location on the second display.

4. generating the user profile includes generating the user profile in response to determining that movement of the second eye exceeds the threshold amount of movement associated with the presentation of the stimulus at the second time; The method of claim 2 , wherein the pupillary distance is determined based on the movement of the second eye.

5. 3. The method of claim 2, further comprising adjusting one or more positions of the first or second displays based on the pupillary distance of the user profile in response to determining that the movement of the first or second eye exceeds the threshold movement amount, such that the adjustment changes a distance between the first and second displays from a first distance to a second distance apart at the head-mounted device.

6. adjusting the one or more positions of the first or second display includes adjusting the one or more positions of the first or second display in response to determining that movement of the second eye exceeds the threshold amount of movement associated with the presentation of the stimulus at the second time based on the pupillary distance of the user profile; The method of claim 5 , wherein the pupillary distance is determined based on the movement of the second eye.

7. adjusting the one or more positions of the first or second display includes adjusting a position of the first display and a position of the second display based on the interpupillary distance of the user profile; The method of claim 5 , wherein the pupillary distance is determined based on the movement of the second eye.

8. causing a first stimulus to be presented on the first display of the head-mounted device and a second stimulus to be presented on the second display of the head-mounted device; determining, via the one or more eye tracking sensors, whether gaze directions of the first and second eyes occurring individually in association with the presentation of the first and second stimuli are consistent with one another; in response to determining that the gaze directions of the first and second eyes are consistent with one another, determining the pupillary distance associated with the user based on (i) a location of the first stimulus on the first display, (ii) a location of the second stimulus on the second display, and (iii) a distance between the first and second displays; The method of claim 2 further comprising:

9. 3. The method of claim 2, wherein determining the pupillary distance includes, in response to determining that the gaze directions of the first and second eyes are both substantially perpendicular to the first and second displays, respectively, determining the pupillary distance associated with the user based on (i) the location of the first stimulus on the first display, (ii) the location of the second stimulus on the second display, and (iii) the distance between the first and second displays.

10. selecting the location of the first or second stimulus based on the movement of the first or second eye; The method of claim 2 , wherein the presentation of the first and second stimuli is based on the selection of the location of the first or second stimulus.

11. selecting the location of the first and second stimuli based on the movement of the first or second eye; The method of claim 2 , wherein the presentation of the first and second stimuli is based on the selection of the locations of the first and second stimuli.

12. The method of claim 2 , wherein the presentation of the stimulus at the second time occurs in a state where no stimulus is presented on the first display.

13. 3. The method of claim 2, wherein the presentation of the stimulus at the first time occurs in a state where no stimulus is presented on the second display, and the presentation of the stimulus at the second time occurs in a state where no stimulus is presented on the first display.

14. 3. The method of claim 2, wherein the presentation of the stimulus at the first time occurs in a state where the stimulus intensity of the second display does not satisfy a stimulus intensity threshold, and the presentation of the stimulus at the second time occurs in a state where the stimulus intensity of the first display does not satisfy the stimulus intensity threshold.

15. One or more non-transitory computer-readable media storing instructions, The instructions, when executed by one or more processors, causing a first eye of a user to gaze at a location on a first display of a user device by causing a stimulus to be presented for the first eye at a first time point at the location on the first display of the user device; causing a stimulus to be presented for a second eye of the user at a location on a second display of the user device at a second time point subsequent to the first time point; determining, via one or more eye tracking sensors, whether gaze directions of the first and second eyes occurring individually in association with the presentation of the stimuli at the first and second time points are consistent with one another; in response to determining that the gaze directions of the first and second eyes are consistent with one another, determining a pupillary distance between pupils of the first and second eyes based on (i) respective distances between the location and edges of the first and second displays, and (ii) a first distance between the first and second displays; 23. A non-transitory computer-readable medium for generating operations including:

16. in response to determining the pupillary distance between pupils of the first and second eyes, adjusting one or more positions of the first or second display based on the pupillary distance, the adjustment changing a distance between the first and second displays from the first distance to a second distance spaced apart on the user device; The non-transitory computer-readable medium of claim 15 , further comprising:

17. Determining whether gaze directions of the first and second eyes occurring individually in association with the presentation of the stimuli at the first and second time points are consistent with one another includes: determining via the one or more eye tracking sensors a first gaze location of the first eye at the first time point and a second gaze location of the second eye at the second time point; determining a first difference, the first difference being based on the location of the presented stimulus for the first eye and the first gaze location; determining a second difference, the second difference being based on the location of the presented stimulus for the second eye and the second gaze location; comparing the first difference to the second difference; determining that the gaze directions of the first and second eyes are consistent with one another in response to the first difference and the second difference satisfying a predetermined threshold difference; 20. The non-transitory computer-readable medium of claim 15, further comprising:

18. 16. The non-transitory computer-readable medium of claim 15, wherein determining the pupillary distance between the pupils of the first and second eyes is further based on a location of the stimulus on the first display and a location of the stimulus on the second display.

19. 16. The non-transitory computer-readable medium of claim 15, wherein the presentation of the stimulus at the second time occurs in a state where no stimulus is presented on the first display.

20. 16. The non-transitory computer-readable medium of claim 15, wherein the presentation of the stimulus at the first time occurs in a state where no stimulus is presented on the second display, and the presentation of the stimulus at the second time occurs in a state where no stimulus is presented on the second display.

Citation Information

Patent Citations

  • Retina direct display device and television receiver using the device

    JP1996205052A

  • Systems, methods, and apparatus for correcting amblyopia and ocular deviation

    JP2010511486A

  • System for measuring visual fixation disparity

    US20160000317A1

  • See-through computer display systems

    US20160018654A1

  • Image projection device

    US20180373024A1

Cited By

  • system

    JP2026140418A