Light Field Processor System

Abstract

To provide various methods and systems for diagnosing, monitoring and treating health conditions and ailments.SOLUTION: A wearable ophthalmic device is disclosed. The device may include an outward facing head-mounted light field camera (e.g., 16) to receive light from a user's surroundings and to generate numerical light field image data. The device may also include a light field processor (e.g., 76) to access the numerical light field image data, to obtain an optical prescription for an eye of the user, and to computationally introduce an amount of positive or negative optical power to the numerical light field image data based on the optical prescription to generate modified numerical light field image data. The device may also include a head-mounted light field display (e.g., 62) to generate a physical light field corresponding to the modified numerical light field image data.SELECTED DRAWING: None

Description

[Technical field]

[0001] (Priority Application) This application claims priority to U.S. Provisional Patent Application No. 62 / 366,524, filed July 25, 2016, and entitled “LIGHT FIELD PROCESSOR SYSTEM,” and U.S. Provisional Patent Application No. 62 / 440,286, filed December 29, 2016, and entitled “LIGHT FIELD PROCESSOR SYSTEM,” the entire contents of each of which are incorporated herein by reference.

[0002] The present disclosure relates to various methods and systems for diagnosing, monitoring, and treating health conditions and diseases. [Background technology]

[0003] Ophthalmic instruments and techniques are routinely used by clinicians to diagnose and treat eye-related ailments. An example of a conventional ophthalmic device is shown in Figure 1. During use of the illustrated device, the patient may be positioned in a specific sitting position for the entire duration of the procedure, which may typically last from a few seconds to a few minutes.

[0004] Unfortunately, ophthalmic devices tend to be large, bulky, and expensive devices, and are typically used exclusively in doctor's offices. Thus, patients may be required to make an appointment with an optometrist and visit the doctor to receive any diagnosis or treatment. This may be a deterrent factor that may cause many patients to delay going to the doctor's office for extended periods of time, potentially until the condition worsens. The worsening condition may require more intensive therapy or procedures to address than would have been more easily remitted if the patient had been diagnosed or treated in a timely manner. Furthermore, the large and bulky nature of most ophthalmic devices forces patients to remain in an uncomfortable position, which in turn may increase the risk of misdiagnosis and patient error.

[0005] Thus, there is a need for a health system that addresses one or more of the difficulties described above. Summary of the Invention [Means for solving the problem]

[0006] A wearable ophthalmic device is described herein. In some embodiments, the wearable ophthalmic device comprises an outwardly facing head-mounted light field camera configured to receive light from a user's surroundings and generate numerical light field image data, a light field processor configured to access the numerical light field image data, obtain an optical prescription for the user's eye, and computationally introduce an amount of positive or negative refractive power into the numerical light field image data based on the optical prescription and generate modified numerical light field image data, and a head-mounted light field display configured to generate a physical light field corresponding to the modified numerical light field image data.

[0007] A method for using a wearable ophthalmic device is also disclosed. In some embodiments, the method includes receiving light from a user's surroundings using a light field camera and generating numerical light field image data, accessing the numerical light field image data, obtaining an optical prescription for the user's eye, computationally introducing an amount of positive or negative refractive power into the numerical light field image data based on the optical prescription to generate modified numerical light field image data, and generating a physical light field corresponding to the modified numerical light field image data using a light field display. The present invention provides, for example, the following: (Item 1) A wearable ophthalmic device, comprising: an outwardly facing head-mounted light field camera configured to receive light from around the user and generate numerical light field image data; a light field processor configured to access the numerical light field image data, obtain an optical prescription for the eye of the user, and computationally introduce an amount of positive or negative refractive power into the numerical light field image data based on the optical prescription to generate modified numerical light field image data; a head-mounted light field display configured to generate a physical light field corresponding to the modified numerical light field image data; A device comprising: (Item 2) Item 1 , wherein the optical power comprises a spherical or cylindrical optical power. (Item 3) 13. The device of claim 1, wherein the modified numerical light field image data is at least partially corrected with respect to the optical prescription. (Item 4) The device of any of the preceding items, wherein the light field processor is configured to identify one or more wavefronts represented by the numerical light field image data and generate the modified numerical light field image data by computationally modifying a curvature of the one or more wavefronts based on the optical prescription. (Item 5) 2. The device of claim 1, wherein the light field processor is further configured to add virtual reality or augmented reality image content to the numerical light field image data. (Item 6) The device of any of the preceding items, wherein the optical prescription comprises a prescription for myopia. (Item 7) The device of any of the preceding items, wherein the optical prescription comprises a prescription for hyperopia. (Item 8) The device of any of the preceding items, wherein the optical prescription comprises a prescription for astigmatism. (Item 9) 2. The device of claim 1, wherein the optical prescription comprises information regarding high-order aberrations of the user's eye. (Item 10) 2. The device of claim 1, wherein the light field camera comprises an integrated imaging camera with a two-dimensional array of microlenses and a corresponding two-dimensional array of photodetectors. (Item 11) 2. The device of claim 1, wherein the light field display comprises an integrated imaging display with a two-dimensional array of microlenses and a corresponding two-dimensional array of light sources. (Item 12) The device of any of the preceding items, wherein the light field display comprises one or more infrared light sources configured to project infrared light into the user's eye and measure the optical prescription. (Item 13) Item 14. The device of any preceding item, wherein the light field processor is configured to generate the modified numerical light field image data in real time. 2. The device of claim 1, wherein the light field processor is configured to generate the optical prescription. (Item 15) The light field processor comprises: computationally introducing a first test amount of positive or negative refractive power into the numerical light field image data; receiving feedback from the user regarding the positive or negative optical power of the first test volume; computationally introducing a positive or negative refractive power of a second test amount into the light field image data; Item 15. The device of item 14, configured to generate the optical prescription by performing the steps of: (Item 16) The device of any of the preceding items, wherein the light field processor is further configured to generate the modified numerical light field image data by removing one or more light rays that, if generated as part of the physical light field, would interact with an impairment in the user's cornea or lens. (Item 17) 1. A method for using a wearable ophthalmic device, the method comprising: computationally introducing an amount of positive or negative refractive power into the numerical light field image data of the user's surroundings to generate modified numerical light field image data, the amount of refractive power being based on an optical prescription for the user; generating a physical light field corresponding to the modified numerical light field image data using a light field display; A method comprising: (Item 18) Item 18. The method of item 17, wherein generating the modified numerical light field image data includes identifying one or more wavefronts represented by the numerical light field image data and computationally modifying a curvature of the one or more wavefronts based on the optical prescription. (Item 19) 5. The method of any preceding item, further comprising adding virtual reality or augmented reality image content to the numerical light field image data. (Item 20) 20. The method of claim 19, further comprising projecting infrared light into the user's eye and measuring the optical prescription. (Item 21) 5. The method of claim 1, further comprising generating the modified numerical light field image data in real time. (Item 22) 13. The method of any preceding claim, further comprising generating the optical prescription. (Item 23) generating the optical prescription computationally introducing a first test amount of positive or negative refractive power into the numerical light field image data; receiving feedback from the user regarding the positive or negative optical power of the first test volume; computationally introducing a positive or negative refractive power of a second test amount into the light field image data; 31. The method according to item 30, comprising: (Item 24) 5. The method of claim 1, further comprising generating the modified numerical light field image data by computationally removing one or more light rays that, if generated as part of the physical light field, would interact with an impairment in the user's cornea or lens. [Brief description of the drawings]

[0008] The drawings illustrate several embodiments disclosed herein and are not intended to limit the invention. It should be noted that the drawings are not drawn to scale and that elements of similar structure or function are represented by like reference numerals throughout the drawings.

[0009] [Figure 1] FIG. 1 illustrates a conventional ophthalmic instrument as used in a clinician's office.

[0010] [Diagram 2] FIG. 2 illustrates a cross section of the human eye.

[0011] [Figure 3A] 3A-3D illustrate various configurations of an exemplary ophthalmic device. [Figure 3B] 3A-3D illustrate various configurations of an exemplary ophthalmic device. [Figure 3C] 3A-3D illustrate various configurations of an exemplary ophthalmic device. [Figure 3D]3A-3D illustrate various configurations of an exemplary ophthalmic device.

[0012] [Figure 4A] 4A-4D illustrate various eye and head measurements taken to configure an ophthalmic device for a particular user. [Figure 4B] 4A-4D illustrate various eye and head measurements taken to configure an ophthalmic device for a particular user. [Figure 4C] 4A-4D illustrate various eye and head measurements taken to configure an ophthalmic device for a particular user. [Figure 4D] 4A-4D illustrate various eye and head measurements taken to configure an ophthalmic device for a particular user.

[0013] [Diagram 5] FIG. 5 shows a schematic diagram of various components of an ophthalmic device, according to some embodiments.

[0014] [Figure 6] FIG. 6 illustrates a light field processor system for capturing light field image data (e.g., photographs and / or videos) from at least a portion of a user's field of view, then processing the captured light field image data and displaying the processed light field image data to the user.

[0015] [Figure 7] FIG. 7 is a schematic diagram of an embodiment of the light field processor system of FIG.

[0016] [Figure 8] FIG. 8 is a flow chart illustrating a method for correcting myopia, hyperopia, and / or astigmatism for a user using the light field processor system shown in FIGS.

[0017] [Figure 9A] 9A-9B illustrate schematic cross-sectional views of a user's eye suffering from myopia. [Figure 9B] 9A-9B illustrate schematic cross-sectional views of a user's eye suffering from myopia.

[0018] [Figure 10A] 10A-10B illustrate schematic cross-sectional views of a user's eye suffering from hyperopia. [Figure 10B] 10A-10B illustrate schematic cross-sectional views of a user's eye suffering from hyperopia.

[0019] [Figure 11A] 11A-11B illustrate schematic cross-sectional views of a user's eye suffering from astigmatism. [Figure 11B] 11A-11B illustrate schematic cross-sectional views of a user's eye suffering from astigmatism.

[0020] [Figure 12] FIG. 12 illustrates an exemplary method for correcting presbyopia using a light field processor system.

[0021] [Figure 13] FIG. 13 illustrates an exemplary method for treating convergence insufficiency, such as that caused by strabismus and / or amblyopia, using a light field processor system.

[0022] [Figure 14] FIG. 14 is a schematic diagram of an embodiment of a light field processor system that includes an outward-facing integrated imaging camera, a light field processor, and an integrated imaging display that also includes one or more photodetectors.

[0023] [Figure 15]FIG. 15 illustrates how the wearable devices described herein can be used to function as an ophthalmometer or refractometer and determine a suitable refraction that will correct or improve the vision of the wearer or patient.

[0024] [Figure 16] FIG. 16 illustrates an example method for determining the optical prescription of a wearer of a light field processor system configured for use as a virtual ophthalmometer.

[0025] [Figure 17] FIG. 17 illustrates an exemplary method for measuring the refractive error of a wearer of a light field processor system configured as an ophthalmic device for performing retinoscopy.

[0026] [Figure 18A] 18A-18C illustrate an example embodiment of an augmented and / or virtual reality system configured as an autorefractometer. [Figure 18B] 18A-18C illustrate an example embodiment of an augmented and / or virtual reality system configured as an autorefractometer. [Figure 18C] 18A-18C illustrate an example embodiment of an augmented and / or virtual reality system configured as an autorefractometer.

[0027] [Figure 19] FIG. 19 illustrates a method for diagnosing, detecting, and / or identifying any areas of macular degeneration using the systems described herein. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0028] Various embodiments of the present invention are directed to devices, methods, systems, and articles of manufacture for implementing a user-wearable health system that may be used to perform health-related diagnostics, monitoring, and therapeutic procedures on a user. Various objects, features, and advantages of certain embodiments of the present invention are set forth in the detailed description, drawings, and claims, but it is not required that any single embodiment include or satisfy all such objects, features, and advantages.

[0029] Various embodiments will be described in detail with reference to the drawings, which are provided as illustrative examples to enable those skilled in the art to practice the present invention. It should be noted that the following figures and examples are not meant to limit the scope of the present invention described herein. Although certain elements of the present invention may be implemented partially or completely using known components (or methods or processes), only those parts of such known components (or methods or processes) necessary for understanding the present invention will be described, and detailed descriptions of other parts of such known components (or methods or processes) will be omitted so as not to obscure the present invention. Furthermore, embodiments of the present invention also encompass current and future known equivalents of the components referenced herein.

[0030] Disclosed herein are methods and systems for diagnosing, treating, and / or monitoring a patient's health abnormalities through a user-wearable health system (e.g., a user-wearable ophthalmic device that interacts with the user's eye). In one or more embodiments, the device may be a head-mounted system capable of implementing one or more diagnostic or treatment plans. In some other embodiments, the device may be stationary (e.g., stationary in a doctor's office). In one or more embodiments, the device may be a virtual reality (VR), augmented reality (AR), and / or mixed reality (MR) system that advantageously combines many VR, AR, and / or MR techniques for health or ophthalmic purposes. A VR system creates a simulated environment for a user to experience. This can be done by presenting computer-generated image data or other light signals to the user through a display. This image data creates a sensory experience that immerses the user in the simulated environment. A VR scenario typically involves the presentation of only computer-generated image data, rather than also including actual real-world image data. An AR system generally supplements a real-world environment with simulated elements. For example, an AR system may provide a user with a view of the surrounding real-world environment via a display. However, computer-generated image data or other light signals may also be presented on the display to augment the real-world environment. The computer-generated image data may include elements that are contextually relevant to the real-world environment. Such elements may include simulated text, images, objects, etc. The simulated elements can often be real-time and interactive. MR scenarios are one type of AR scenario that typically involve virtual objects that are integrated into and responsive to the natural world. For example, in MR scenarios, AR image content may be presented in a way that is perceived to interact with objects in the real world.

[0031] In some other embodiments, a clinician may wear the device for diagnostic and / or simulation and training purposes. Although the various embodiments described below discuss a new paradigm of health systems in conjunction with AR systems, it should be understood that the techniques disclosed herein may be used independently of any existing and / or known AR system. Thus, the examples discussed below are for illustrative purposes only and should not be read as limited to AR systems.

[0032] As mentioned above, embodiments of the present invention present a new paradigm in which a user-wearable health diagnostic or therapy system, such as an ophthalmic instrument (generally referred to herein as a health system), can be worn by a patient and programmed with one or more applications specific to various health-related (e.g., eye-related) ailments. In some embodiments, diagnosis and / or treatment may be provided by optical devices, mechanical structures, processing algorithms, or any combination of the foregoing. In some other embodiments, the patient-worn health system may further involve sensing and / or stimulation capabilities for enhanced treatment or diagnostic purposes. In some embodiments, a head-mounted augmented reality system may be used to provide various health-related (e.g., ophthalmic) measurements, assessments, diagnoses, or treatments.

[0033] Given that the head-mounted augmented reality display system interacts with the user's eyes, many applications can be envisioned for eye-related diagnosis and therapy. Furthermore, many other applications in non-ocular diagnosis and therapy can be envisioned as well. Thus, the disclosure presented herein is not limited to eye diagnosis, monitoring, and / or treatment. The embodiments disclosed herein may also be applied to diagnose, monitor, and / or treat other areas of the user's health, including, but not limited to, the user's cardiovascular and neurological health.

[0034] Many embodiments of the health system will be discussed in relation to various eye-related and other ailments. Prior to detailed description of various embodiments of the health system, the biology of the human eye will be briefly discussed below to provide context for common ailments that may affect patients.

[0035] Referring to FIG. 2, a simplified cross-section of the human eye is depicted, featuring the cornea 42, iris 44, lens or "crystalline lens" 46, sclera 48, choroid layer 50, macula 52, retina 54, and optic nerve pathway 56 to the brain. The macula is the center of the retina, which is used to see medium details. At the center of the macula is a portion of the retina called the "fovea," which is used to see the finest details and contains more photoreceptors (approximately 120 cones per diopter) than any other part of the retina. The human visual system is not a passive sensor type system; that is, it is configured to actively scan the environment. In a manner somewhat similar to the use of a flatbed scanner to capture an image or a finger to read Braille from paper, the eye's photoreceptors fire in response to changes in stimuli, rather than a constant response to a constant state of stimuli. Thus, movement is required to present the photoreceptor information to the brain. Indeed, experiments with substances such as cobra venom, which are used to paralyze the eye muscles, have shown that human subjects will suffer from blindness if positioned with their eyes open and viewing a static scene with ocular venom-induced paralysis. In other words, in the absence of a change in stimulus, the photoreceptors provide no input to the brain and suffer from blindness. This is thought to be at least one reason why normal human eyes have been observed to move back and forth, or jerk, in lateral movements called "microsaccades."

[0036] As mentioned above, the fovea of ​​the retina contains the greatest density of photoreceptor cells, and although humans typically have perception with high-resolution visualization capabilities throughout their entire field of vision, by and large, they actually only have a small high-resolution center that mechanically sweeps over and over again with a persistent memory of the high-resolution information captured in the immediate vicinity at the fovea. In a somewhat similar manner, the eye's focal length control mechanism (the ciliary muscle operatively coupled to the crystalline lens in a manner in which ciliary relaxation causes tensioning of the ciliary connective fibers, flattening the lens for longer focal lengths used for viewing at greater distances, while ciliary contraction causes relaxation of the ciliary connective fibers, allowing the lens to assume a rounder geometry for shorter focal lengths used for viewing at shorter distances) moves back and forth by approximately 1 / 4 to 1 / 2 diopters to periodically induce a small amount of what is called "optical refractive blur" both near and far from the target focal length. This is utilized by the brain's accommodation control functionality as a periodic negative feedback that constantly corrects accommodation and serves to keep the retinal image of the fixated object approximately in focus.

[0037] The brain's visualization centers also derive useful perceptual information from the relative movement of the eyes and their components. The relative vergence of the eyes (i.e., the rotational movement of the pupils toward or away from each other to converge the gaze of the eyes and fixate on an object) is closely linked to the focusing of the eye's lenses (or "accommodation"). Under normal conditions, changing the focus of the eye's lenses, i.e., accommodating the eyes to focus on objects at different distances, will automatically produce a matching change in vergence to the same distance, a relationship known as the "accommodation-vergence reflex." Similarly, a change in vergence will also induce a matching change in accommodation, under normal conditions. Working against this reflex is known to cause eye fatigue, headaches, or other forms of discomfort to the user (as in some conventional stereoscopic AR or VR configurations).

[0038] Head movement, which retracts the eyes, also has a significant effect on the visualization of objects. Humans move their heads to visualize the world around them. They are often in a state of very constant head repositioning and reorientation relative to the object of interest. Furthermore, most people prefer to move their heads when their eye gaze needs to move more than about 20 degrees off-center to focus on a particular object (i.e., people typically do not like to see things "out of the corner of the eye"). Humans also typically scan or move their heads in tandem with sounds to improve audio signal capture and take advantage of the geometry of the ears relative to the head. The human visual system derives excellent depth cues from what is called "head motion parallax", which relates to the relative motion of objects at different distances as a function of head motion and eye convergence distance (i.e., if a person moves their head laterally and maintains fixation on an object, items further from the object will appear to move in the same direction as the head, while items in front of the object will appear to move in the opposite direction to the head motion. These are very salient cues regarding the location of things in the environment relative to a person, and are probably as good as stereopsis). Head motion is also, of course, used to orient objects around them.

[0039] Furthermore, head and eye movements are coordinated with what is called the "vestibulo-ocular reflex", which stabilizes image information on the retina during head rotation, thus keeping object image information approximately centered on the retina. In response to head rotation, the eyes are reflexively and proportionally rotated in the opposite direction to maintain a stable fixation on the object. As a result of this compensation relationship, many people can read a book while oscillating their head back and forth (interestingly, the same is generally not true when a book is turned back and forth at the same speed with the head remaining approximately stationary; that is, the person is unlikely to be able to read a book that is being turned. The vestibulo-ocular reflex is one of the motor coordination of the head and eye, and is generally not developed for hand movements). This paradigm can be important for patient-worn health systems, since the user's head movements can be relatively directly related to eye movements, and the system will preferably be ready to work with this relationship. Thus, when designing a patient-worn or stationary display-based health system, the characteristics, and sometimes limitations, of the human eye are preferably taken into account to provide meaningful virtual content that works with the natural mechanisms of the eye rather than imposing stresses on them. Furthermore, in the context of health-related applications of AR display systems, this can provide various advantages as disclosed herein. As mentioned above, the display of the health system may be implemented independently of the AR system, although many of the following embodiments are described in conjunction with an AR system for illustrative purposes only.

[0040] 3A-3D, some general component options are illustrated. While the embodiment of FIGURES 3A-3D illustrates a head mounted display, it should be understood that the same components may be incorporated into a stationary wellness system as well, in some embodiments.

[0041] As shown in FIG. 3A, a user 60 is depicted wearing a patient-worn ophthalmic device that includes a frame 64 structure coupled to a display system 62 positioned in front of the user's eye. The frame 64 may be coupled to several ophthalmic-specific measurement subsystems depending on the application of the health system. Some embodiments may be constructed for one or more ophthalmic applications, while other embodiments may be general AR systems that are also compatible with ophthalmic applications. In either case, the following disclosure describes possible components of a health system or an AR system used for ophthalmic instruments and / or procedures.

[0042] In one or more embodiments, the health system is worn by a patient, i.e., a user. In some other embodiments, the health system may be worn by another person (e.g., a doctor or clinician) and used to perform a set of diagnostic tests and / or treatment protocols on a patient who is not the wearer of the system. It should be understood that any of the following applications may be used for a health system worn by another person to perform diagnostic tests, treatment protocols, and / or monitoring (real-time or longitudinal) on a patient as well.

[0043] A speaker 66, in the depicted configuration, may be coupled to the frame 64 and positioned adjacent the user's ear canal. (In one embodiment, a separate speaker (not shown) is positioned adjacent the user's other ear canal to provide stereo / adjustable sound control.) A microphone 55 may also be coupled to the frame and detect sounds from the user or the surrounding environment. In some embodiments, another microphone (not shown) may also be provided (e.g., coupled to the frame 64 on the right hand side of the user). In one or more embodiments, the wellness system may have a display 62 operably coupled to a local processing and data module 70, which may be mounted in a variety of configurations, such as fixedly attached to the frame 64, such as by wired leads or wireless connectivity 68, fixedly attached to a helmet or hat 80 as shown in the embodiment of FIG. 3B, built into headphones, removably attached to the torso 82 of the user 60 in a backpack-style configuration as shown in the embodiment of FIG. 3C, or removably attached to the waist 84 of the user 60 in a belt-coupled configuration as shown in the embodiment of FIG. 3D.

[0044] The local processing and data module 70 may include a power efficient processor or controller and digital memory, such as flash memory, both of which may be utilized to aid in the processing, caching, and storage of data a) captured from sensors, which may be operatively coupled to the frame 64, such as image capture devices (such as cameras), microphones, inertial measurement units, accelerometers, compasses, GPS units, wireless devices, and / or gyroscopes, and / or b) obtained and / or processed using the remote processing module 72 and / or remote data repository 74, possibly for processing or readout and then passing to the display 62. The local processing and data module 70 may be operatively coupled to the remote processing module 72 and the remote data repository 74 via wired or wireless communication links 76, 78, etc., such that these remote modules 72, 74 are operatively coupled to each other and available as resources to the local processing and data module 70.

[0045] In some embodiments, the remote processing module 72 may include one or more relatively powerful processors or controllers configured to analyze and process the data and / or image information. In some embodiments, the remote data repository 74 may include a relatively large-scale digital data storage facility, which may be available through the Internet or other networking configurations in a "cloud" resource configuration. In some embodiments, all data is stored and all calculations are performed in the local processing and data module, allowing for fully autonomous use from the remote module.

[0046] Advantageously, a health system (or an AR system, having an ophthalmic application), similar to that described in Figs. 3A-3D, provides unique access to the user's eyes and head. Given that the health system interacts with the user's eyes, enabling the user to perceive 3D virtual content and, in many embodiments, tracking various biometrics related to the user's eyes (e.g., eye convergence and divergence, eye movements, retinal structure, anterior and posterior eye geometry, patterns of eye movement, etc.), the resulting tracked data may advantageously be used in health-related applications, as described in further detail herein. This unprecedented access to the user's eyes is beneficial for the implementation of various health applications. Depending on the type of health anomaly, the health system may be configured to image, sense (including measure), and / or provide stimulation to the user's eyes to diagnose and / or treat the disease.

[0047] In one or more embodiments, the augmented reality display system may be used as a patient-worn, i.e., user-worn, ophthalmic device. Ophthalmic instruments are used by clinicians to view and examine a patient's eye, perform medical procedures, and / or administer tests or therapies to a user's eye. Traditionally, ophthalmic devices have large and bulky stationary devices and often require the patient to visit a doctor's office, where the clinician or doctor performs eye-related tests on the patient. Typically, the patient is restrained in the ophthalmic instrument device (e.g., with the chin resting on the chin rest component of the ophthalmic device and the head facing forward, etc.) until the clinician completes a series of tests. Thus, current approaches have several limitations.

[0048] In addition to using heavy and bulky devices for testing, the traditional approach requires physician supervision, and the patient may need to repeatedly return to the clinician's office for further testing / progression assessment, and may need to remain in an uncomfortable or restrictive position for extended periods of time. Furthermore, there is a limit to the amount of data the clinician can collect to diagnose or treat the patient, given the short duration of time the patient is exposed to the ophthalmic device. The traditional approach does not consider the dynamic changes in the user's behavior and user's orientation. Many tests performed under the traditional approach require the user to be constrained to a specific, usually static, position. However, if the user is, for example, undergoing a visual field test and has a limited attention span, they may move their head and eyes, thereby generating noise and potentially causing inaccurate test results. In addition, the traditional approach is not engaging, engaging, or interactive.

[0049] In one or more embodiments, a head-mounted health (e.g., ophthalmic) device similar to that shown in FIGS. 3A-3D may be used by a patient to track data, identify and correct one or more eye-related ailments, and / or help prevent other health problems. In one or more embodiments, an AR display system may be used as a head-mounted health (e.g., ophthalmic) device. It should be understood that some embodiments described below may be implemented in head-mounted embodiments, while other embodiments may be implemented in stationary devices. Furthermore, some embodiments may utilize AR technology to implement systems and methods for diagnosis, monitoring, and / or treatment with physician oversight (e.g., for medical safety concerns, regulatory concerns, etc.), while other embodiments may be implemented for self-diagnosis and / or monitoring through a head-mounted health device or AR device, or as part of a treatment protocol for a particular ailment, as described herein. For illustrative purposes, the present disclosure will focus primarily on head-mounted health devices, particularly AR devices, but it should be understood that the same principles may be applied to non-head-mounted embodiments as well.

[0050] In one or more embodiments, the AR display device may be used as a patient-worn health device. The device may typically be fitted for a particular user's head and / or facial features, and the optical components are aligned to the user's eyes. These configuration steps may be used to help ensure that the user is generally provided with an augmented reality experience without physiological side effects such as headaches, nausea, discomfort, etc. Thus, in one or more embodiments, the patient-worn health system may be configured for each individual user (both physically and digitally), and a set of programs may be specifically calibrated for the user. In other scenarios, the AR device may be used comfortably by a variety of users. For example, in some embodiments, the patient-worn health system knows one or more of the distance between the user's eyes, the distance from the head-mounted display to the user's eyes, the curvature of the user's forehead, the distance to the ears, or the height of the bridge of the nose for fitting correction purposes. All of these measurements may be used to provide the correct head-mounted display system for a given user. In some other embodiments, such measurements may not be necessary to perform ophthalmic functions. In the context of a patient-worn health system, this aspect of the head-worn device can be advantageous because the system already has a set of measurements about the user's physical characteristics (e.g., eye size, head size, distance between the eyes, etc.) and other data that can be used for patient therapy and diagnosis.

[0051] 4A-4D, the health system may be customized for each user. The user's head shape 402 may be taken into consideration when fitting the head-mounted patient-worn health system, as shown in FIG. 4A. Similarly, the eye components 404 (e.g., optics, structures for optics, etc.) may be rotated or adjusted for the user's comfort both horizontally and vertically, as shown in FIG. 4B. In one or more embodiments, the head rotation point set for the user's head may be adjusted based on the shape of the user's head, as shown in FIG. 4C. Similarly, the interpupillary distance (IPD) (i.e., the distance between the user's eyes) may be compensated, as shown in FIG. 4D.

[0052] In addition to various measurements and calibrations performed on the user, the patient-worn health system may be configured to track a set of biometric data about the user for patient identification and secure communication. For example, the system may perform iris recognition and / or retina matching for patient identification, and track eye movements, eye movement patterns, blink patterns, eye convergence and divergence movements, fatigue parameters, eye color changes, focal length changes, and many other parameters that may be used in providing an optical augmented reality experience to the user. In the case of an AR device used for health care applications, it should be understood that some of the aforementioned aspects may be part of a commonly available AR device, while other features may be incorporated for a specific health-related application.

[0053] Referring now to FIG. 5, various components of an exemplary patient-worn health display device will be described. It should be understood that other embodiments may have additional or fewer components depending on the application (e.g., a particular diagnostic tool) for which the system is used. However, FIG. 5 provides a basic idea of ​​some of the various components and types of biometric data that may be collected and stored through a patient-worn health system or AR device. FIG. 5 shows a simplified version of the head-mounted health system 62 in a block diagram on the right side for illustrative purposes.

[0054] 5, one embodiment of a suitable user display device 62 is shown to include a display lens 106 that may be mounted to the user's head or eye by a housing or frame 108, corresponding to the frame 64 (FIGS. 3A-3D). The display lens 106 is positioned by the housing 108 in front of the user's eye 20 and may include one or more semi-transparent mirrors configured to reflect the projected light 38 into the eye 20 and facilitate beam shaping while also allowing transmission of at least some light from the local environment. As shown, two wide-field machine vision cameras 16 are coupled to the housing 108 to image the environment around the user. In some embodiments, these cameras 16 are dual capture visible / non-visible (e.g., infrared) light cameras.

[0055] Continuing with reference to FIG. 5, a pair of scanning laser shaped wavefront (i.e. for depth) light projector modules with display mirrors and optics configured to project light 38 into the eye 20 are shown. The depicted embodiment also includes two miniature infrared cameras 24 paired with infrared light sources 26, such as light emitting diodes "LEDs," configured to track the user's eye 20 and support rendering and user input. The system 62 further features a sensor assembly 39, which may include three-axis accelerometer capabilities as well as magnetic compass and three-axis gyroscope capabilities, and preferably provide data at a relatively high frequency, such as 200 Hz. The depicted system also includes a sensor head pose processor 32, such as an ASIC (application specific integrated circuit), FPGA (field programmable gate array), and / or ARM processor (advanced reduced instruction set machine), which may be configured to perform digital and / or analog processing and calculate the user head pose in real time or near real time from the gyroscope, compass, and / or accelerometer data from the sensor assembly 39. The system may also include an image head pose processor 36, which may use the wide field of view image information output from the capture device 16 to determine the user's head pose.

[0056] The depicted embodiment also features a GPS (Global Positioning System) subsystem 37 to aid in attitude and positioning analysis. In addition, the GPS may further provide remote-based (e.g., cloud-based) information about the user's environment. This information may be used for diagnostic purposes. For example, if the user is located in an area with high pollen in the surrounding air, this information may be useful for diagnosing and / or treating a particular illness. Or, in another example, information about air pollution in a particular area may be advantageously used in considering treatment options for a particular user. Other types of information (e.g., pollen counts, pollution, demographics, environmental toxins, weather and air quality conditions, lifestyle statistics, proximity to health care providers, etc.) may be used in one or more applications as well.

[0057] The depicted embodiment may also include a rendering engine 34, which may feature hardware that runs software programs configured to provide virtual content for display to a user. The rendering engine 34 is operatively coupled to the sensor head pose processor 32, the image head pose processor 36, the eye tracking camera 24, and / or the projection subsystem 18 via wired or wireless connectivity (e.g., 105, 94, 100, 102, 104) such that rendered image data may be projected to a user (e.g., using the scanning laser array 18 in a manner similar to a retinal scanning display). The wavefront of the projected light beam 38 may be modified to match a desired focal length of the projected light.

[0058] A camera 24 (e.g., a small infrared camera) may be utilized to track the eyes and support rendering and user input (e.g., where the user is looking, what depth they are focusing at (eye convergence-divergence movements may be utilized to estimate focal depth / accommodation), etc.). A GPS 37, gyroscope, compass, and accelerometer 39 may be utilized to provide rough and / or fast pose estimation. The camera 16 images and pose, along with data from associated cloud computing resources, may be utilized to map the local world and share the user view with others and / or virtual or augmented reality communities and / or health care providers. In one or more embodiments, the camera 16 may be used to analyze the foods, drugs, nutrients, and toxins ingested by the user as part of a comprehensive health care and / or wellness or monitoring system.

[0059] With continued reference to FIG. 5 , the display device 62 may include a drug dispensing module 21 to deliver a drug to a user. The drug dispensing module 21 may include one or more outlets 22 and at least one drug container 23, which may be a reservoir for storing drug dispensed through the outlet 22. The outlet 22 may be connected to the container 23 by one or more channels 22a, which transfer the drug (e.g., liquid or gas) from the container 23 to the outlet 22. In some embodiments, the outlet 22 may simply be an opening in the frame 108, or may be a nozzle attached to or integral with the frame 108. In some embodiments, the nozzle may be a sprayer. In some embodiments, the channel 22a is formed in the frame 108 and / or tubing by an opening.

[0060] In one or more embodiments, the display device may include a light emitting module 27 to selectively administer light to the wearer, such as for treatment of the wearer's eye based on a treatment protocol. The light emitting module 27 may comprise a light source, which may include a light emitter that emits polychromatic polarized light, a laser, a light emitting diode, a fluorescent lamp, a dichroic lamp, a full spectrum light source, or the like. In some embodiments, one light emitting module 27 may be provided for both eyes. In some other embodiments, the display device may include multiple light emitting modules 27, with each eye having at least one light emitting module configured to direct light to that eye.

[0061] Although much of the hardware in the display system 62 featured in Figure 5 is depicted as being directly coupled to the housing 108 adjacent the display 106 and the user's eye 20, the depicted hardware components may be mounted or stored within other components, such as, for example, a belt-mounted component as shown in Figure 3D. Additionally, as described herein, multiple sensors and other functional modules are shown together for ease of illustration and explanation. However, it should be understood that some embodiments may include only one or a subset of these sensors and / or modules.

[0062] In some embodiments, all of the components of system 62 featured in Figure 5 are directly coupled to the display housing 108, except for the image head pose processor 36, the sensor head pose processor 32, and the rendering engine 34, and communication between the latter three and the remaining components of the system may be by wireless communication, such as ultra-wideband, or by wired communication. The depicted housing 108 is preferably head-mounted and wearable by a user. It may also feature speakers (e.g., speakers 66, Figures 3A-3D), such as those that may be inserted into the user's ears and utilized to provide sound to the user.

[0063] With regard to the projection of light 38 into the user's eye 20, in some embodiments, a camera 24 may be utilized to measure where the user's eye 20 is looking (e.g., where the lines of sight of the two eyes intersect), and that information may be used to determine the state of focus or accommodation of the eye 20. The three-dimensional surface of all points focused on by the eye is called the "single visual locus." The focal length may take on a finite number of depths or may vary infinitely. Light projected from the vergence distance, either physically or virtually, appears focused to the target eye 20, while light in front of or behind the vergence distance is blurred.

[0064] Further, without being limited by theory, it has been discovered that spatially coherent light with a beam diameter of less than about 0.7 millimeters is properly resolved by the human eye regardless of where the eye focuses. Given this understanding, to create the illusion of proper depth of focus, eye vergence-divergence may be tracked using camera 24, and rendering engine 34 and projection subsystem 18 may be utilized to render all virtual objects on or near the monocular locus in focus and all other virtual objects out of focus to a variable degree (i.e., with intentionally created blur). Preferably, system 62 renders to the user at a frame rate of about 60 frames per second or greater. As explained above, preferably camera 24 may be utilized for eye tracking, and software may be configured to screen not only vergence-divergence geometry, but also focus location cues to serve as user input. Preferably, such a display system is configured with a brightness and contrast suitable for daytime or nighttime use.

[0065] In some embodiments, the display system preferably has a latency of less than about 20 milliseconds of visual object alignment, less than about 0.1 degrees of angular alignment, and about 1 arc minute of resolution, which is believed to be approximately the limit of the human eye, without being limited by theory. The display system 62 may be integrated with a localization system, which may involve GPS elements, optical tracking, a compass, an accelerometer, and / or other data sources to aid in position and attitude determination. Localization information may be utilized to facilitate accurate rendering within the user's view of the relevant world (e.g., such information would facilitate the display system knowing where it is located relative to the real world).

[0066] Having described the general components of some embodiments of a user-worn health system (e.g., an ophthalmic system), additional components and / or features related to medical care and diagnosis will be discussed below. It should be understood that some of the features described below will be common to various embodiments of a user-worn health system or many embodiments of an AR system used for health purposes, while others will require additional or fewer components for health diagnosis and treatment purposes.

[0067] In some embodiments, the user-worn health system is configured to display one or more virtual images based on the accommodation of the user's eyes. Unlike previous 3D display approaches that force the user to focus where the image is projected, in some embodiments, the user-worn health system is configured to automatically vary the focus of the projected virtual content to allow a more comfortable viewing of the one or more images presented to the user. For example, if the user's eye has a current focus of 1m, the image may be projected to match the user's focus. Or, if the user shifts the focus to 3m, the image is projected to match the new focus. Thus, rather than forcing the user to a predetermined focus, the user-worn health system or AR display system of some embodiments allows the user's eyes to function in a more natural manner.

[0068] Such user-worn health systems may eliminate or reduce the occurrence of eye strain, headaches, and other physiological symptoms typically observed with virtual reality devices. To achieve this, various embodiments of patient-worn health systems are configured to project virtual images at variable focal distances through one or more variable focus elements (VFEs). In one or more embodiments, 3D perception may be achieved through a multi-planar focus system that projects images onto a fixed focal plane away from the user. Other embodiments employ a variable planar focus, where the focal plane is moved back and forth in the z-direction to match the user's current state of focus.

[0069] In both multi-plane and variable-plane focus systems, the patient-worn health system may employ eye tracking to determine the vergence-divergence movement of the user's eyes, determine the user's current focus, and project the virtual image at the determined focus.

[0070] Any wavelength of light may be projected into the user's eye. In addition to visible light, infrared light or other wavelengths of light may be projected through the patient-wearable health system as well. This aspect of the patient-wearable health system may be used for imaging, diagnosis, treatment, and / or compensation for health anomalies, as will be described below.

[0071] In healthcare and diagnostic contexts, the type, frequency, color scheme, location, etc. of one or more images presented to a user may be advantageously manipulated for the diagnosis, patient monitoring, and / or treatment of one or more disorders. For example, certain ailments may require the augmentation of one eye in relation to the other eye. To this end, treatment protocols may be devised to "train" the weaker eye, for example, by providing increased stimulation to the weaker eye compared to the stronger eye. Or, in another example, certain parts of the retina may have reduced sensitivity due to macular degeneration. To counter this, images may be modulated or reformatted and projected to the periphery of the retina, thereby compensating for the user's reduced field of view. Thus, as will be explained in more detail below, the health system's ability to modulate certain parameters associated with virtual image projection may be used to diagnose, monitor, and / or treat certain health abnormalities.

[0072] In addition, using the various principles outlined above, the health system may be designed to provide a diagnosis using a stimulus-response measurement analysis process. Devices such as these may be used by a clinician, or in other embodiments, a disease may simply be assessed or symptoms may be noted by the patient (e.g., eye strain, dry eye, hypertension, onset of stroke or attack, etc.). This may greatly help the user to actively control their health and prevent the onset of disease by proactively addressing certain symptoms as they occur. Such diagnosis and / or assessment may be performed, for example, by analyzing current data together with one or more tracked biometric parameters and historical data related to environmental changes. In one or more embodiments, the health system may also be configured to provide information cues, send alerts to the user and / or physician or others, or assist with other response measures.

[0073] It should be understood that the health system may be configured to be autonomous (i.e., provide results directly to a user or other person or entity without input or control from a clinician or other person) or semi-autonomous (i.e., some degree of input or control from a clinician or other person), such as in a software-controlled implementation of the health system to diagnose, monitor, or treat a user. In other embodiments, the health system may be fully controlled by a clinician or other person (e.g., on a networked or any remote basis, such as a cloud-based implementation, or in an implementation in which the health system is worn by a clinician to test a patient).

[0074] In some embodiments, the wellness system may be designed with a number of additional health-related sensors, including, for example, accelerometers, gyroscopes, temperature sensors, pressure sensors, light sensors, non-invasive glucose sensors, ETCO2, EEG, and / or other physiological sensors, to monitor one or more physiological responses of the user.

[0075] As described herein, the health system, in one or more embodiments, includes an eye tracking module. The eye tracking module may be configured to determine the convergence and divergence of the user's eyes to determine proper normal accommodation (through a direct relationship between convergence and accommodation) for the projection of one or more virtual images, and may also be configured to track one or more eye-related parameters (e.g., eye position, eye movement, eye patterns, etc.). This data may be used for several health-related diagnostic and treatment applications, as will be described below.

[0076] As will be apparent from the description herein, the health system may be used for diagnosis, monitoring, and therapy, which may include eye-related diagnosis, monitoring, and therapy. In eye-related applications, the health system may be referred to as an ophthalmic system. Also, as will be apparent from the description herein, the user (or wearer) of the device may be referred to as the patient, and the diagnosis, monitoring, and therapy are performed by the device on the user. In some other embodiments, the user may be a clinician, and the patient may be a third party who may be evaluated and treated by the user. It should also be understood that diagnosis and monitoring may be generally referred to as health analysis. Light Field Processor System

[0077] FIG. 6 illustrates a light field processor system 600 for capturing light field image data (e.g., photographs and / or videos) from at least a portion of a user's field of view, then processing the captured light field image data and displaying the processed light field image data to the user. The light field processor system 600 may include one or more integrated outwardly facing imaging cameras 16, a light field processor 70, and one or more integrated imaging displays 62. Each of these components may be provided as part of the wearable virtual / augmented reality and wellness systems discussed herein. In some embodiments, the light field image data is captured, processed, and displayed in substantially real time such that the user does not perceive a lag or delay in the displayed light field image data.

[0078] As mentioned immediately above, one aspect of the light field processor system 600 is to capture, process, and display light field image data rather than traditional two-dimensional photographs and / or videos. The spatial world is three-dimensional, but traditional photographs and videos record only two-dimensional images. Thus, traditional cameras reduce the complex interactions of light and things in the three-dimensional world to a flat two-dimensional map of light intensity detected from the object space in the camera's field of view. This flattening effect is a result of imaging, where light rays emitted, reflected, and / or scattered at different points on the object plane in the camera's field of view are focused by a lens to corresponding points on the image plane. Angular information is lost in this process. That is, the light intensity recorded at a given pixel in a traditional image does not show the individual intensity contributions of light rays that originate with different angular orientations from the corresponding point on the object plane in the field of view. Instead, in traditional photographs and videos, the intensity measured at each point in the image plane shows the combined intensity of various light rays that enter the camera with different angular orientations from the corresponding point on the object plane in the field of view. The flattening of three dimensions to two dimensions in conventional cameras significantly limits the information content of the image.

[0079] In contrast, light field data contains information about the direction and intensity of light emanating from various points in the field of view. Thus, while a typical photograph captured with a conventional camera is a two-dimensional image irradiance map, light field data is a map of light emitted from multiple (x,y) points in the field of view at multiple angles (θx,θ y ), may be considered four-dimensional because it may contain information about the intensity or irradiance of light rays emanating from the light source. When reproduced using a light field display, a light field image provides the viewer with a three-dimensional representation that resembles direct viewing of the actual physical scene.

[0080] The outwardly facing integrated imaging camera 16 can be any camera or device capable of capturing light field data. An example of one such device is described herein with respect to FIG. 7. In some embodiments, one or more outwardly facing integrated imaging cameras 16 can be integrated with the wearable system, as shown in FIGS. 3-5. Two (or more) outwardly facing integrated imaging cameras 16 can be provided, each generally used to capture numerical light field image data, which will be displayed to one of the user's eyes. Specifically, a left outwardly facing integrated imaging camera 16 can be provided and aligned to have a field of view that overlaps with the normal field of view of the user's left eye. Similarly, a right outwardly facing integrated imaging camera 16 can be provided and aligned to have a field of view that overlaps with the normal field of view of the user's right eye.

[0081] The light field processor 70 can be implemented, for example, using the local processing and data module 70 described herein. The light field processor 70 receives the captured four-dimensional numerical light field image data from the integrated imaging camera 16 and / or non-transitory computer memory. The light field processor can computationally perform various operations on the captured light field image data. Such operations can include computationally adding positive or negative spherical and / or cylindrical powers to the numerical light field image data, identifying one or more wavefronts represented by the numerical light field image data, computationally altering the curvature of such wavefronts, focusing the light field image data to a selected depth plane, adding prismatic powers to the light field image data, adding virtual objects or other VR / AR content to the image data, removing objects or other content from the image data, shifting the light field image data up, down, left, or right, magnifying selected subportions of the captured light field image data, performing spatial and / or wavelength filtering of the image data, and the like.

[0082] In some embodiments, the light field processor 70 processes the captured numerical light field image data based on an ophthalmic prescription or other characteristics of the user's eye. (The ophthalmic prescription or other ocular characteristics can be input to the light field processor 70 from the user or another device (e.g., non-transitory computer memory), or the ophthalmic prescription or other ocular characteristics can be measured by the light field processor system 600 itself, as described further herein.) For example, the captured numerical light field image data can be processed to improve at least one aspect of the user's vision. In some embodiments, the numerical light field image data can be processed to at least partially correct the user's myopia, hyperopia, astigmatism, presbyopia, strabismus, amblyopia, macular degeneration, higher order refractive errors, chromatic aberrations, or micro-disorders. Other types of corrections are also possible.

[0083] The integrated imaging display 62 can be any device capable of displaying light field image data. An example of one such device is described herein with respect to FIG. 7. In some embodiments, one or more integrated imaging displays 62 can be integrated with the wearable system, as shown in FIGS. 3-5. The integrated imaging display 62 can be partially transparent to allow the user to view its real-world environment. Alternatively, the integrated imaging display may instead rely entirely on the outward-facing integrated imaging camera 16 to provide the user with image data from its real-world environment. Two integrated imaging displays 62 can be provided, each generally used to display processed light field image data to one of the user's eyes. Specifically, a left integrated imaging display 62 can be provided and matched to display light field image data to the user's left eye. Similarly, a right integrated imaging display 62 can be provided and matched to display light field image data to the user's right eye.

[0084] FIG. 7 is a schematic diagram of an embodiment of the light field processor system 600 of FIG. 6. The outwardly facing integrated imaging camera 16 can include a two-dimensional array of microlenses 702 and a corresponding two-dimensional array of photodetectors 704 with light-sensitive pixels. The photodetector array 704 can be, for example, a complementary metal oxide semiconductor (CMOS) sensor, a charge-coupled device (CCD) sensor, an array of CMOS or CCD sensors, or any other device capable of detecting and digitizing light energy. The array of photodetectors can form a planar surface or a curved surface (e.g., the array of photodetectors can be curved radially around the eye to facilitate capturing light from anywhere the eye may be looking). The microlens array 702 collects light from a point 706 in object space within the user's field of view. Each microlens, or lenslet, samples a spatially localized region of the wavefront of light entering the integrated imaging camera 16 from the field of view, allowing local angular information to be recorded on the photodetector array 704. In this manner, the photodetector array 704 can detect the individual intensities of light rays arriving at the instrument from different angular directions. For example, as shown in FIG. 7, various cones of light 708 are incident at different angles (θ x , θ y ) of the scene. Each lenslet 702 focuses a cone of light 708 emanating from point 706 with a different angular orientation onto the photodetector array 704. Although only a single point 706 is illustrated, each lenslet 702 performs this function for each point on the (x,y) plane in object space. In other words, each lenslet 702 creates an elementary image of the scene in object space with a slightly different viewpoint than the elementary images created by the other lenslets 702 in the array. Each elementary image is recorded by a group of photodetectors in the photodetector array 704. Collectively, this two-dimensional array of elementary images forms what is referred to as an integral image. The integral image is one of various possible light field representations of the scene. It contains a two-dimensional array of elementary images, each captured from a slightly different viewpoint.

[0085] The captured light field image data from the integrated imaging camera 16 can be represented in various numerical forms after being digitized by the integrated imaging camera. This allows the light field processor 70 to perform mathematical operations (such as those described above) on the captured light field image data. Once the light field processor 70 has performed the desired mathematical operations on the captured light field image data, the processed light field image data is then passed to the integrated imaging display 62. In some embodiments, the processed light field image data can be passed to the integrated imaging display 62 in the form of one or more processed integrated images, each consisting of a set of processed base images.

[0086] As shown in FIG. 7, the integrated imaging display 62 includes a two-dimensional array of microlenses 712 and a two-dimensional array of light sources 714. The integrated imaging display 62 includes a controller, which may, for example, use the processed light field image data to modulate the array of light sources 714. The array of light sources 714 may, for example, be a red, green, blue (RGB) array of light emitting diodes (LEDs). Some embodiments may also include an infrared emitting light source and project infrared light into the wearer's eye for any of the reasons discussed hereafter herein. Alternatively, the array of light sources 714 may be implemented as a liquid crystal display (LCD) panel or some other type of display panel. Each light source may be used to emit light corresponding to a pixel or subpixel of the processed integrated image. The light emitted from each light source 714 is then projected by one of the lenslets 712 to a corresponding point 716 in space in front of the user's eye. In some embodiments, each lenslet 712 projects one of the processed elementary images. The overlap of light from each of the projected elementary images recreates a physical light field that can be viewed by the user. This physical light field is perceived by the user as three-dimensional, similar to when viewing an actual three-dimensional scene. If the light field processor 70 is not to modify the integral image collected by the integral imaging camera 16, the physical light field created by the integral imaging display 62 will be a representation of the same physical light field that exists in the object space in the field of view of the camera 16. This is shown in FIG. 7, where the intersection of all the projected cones of light 718 results in an irradiance distribution that reproduces the radiance of the original three-dimensional scene. Alternatively, if the light field processor 70 is used to modify the integral image, the physical light field projected by the integral imaging display 62 will be a similarly modified version of the physical light field that exists in the object space in the field of view of the camera 16.

[0087] 7 illustrates one embodiment of a light field capture / processing / display system 600, although other embodiments are possible. For example, the integrated imaging camera 16 shown in FIG. 7 could be replaced with any type of device capable of capturing light field image data. Similarly, the integrated imaging display 62 shown in FIG. 7 could also be replaced with any type of device capable of displaying light field image data. Correction of myopia, hyperopia, astigmatism, and higher order aberrations

[0088] FIG. 8 is a flow chart illustrating a method 800 for correcting myopia, hyperopia, and / or astigmatism for a user using the light field processor system 600 shown in FIGS. 6 and 7. The method 800 begins at block 810, where the light field processor 70 of the system 600 receives a refractive power measurement of the user. The refractive power measurement may comprise the user's optical prescription for eyeglasses, including spherical power, cylindrical power, and / or cylindrical power axis measurements. These measurements may be performed by the system 600 itself (as discussed further herein) or may be received from the user or a separate diagnostic device. The optical prescription may be determined during initialization of the light field processor system 600, during calibration of the system 600, as part of an eye prescription configurator program (e.g., using an ophthalmometer or other vision test), or at any time during use of the light field processor system. Biometric data may be used to identify the user and the associated optical prescription.

[0089] As mentioned immediately above, there are many ways in which the refractive power measurements of the user's eye may be input to the light field processor system 600. In some embodiments, the user may simply provide the information to the light field processor system 600. For example, the user may input a prescription into a user interface. Or, in other embodiments, the light field processor system 600 may be configured to automatically or interactively determine the user's optical prescription. For example, the light field processor system may determine the user's prescription manually and interactively via an eye prescription configurator program. The light field processor 70 may be pre-programmed with discrete granularity steps in adjusting the focus or modified wavefront of the collected light field image data. The focus adjustment or wavefront modification may include spherical power adjustment and / or cylindrical power / axis adjustment. The user may specify the desired amount of refractive power modification to the light field processor system through an appropriate feedback mechanism (e.g., a user interface). Or, in some embodiments, the user may have the option to incrementally increase or decrease the prescription value until the user arrives at a viewing prescription that is comfortable for them.

[0090] In some embodiments, the light field processor system 600 may automatically measure the user's optical prescription (and gradually vary the applied correction in real time, eye by eye) without requiring user input, based on monitoring and tracking the eyes via an eye tracking system, autorefraction, aberrometry, or other systems as described herein. In some embodiments, the light field processor system 600 may utilize a biofeedback system (discussed herein) to automatically vary the applied correction prescription.

[0091] In some embodiments, the light field processor system 600 may be configured to receive a user's optical prescription from a third party system. For example, a physician may be able to wirelessly transmit a user's optical prescription (e.g., via the Internet, a Bluetooth connection, etc.), which is received by a receiver and stored in a digital memory of the light field processor 70.

[0092] At block 820, the outwardly facing integrated imaging camera 16 of the system 600 captures light field image data and inputs the captured light field image data to the light field processor 70. As discussed herein, the captured light field can be mathematically manipulated prior to being displayed by the integrated imaging display 62. A variety of mathematical manipulations can be performed, some of which may be used to correct the captured light field image data, at least in part, based on one or more characteristics of the user's eye, such as its optical prescription, corneal shape or curvature, ocular length, etc. The light field processor 70 can then be used to make real-time modifications to the user's incident light field before displaying the modified light field to the user.

[0093] In some embodiments, the specific correction applied by the light field processor can be adjusted to dynamically correct vision impairments as the user's vision changes over time. For example, in some embodiments, the light field processor system 600 may implement dynamic vision correction by initiating an eye prescription configurator program. The light field processor system may be configured to determine the user's prescription at intervals of time with or without user activation. Thus, the light field processor system may dynamically identify a first optical prescription at a first time and adjust the vision correction based on that prescription, and may identify a second optical prescription at a second time and adjust the vision correction based on that second prescription.

[0094] Common vision defects that may be corrected, at least in part, by the light field processor 70 include myopia (i.e., nearsightedness), hyperopia (i.e., farsightedness), and astigmatism. Traditionally, these defects are corrected using spherical and / or cylindrical lenses. However, the light field processor system 600 may alternatively correct these defects through computational manipulation of the captured light field.

[0095] In the case of myopia, light associated with one or more objects in the user's field of view is focused in front of the retina, rather than on the retina, as shown in FIG. 9A. This makes the object appear out of focus. Now referring to FIG. 9B, conventionally, a negative power concave lens can be used to compensate for the impairment and introduce a negative power, which focuses the light on the retina. The light field processor 70 can replace the corrective concave lens shown in FIG. 9B with mathematical manipulation of the numerical light field image data captured by the outward facing integrated imaging camera 16 shown in FIGS. 6 and 7. For example, as shown in block 840 of FIG. 8, if the user's optical prescription indicates that he has myopia, the light field processor 70 selects and computationally introduces an amount of negative spherical power wavefront curvature into the captured light field image data to at least partially correct the user's myopia.

[0096] In the case of hyperopia, light associated with one or more objects is focused behind the retina, rather than on the retina, as shown in FIG. 10A. This causes the objects to appear out of focus. Now referring to FIG. 10B, conventionally, a positive power convex lens can be used to compensate for the impairment. Again, the light field processor 70 can replace the corrective convex lens shown in FIG. 10B with the mathematical manipulation of the light field captured by the outward facing integrated imaging camera 16 shown in FIGS. 6 and 7. For example, as shown in block 830 of FIG. 8, if the user's optical prescription indicates that he or she has hyperopia, the light field processor 70 selects and computationally introduces an amount of positive spherical power wavefront curvature into the captured light field image data to at least partially correct the user's hyperopia.

[0097] Astigmatism is a medical condition in which the eye exhibits a cylindrical refractive power (i.e., the refractive power of the eye, when measured relative to one meridian, is different from that relative to the orthogonal meridian). As illustrated diagrammatically in FIG. 11A, this can be caused by an irregularly shaped cornea. For example, the curvature along one meridian can be different from the curvature along the perpendicular meridian. Referring now to FIG. 11B, conventionally, astigmatism can be corrected by the introduction of a suitably oriented cylindrical lens. Again, the light field processor system 600 can substitute the corrective lens shown in FIG. 11B for the mathematical manipulation of the light field captured by the outwardly facing integrated imaging camera 16 shown in FIGS. 6 and 7. Specifically, as shown in block 850 of FIG. 8, the light field processor 70 can select and computationally introduce an amount of cylindrical refractive power (oriented to the angular axis indicated by the user's prescription) to correct the captured light field image data, at least in part, for the user's astigmatism.

[0098] However, the light field processor system 600 is not limited to correcting simple low-order optical aberrations, such as defocus and / or astigmatism. In fact, the light field processor 70 can also apply computational corrections to the captured light field for complex higher-order optical aberrations as well. In some cases, these higher-order aberrations may account for approximately 10% of all refractive errors. These higher-order refractive errors may be caused by irregularly shaped optical surfaces in the eye, and are particularly common after refractive surgery. For example, shape irregularities in the cornea and / or lens of the eye may introduce higher-order refractive errors into light passing through the eye to the retina. Such higher-order aberrations may be reduced using appropriate refractive corrections.

[0099] Various implementations of the light field processor system 600 described herein may be applicable to provide computational correction to the incident wavefront for these higher order aberrations. Such computational correction may be applied by the light field processor 70 based on measurements of the higher order aberrations of the user's eye. Wavefront correction, including correction of all aberrations described by the Zernike modes (e.g., astigmatism, coma, trefoil, spherical aberration, quatrofoil, etc.), may be performed utilizing the light field processor system 600 described herein.

[0100] In some embodiments, the light field processor system 600 may have the capability to measure high order aberrations of the user's eye as described herein. These measurements can then be used by the light field processor 70 to computationally alter the light field image data collected by the integrated imaging camera 16. In other embodiments, the light field processor 70 receives the high order aberration measurements of the user's eye from an external device. For example, the light field processor system 600 may include one or more transmitters and receivers to enable transmission and reception of data between the system and the remote processing module 72 and / or the remote data repository 74. The transmitters and receivers may be combined into a transceiver. In some embodiments, the remote processing module 72 and / or the remote data repository 74 may be part of a third party server and database that enables a third party (e.g., a physician or other medical administrator) to transmit data, such as, for example, an optical prescription, to an ophthalmic device.

[0101] In some embodiments, a method for correcting high-order aberrations may be similar to the method 800 illustrated in FIG. 8 for correcting myopia, hyperopia, or astigmatism. The light field processor 70 first receives measurements of the high-order aberrations of the user's eye. These measurements can be provided as Zernike coefficients, as a map of the corneal topography, or in any other suitable format. The light field processor 70 then receives the light field image data collected by the integrated imaging camera 16. Finally, the light field processor 70 selects one or more adjustments to make to the wavefront of the captured light field data to, at least in part, reduce one or more high-order aberrations. These corrections are applied computationally.

[0102] A user's eye may experience higher order refractive errors, which vary with changes in accommodation. Thus, in some embodiments, light field processor 70 may select a different set of wavefront adjustments to make to the incident light field based on the current accommodation of the user's eye and / or based on the focal plane of the light field image data.

[0103] In some implementations, the light field processor system 600 may also be configured to modify the collected light field image data to reduce the effects of microscopic obstructions in the cornea, lens, and / or other anatomical structures of the eye. These obstructions may produce complex refraction, reflection, and scattering patterns that impair visual quality. The light field processor 70 may spatially filter light rays that would interact with these obstructions, thus blocking those optical paths that contribute to impairing visual quality. This may be done by identifying light rays in the collected light field image data that would interact with obstructions in the user's eye, and then computationally removing those selected rays from the processed light field.

[0104] Once the light field processor 70 has performed the desired computational corrections to the captured light field image data, in block 860, the integrated imaging display 62 displays the corrected light field image data to the user.

[0105] In some embodiments, the light field processor system 600 may include a biofeedback system configured to determine a user's comfort level in viewing an object or image. For example, if the user's eyes shift, change accommodation, change pupil size, change convergence and divergence, etc., these may be indicators that the user is unable to comfortably view an object or image. Instability or oscillations in accommodation or behavior associated with accommodation may be a sign that the user is having difficulty focusing on an object. Thus, the biofeedback system may receive real-time input related to the state or nature of the user's eyes.

[0106] In various embodiments, the light field processor system 600 includes one or more eye tracking cameras or other cameras or imaging systems to track one or more eyes of a user. For example, some embodiments may utilize a camera 24 (e.g., an infrared camera) paired with a light source 26 (e.g., an infrared light source) configured to monitor and track the user's eyes. These cameras and sources may be operatively coupled to the light field processor system 600. Such cameras and / or imaging systems may monitor eye orientation, eye pupil size, eye convergence and divergence, and corresponding directions of the eyes' individual gazes. In some embodiments, the camera 24 may be used to determine the eye's convergence point. Additionally, as described herein, the system 600 may be configured to determine the accommodation of the user's eyes. The eye tracking system may detect variations in the eye's accommodation by comparing multiple measurements. In some embodiments, accommodation may be monitored based on the shape of the eye lens, eye convergence and divergence, pupil size, etc. In some embodiments, monitoring the accommodation state may include projecting a small image (e.g., a dot or dots) into the eye and using an inward-facing camera to monitor whether the image is focused on the retina. Fluctuations in accommodation may indicate an uncomfortable depth of focus or a blurred image. Thus, the light field processor system may computationally increase or decrease the prescription until the fluctuations stop or decrease, thereby reaching a comfortable viewing prescription.

[0107] In some embodiments, the ophthalmic device includes a gyroscope sensor, an accelerometer, other sensors, or a combination thereof to monitor changes in head position, head pose, or orientation. In some embodiments, the system 600 may include a sensor assembly 39 configured to detect movement and / or orientation of the system due to movement of the user's head. A biofeedback system may be configured to receive the detected head movement, and if the frequency and / or magnitude of the movement exceeds a threshold, the system may be configured to determine that the user is unable to comfortably view the image. For example, constant head movement may indicate a search for a comfortable viewing position for the image. If there are indications that the person may not be well focused, the light field processor system may be configured to alert the user thereto, perform a subjective visual test, or objectively and automatically evaluate the user's prescription and improve the visual quality.

[0108] The following paragraphs describe various exemplary embodiments of the devices, systems, and methods described herein.

[0109] 1. A wearable ophthalmic device comprising: an outwardly facing, head-mounted light field camera configured to receive light from a user's surroundings and generate numerical light field image data; a light field processor configured to access the numerical light field image data, obtain an optical prescription for the user's eye, and computationally introduce an amount of positive or negative refractive power into the numerical light field image data based on the optical prescription to generate modified numerical light field image data; and a head-mounted light field display configured to generate a physical light field corresponding to the modified numerical light field image data.

[0110] The device of the preceding paragraph, wherein the optical power comprises a spherical or cylindrical optical power.

[0111] The device of any of the preceding paragraphs, wherein the modified numerical light field image data is at least partially corrected with respect to an optical prescription.

[0112] The device of any of the preceding paragraphs, wherein the light field processor is configured to generate modified numerical light field image data by identifying one or more wavefronts represented by the numerical light field image data and computationally modifying the curvature of the one or more wavefronts based on the optical prescription.

[0113] The device of any of the preceding paragraphs, wherein the light field processor is further configured to add virtual reality or augmented reality image content to the numerical light field image data.

[0114] The device of any of the preceding paragraphs, wherein the optical prescription comprises a prescription for myopia.

[0115] The device of any of the preceding paragraphs, wherein the optical prescription comprises a prescription for hyperopia.

[0116] The device of any of the preceding paragraphs, wherein the optical prescription comprises a prescription for astigmatism.

[0117] The device of any of the preceding paragraphs, wherein the optical prescription comprises information regarding high-order aberrations of the user's eye.

[0118] The device of any of the preceding paragraphs, wherein the light field camera comprises an integrated imaging camera with a two-dimensional array of microlenses and a corresponding two-dimensional array of photodetectors.

[0119] The device of any of the preceding paragraphs, wherein the light field display comprises an integrated imaging display with a two-dimensional array of microlenses and a corresponding two-dimensional array of light sources.

[0120] The device of any of the preceding paragraphs, wherein the light field display comprises one or more infrared light sources configured to project infrared light into the user's eye and measure the optical prescription.

[0121] The device of any of the preceding paragraphs, wherein the light field processor is configured to generate modified numerical light field image data in real time.

[0122] The device of any of the preceding paragraphs, wherein the light field processor is configured to generate an optical prescription.

[0123] The device described in the preceding paragraph, wherein the light field processor is configured to generate an optical prescription by computationally introducing a positive or negative refractive power of the first test amount into the numerical light field image data, receiving feedback from a user regarding the positive or negative refractive power of the first test amount, and computationally introducing a positive or negative refractive power of the second test amount into the light field image data.

[0124] The device of any of the preceding paragraphs, wherein the light field processor is further configured to generate modified numerical light field image data by removing one or more light rays that, if generated as part of the physical light field, would interact with an obstruction in the user's cornea or lens.

[0125] A method for using a wearable ophthalmic device, the method comprising: computationally introducing an amount of positive or negative refractive power into numerical light field image data of a user's surroundings to generate modified numerical light field image data, the amount of refractive power being based on an optical prescription for the user; and generating a physical light field corresponding to the modified numerical light field image data using a light field display.

[0126] The method of the preceding paragraph, wherein the optical power comprises a spherical or cylindrical optical power.

[0127] The method of any of the preceding paragraphs, wherein generating modified numerical light field image data includes correcting the numerical light field image data, at least in part, for an optical prescription.

[0128] The method of any of the preceding paragraphs, wherein generating the modified numerical light field image data includes identifying one or more wavefronts represented by the numerical light field image data and computationally modifying a curvature of the one or more wavefronts based on an optical prescription.

[0129] The method of any of the preceding paragraphs, further comprising adding virtual reality or augmented reality image content to the numerical light field image data.

[0130] The method of any of the preceding paragraphs, wherein the optical prescription comprises a prescription for myopia.

[0131] The method of any of the preceding paragraphs, wherein the optical prescription comprises a prescription for hyperopia.

[0132] The method of any of the preceding paragraphs, wherein the optical prescription comprises a prescription for astigmatism.

[0133] The method of any of the preceding paragraphs, wherein the optical prescription comprises information regarding high order aberrations of the user's eye.

[0134] The method of any of the preceding paragraphs, further comprising receiving light from the user's surroundings and generating numerical light field image data using a light field camera, the light field camera comprising an integrated imaging camera with a two-dimensional array of microlenses and a corresponding two-dimensional array of photodetectors.

[0135] The method of any of the preceding paragraphs, wherein the light field display comprises an integrated imaging display with a two-dimensional array of microlenses and a corresponding two-dimensional array of light sources.

[0136] The method of any of the preceding paragraphs, further comprising projecting infrared light into the user's eye to measure the optical prescription.

[0137] The method of any of the preceding paragraphs, further comprising generating modified numerical light field image data in real time.

[0138] The method of any of the preceding paragraphs, further comprising the step of generating an optical prescription.

[0139] The method of the preceding paragraph, wherein generating an optical prescription includes computationally introducing a positive or negative refractive power of a first test amount into the numerical light field image data, receiving feedback from a user regarding the positive or negative refractive power of the first test amount, and computationally introducing a positive or negative refractive power of a second test amount into the light field image data.

[0140] The method of any of the preceding paragraphs, further comprising generating modified numerical light field image data by computationally removing one or more light rays that, if generated as part of the physical light field, would interact with an obstruction in the user's cornea or lens.

[0141] Chromatic aberration correction In various embodiments, the light field processor system 600 may be used to at least partially correct or compensate for chromatic aberration. Chromatic aberration results from different wavelengths of light interacting differently with the optics of the light field processor system or the user's eye. Chromatic aberration typically results from the fact that the refractive index of optical materials usually varies somewhat as a function of the wavelength of light. Generally, there are two types of chromatic aberration: longitudinal chromatic aberration and lateral chromatic aberration. Longitudinal chromatic aberration occurs when different wavelengths of light are focused at different distances from the lens. Lateral chromatic aberration occurs when different wavelengths of light are focused at different positions on the focal plane of the optical system. Chromatic aberration can cause the color components of an image to shift or displace relative to each other.

[0142] The light field image data projected by the light field processor system 600 and viewed by a user may typically comprise multiple color components. Different color components of the image data may be projected using different wavelengths of light. For example, the color light field image data may include a red component, a green component, and a blue component (although a different number of color components may be used, and different primary colors may also be used). In some embodiments, the light field processor system 600 may be configured to compensate for longitudinal chromatic aberration by varying the focus of the different wavelengths of light. As already discussed, the light field processor system 600 may computationally adjust the focus of the light field image data by introducing a positive or negative refractive power. By computationally introducing a positive or negative refractive power to the light field image data for each color (i.e., computationally introducing a first amount of refractive power to a first color component, a different second amount of refractive power to a second color component, etc.), the light field processor system 600 may adjust the focus of the different colors of the image data to at least partially reduce longitudinal chromatic aberration.

[0143] In addition, the light field processor system 600 can be configured to computationally alter the angles at which different colored light is projected to, at least in part, reduce lateral chromatic aberration. The amount of this angular shift can be different for different color components. In some embodiments, the change in projection angle for a color component is constant across the entire image, while in other embodiments, the change in projection angle for a color component may vary as a function of position within the image.

[0144] In some embodiments, the light field processor system 600 may be configured to compensate for chromatic aberration based on a user's optical prescription, or based on chromatic aberration introduced by the optics of the light field processor system 600 itself, or both.

[0145] The user's eye may impose chromatic aberrations on the image data received at the retina. The chromatic aberrations caused by the eye may be determined based on an optical prescription. The optical prescription may comprise a prescription for longitudinal chromatic aberrations and / or lateral chromatic aberrations. In some embodiments, the light field processor system 600 itself may determine the user's optical prescription. Additionally or alternatively, the system may include a user interface whereby the user inputs an optical prescription, or the light field processor system 600 may determine the chromatic aberrations of the eye via an eye prescription configurator program. In some embodiments, the light field processor system 600 may be configured to objectively monitor and dynamically adjust the user's optical prescription (e.g., in real time) based on inputs received from a biofeedback system, as described herein.

[0146] Similarly, the light field processor system 600 may also impose chromatic aberrations on the light field image data passing through its optics. The chromatic aberrations introduced by the system 600 may be determined based on the system's specifications, manufacturing requirements, and / or characterization measurements. The light field processor system 600 may be configured to compensate, at least in part, for these predetermined aberrations when projecting an image to the wearer's eye.

[0147] Once information about the chromatic aberrations introduced by the user's eyes and / or the light field processor system 600 itself is known, the system can select an appropriate image correction program to apply to image data projected to the user by the display of the light field processor system 600. As discussed further below, the image correction program may, for example, change the focus of one or more color components of the image to compensate for longitudinal chromatic aberration. In another embodiment, alternatively or in combination, the image correction program may change the projection angle of one or more color components of the image to compensate for lateral chromatic aberration.

[0148] The light field processor 70 may be configured to correct chromatic aberration by providing a different amount of refractive power for each color component of the light field image data for display to a user. For example, the light field processor 70 may be configured to computationally provide a first refractive power adjustment for a first color component of the light field image data. The light field processor 70 may be configured to computationally provide a second refractive power adjustment, different from the first refractive power adjustment, for a second color component of the light field image data. Similarly, the light field processor 70 may be configured to computationally provide a third refractive power adjustment, different from the first and second refractive power adjustments, for a third color component of the light field image data. Any of the refractive power adjustments may be positive or negative. The specific amount of refractive power adjustment provided to any color component of the image data may be selected to bring the individual focal planes associated with the color components of the image data closer together. In this manner, the focal depth of each color component of the image may be selectively altered to reduce longitudinal chromatic aberration.

[0149] In some embodiments, the light field processor 70 may be configured to computationally vary the projection angles or positions of different color components of the light field image data. The projected angles / positions may be varied to substantially align the different color components of the image data. In some embodiments, the light field processor 70 may computationally modify the projection angle and / or position for a first color component of the light field image data by a first adjustment amount. The same can be done for other color components of the image data to reduce lateral chromatic aberration (i.e., the projection angle and / or position for a second color component can be computationally modified by a second adjustment amount, and the projection angle and / or position for a third color component can be computationally modified by a third adjustment amount).

[0150] After selecting an appropriate image correction program for making the aforementioned adjustments, the light field processor system 600 may apply the image correction program to computationally compensate for chromatic aberrations in the projected light field image data. This may be done on a real-time basis. Although the system 600 compensates for chromatic aberrations, not all chromatic aberrations necessarily need to be removed. For example, some chromatic aberrations may contribute to creating a realistic focus of depth cues for a given user. Thus, the compensation of chromatic aberrations may be controlled to allow some aberrations while correcting others to provide improved or optimal visual quality.

[0151] The following paragraphs describe various exemplary embodiments of the devices, systems, and methods described herein.

[0152] 1. A wearable ophthalmic device comprising: an outwardly facing, head-mounted light field camera configured to receive light from a user's surroundings and generate numerical light field image data; the numerical light field image data comprising a plurality of color components; a light field processor configured to computationally correct the numerical light field image data to generate corrected numerical light field image data, wherein different color components are corrected differently to, at least in part, compensate for chromatic aberration; and a head-mounted light field display configured to generate a physical light field corresponding to the corrected numerical light field image data.

[0153] The device of the preceding paragraph, wherein the chromatic aberration comprises longitudinal chromatic aberration.

[0154] The device described in the preceding paragraph, wherein the light field processor is configured to computationally introduce a first amount of refractive power into a first color component of the numerical light field image data and a second amount of refractive power into a second color component of the numerical light field image data, the second amount of refractive power being different from the first amount of refractive power.

[0155] The device of the preceding paragraph, wherein the first amount of refractive power and the second amount of refractive power reduce a difference between a focal plane of the first color component and a focal plane of the second color component.

[0156] The device of any of the preceding paragraphs, wherein the chromatic aberration comprises lateral chromatic aberration.

[0157] The device described in the preceding paragraph, wherein the light field processor is configured to project a first color component of the light field image data at a first angle and project a second color component of the light field image data at a second angle different from the first angle to compensate for lateral chromatic aberration.

[0158] A device according to any of the preceding paragraphs, wherein color components of the light field image data are modified based on an optical prescription for the user's eye.

[0159] The device of any of the preceding paragraphs, wherein color components of the light field image data are corrected based on a measurement of chromatic aberration of the device.

[0160] The device of any of the preceding paragraphs, wherein the color components comprise red, green, and blue color components.

[0161] 1. A method for using a wearable ophthalmic device, the method comprising: receiving light from a user's surroundings and generating numerical light field image data using an outward-facing head-mounted light field camera, the numerical light field image data comprising a plurality of color components; computationally correcting the numerical light field image data using a light field processor to generate corrected numerical light field image data, different ones of the color components being corrected differently to at least in part compensate for chromatic aberration; and generating a physical light field corresponding to the corrected numerical light field image data using a head-mounted light field display.

[0162] The method of the preceding paragraph, wherein the chromatic aberration comprises longitudinal chromatic aberration.

[0163] The method of the preceding paragraph, further comprising using a light field processor to computationally introduce a first amount of refractive power into a first color component of the numerical light field image data and a second amount of refractive power into a second color component of the numerical light field image data, the second amount of refractive power being different from the first amount of refractive power.

[0164] The method of the preceding paragraph, wherein the first amount of refractive power and the second amount of refractive power reduce a difference between a focal plane of the first color component and a focal plane of the second color component.

[0165] The method of any of the preceding paragraphs, wherein the chromatic aberration comprises lateral chromatic aberration.

[0166] The method of the preceding paragraph, further comprising using a light field processor to project a first color component of the light field image data to a first angle and project a second color component of the light field image data to a second angle different from the first angle to compensate for lateral chromatic aberration.

[0167] The method of any of the preceding paragraphs, wherein color components of the light field image data are modified based on an optical prescription for the user's eye.

[0168] The method of any of the preceding paragraphs, wherein color components of the light field image data are modified based on a measurement of chromatic aberration of the device.

[0169] The method of any of the preceding paragraphs, wherein the color components comprise red, green, and blue color components.

[0170] Correction of farsightedness In some embodiments, the light field processor system may be used to compensate for presbyopia, which is a reduction in the amplitude of accommodation of the eye's lens, typically associated with aging. For close objects, the ciliary muscle changes the shape of the lens to provide accommodative refractive power, which focuses the light received by the eye onto the retina and forms an image thereon. With age, the ability of the eye's lens to change shape and accommodate for near vision is reduced. Traditionally, presbyopia is treated by using multifocal corrective lens systems with segmented lenses (e.g., bifocal, trifocal, etc.), or by using lenses with continuous focal length gradients (e.g., progressive lenses) or variable focus mechanically deformable or liquid crystal lenses.

[0171] In some embodiments, the light field processor system 600 may be configured to assist with presbyopia. In various embodiments, the light field processor system 600 may computationally function as a variable focus lens. As described above, for example, when correcting for myopia, hyperopia, or astigmatism, the light field processor system 600 may apply a computational transformation to the captured light field image data and add a spherical wavefront curvature, which may be dynamically modified by varying the computational transformation applied. By using the light field processor 70 to add a positive spherical power for near viewing, the system can provide a presbyopia correction. The correction added for near viewing can be applied to the light field image data of the entire field of view or only a portion thereof (e.g., the lower portion of the field of view).

[0172] In some embodiments, a user may be able to manually adjust the focusing optical power added to the light field captured by the light field processor 70. For example, in some embodiments, the light field processor system 600 may have a feedback mechanism (e.g., a user interface control) to increase or decrease the optical power, which is computationally applied to the light field captured by the light field processor 70. The user may provide feedback based on the focus of the image data being presented to them.

[0173] Alternatively or in addition, the light field processor system 600 may be configured to automatically or bidirectionally adjust the refractive power added by the light field processor 70 based on subjective or objective measurements of the user's eyes. For example, in some embodiments, the light field processor system 600 includes one or more eye tracking cameras or imaging systems to track the eyes. Such cameras and / or imaging systems may monitor the orientation of the eyes and the corresponding direction of gaze of the individual eyes. For example, if the user's eyes are looking downward, the user may be looking at a nearby object, such as a book, or may be looking at projected image content that corresponds to an image placed in a location typically associated with a nearby object (the lower part of the field of view). The gaze may also be determined based on eye convergence-divergence, i.e., how the gaze of the pair of eyes converges at a location and the distance of that location relative to the wearer. Thus, by monitoring the convergence-divergence, the distance at which the viewer intends to view an object may be determined. When the light field processor system 600 detects that the user is looking down or viewing a nearby object, the light field processor 70 can add spherical power to the captured light field image data to help correct for presbyopia.

[0174] In some embodiments, the light field processor system 600 may include a sensor that detects the user's head position. In one embodiment, the distance at which the viewer intends to view an object may be estimated or detected based on the user's head position (e.g., head pose, or orientation). For example, if the user's head is tilted forward and / or downward, the wearer may be looking at a relatively close object, such as a book, or may be looking at projected image content that corresponds to an image located within a location typically associated with nearby objects (the lower portion of the field of view).

[0175] In some embodiments, the light field processor system 600 may comprise a sensor assembly 39 with one or more accelerometers, gyroscope sensors, etc., configured to determine the user's head position (e.g., head pose or head orientation, i.e., straight, tilted down, looking up, etc.) and / or head movement, as discussed elsewhere herein. The light field processor system 600 may also include a processor 32 (e.g., a head pose processor) operably coupled to the sensor assembly 39 and configured to perform digital and / or analog processing to derive head position, head pose, and / or orientation from movements detected by the sensor assembly 39. In one embodiment, the sensor assembly 39 may generate movement data, which is stored in a digital memory. In some embodiments, the movement data may be used to reduce signal noise while diagnosing vision disorders (e.g., detection of head movement during a test may indicate an erroneous test and result). The processor 32 may read this movement data, execute processing logic, and determine head position (e.g., head pose or orientation).

[0176] In one or more embodiments, the gaze orientation may also be based on tracking eye movements through an eye tracking system. In one embodiment, the prescription may be correlated with a set of user eye convergence points, which indicate the focal depth of the eyes. For example, the user's eyes may be tracked to a convergence point below the horizon while the user's head position may remain unchanged. Such a movement may indicate that the eyes are focused on an object located at a near focal depth. Also, as mentioned above, eye convergence-divergence movements can help determine the distance at which the viewer is directing (e.g., focusing) their attention. This distance may be ascertained from the convergence of the eyes' gaze. Thus, in various embodiments, the user's eyes may be tracked to a convergence point at a particular distance from the wearer.

[0177] Similarly, in various embodiments, the light field processor system may be configured to determine a focal depth at which the eyes are focused or accommodated. In some embodiments, the eye tracking system may be used to triangulate the user's convergence point and adjust the focus of the image to be presented to the user accordingly. For example, the eye tracking system may determine the direction (e.g., a line extending from each eye) in which each eye is looking and determine the convergence angle at which the directions intersect. The convergence point may be determined from the determined angle of convergence. In some embodiments, the eye tracking system may be included as part of a biofeedback system. As described above, in various embodiments, the light field processor system may track the position of each eye utilizing a camera 24 paired with a light source 26 (e.g., an infrared light source and an infrared camera), which may be operably coupled to a local processing module 70. The local processing module 70 may include software that, when executed, may be configured to determine the convergence point of the eyes. From this determination, the light field processor system may also execute a logic device to determine a focal location or depth based on the orientation or direction of the user's gaze.

[0178] In another embodiment, gaze orientation may be determined through flash detection. An eye tracking system may be configured to determine one or more flashes or reflections from the eye and determine the location of one or more flashes on the eye relative to eye features (e.g., pupil, cornea, etc.). As the eye is moved, the relative location of the flash on the eye may change. For example, if the flash is located at the top of the eye and the space between the flash and the pupil increases, this may indicate that the gaze orientation is tilted downward and the eye may be accommodating to a near focal depth.

[0179] FIG. 12 illustrates an exemplary method for correcting presbyopia using the light field processor system 600. The method 1200 is directed to modifying light field image data presented to a user based on the user's prescription. In some embodiments, the method 1200 may be implemented by a patient-worn ophthalmic device such as that described in connection with FIGS. 3A-5. In various embodiments, the light field processor system described in FIGS. 6 and 7 may be used to provide correction for presbyopia based on the user's optical prescription.

[0180] At 1202, a presbyopia prescription is determined for the user. The prescription may be determined by receiving information from the user, or may be determined by the light field processor system itself by adjusting the wavefront presented to the user and the user selecting the desired prescription. For example, the light field processor system may be configured to test for optical prescriptions for different focal planes of accommodation. In some embodiments, the light field processor system may be pre-programmed with discrete granularity steps in modifying the wavefront of the light field image data presented to the user through the display device 62 for multiple focal depths. For example, the light field processor system may employ ophthalmometer technology as described herein. The prescription may be entered by the user through a user interface or may be stored in the remote data repository 74. The prescription may be read by one or more processors of the light field processor system, for example, the remote processing module 72.

[0181] In another embodiment, the light field processor system may automatically, and possibly gradually, change the user's prescription via a biofeedback system. The biofeedback system may be configured to determine the user's comfort level in viewing an object or image. For example, if the user's eyes are unstable, shift, oscillate, change accommodation (e.g., in an unsteady or random manner), etc., these may be indicators that the user is unable to comfortably view the object. Thus, accommodation, convergence and divergence movements, pupil size, etc. may be monitored and / or an auto-refractometer or other device (e.g., a scanning laser ophthalmoscope (SLO)) may be used to ascertain whether the retina is focused on the fovea.

[0182] In some embodiments, the light field processor system may be configured to receive an optical prescription for presbyopia correction (e.g., additional power) from a third party. For example, a physician may be able to wirelessly transmit (e.g., via the Internet, a Bluetooth® connection, a hospital network, etc.) the user's optical prescription, which is received by a receiver or transceiver and stored in a digital memory of the local processing module 70.

[0183] At 1204, the system may store information about the compensation wavefront. The compensation wavefront may be based on the user's optical prescription for various focal depths of accommodation. In some embodiments, the information may be input parameters for varying the focal depth.

[0184] In some embodiments, each correction function comprises a set of input parameters that define the accommodation to achieve the desired wavefront correction. The input parameters may be based on the optical prescription of the user for correcting presbyopia when the wearer focuses to a near focal depth. The system may correlate different viewing distances with corresponding correction functions to compensate for refractive errors associated with accommodation at variable focal depths. Thus, different convergence points and focal depths of accommodation may be correlated with different prescription corrections.

[0185] In some embodiments, this information may be further correlated with a set of head positions (e.g., head pose, orientation) and / or gaze directions. For example, if the user's head is tilted downward, the eyes may accommodate at a certain (e.g., closer) focal depth. Thus, in various embodiments, a program specific to presbyopia may be pre-programmed (or downloaded) into the light field processor system so that the correct head position (e.g., head pose, orientation) and / or gaze orientation can be correctly matched with a certain amount of presbyopia correction. Different gaze directions and head orientations can be correlated with different prescription corrections to assist with accommodation. The light field processor system may thus be configured to provide different optical corrections depending on the measured gaze direction and / or head position, head pose, or orientation.

[0186] At 1206, the system may detect the user's gaze orientation and / or head position, head pose, or orientation through a gyroscope, accelerometer, inertial measurement unit (IMU), other sensors, or combinations thereof. The gaze orientation (including, for example, eye convergence point) and / or head position, head pose, or orientation may indicate whether the wearer is looking near or far, and therefore whether the wearer needs to accommodate as described above. Thus, different correction functions may be used for different focal depths of accommodation.

[0187] In some embodiments, the light field processor system comprises a sensor assembly 39, a processor 32, and a head pose processor 36, which may be configured to detect the movement, tilt, and orientation of the user's head. In some embodiments, the sensor may be operatively coupled to a local processing module 70, which may execute logic and read out the detected head movement and / or the determined head position, head pose, or orientation. A downward movement of the user's head may indicate focusing on an object in the near distance.

[0188] In some embodiments, the gaze orientation may be based on tracking eye movements as described above. For example, a downward movement of one or more of the user's eyes may indicate focusing on an object in the near distance (e.g., shifting the viewpoint from the horizon to a book held below the horizon). Thus, if the eye tracking system determines that the user's eyes have shifted downward, an appropriate correction function may be determined based on the presbyopia prescription. In another embodiment, the eye tracking system may be configured to determine whether the angle of convergence is increasing (i.e., the convergence point is closer to the user). Such a determination may indicate focusing on an object in the near distance. Thus, the light field processor system may be capable of determining the focal depth of accommodation.

[0189] In some embodiments, the system may track the position of each eye using the camera 24, which may be operatively coupled to the light field processor 70. In another embodiment, the system may perform flash detection and monitoring using the camera 24, for example, the camera 24 tracks the location of the flash relative to eye features (e.g., eye margins, eye and eyelid intersection, pupil, etc.). The light field processor 70 may include software that, when executed, may be configured to track eye movement, flash movement, and / or determine eye convergence points. In some embodiments, the gaze orientation may be stored in a remote data repository 74.

[0190] In some embodiments, the remote processing module 72 may be configured to correlate gaze orientation, angle of convergence, and / or head position information with the optical prescription, all of which information may be stored locally or in a remote data repository 74.

[0191] At 1208, based on the detected gaze orientation and / or head position, the system consults a mapping table (e.g., the information stored at 1204) to determine an appropriate correction function and computationally apply it to the captured light field image data to produce a compensated wavefront. For example, based on the gaze orientation or direction and / or head position, the system may determine a focal depth to which the eye is accommodating. In various embodiments, an optical prescription may be correlated with one or more focal depths associated with accommodation. For example, different convergence points or focal depths associated with different amounts of accommodation may be correlated with different optical prescriptions and correction functions.

[0192] In various embodiments, at 1208, the system may read the detected focal depth of accommodation and look up the mapping stored at 1204. Based on the mapping, the system may determine an appropriate correction function for the identified focal depth.

[0193] In some embodiments, the light field processor 70 may read the detected gaze orientation stored in the digital memory at 1206. Alternatively, the detected gaze orientation may be received directly from the eye tracking system, the sensor assembly 39, and / or the head pose processor 36. The light field processor 70 executes logic to access the mapping table stored at 1206 and selects an appropriate corresponding correction function based on the gaze orientation, angle of vergence, and / or head position to computationally apply to the collected light field image data to compensate for presbyopia.

[0194] At 1210, an appropriate compensating wavefront (e.g., positive spherical power) is computationally applied to the collected light field image data. At 1212, the modified light field image data is presented to a user. The processed light field image data may have a wavefront that is modified based on the selected correction function as applied at block 1210.

[0195] Thus, the method 1200 may be implemented to dynamically correct a user's presbyopia. The correction applied by the light field processor may be dynamically reconfigured (e.g., reconfigured in real time as the user's optical prescription changes, and thus updated repeatedly during use of the light field processor system). For example, the presbyopia prescription correction may be adjusted at various time intervals. The light field processor system may thus be configured to dynamically correct changes in the user's presbyopia prescription. The intervals may be determined in advance and based on an expected rate or occurrence of vision impairment, deterioration, or change. For example, the user's presbyopia may change as the user ages.

[0196] In some embodiments, at 1210, the light field processor system may implement an eye prescription configuration program. At 1210, the light field processor system may be configured to return to block 1202 at each interval without user activation and to manually and interactively determine the user's prescription, updating or adjusting the prescription based on input from a biofeedback system, as described above. Such procedures may be scheduled by protocol (e.g., configured to be checked once a month, several times a year, etc.) or when vision, e.g., myopia, is determined to be worsening. In another embodiment, as described above, the biofeedback system may monitor the movement and changes of the user's eyes (e.g., via the camera 24 and light source 26) to determine that the user is struggling with accommodation. For example, the light field processor system may monitor convergence and divergence movements, pupil dilation, and / or movement and / or shape of the eye's natural lens. The light field processor system may also monitor the image formed on the fovea of ​​the retina using an autorefractometer or other technology as described herein. The light field processor system may then initiate an eye prescription configuration program, determine a new optical prescription, and / or adjust the correction function (e.g., update the depth of focus and correction function mapping table).

[0197] In one or more embodiments, the light field processor system may allow a user to manually adjust the focus of one or more images presented to the user. For example, the system may be pre-programmed with discrete steps in adjusting the focus. The user may then specify the desired focus to the light field processor system through a user interface. In some embodiments, the user may have the option to incrementally increase or decrease the prescription (e.g., change the focus) until the user arrives at a comfortable viewing focus. Alternatively, the light field processor system may incrementally increase or decrease the prescription automatically, possibly by utilizing biofeedback systems and other diagnostic techniques (e.g., see the description of oculometer and autorefractor techniques herein). In some embodiments, such user-input prescriptions may be associated with a particular gaze or head orientation and provided when the wearer has such gaze or head orientation. However, in some embodiments, such user-input prescriptions may be applied independently of gaze or head orientation and do not change with changes in gaze, gaze, and / or head orientation.

[0198] The following paragraphs describe various exemplary embodiments of the devices, systems, and methods described herein.

[0199] 1. A wearable ophthalmic device comprising: an outwardly facing, head-mounted light field camera configured to receive light from a user's surroundings and generate numerical light field image data; a light field processor configured to generate modified numerical light field image data by computationally introducing an amount of refractive power into the numerical light field image data based on a viewing distance from the user to objects in the numerical light field image data; and a head-mounted light field display configured to generate a physical light field corresponding to the modified numerical light field image data.

[0200] The device of the preceding paragraph, further comprising a head-mounted accelerometer sensor for determining a user's head orientation, and wherein the viewing distance is estimated based on the user's head orientation.

[0201] The device of any of the preceding paragraphs, further comprising one or more head-mounted eye-tracking cameras to determine a user's gaze direction, and wherein the viewing distance is estimated based on the user's gaze direction.

[0202] The device of any of the preceding paragraphs, further comprising one or more head-mounted eye-tracking cameras for determining a convergence point of the user's gaze, and wherein the viewing distance is estimated based on the convergence point of the user's gaze.

[0203] A device according to any of the preceding paragraphs, wherein the amount of refractive power is based on an optical prescription for the user.

[0204] The device of any of the preceding paragraphs, wherein the light field camera comprises an integrated imaging camera.

[0205] The device of any of the preceding paragraphs, wherein the light field display comprises an integral imaging display.

[0206] The device of any of the preceding paragraphs, further comprising a manual control to define an amount of refractive power to introduce into the numerical light field image data.

[0207] The device of any of the preceding paragraphs, wherein the light field processor is configured to introduce that amount of refractive power into the entire field of view of the numerical light field image data.

[0208] The device of any of the preceding paragraphs, wherein the light field processor is configured to introduce that amount of refractive power into only a lower portion of the field of view of the numerical light field image data.

[0209] 1. A method for using a wearable ophthalmic device, the method comprising: receiving light from a user's surroundings and generating numerical light field image data using an outward-facing head-mounted light field camera; generating modified numerical light field image data using a light field processor by computationally introducing an amount of refractive power into the numerical light field image data, the amount of refractive power introduced being based on a viewing distance from the user to an object in the numerical light field image data; and generating a physical light field corresponding to the modified numerical light field image data using a head-mounted light field display.

[0210] The method of the preceding paragraph, further comprising determining a user's head orientation using a head-mounted accelerometer sensor, and wherein the viewing distance is estimated based on the user's head orientation.

[0211] The method of any of the preceding paragraphs, further comprising determining a user's gaze direction using one or more head-mounted eye-tracking cameras, and wherein the viewing distance is estimated based on the user's gaze direction.

[0212] The method of any of the preceding paragraphs, further comprising determining a convergence point of the user's gaze using one or more head-mounted eye tracking cameras, and the viewing distance is estimated based on the convergence point of the user's gaze.

[0213] The method of any of the preceding paragraphs, wherein the amount of refractive power is based on an optical prescription for the user.

[0214] The method of any of the preceding paragraphs, wherein the light field camera comprises an integrated imaging camera.

[0215] The method of any of the preceding paragraphs, wherein the light field display comprises an integral imaging display.

[0216] The method of any of the preceding paragraphs, further comprising receiving from a manual control an amount of refractive power to introduce into the numerical light field image data.

[0217] The method of any of the preceding paragraphs, further comprising introducing that amount of refractive power into the entire field of view of the numerical light field image data.

[0218] The method of any of the preceding paragraphs, further comprising introducing that amount of refractive power only into a lower portion of the field of view of the numerical light field image data.

[0219] Strabismus / Amblyopia correction Another common visual condition is strabismus, in which both eyes are unable to align at a single point of convergence to produce a fused stereoscopic image. This typically results from an eye with weakened eye muscles being unable to coordinate its movements with its normal counterpart. Similarly, amblyopia is a visual condition in which there is reduced vision in one or both eyes. This reduced vision can result from abnormal development of vision during infancy or early childhood. Amblyopia is sometimes also referred to as the "lazy eye."

[0220] In some embodiments, the wearable light field processor system 600 may be used to treat or correct convergence insufficiencies, such as those resulting from strabismus or amblyopia. As an example, if the convergence is offset in an angular manner, a compensating prism correction may be computationally applied to bring the convergence points of both eyes together. The compensating prism correction may be computationally applied to the collected light field image data by the light field processor.

[0221] If the convergence of the two eyes is angularly offset, one or more of the following techniques may be used: In some embodiments, an eye tracking system may determine the gaze vector and / or focus of the healthy eye. This information may be extrapolated to determine a target convergence point for the two eyes. In an embodiment, an eye tracking system and a depth sensing system may be used in combination to determine the convergence point of the two eyes. In some implementations, one or more eye muscles may be "retrained" through a treatment protocol to gradually align the focus and / or convergence points of the two eyes. The treatment protocol may include methods described herein, including methods designed to strengthen the muscles of the weaker eye and / or stimulate neural responses to optical signals from the weaker eye.

[0222] The light field processor system 600 can be configured to selectively introduce prism effects or angular shifts into the image provided to the wearer. This can be done in several ways and for different purposes. For example, a computational compensation prism correction can be applied as an optical correction for the wearer (e.g., to compensate for convergence insufficiency in one or both of the wearer's eyes). This correction can be applied to the collected light field image data and take into account the impairments of the wearer, so that the wearer can achieve or approximate binocular single vision, even if the wearer suffers from strabismus and / or amblyopia.

[0223] Compensatory prism correction can be achieved by adjusting the location of image data provided to the wearer. The light field processor can achieve this by shifting the location of the image presented on the display laterally (orthogonal to the normal line of sight and normal to the optical axis) (compared to where the image data would be projected in the absence of such intervention). The system 600 can be configured to detect the focus or alignment of the wearer's eyes and adjust the location of the individual left and right images to be at a target point within the field of view of each eye. For example, the system 600 can include eye tracking to determine the line of sight of each eye. Once the line of sight is determined, the system can be configured to center and position the individual left and right images within the field of view of the individual left and right eyes. In some embodiments, to retrain the weaker eye, the system can gradually move the image presented to the weaker eye toward a desired or target convergence point. In this way, the weaker eye can be retrained to see properly at the same point as the stronger eye.

[0224] To determine the amount of prism correction to computationally apply, the light field processor system 600 can be configured to monitor where the image and light were projected onto the retina of the wearer's weaker eye. (This can be done using integrated ocular imaging capabilities as described herein.) If the prism correction allows the light to illuminate the desired portion of the retina, this is the correct amount. If not, a greater or lesser amount is needed. As described herein, autorefractometer techniques or other techniques (e.g., Scanning Laser Ophthalmoscopy (SLO)) can be used to determine whether the compensatory prism correction reduced or corrected the wearer's misaligned vision. In some embodiments, the system 600 is configured to determine (or receive input indicative of) whether the user has an exotropia or esotropia deviation. Once this deviation is captured by the device, prism correction can be computationally applied to the light field image data until the vision defect is substantially corrected. This may be determined automatically or based on user input. In some embodiments, to automatically determine the correction or preferred prism correction, the system 600 can include one or more inward facing cameras to measure the angular deviation (e.g., shift in fixation) and use a display to vary the prism prescription in one eye while occluding the other eye. In some embodiments, to determine the correction or preferred prism correction using user input, the system 600 can be configured to implement a test similar to the Maddox rod test. For example, the system 600 can provide a digital filter to filter light for the test. As another example, the system 600 can provide image sources from two different depth planes. (This can be done using a light source 714 configured to project light into the wearer's eye.) Based on the user input, the system 600 can adjust the prism correction until a satisfactory condition is met (e.g., the first image is aligned with the second image).

[0225] As another example, computational compensatory prism correction can be applied for therapeutic purposes (e.g., to gradually retrain the eyes to arrive at a target convergence point). Methods for retraining the eyes that emphasize the presentation of images with different characteristics to the wearer's individual eyes are also discussed herein.

[0226] In some embodiments, the light field processor system 600 includes an eye tracking system. The eye tracking system can be configured to determine the line of sight of the wearer's eye. The eye tracking system can include one or more sensors configured to sense properties of the wearer's eye. In some embodiments, the one or more sensors include a camera as described herein. In various embodiments, the one or more sensors, including a camera, can be configured to image phosphenes and / or Purkinje images and determine the line of sight. The eye tracking system can include an analysis module configured to determine the direction of the line of sight of the wearer's eye based at least in part on information obtained using the one or more sensors.

[0227] In some embodiments, system 600 includes one or more outwardly facing cameras. In an implementation, the one or more outwardly facing cameras can be similar to camera 16 described herein with reference to FIG.

[0228] The system 600 can include one or more user interface features configured to enable the wearer or other person to provide input to the device. The user interface features can be integrated with the system. In some implementations, the user interface features are provided by devices or components that are not physically integrated with the system. For example, the user interface features can be provided by a device or system that communicates with the device. This can be a smartphone, computer, tablet, or other computing device that communicates with the device by wire or wireless communication. In some embodiments, the user interface features can be provided by a combination of different devices and systems that are linked to the device (e.g., through a wire or wireless communication network, or through a component that is physically linked to or integrated with the device). The user interface features can be presented on a device with a touch screen, and interaction with the touch screen provides input to the system 600. Voice recognition and / or virtual touch screen technology can also be employed. The user interface features can include touch-sensitive capacitive features, keyboards, buttons, microphones, photodetectors, or various software-implemented features provided by a graphical user interface. In some embodiments, the user interface features include a gesture detection component, allowing the wearer to provide user input through gestures. In some embodiments, the user interface features include a gaze detection component, allowing the wearer to provide user input through eye gaze (e.g., this can include selecting a button or other element when the wearer fixates the button for a period of time or when the wearer blinks while fixating on the button). Such a system can also be used for other devices and systems described herein.User interface features can be presented on the device using a touch screen, and interaction with the touch screen provides input to the wearable light field processor system.

[0229] In some implementations, the wearer, clinician, physician, or other user can use the interface features to control aspects of the vision test and / or therapy, for example to vary the amount of computational prism correction applied, the amount of computational lateral shift of the image, modify characteristics of the enhanced image, or otherwise configure the convergence insufficiency test or treatment.

[0230] FIG. 13 illustrates an exemplary method 1300 for treating convergence insufficiency, such as that caused by strabismus and / or amblyopia, using a light field processor system. For ease of explanation, the method 1300 will be described as being implemented by the light field processor system 600 as described herein. The method 1300 includes a process of "retraining" the lazy or misaligned eye by occluding or deemphasizing the stronger eye. It should be understood that many treatment protocols can be devised based on the user's particular prescription, and the exact parameters and / or techniques may vary.

[0231] In block 1302, the light field processor system 600 determines the difference in the focus and / or convergence points of the two eyes. As discussed herein, this difference may be determined based on user input or based on prescription testing performed by the light field processor system. Eye tracking and / or gaze detection may also be used. The light field processor system may be configured to perform any of the methods described herein, for example, to determine the focus and / or convergence points.

[0232] In block 1304, the light field processor system 600 selects a treatment protocol to help treat the wearer's vision impairment. In some embodiments, the treatment protocol may be devised by a doctor or clinician, or the treatment protocol may be devised at an external location and downloaded onto the light field processor system. The treatment protocol may comprise various parameters. For example, the treatment protocol may involve the frequency at which the treatment is administered. The treatment protocol may include information regarding the type of image to be presented to the wearer and / or the difference in the two displays or images shown to each eye. For example, the treatment protocol may be based on binocular split vision presentation, where images with different characteristics are displayed to the wearer (e.g., different images or the same image with modified versions of the image shown to the left and / or right eye). In some implementations, the image shown to the weaker eye may be enhanced and / or the image shown to the stronger eye may be attenuated. For example, the image shown to the weaker eye can be altered to be more interesting or attention-grabbing to the wearer (e.g., brightened, color enhanced, three-dimensionally enhanced, sharpened focus, higher resolution, enhanced contrast, moving, higher refresh rate, etc.). Similarly, the image shown to the stronger eye can be altered to be less interesting or attention-grabbing to the wearer (e.g., darkened, dull color, flattened, blurred, lower resolution, lower contrast, static, lower refresh rate, etc.). In various implementations, only the image shown to the weaker eye is altered while the image shown to the stronger eye is not altered. In various implementations, only the image shown to the stronger eye is altered while the image shown to the weaker eye is not altered. In various implementations, the images shown to both the weaker and stronger eyes are altered. The treatment protocol may include information regarding the duration of the protocol. The treatment protocol may employ interactive virtual objects, thereby making the "treatment" more enjoyable for the user and increasing user compliance with the therapy regimen.Treatment protocols may employ dynamic imagery (eg, movies, games, etc.) to make treatment more enjoyable, thereby increasing compliance.

[0233] In block 1306, the light field processor system may detect or determine a time or time frame for initiating the treatment protocol based, at least in part, on a scheduler associated with the treatment protocol. For example, the treatment protocol may be programmed such that eye retraining is performed daily between 10 PM or 8 AM-9 AM. The treatment protocol may be prescribed, for example, only once per week. The treatment protocol may involve more treatment sessions, such as at least twice per week, at least five times per week, daily, and / or multiple times per day, such as at least once, two, three, four, or five times per day. In some embodiments, the treatment protocol may be programmed to respond to detection of the eye becoming more or less misaligned. In various embodiments, the treatment protocol may be programmed to occur when the eye has recovered from a previous treatment protocol.

[0234] In block 1308, the light field processor system modifies the field of view of one or both eyes. This may include, for example, partially or completely occluding the eyes computationally (e.g., by blacking out all or part of the image data presented to each eye). In some embodiments, the light field processor system may present different images to each eye, sometimes at different locations in the field of view, either to strengthen the weaker eye or to encourage proper convergence and divergence movements for image fusion in the brain. It should be understood that this is merely one exemplary technique and that many other techniques may be used to strengthen or retrain the eye muscles. The occlusion may be partial or complete occlusion. Complete or partial defocusing, blurring, attenuation, or other modifications to the images presented to the wearer may be used as well.

[0235] In some embodiments, the light field processor system may skip block 1308. For example, in some treatment protocols, the individual fields of view of the stronger and weaker eyes are not altered. The light field processor system may present images of different visual characteristics to the wearer's eyes to encourage the weaker eye to gain strength.

[0236] As described herein, the content shown to the individual eyes can be different as well, regardless of whether the field of view of one or both of the eyes is altered. For example, more intriguing content can be shown to the weaker eye. This may be accomplished by projecting an image to the weaker eye that is brighter, higher resolution, more complete, moving, higher contrast, three-dimensional, color enhanced, resulting from multiple depth planes, etc. As another example, less intriguing content can be shown to the stronger eye. This may be accomplished by projecting an image to the stronger eye that is darker, lower resolution, missing parts, static, lower contrast, flattened, dull color, resulting from a single depth plane, etc. The intriguing content can be projected to the weaker eye at the same time as the less intriguing content that may be projected to the stronger eye, thereby encouraging the weaker eye to gain intensity.

[0237] In block 1310, the light field processor system projects stimulating images that attract the weaker eye's attention. These stimulating images may be presented at prescribed locations and / or with enhanced visual properties, i.e., color saturation, contrast, resolution, depth cues, three-dimensional effects, brightness, intensity, focus, etc., to encourage the eyes to focus and / or converge at the target location and / or the visual content from the weaker eye to reinforce it. The virtual images may be moved over time and at rates specified by the treatment protocol to bring the eyes together at the point of convergence and divergence across multiple depth planes. When the eyes align at a common point of convergence and divergence, the images from each eye are fused and the brain sees one image instead of two. This may be accomplished, for example, in the form of a game.

[0238] In various embodiments, for example, the light field processor system can be configured to present images to both eyes (binocular dissection) simultaneously for treatment or therapeutic purposes. Images presented to individual eyes can differ in visual characteristics. This difference can increase the performance of the weaker eye over time. For example, left and right images can be presented to the wearer to provide the wearer with a stereoscopic image. During treatment, the image corresponding to the weaker eye can be enhanced relative to the stronger eye. Image enhancement can include, for example, but not limited to, increasing the brightness of the image, increasing the contrast of the image, increasing the color saturation of the image, increasing the intensity of the image, increasing the three-dimensional effect of the image, adding content to the image, and the like. Similarly, the image corresponding to the stronger eye can be attenuated. Image attenuation can include, for example, but not limited to, decreasing the color saturation of the image, attenuating or reducing the intensity of the image, flattening the image, blurring the image, defocusing the image, shading the image, partially or completely occluding the image, and the like. In some implementations, defocusing the image can be accomplished by computationally presenting the image to the individual eyes from different depth planes. In some embodiments, the treatment can include enhancing the image to the weaker eye and attenuating the image to the stronger eye. In some embodiments, the treatment can include enhancing the image to the weaker eye while not altering the image to the stronger eye. In various embodiments, the treatment can include attenuating the image to the stronger eye while not altering the image to the weaker eye. The enhancement and / or attenuation of the image can be applied gradually and / or intermittently. For example, the image quality can be gradually enhanced or attenuated every 30-60 seconds and / or when the light field processor system detects that the eyes have become more misaligned. As another example, the image can be enhanced or attenuated for a period of time, and then the effect can be removed for a second period of time. This can be alternated over the treatment period.

[0239] The treatment may also include, in some embodiments, varying the depth plane at which the image is presented. This may be similar to a block string, where multiple depth planes are used to retrain eyes with convergence insufficiency. Images are projected from various depth planes, thereby allowing the eyes to converge and focus on the image at variable depths. Variable depth planes may also be used to provide a treatment similar to pencil push-ups. The treatment may include presenting an image at a first depth plane (e.g., about a foot away or further away), and then moving the image closer to the wearer to a second depth plane. Moving the image may include gradually moving the depth plane from the first depth plane to closer to the wearer to the second depth plane. While the image is presented at a closer depth plane than a book, the depth of the image may be adjusted so that the wearer may practice focusing on the image in areas where it is difficult to focus (e.g., the wearer has difficulty converging on the image). The treatment also includes providing an image at a third depth plane, which is further away from the first and second depth planes. While the image is presented at the second depth plane, the wearer can alternate between focusing on the image at the second depth plane and the image presented at the third depth plane. This can, for example, strengthen the eye muscles. These methods can be combined with the enhancement and / or attenuation of the image during treatment.

[0240] The treatment may, in some embodiments, include selective occlusion of one or both eyes. This may be done to present visually stimulating images to a targeted portion of the retina and increase the effectiveness of the treatment. In some embodiments, the light field processor system is configured to use selective occlusion to block a portion of an object seen by the wearer. Selective occlusion may also include intermittently occluding an image in the eye. This may be done to alternate images to the eyes (e.g., alternating between presenting an image to the left eye and then the right eye).

[0241] The treatment, in some embodiments, can include fine adjustments to the compensation prism correction to gradually affect the convergence of the eyes. For example, the amount of compensation prism correction and / or lateral image shift can be reduced during treatment to affect the weaker eye to converge to a target point. The amount of compensation prism correction and / or lateral image shift can be reduced over time during a single treatment or over multiple treatments.

[0242] At block 1312, the system detects the end of the time prescribed for the treatment protocol, possibly by detecting normal characteristics in ocular function for accommodation, convergence, divergence, etc. At such time, the light field processor system discontinues treatment at block 1314. This may include terminating occlusion or de-enhancement of image data provided to the wearer, if occlusion or de-enhancement was applied. In some embodiments, the wearer may manually administer the treatment protocol based on the wearer's schedule. Similarly, many other such treatment protocols may be envisioned.

[0243] Additionally or alternatively, the prescribed treatment protocol can be terminated or adjusted based on monitoring of the performance parameter. When the performance parameter reaches a desired value, state, or range, the treatment protocol can be terminated or adjusted. In some embodiments, the performance parameter is a biofeedback parameter. For example, in some embodiments, the light field processor system determines that the treatment should be interrupted by tracking the wearer's performance during the treatment in block 1312. When the wearer shows signs of fatigue or lack of compliance, the light field processor system interrupts the treatment in block 1314. For example, the light field processor system can include an eye tracking system configured to detect the wearer's gaze. The eye tracking system can detect that the wearer's performance (e.g., the wearer's ability to successfully focus on the image being presented) is deteriorating over time during the treatment. This can indicate that the wearer is fatigued and that further training or treatment will have limited benefit. In some embodiments, the light field processor system can track the wearer's performance over time and determine whether the wearer's convergence insufficiency is deteriorating over time (e.g., during a single treatment and / or across multiple treatment sessions). If the lazy eye is more swaying than drifting, the system may initiate stronger therapy, or if the lazy eye is nearly aligned with the stronger eye, the system may terminate treatment.

[0244] In some embodiments, the light field processor system receives input from a user, clinician, or external source indicating that treatment should be interrupted at block 1312. If such user input is received, the light field processor system interrupts treatment at block 1314.

[0245] In some embodiments, at block 1312, the light field processor system automatically detects the user's performance during the administered treatment protocol. At block 1312, the light field processor system can be configured to return to block 1304 and update or adjust the treatment protocol based on the detected performance of the wearer during the administered treatment protocol. For example, if the angle of convergence of the weaker eye does not improve during treatment, the light field processor system can adjust the treatment protocol parameters and proceed with a resumption of method 1300 at block 1304. In this manner, the light field processor system can use the wearer's performance during the test as feedback to adjust the treatment protocol and / or determine when to terminate treatment.

[0246] In various implementations, the light field processor system can be configured to terminate the treatment protocol prior to termination. For example, if the light field processor system detects eye strain (e.g., the angle of convergence worsens for the weaker eye), the light field processor system can be configured to terminate the treatment protocol at block 1312 by proceeding to block 1314.

[0247] Amblyopia or "lazy eye" is a medical condition in which one of the eyes is weaker than the other. This may result from a brain preference that biases input to the stronger eye over the weaker eye. In some embodiments, the light field processor system may be programmed to perform a method similar to method 1300 described with reference to FIG. 13 to enhance visual stimuli, thereby recreating the effect of an eye patch by gradually strengthening the weaker eye and / or selectively dimming the intensity of light or decreasing the level of visual stimuli incident on the stronger eye. Other treatment and training systems and techniques, such as those described above with reference to strabismus, may also be employed.

[0248] In various embodiments, to reduce distraction, the view of the world in front of the wearer's eye through the light field processor system is blacked out or otherwise made invisible during the examination and / or procedure.

[0249] 14 is a schematic diagram of an embodiment of a light field processor system that includes an outwardly facing integrated imaging camera 16, a light field processor 70, and an integrated imaging display 62 that also includes one or more photodetectors. The outwardly facing integrated imaging camera 16 and light field processor 70 shown in FIG. 14 can be similar to those described anywhere herein.

[0250] As discussed above with respect to FIG. 7, the integrated imaging display 62 includes a two-dimensional array of microlenses 712 and a two-dimensional array of light sources 714. The array of light sources 714 can be, for example, a red, green, blue (RGB) array of light emitting diodes (LEDs). Some embodiments may also include an infrared emitting light source for projecting infrared light into the wearer's eye for any of the reasons discussed herein. Alternatively, the array of light sources 714 can be implemented as a liquid crystal display (LCD) panel or some other type of display panel. Each light source can be used to emit light corresponding to a pixel or sub-pixel of the processed integrated image. The light emitted from each light source 714 is then projected by one of the lenslets 712 to a corresponding point 716 in space in front of the user's eye. In some embodiments, each lenslet 712 projects one processed base image. The overlap of light from each of the projected base images recreates a physical light field, which can be viewed by the user.

[0251] In FIG. 14, the integrated imaging display 62 additionally includes a two-dimensional array of photodetectors 715 (illustrated as shaded boxes interposed with the light sources 714). The photodetectors 715 can be sensitive to visible and / or infrared light. The photodetectors 715 can be used to collect light from the wearer's eye and digitize the collected light. In some embodiments, the photodetectors 715 images the wearer's eye, including any anatomical portions thereof. In some embodiments, the photodetectors 715 are interposed with and between the light sources 714. The photodetectors 715 and the light sources 714 may receive light from the same set of lenslets 712. Alternatively, the photodetectors 715 and the light sources 714 may be sufficiently physically separated (e.g., in separate clusters) that the photodetectors 715 use one or more lenses that are different from the light sources 714. One or more groups of photodetectors 715 with their associated lenses can form one or more cameras to capture light from the wearer's eye. In some embodiments, the one or more cameras are light field cameras to capture light field image data from the wearer's eye. The photodetectors 715 shown in FIG. 14 can be used to collect light from the wearer's eye for any of the purposes discussed herein. For example, in the autorefractometer and aberrometer embodiments discussed herein, the photodetectors 715 can be used to capture and digitize light reflected from the wearer's retina. These signals can then be processed by the light field processor 70 to determine information about the wearer's vision (e.g., its optical prescription and / or higher order aberrations). In some embodiments, the photodetectors 715 capture light field data. In other embodiments, the photodetectors 715 capture traditional 2D images.

[0252] 14 illustrates an embodiment in which the photodetectors 715 are integrated with the integrated imaging display 62, but this is not required. In other embodiments, the photodetectors 715 can be provided on the sides of the integrated imaging display 62 or anywhere they can be configured (e.g., using beam splitters, mirrors, prisms, etc.) to obtain a proper view of the wearer's eye.

[0253] The following paragraphs describe various exemplary embodiments of the devices, systems, and methods described herein.

[0254] 1. A wearable ophthalmic device comprising: a left outward-facing, head-mounted light field camera configured to receive light from a user's surroundings and generate numerical light field image data for a user's left eye; a right outward-facing, head-mounted light field camera configured to receive light from a user's surroundings and generate numerical light field image data for the user's right eye; a light field processor configured to generate modified numerical light field image data for one of the user's eyes by introducing a translational shift into the numerical light field image data captured by the light field cameras based on a convergence point of the user's gaze; and a head-mounted light field display configured to generate a physical light field corresponding to the modified numerical light field image data.

[0255] The device of any of the preceding paragraphs, wherein computationally introducing a translational shift includes computationally introducing a prismatic power.

[0256] The device of any of the preceding paragraphs, wherein the light field camera comprises an integrated imaging camera.

[0257] The device of any of the preceding paragraphs, wherein the light field display comprises an integral imaging display.

[0258] 1. A method of using a wearable ophthalmic device, the method comprising: receiving light from the user's surroundings and generating numerical light field image data for a user's left eye using a left outward-facing head-mounted light field camera; receiving light from the user's surroundings and generating numerical light field image data for the user's right eye using a right outward-facing head-mounted light field camera; generating modified numerical light field image data for one of the user's eyes using a light field processor by introducing a translational shift into the numerical light field image data captured by the light field camera based on a convergence point of the user's gaze; and generating a physical light field corresponding to the modified numerical light field image data using a head-mounted light field display.

[0259] The method of the preceding paragraph, wherein the step of computationally introducing a translational shift includes the step of computationally introducing a prismatic power.

[0260] The method of any of the preceding paragraphs, wherein the light field camera comprises an integrated imaging camera.

[0261] The method of any of the preceding paragraphs, wherein the light field display comprises an integral imaging display.

[0262] Correction of other ocular abnormalities The systems described herein (e.g., system 600 in FIGS. 6, 7, and 14) can also be used to detect, diagnose, assess, correct, and / or treat various other ocular abnormalities or distortions using any of the techniques described in U.S. patent application Ser. No. 15 / 072,290, entitled "METHODS AND SYSTEMS FOR DIAGNOSING AND TREATING HEALTH AILMENTS," filed March 16, 2016, the entire contents of which are incorporated herein by reference. For example, the systems described herein can be used to perform contrast tests, visual field tests, or color blindness tests, as described in U.S. patent application Ser. No. 15 / 072,290 and briefly summarized herein.

[0263] In some embodiments, a system described herein (e.g., system 600) may be configured to test the contrast sensitivity of a wearer. The contrast sensitivity test may be used to assess the wearer's ability to distinguish between different luminances in an image. The contrast sensitivity test may indicate the presence of a medical condition, such as age-related macular degeneration, amblyopia, and / or cataracts.

[0264] The system (e.g., system 600) may administer a contrast sensitivity test by projecting a static or variable image using the integrated imaging display 62. The system may then detect a response from the wearer. The image may be a high-contrast image whose contrast is gradually decreased or a low-contrast image whose contrast is gradually increased. For example, a dark gray image presented against a white background may be gradually lightened until the image is white or nearly white. The wearer may be instructed to indicate when the image can no longer be distinguished due to its similarity to the background color. The test may be repeated multiple times with the same or different images to obtain a more accurate estimate of the wearer's contrast sensitivity. For example, the image may change to a different number / letter / shape each time it is lightened, and the wearer may be asked to report the number / letter / shape of the image after each change. Color variation between images and / or glare tests may be incorporated as well.

[0265] In some embodiments, the system may use a static image to assess the wearer's contrast sensitivity. For example, the system may use an image such as a Pelli-Robson contrast sensitivity chart. A Pelli-Robson chart contains multiple rows of capital letters against a white background. The top left letters are printed in black, each successive row and / or letter is printed in a lighter shade of gray, and the letters in the bottom right row are printed in a shade closer to white. The system may project increasing or decreasing contrast of a Pelli-Robson chart or a similar sequence of letters, numbers, shapes, or other patterns. The wearer may be asked to read the sequence of letters or numbers, or describe the shape or pattern, and provide a response via any of the response methods described herein. The system may then determine the wearer's contrast sensitivity based on the lowest contrast at which the wearer can accurately detect the presence of a letter, number, shape, or other pattern.

[0266] In some embodiments using variable images, the wearer may be prompted, automatically or by another user, such as a medical professional, to indicate when an image appears or disappears and / or whether the wearer can distinguish between images of different intensities. In other embodiments using static images, the user may be prompted to indicate the observed content of the image, such as visible letters, numbers, shapes, or other patterns. The user may then input the wearer's response through a user interface.

[0267] Contrast sensitivity testing may be performed in discrete tests on demand, or may be performed periodically and / or repeatedly over time. Repeated analysis may allow tracking of gradually decreasing or increasing contrast sensitivity through historical analysis of previous results, as well as monitoring or detection of abnormalities. Thus, contrast sensitivity testing functionality may be incorporated into a device worn solely for ophthalmic diagnosis, or may be part of a device worn periodically, such as for entertainment, work, or other purposes, such that testing may be performed automatically at regular intervals and / or at various times of the day, week, month, year, etc. In some embodiments, the frequency of regularly scheduled testing may be automatically adjusted based on trends in contrast sensitivity test results. If the system detects that the wearer is suffering from reduced contrast sensitivity, the system may initiate further testing and / or contact a clinician. For example, the system may contact a clinician if it detects that the wearer has difficulty seeing in dark conditions or exhibits accommodation / vergence-divergence fluctuations associated with having difficulty focusing. In an augmented or virtual reality system, the system may provide alerts to the wearer using the outward facing integrated imaging camera 16. The alerts may be based on detection of a hazardous condition that is invisible to the wearer due to impairments in contrast sensitivity. The system may determine the presence of an invisible hazard based on correlation of images from the integrated imaging camera 16 with the wearer's known contrast sensitivity data, such as light conditions in which the wearer has reduced contrast sensitivity. When a hazard that is unlikely to be visible to the wearer, such as an incoming dark object or a hole in the ground during dark nighttime conditions, is detected, the system may alert the wearer. The alerts may include visual, audio, or tactile notifications.

[0268] In some embodiments, the systems described herein may be configured to detect, diagnose, and / or compensate for visual field defects. Visual field testing may be used to detect visual impairments in central and / or peripheral vision by analyzing a subject's ability to see stationary and / or moving objects and / or images at various locations in the subject's visual field. Visual field testing may indicate the presence of various pathologies, such as scotoma, trauma to the cornea, vitreous tears, trauma-induced cataracts, retinal hemorrhage, retinal detachment, macular degeneration, or intraocular hemorrhage (torsion syndrome).

[0269] A system (e.g., system 600) may administer a visual field test by determining the subject's ability to detect images at various locations in the visual field. The system may, for example, use the integrated imaging display 62 to project light into the wearer's eye to form an image within the eye. The system may then detect a response from the wearer. The image may, for example, be a small dot that is clearly visible when projected within a healthy portion of the visual field, but likely would not be visible if projected within an impaired portion. In some embodiments, the wearer may be prompted, automatically or by another user, such as a medical professional, to indicate whether the wearer saw the image and / or the time the wearer observed the image. The user may then input the wearer's response, for example, through any of the user interfaces discussed herein. In some embodiments, the system may require the user to match characteristics of the image (e.g., numbers, colors, letters, shapes, etc.) to ensure that the wearer saw the image.

[0270] In some embodiments, the system may be used to evaluate the wearer's visual field perception at the periphery of the visual field. For example, the system may provide a stationary fixation target near the optical axis. While the wearer's gaze is fixated on the fixation target, an image may be projected onto an outer portion of the display outside the wearer's visual field. The image may then be moved inward toward the fixation target until it enters the visual field. The wearer may be instructed to indicate when the target becomes visible, such as by any of the response methods described herein. In some embodiments, the wearer may be instructed to describe characteristics of the image, such as shape, number of apparent objects, or other features. The test may be repeated in various quadrants or locations at the periphery of the wearer's visual field, such as left, right, top, and / or bottom of the visual field.

[0271] Visual field testing may be performed in discrete tests on demand, or may be performed periodically and / or repeatedly over time. Repeated analysis may allow tracking of progression of visual field defects through historical analysis of previous results. Thus, visual field testing functionality may be incorporated into a device worn solely for ophthalmic diagnosis, or may be part of a device worn periodically for entertainment, work, or other purposes, etc., such that tests may be performed automatically at regular intervals and / or at various times of the day, week, month, year, etc. In some embodiments, the frequency of regularly scheduled tests may be automatically adjusted based on trends in visual field test results.

[0272] In an augmented or virtual reality system, the system may use the outward facing integrated imaging camera 16 to provide an alert to the wearer. The alert may be based on detection of a hazard condition that is invisible to the wearer due to visual field obstruction. The system may determine the presence of an invisible hazard based on correlation of images from the integrated imaging camera 16 with the wearer's known visual field data, such as quadrants in which the wearer has reduced peripheral vision. If a hazard in the obstruction quadrant, such as an incoming object, hole in the ground, or other condition, is detected, the system may alert the wearer. The alert may include a visual, audio, or tactile notification.

[0273] The systems described herein may also administer color tests to test the wearer's impairment in detecting specific colors. For example, the system (e.g., system 600) may administer the Ishihara color test, which is designed to test for red-green color perception deficiency. The test involves presenting a series of colored boards ("Ishihara boards"). The color boards contain circles of dots, which appear in random sizes and random or uniform colors. Within each circle are patterns of dots, which form numbers or shapes. In some circles, the numbers or shapes are clearly visible only to viewers with normal color vision, but are difficult or impossible to see to viewers with red-green color vision deficiency. In other circles, the numbers or shapes are only visible to viewers with red-green color deficiency. In some embodiments, the color boards may be selected and / or modified based on the wearer's known medical condition and / or previous responses. The colors or other stimuli may be gradually changed to determine the wearer's color vision deficiency boundaries. That is, the hue may change from a first color to a second color through multiple hues. For example, a wearer with a detected red-green color disorder may be presented with a board that gradually changes from red to orange or purple, and the wearer's response to each gradual change may be recorded.

[0274] In one or more embodiments, the system may be programmed to administer the stone primary color test by providing a virtual image of each of the color boards and receiving user input regarding the color boards in a manner similar to the process flow described above. The images of the color boards may be provided by an integral imaging display 62.

[0275] The system may automatically determine whether the user has a red-green color vision deficiency or other impairment based on input received from the wearer in response to the color test. User input for the color board may be produced and / or received by any suitable method for a user to input numbers, letters, shapes, or descriptions of other image characteristics. For example, input may be received through a user interface such as a keyboard, number pad, or a touch screen having keys or virtual buttons that correspond to the numbers utilized on the stone board. In some embodiments, the system may be configured to receive spoken input from the wearer and use voice recognition to determine the wearer's response to the test. The user interface may further have an option for the wearer to indicate that no numbers or shapes were observed.

[0276] In some embodiments, the system may be configured to determine whether the wearer sees the numbers or shapes in the projected stone slab without conscious input from the wearer. For example, the system may detect the wearer's gaze fixation at the location of the numbers or shapes in the stone slab (e.g., using an inward-facing camera) for a sufficiently long period that is an indication that the numbers or shapes are seen, while the wearer's scanning of the image for an extended period may indicate that the numbers or shapes are not seen. For example, the system may track the wearer's gaze for a long period of time, up to 1 second, 5 seconds, 10 seconds, or more.

[0277] In various embodiments, the system may use patterns and / or images other than stone slabs that test the wearer's ability to detect different colors. For example, the system may be configured to function as an RGB anomaloscope. A test based on color matching of two images and / or light sources is used to provide a color detection test. One source or image may have a fixed control color while the other is adjustable by the viewer (e.g., a fixed spectrum image whose brightness may be adjusted). The viewer may be presented with various control colors and attempt to match the adjustable image to the control image or determine that a match cannot be made.

[0278] In various embodiments, the system may be configured to administer color tests repeatedly and / or periodically. For example, the system may test the wearer's color vision periodically, such as several times per day, or once or more per week, month, or year, and compare the results over time. In some embodiments, the frequency of regularly scheduled tests may be automatically adjusted based on trending color blindness test results and / or based on detection that the wearer has difficulty distinguishing colors. In such cases, the system may be able to better detect the severity or time variation of the wearer's color detection impairment by testing the wearer's color vision at different times of day and in different light conditions. Similarly, the system may be able to obtain more complete and / or accurate results by repeatedly testing the wearer's color vision at different depth planes, degrees of accommodation, and / or areas of the retina. Thus, the system may be able to vary the depth planes, accommodation, and / or areas of the retina when administering color vision tests. Repeated testing over a longer period of time, such as months or years, may allow tracking of any improvement or deterioration of the wearer's color detection impairment, such as due to macular degeneration or any other progressive condition.

[0279] The system may also be configured for therapeutic functions such as compensating for the wearer's color detection impairments. The therapeutic functions may include computationally modifying the color, intensity, and / or other qualities of the light from the image and / or the world. For example, the system may function as a color boosting mechanism by computationally increasing the intensity of light in portions of the image that contain colors of reduced detection. In some embodiments, the system may computationally shift the color of such regions, such as by computationally changing the wavelength of light or adding light of a different wavelength, to present light of a color that is better detectable by the wearer.

[0280] In embodiments with an augmented reality device, the system may similarly modify the wearer's view of light from the surrounding world. The augmented reality system may detect the color of light incident on the device in real time or near real time and may computationally modify some of the light or project additional light to compensate for the wearer's color detection impairment. For example, the system may use an outwardly facing integrated imaging camera 16 to image the world around the wearer. The system may use an integrated imaging display 62 to project additional light of the same or different color to enhance intensity in areas of reduced detection ability, at least in part, to alleviate the wearer's color detection impairment. The system may further incorporate a labeling function, whereby the name of a known impairment color may be extended over an area of ​​ambient light determined to be that color. In some embodiments, superimposition may be used to enhance color in a portion of the display by projecting light of a desired amplitude, such as by adding a tint to the image.

[0281] eye position meter FIG. 15 illustrates how a wearable device described herein can be used to function as an ophthalmometer or refractometer and determine a suitable refraction that corrects or improves the vision of a wearer or patient. For example, the light field processor system described herein can be used to implement the ophthalmometer shown in FIG. 15. The results of the test from the ophthalmometer can be used, for example, to determine the optical prescription of the wearer or patient (e.g., for corrective lenses in glasses or contact lenses, or for use in a light field processor system). It should be understood that such a system may be used to administer an eye test, which may typically be administered in a doctor's or clinician's office. In one or more embodiments, the patient's individual light field processor system may be used, possibly with physician supervision, or the doctor's office may have its own version of the light field processor system that may be used for diagnostic purposes. Although FIG. 15 is discussed in the context of augmented reality, similar features may likewise be included in virtual reality devices, such as virtual reality eyewear.

[0282] Conventional ophthalmometers are used by optometric medical practitioners for eye examinations to determine the patient's refractive error and therefore correct the refraction and compensate for any ocular abnormalities. Using this information, the optometric medical practitioner can determine the optical prescription for the patient's eye and improve or correct the patient's vision. Conventional ophthalmometers typically involve presenting an alphabetical eye test chart, usually of variable size, with different lenses that can be tested, and testing the patient's vision. The patient looks at the eye test chart and lenses of different refractive powers are placed in front of the patient's eye to determine whether the patient's vision has improved. Conventional setups tend to be bulky and require the doctor to individually select the next step size in the lens. The clinician typically asks for the patient's feedback regarding image clarity and changes the lens accordingly.

[0283] In contrast, some embodiments of the wearable light field processor system 1500 may be used to perform these same functions without the bulky eye positioning setup. The wearable light field processor system 1500 includes an augmented (or virtual) reality display platform 1502 configured to project images to the wearer's eyes. In some embodiments, the augmented or virtual reality display platform 1502 can be any of the light field processor systems 600 described herein. The display platform 1502 can be configured similarly to the display lens 106, for example, as described herein with reference to FIG. 5. In some implementations, such as for a light field processor system, the display platform 1502 can also be configured to pass light from the world or environment through the display platform 1502 (e.g., through a lens in front of it and / or via an integrated imaging camera 16) to the wearer's eyes. In this way, the wearer sees images projected using the display platform 1502 superimposed on what the wearer sees in the world. In some embodiments, rather than a physical eye test chart, a virtual eye test chart 1520 may be projected to the wearer using the display platform 1502 (e.g., via the integrated imaging display 62). To provide functionality similar to an ophthalmometer, the focus of the image may be computationally varied by the light field processor 70 (as discussed herein). The wearable light field processor system 1500 may be configured to administer or provide an eye test by automatically providing incremental changes in the optical prescription by varying the focus of the image. This may be done by computationally varying the refractive power applied to the image data by the light field processor 70 in the display platform 1502.

[0284] As described herein, the light field processor system 1500 can include a light field processor 70 configured to computationally modify one or more optical properties of image data provided to it. For example, the light field processor 70 can add or subtract refractive power to the image data, for example, by modifying the curvature of a wavefront in the light field image data, or otherwise introducing positive or negative spherical / cylindrical refractive power. The light field processor 70 can thus provide variable optical corrections or higher order aberration corrections, such as spherical, cylindrical, and axial.

[0285] In one implementation, the display platform 1502 includes a display 62, which may also include the ability to not only project light into the wearer's eye, but also capture light therefrom (e.g., as discussed herein with respect to FIG. 14). The received light can be imaged or otherwise detected. This may be utilized in eye testing, where the display 62 is configured to image the wearer's retina and monitor the image formed on the retina. To determine the accommodation state of the eye, reflections from the retina (e.g., reflections from images projected into the wearer's eye) can be measured using the display 62. This can result in a rapid measurement of the accommodation state of the eye. For example, a sequence of point source images (e.g., outside the visible wavelength band in the infrared band) can be projected into the wearer's eye from various apparent depths by the display 62, which can simultaneously measure reflections from the retina. The light field processor system 1500 can be configured to determine the accommodation state of the eye in real time by determining the depth plane corresponding to the brightest, least, or best-focused image based on the most collimated reflection. (The light field processor 70 can be used to provide image data from different apparent depth planes.) The light field processor system 1500 can be configured to project a beam of light from an optical source (e.g., the display 62) into the wearer's eye. Portions of the projected beam can be reflected, scattered, and / or diffracted by various anatomical features of the wearer's eye and received by one or more imaging devices. An electronic hardware processor (e.g., the light field processor 70) can be used to analyze the light received from the wearer's eye and examine various structures of the wearer's eye. This can result in a determination or approximation of the accommodation state of the eye, which can include lens shape, pupil state, convergence-divergence movement, dynamic accommodation, etc. In some embodiments, computational refraction correction can be provided by the augmented or virtual reality device 1500 using the light field processor 70.

[0286] In some implementations, to perform the eye test, the user / wearer may be presented with various images of variable size, shape, color, brightness, contrast, etc., and the user / wearer may provide input regarding image clarity through the user interface 1504 of the light field processor system. In some embodiments, the light field processor system is configured to automatically determine image clarity based, at least in part, on detecting whether the image is focused on the wearer's retina. This can be done by imaging the user's retina and measuring, analyzing, and / or observing the reflection of the projected image from the retina. If the user / wearer indicates that a particular image is not clear or does not look comfortable, rather than physically changing the lenses as in traditional ophthalmometers, the focus of the image may be automatically computationally varied to provide a gradual change in the corresponding or equivalent optical prescription. Thus, the eye test may be seamlessly performed through the light field processor system, locally or remotely. For example, a clinician or physician may administer the eye test remotely, such as, by way of example and not limitation, via telephone, using video conferencing, via a networked communication program, or the like. It should also be understood that in some implementations the eye examination may be performed with or without direct interaction from a clinician, or with little or minimal effort and time from the clinician.

[0287] In some embodiments, adjustments to the optical prescription may be performed automatically by the light field processor system based on physical changes in the eye during accommodation and / or convergence attempts. For example, the light field processor system may be programmed to detect certain patterns of eye behavior that are symptomatic of a weakened eye. Based at least in part on the tracked eye behavior, the eye accommodation may be performed automatically by the light field processor system. In response to detecting that the wearer is struggling to accommodate, for example, the system may initiate an ophthalmometry test as described herein, or the system may alert the wearer or a clinician that the wearer is struggling to accommodate. In some embodiments, the system detects that the wearer is struggling to accommodate by detecting minute variations in accommodation (e.g., small and / or fast changes in lens shape, convergence, pupil size, etc.). In some implementations, struggling to accommodate can be detected by monitoring the focus state of an image projected onto the wearer's retina, as described herein. If the reflection from the retina is fluctuating, the system can be configured to determine that the wearer is struggling to accommodate. The system may present an image to the wearer, test different optical corrections, and ask the user to provide feedback on whether the optical corrections improve the image. In some implementations, rather than or in addition to asking the wearer to provide feedback, the system may be configured to determine whether the image is in focus by observing, measuring, and / or analyzing the image on the wearer's retina. As previously described, computational optical correction may be used to implement different test corrections during testing. In some embodiments, the system may be used to determine the wearer's heterophoria by presenting images to the wearer's eyes, one eye at a time. In some implementations, the system may be used to monitor the wearer's convergence-divergence movements on a target.If the wearer is in exotropia when attempting to focus on a near image (e.g., an image projected from a near depth plane), the system can be configured to determine that the wearer may be presbyopic or fatigued and / or whether the wearer may have strabismus or amblyopia. The system can also be configured to administer various tests of the visual field by providing images from a variety of different depth planes.

[0288] The images provided by the light field processor system can be stored images. The wearable light field processor system 1500 can include a data storage device that includes one or more stored images suitable for performing an eye test or for determining an optical prescription for the wearer. The stored images can be letters, numbers, symbols, etc., used in eye charts, etc. The images can be presented to the viewer at a desired depth plane, such as at infinity or otherwise at a large apparent distance, e.g., at least 20, 40, 60, 80, or 100 feet away. As described herein, the stored images can be processed to produce a corrected wavefront for projection to the wearer. The corrected wavefront can be configured to take into account the optical prescription or anomalies of the wearer's eye. In some embodiments, a light field processor 70 in the display platform is used to computationally provide adjustments to the image seen by the wearer to take into account the optical prescription. The light field processor 70 can also, for example, modify the intensity patterns that make up the image, to compensate for distortion or fisheye, e.g., where straight lines appear curved. For example, to compensate for pincushion distortion, a degree of barrel distortion may be introduced into the intensity pattern that constitutes the image. Similarly, to compensate for barrel distortion, a degree of pincushion distortion may be introduced into the intensity pattern that constitutes the image. Other types of modifications to the intensity pattern that constitutes the image may also be introduced by software used to drive spatial light modulators or light sources to produce the desired intensity pattern. In some embodiments, the wearable augmented or virtual reality device 1500 can be configured to project images of variable size to the wearer or images from variable depth planes to the wearer using the display platform 1502. In some implementations, the images can include letters or shapes of variable size and / or be projected from variable depth planes. In various implementations, the size and / or depth plane of the letters and / or shapes projected to the wearer can be varied during the eye test.In some embodiments, the system can be configured to administer a brightness or glare test that includes an objective measurement of functional visual acuity under different brightness and glare conditions. In various embodiments, the system can be configured to administer a brightness visual acuity test to determine functional visual acuity under various brightness conditions. For example, the system can be configured to simulate three or more brightness conditions: 1) high-direct sunlight from above, 2) medium-partially cloudy day, and 3) low-bright commercial electric light from above. Visual acuity measurements can be similar to those that would be measured under these three conditions using a standard eye chart (e.g., eye chart 1520). The results of such a test can be an assessment of functional visual acuity. Such tests can be used to test sensitivity to bright light, photophobia, impaired scotopic vision, and the like. In some embodiments, the system can be configured to test individual colors. For example, the light field processor system can be configured to determine refractive error for individual colors (e.g., red, green, blue, yellow, etc.). In some embodiments, the system can be configured to test various depth planes. For example, the light field processor system can be configured to determine a refractive error for each depth plane. This can result in an optical prescription that varies, at least in part, based on the depth plane. Refractive corrections for presbyopia may also be determined.

[0289] In some embodiments, the wearable augmented (or virtual reality) device 1500 can include one or more outwardly facing cameras. In some implementations, the one or more outwardly facing cameras can be similar to the camera 16 described herein with reference to FIG. 5 or the integrated imaging camera 16 described herein with reference to FIGS. 6-7. The outwardly facing camera in the augmented reality display device can be configured to capture an image of the surrounding environment and determine where to superimpose a test image, such as, for example, a letter or symbol. For example, the light field processor system can superimpose an image of an eye chart, such as a standard Snellen chart or other visual acuity chart, over an area in the wearer's field of view that corresponds to the wall of an optometrist's office. In another example, the outwardly facing camera can be configured to capture an image of an eye chart, such as a standard Snellen chart or other visual acuity chart, that is actually on the wall of an optometrist's office. The wearable augmented or virtual reality device 1500 can then be configured to modify the captured image based, at least in part, on the desired depth plane to present the image. For example, the acquired image can be projected by the display platform 1502 to accommodate infinity. Then, through the user interface 1504, the light from the image can be computationally manipulated by the light field processor 70 to provide functionality similar to physically changing the lens with a conventional ophthalmometer. For example, spherical, cylindrical, or higher order aberration correction can be introduced. If cylinder is to be added, the appropriate axis can also be determined. In this manner, an eye test can be performed to determine the wearer's optical prescription. In some embodiments, the system is configured to objectively measure or estimate the optical prescription through observation, measurement, and / or analysis of the manipulated image as described anywhere herein, and determine whether it is focused on the wearer's retina.

[0290] The wearable light field processor system 1500 can include one or more user interface features 1504 configured to allow the wearer or other person to provide input to the device. The user interface features 1504 can be integrated with the device 1500 as illustrated in FIG. 15. In some implementations, the user interface features 1504 are provided by a device or component that is not physically integrated with the device 1500. For example, the user interface features 1504 can be provided by a device or system that communicates with the device 1500. This can be a smartphone, computer, tablet, or other computing device that communicates with the device 1500 in wired or wireless communication. In some embodiments, the user interface features 1504 can be provided by a combination of different devices and systems linked to the device, for example, through a wired or wireless communication network, or through a component that is physically linked to or integrated with the device. A touch screen, a voice recognition system, or a virtual touch screen are some examples of interfaces. Thus, the user interface features 1504 can include touch-sensitive capacitive features, keyboards, buttons, microphones, photodetectors, cameras, and / or various software-implemented features provided by a graphical user interface. The user interface features 1504 can be presented on a device with a touch screen, where interaction with the touch screen provides input to the wearable augmented or virtual reality device 1500. In various embodiments, a virtual touch screen is provided through an image projected to the user's eyes, and sensors sense the user moving their body, e.g., fingers. In some embodiments, the user interface features 1504 include a gesture detection component, allowing the wearer to provide user input through gestures.In some embodiments, the user interface features 1504 include an eye gaze detection component, allowing the wearer to provide user input through eye gaze (e.g., this can include selecting a button or other element when the wearer fixates the button for a period of time, or when the wearer blinks while fixated on the button). Such user interface systems can be employed for other devices and systems described herein.

[0291] In the light field processor system, the user interface features 1504 can be used by the wearer to provide feedback regarding the quality of the image as perceived by the wearer. The wearer can provide feedback through the user interface features 1504, for example, regarding whether the wearer can comfortably view the image being projected to the user as changes to the applied optical power (e.g., gradual values ​​of sphere, cylinder, and axis, and / or high-order aberration correction) are gradually provided. In this way, a suitable optical prescription for the wearer can be determined.

[0292] In some implementations, the clinician or physician can also use the interface feature 1504 to vary the focus and / or depth plane at which the image is projected onto the wearer or the size of the image being projected. Such variations can be used incrementally, as desired.

[0293] 16 illustrates an example method 1600 for determining an optical prescription of a wearer of a light field processor system configured for use as a virtual oculometer. For ease of explanation, the method 1600 will be described as being performed by a light field processor system such as the augmented (or virtual) device 1500 described herein with reference to FIG. 15. However, it should be understood that components or sub-portions of various augmented reality (or virtual) devices or other similar devices disclosed herein may be used to perform any step, combination of steps, or portions of steps in the method 1600.

[0294] In block 1602, the ophthalmic device initiates an eye test program. The eye test program can be a stored process or sequence of functions provided by the ophthalmic device. Initiating the eye test program can include determining or retrieving a starting optical prescription of a wearer who has previously undergone an eye test or other eye examination, etc. In some implementations, the eye test program can integrate information about ocular abnormalities of the wearer's eye, which can be typed in by the wearer or clinician, determined from a previous eye test program, or retrieved from a data store (e.g., a data store that is part of the light field processor system or a networked data store). Initiating the eye test program can include determining an image or sequence of potential images to be projected to the wearer. Initiating the eye test program can include determining whether a clinician or doctor will administer an eye test or whether the test will be self-administered by the wearer. In some embodiments, the ophthalmic device initiates the eye test program in response to input received from the wearer or clinician.

[0295] In block 1604, the light field processor system projects an image to the wearer's eye. For example, the light field processor system can project alphabets, letters, and / or shapes of target size to the wearer. The image can be a stored image, or the image can be obtained by the light field processor system. The image can include elements configured to assist in determining the wearer's visual acuity, the visual acuity elements comprising, for example, but not limited to, icons, symbols, letters, shapes, or the like. The visual acuity elements of the image can have different sizes within the image, and / or the size of the visual acuity elements can be varied by the light field processor system.

[0296] At block 1606, the light field processor system receives user input regarding the image. The user input may indicate whether the wearer can clearly see the image. In one or more embodiments, the light field processor system may begin by projecting relatively small letters that increase in size until the received user input indicates that the wearer can clearly see the projected image. In some embodiments, the light field processor system is configured to present an eye chart, such as a traditional eye chart like a Snellen chart. In such embodiments, the received user input may include a portion or portions of the projected chart that are clearly visible to the wearer.

[0297] At block 1608, the light field processor system determines whether the user can comfortably view the image (e.g., the projected eye chart). In some embodiments, the system is configured to receive user input regarding whether the user can comfortably view the image through a user interface. As described above, examples of such user interfaces may include a voice recognition system, a touch screen, or a virtual touch system.

[0298] In some embodiments, the user input received in block 1606 is automatically determined through analysis of the wearer's physical and / or optical characteristics. For example, automatically determined user input includes analysis of whether an image is in focus by observing, measuring, and / or analyzing the wearer's retina. As described herein, by measuring reflections from the retina, the light field processor system can be configured to assess the quality or degree of focus of an image formed by the wearer's eye. In some embodiments, the light field processor system can be configured to project a pair of spots into the wearer's eye. Reflections of these projected spots can be measured and analyzed to determine the quality of focus of the image. For example, if the images of the projected spots on the retina are aligned, the light field processor system can determine that the wearer is focusing on the projected image or that the wearer is properly accommodating at the target location.

[0299] In some embodiments, the light field processor system is configured to determine that the wearer can comfortably view the image based at least in part on detection of relaxed accommodation and / or convergence. As described herein, the light field processor system can include an eye detection and / or tracking component configured to monitor the eye. Such a component can be capable of detecting the wearer's accommodation, convergence, and / or pupil size. Lens accommodation may be detected, for example, by an autorefractometer (as described in more detail elsewhere herein) that measures the size of an image on the retina. Convergence and pupil size can be measured using one or more inward-facing cameras. In some embodiments, the light field processor system monitors eye fluctuations as the user attempts to focus on a target object or image. For example, as the eye focuses on a stationary stimulus, the refractive power of the eye's lens changes rapidly and persistently. These fluctuations may increase when a person struggles to focus on a stationary object. This increase in variation can be measured and / or observed by the light field processor system to determine that the wearer is not focused on the target image or object. In some embodiments, the light field processor system can be configured to monitor these variations and move the projected image (e.g., change the apparent depth plane being projected) until the wearer successfully focuses on the object. For example, the light field processor system can project the image from an apparent depth plane relatively close to the wearer and move the image back (e.g., increase the distance between the wearer and the depth plane) until the light field processor system determines that the wearer is correctly focused on the image.

[0300] Monitoring the eye's accommodation reflex can be employed by the light field processor system to determine whether the wearer is comfortable with the current prescription or whether the wearer needs optical correction. An autorefractometer can be used to determine whether the wearer is accommodating by monitoring the size of the image projected through the lens onto the retina, an example of which is described elsewhere herein. Such a method of monitoring accommodation can be used to determine whether the wearer is accommodating, which can be useful in assessing whether the wearer needs optical correction.

[0301] Vergence movements can be employed to help determine whether optical correction is required as well when vergence movements are monitored to test the accommodation reflex. Using an inward-facing camera and processing electronics, the light field processor system can be configured to track the individual gaze changes of the left and right eyes and determine vergence movements. Such vergence movement information can be used to determine whether the wearer's eyes respond as expected to images presented at various depth planes. For example, if the eyes are approximately parallel and not verging when images at relatively close depth planes are presented, the light field processor system can be configured to interpret this result as an indication that the wearer is not viewing the images comfortably or that the wearer has a vision disorder. Vergence movements for different depth planes can be determined whether the eyes that match the appropriate vergence movements for the particular depth plane can be assessed. Similarly, potentially, if vergence movements are observed to be inward for a depth plane at infinity, the wearer may require optical correction.

[0302] As another example, the light field processor system can test the accommodation reflex by determining the size of the wearer's pupil or the change in the size of the wearer's pupil when an image is projected on the wearer. The light field processor system can be configured to track the change in pupil size using an inward-facing camera that images the eye, specifically the iris. This information can be used to determine whether the wearer's eye responds as expected to images presented at different depth planes. For example, the size of the pupil is expected to constrict when viewing closer objects (compared to more distant objects). Thus, the light field processor system can be configured to present images from a near depth plane and track the response of the wearer's pupil. If the pupil does not constrict, or does not constrict sufficiently, the light field processor system can be configured to interpret this result as an indication that the wearer is not viewing the image comfortably.

[0303] Thus, to determine the wearer's comfort in viewing an image, the light field processor system can determine the wearer's accommodation, convergence, and / or pupil size as part of a test of the accommodation reflex when a particular image is projected to the user. Similarly, the wearer's comfort can be determined when images are projected through various depth planes. In some embodiments, the light field processor system can compare the measured accommodation, convergence, and / or pupil size with the expected accommodation, convergence, and / or pupil size. If one or more of the measured characteristics are within a target range of one or more expected characteristics, the light field processor system can determine that the wearer is viewing the image comfortably or correctly (e.g., the wearer is viewing the image with expected, proper, or normal vision). If one or more of the measured characteristics are outside of a target range of one or more expected characteristics, the light field processor system may determine that the wearer is not viewing the image comfortably or correctly (e.g., the wearer is viewing the image with impaired vision). In some embodiments, the light field processor system combines information about the measured characteristics with information received or determined from the user input in block 1606 to determine whether the wearer is viewing the projected image comfortably. For example, when viewing an image from a relatively close apparent depth plane, the wearer's eyes are expected to converge or move towards each other, the pupils are expected to contract, and the convexity of the lens is expected to increase. Deviations from one or more of these expectations can be interpreted as an indication that the user is not viewing the projected image comfortably or correctly (e.g., the wearer is viewing the image with impaired vision).

[0304] If the light field processor system determines that the wearer cannot view the image comfortably, for example, by either receiving input from the user through a user interface or by assessing the user's accommodation and / or convergence-divergence movements, the light field processor system proceeds to block 1610 and changes the focus and incrementally increases the prescription (e.g., adding or subtracting more positive or negative sphere to obtain a more positive or negative spherical wavefront). The system may also test for astigmatism and therefore incrementally change the axis and cylinder. The light field processor system then returns to block 1606 to receive or determine user input and again determine whether the user can view the image comfortably or uncomfortable (e.g., with normal vision). This loop can be repeated until the image appears comfortable to the user.

[0305] In some embodiments, the light field processor system is configured to adjust the optical correction at block 1610 based at least in part on the feedback received from the user input at block 1606 and / or the objective assessment determined at block 1608 or as described anywhere herein. In some embodiments, the light field processor system is configured to adjust the optical correction at block 1610 based at least in part on measurements of the wearer's accommodation, convergence-divergence movement, accommodation reflex, and / or pupil size when viewing the projected image. Thus, in an implementation, the light field processor system can be configured to perform an eye examination and determine an optical prescription for the wearer using subjective and objective tests.

[0306] In some embodiments, the subjective element of the eye test can include projecting an image and then projecting the image with a diopter change (e.g., ±0.01 D, ±0.1 D, ±0.125 D, ±0.25 D, ±0.5 D, ±1.0 D, or a substantially continuous change in diopter power) through a computer light field processor 70 or the like, and receiving user input as to whether the image quality has changed. In one implementation, the light field processor system can also be configured to determine changes in the wearer's eye (e.g., accommodation, convergence and divergence, pupil size, etc.) as the diopter value is changed. This objective test data can be combined with the wearer's subjective response to determine whether the change in diopter value results in a change in visual quality for the wearer.

[0307] If the light field processor system determines that the wearer can comfortably view the image, the light field processor system proceeds to block 1612 and determines the prescription for the wearer's eyes. It should be understood that in some implementations, the same process may be repeated for both eyes (e.g., both eyes can be treated to apply the same correction to each eye together, or to apply different corrections to the left and right eyes individually). In some embodiments, this can be used to treat anisometropia, where two different optical prescriptions are applied to the wearer's two separate eyes. In some embodiments, the light field processor system can be configured to dynamically switch between optical prescriptions depending on what is being viewed by the wearer and / or the activity being performed by the wearer. For example, lenses for anisometropia may fatigue the wearer when he or she views primarily near or primarily far images compared to when he or she views images that are a mixture of near, medium, and far distances. Thus, the light field processor system can be configured to dynamically vary the applied optical prescription in real time based, at least in part, on the wearer's known optical prescription for treating anisometropia, myopia, or hyperopia.

[0308] Method 1600 can be used to provide information to a light field processor system, such as device 1500 described herein with reference to FIG. 15 or other similar devices described herein. Thus, the light field processor system can be configured to vary the focus or other aspects of the image being projected based on the wearer's optical prescription, as described herein. In some embodiments, the same wearable light field processor system that the wearer uses for entertainment, work, or other purposes can be used to perform the eye tests described herein.

[0309] Method 1600 can be used to determine optical prescriptions, optical corrections, or refractive corrections for different depths. For example, there can be a first optical prescription for a far apparent depth plane, a second optical prescription for a mid-depth (intermediate) plane, and a third optical prescription for a near apparent depth plane, or far and near, far and mid, near and mid.

[0310] The method 1600 and system can be used to correct or improve the vision of a wearer who suffers from presbyopia. Different optical corrections can be applied to different depth planes and associated content projected from those depth planes. Or, the wearable light field processor system can be configured to switch between providing prescriptions for far distances (if applicable) and near distances based on a sensed orientation of the user's head or eyes. As described herein, an orientation sensor or other sensor can be used to determine the orientation of the user's head or eyes.

[0311] The method 1600 may be performed in real time to automatically determine the wearer's optical prescription. This information can be stored within the light field processor system and used for future testing. For example, the light field processor system can be configured to update the wearer's current optical prescription for the wearer based on the test. For example, the light field processor system can be configured to monitor the eye and record the wearer's eye behavior over time. Based at least in part on this information, the light field processor system can dynamically adjust the wearer's optical prescription over time. For example, the light field processor system can measure the eye's behavior when presented with an image at a known depth. The light field processor system can determine deviations from expected eye responses to the images and determine whether the eye behaves as expected. The light field processor system can be configured to initiate a test and / or update the wearer's optical prescription, or initiate or schedule an updated eye test if the light field processor system determines that the determined deviations are outside of range (e.g., target, i.e., acceptable range of expected behavior).

[0312] In some embodiments, the light field processor system can be configured to prominently determine the optical prescription. For example, this may occur when the light field processor system does not provide alternative functionality to the wearer while the eye test is being performed. In other words, this may occur when the wearer is required to focus solely on the eye test.

[0313] In some embodiments, the light field processor system can be configured to unobtrusively determine the optical prescription. For example, this may occur when the light field processor system is configured to obtain measurements of the wearer's eye behavior while the wearer is engaged in other activities (e.g., watching a movie, reading a text, viewing an image, etc.). The light field processor system can be configured to measure characteristics of the wearer's eye while the wearer is engaged in these other activities, compare the measured characteristics of the wearer's eye, and determine deviations from expected characteristics of the wearer's eye. In some embodiments, the system can be configured to determine the optical prescription based, at least in part, on these determined deviations. In some embodiments, the expected characteristics of the wearer's eye can be based, at least in part, on the apparent depth plane and image properties being projected to the wearer. In some embodiments, if such deviations are detected, the system may ask the user to undergo an examination or to confirm that the test optical correction applied by the system is sufficient or insufficient. In some implementations, the light field processor system can be configured to track physical changes in the wearer's eye while attempting accommodation and convergence movements while engaging in these other activities. In some embodiments, this information can be compared to measurements obtained during attempted accommodation and convergence movements to determine the optical prescription.

[0314] In some embodiments, the light field processor system can be configured to objectively measure the wearer's optical prescription. In various implementations, this may be accomplished without receiving feedback from the wearer regarding image quality. In some implementations, this may be accomplished without projecting a variable-sized image to the user. For example, the light field processor system can be configured to project an image from a virtual infinite depth (e.g., the light field processor system places the image at infinity). The light field processor system then measures the wearer's accommodation reflex, accommodation, convergence-divergence, and / or pupil size. Based at least in part on the wearer's accommodation, convergence-divergence, and / or pupil size and deviations of the wearer's accommodation, convergence-divergence, and / or pupil size from what is expected, the light field processor system can objectively determine the wearer's optical prescription. For example, if the wearer's eyes are accommodating +1D when the image is placed at infinity, the light field processor system can objectively determine the optical prescription.

[0315] In some embodiments, the light field processor system can be calibrated to determine the appropriate diopter correction and / or to take into account the appropriate diopter correction based, at least in part, on the configuration of the display platform. For example, when adjusting the depth plane of the image being projected to the wearer, the light field processor system can be configured to be properly calibrated and correlate the change in the depth plane with the change in diopter or refractive power. In some embodiments, during calibration, the iris of the wearer is analyzed. The iris can be used to uniquely identify the patient, and this unique identification can be used to access the associated patient record and correlate with the person and their medical records / prescriptions, etc.

[0316] In various embodiments, to reduce distraction, the view of the world in front of the wearer's eyes through the light field processor system is occluded or otherwise invisible during testing. This may occur, for example, when the image is presented to the viewer, but this approach is not required.

[0317] The light field processor system may be a system provided by a doctor or clinician for testing in a medical facility or optometrist's office or other location. In other embodiments, the system may belong to a user and may be employed for other purposes, such as entertainment (e.g., games and movies) and / or professional tasks. As explained above, one advantage of implementing the test on the user's system is that the test may be conveniently taken multiple times (at least 2, 3, 4, 5, 6, 8, 10, 12, 16, 18, 24, or more times) throughout the year. In some embodiments, the frequency and / or schedule of the test may be based, at least in part, on the wearer's rate of decline in vision. For example, if the rate of decline increases, the frequency of the test may be increased. Similarly, the test may be performed with or without a medical professional, such as an optometrist, ophthalmologist, nurse, technician, medical assistant, etc.

[0318] The following paragraphs describe various exemplary embodiments of the devices, systems, and methods described herein.

[0319] A wearable ophthalmic device comprising: a light field processor configured to access numerical light field image data and generate modified numerical light field image data by computationally introducing graduated amounts of positive or negative refractive power into the numerical light field image data, thereby performing an eye examination; and a head-mounted light field display configured to generate a physical light field corresponding to the modified numerical light field image data.

[0320] The device of the preceding paragraph, wherein the light field display comprises an integrated imaging display with a two-dimensional array of microlenses and a corresponding two-dimensional array of photodetectors.

[0321] The device of any of the preceding paragraphs, wherein the light field image data comprises characters of variable size.

[0322] The device of any of the preceding paragraphs, further comprising a user interface configured to receive input from a user regarding distinctness of the physical light field.

[0323] The device of the preceding paragraph, wherein the light field processor is configured to gradually vary a refractive power of the light field image data based on user input.

[0324] The device of any of the preceding paragraphs, wherein the light field processor is configured to gradually vary the refractive power of the light field image data based on observation of the user's eye.

[0325] A device described in any of the preceding paragraphs, wherein the light field display comprises one or more infrared light sources configured to project infrared light into the user's eye and measure its accommodation state, and wherein graduated amounts of positive or negative refractive power are determined based on the accommodation state.

[0326] The device described in the preceding paragraph, wherein the one or more infrared light sources are configured to project two or more infrared beams of different convergence and divergence movements into the user's eye to measure its accommodation state.

[0327] The device of any of the preceding paragraphs, further comprising an outwardly facing, head-mounted light field camera configured to receive light from around the user and generate additional numerical light field image data, and the light field processor configured to computationally modify the additional numerical light field image data based on results of the eye test.

[0328] A method of using a wearable ophthalmic device, the method including the steps of: using a light field processor to access numerical light field image data, generating modified numerical light field image data, and computationally introducing graduated amounts of positive or negative refractive power into the numerical light field image data, thereby performing an eye examination; and using a head-mounted light field display to generate a physical light field corresponding to the modified numerical light field image data.

[0329] The method of any of the preceding paragraphs, wherein the light field display comprises an integrated imaging display with a two-dimensional array of microlenses and a corresponding two-dimensional array of photodetectors.

[0330] The method of any of the preceding paragraphs, wherein the light field image data comprises characters of variable size.

[0331] The method of any of the preceding paragraphs, further comprising receiving input from a user regarding the distinctness of the physical light field using a user interface.

[0332] The method of the preceding paragraph, further comprising gradually varying a refractive power of the light field image data based on user input.

[0333] The method of any of the preceding paragraphs, further comprising gradually varying the refractive power of the light field image data based on an observation of the user's eye.

[0334] The method of any of the preceding paragraphs, further comprising using one or more infrared light sources in the light field display to project infrared light into the user's eye and measure its accommodation state, and determining incremental amounts of positive or negative refractive power based on the accommodation state.

[0335] The method of the preceding paragraph, further comprising using one or more infrared light sources to project two or more infrared beams of different convergence and divergence movements into the user's eye to measure its accommodation state.

[0336] The method of any of the preceding paragraphs, further comprising the steps of using an outward-facing, head-mounted light field camera to receive light from around the user and generate additional numerical light field image data, and computationally modifying the additional numerical light field image data based on results of the eye test.

[0337] Screening method The wearable light field processor system described herein can be used as a retinoscope to determine the vision defect of the wearer or patient. In particular, when operating as a retinoscope, the light field processor system can be used to detect myopia, hyperopia, astigmatism, and / or other vision defects. The light field processor system can be configured to determine the refractive error of the patient's eye, for example, by using retinoscopy techniques such as neutralization. Neutralization involves adjusting the refractive power of the front of the eye until a light beam or spot swept across the eye forms an image on the retina and substantially stops moving across the retina. The light field processor system may be configured to provide beams with different optical corrections until neutralization is achieved. The light field processor system can thus be configured to determine an optical prescription to correct the identified vision defect. It is to be understood that such a device may be used to administer an eye test, which may typically be administered in a doctor's or clinician's office or automatically at home by the wearer. In one or more embodiments, a patient's individual light field processor system may be used, possibly with physician supervision, or a physician's office may have its own version of a light field processor system that may be used for diagnostic purposes. In various embodiments, the light field processor system may be configured similarly to the devices disclosed herein.

[0338] In some embodiments, the wearable light field processor system may be used to perform retinoscopy and identify vision defects using light swept across the wearer's eye. Examples of devices that may be configured to perform retinoscopy are described herein, including, for example, but not limited to, the devices described herein with reference to Figs. 5, 6, 7, 14, and 15. The wearable light field processor system may be configured to project a beam of light into the wearer's eye. In some implementations, the light field processor system may be configured to display image data from the world or environment to the wearer's eye. In this way, the wearer sees objects in the world in front of the wearer and can potentially fixate on distant or near objects, for example, depending on the type of retinoscopy test. The focus of the beam projected into the eye may be varied or otherwise provided with optical corrections. Thus, the wearable light field processor system may be configured to perform retinoscopy and measure refractive errors of the wearer's eye.

[0339] In some embodiments, the wearable light field processor system includes at least one light source (e.g., 714 in FIG. 14) configured to project light into the wearer's eye. The light source used for retinoscopy can be configured to provide a light beam that is swept across the wearer's eye. In some embodiments, the wearable light field processor system also includes one or more sensors (e.g., 715 in FIG. 14) configured to measure light reflected from the retina in response to the light beam from the at least one light source being swept across the wearer's eye. In various embodiments, the sensor images the eye. The sensor may comprise an eye tracking sensor or other inwardly facing optical sensor or camera mounted on the eye and configured to image the eye, for example. The wearable light field processor system can be configured to perform retinoscopy and measure the refractive error of the wearer's eye. For example, the light field processor system can be configured to sweep a light beam in one or more directions across the wearer's eye, detect, measure, or image a reflection, i.e., a reflection from the back of the wearer's eye (e.g., the retina, the fundus, etc.), and determine a vision defect through observation or measurement of the reflection. Optical corrections can be computationally introduced into the light beam by the light field processor 70 and the reflections observed to determine when such optical corrections are sufficient to offset the wearer's refractive error.

[0340] At least one light source (e.g., 714 in FIG. 14) can be configured to provide a light beam that is moved across or around the eye of a wearer of the light field processor system. In some implementations, the light source provides a relatively narrow beam or strip of light that is moved across the eye. The beam of light may have a cross section perpendicular to the direction of its optical path that is elongated, the cross section being longer in one direction than the perpendicular direction. Thus, the beam of light may form a strip of light on the eye in some embodiments. The light provided by the light source can be configured to be moved in one or more directions. When the light provided by the light source is a relatively narrow beam or strip of light, the orientation of the beam or strip of light can be changed. Thus, the light source can be configured to provide light that can be used to identify myopia, hyperopia, astigmatism, pigmentary, age-related macular degeneration, and other vision disorders.

[0341] The sensor (e.g., one or more of the photodetectors 715 in FIG. 14) can be configured to sense light reflected from the back or retina of the wearer's eye of the light field processor system (e.g., retinoscopy reflex, retinal reflex, or reflex) and form an image of the eye in various embodiments. Thus, the sensor can be an image sensor or camera, one or more photodetectors, or other devices that can provide a signal in response to the emitted light, and possibly an image of the eye. In some embodiments, the sensor can include one or more filters (e.g., bandpass filters tuned to pass a band of wavelengths expected for reflection) that are tuned to preferentially pass reflections from the wearer's eye and preferentially block light in other wavelength bands. The filters can be physical filters and / or filters applied within signal or image processing software.

[0342] The light sources and sensors (e.g., cameras) can be coupled to a control system (e.g., light field processor 70) configured to process information about the characteristics, direction, orientation, and / or position of the light provided by the light sources and to process information about the characteristics, direction, orientation, and / or position of the light detected by the sensors. From this information, the control system can be configured to determine a vision defect in one or more of the wearer's eyes. In some embodiments, the control system can be configured to modify the light provided by the light sources (e.g., direction, orientation, optical corrections provided to the beam, etc.) based, at least in part, on an analysis of the light detected by the sensors. In some embodiments, the control system is configured to implement a predefined routine to determine a vision defect. In some embodiments, the control system can adapt a retinoscopy routine based on the results of an analysis of the light detected by the sensors at any point during the routine.

[0343] The light field processor system can be configured to project images to the wearer. As described herein, the light field processor system can provide the wearer with images corresponding to different apparent depth planes, both far and near. Thus, the wearer can fixate the display and see images that simulate distant and near objects. In this way, the wearer can either have relaxed accommodation or may exhibit accommodation in response to testing.

[0344] The light field processor system can thus be configured to provide static and / or dynamic retinoscopy. For static retinoscopy, for example, the light field processor system can be configured to determine the refractive error when the wearer has relaxed accommodation. For dynamic retinoscopy, for example, the light field processor system can be configured to perform retinoscopy while the wearer is accommodating to different distances. This can be accomplished by providing a virtual image or object on which the wearer focuses while performing retinoscopy. The apparent distance to the image or object can be varied while the accommodation of the eye is tracked through the methods and systems described herein. The distance to the image can be varied by the light field processor 70 by varying the apparent depth plane in a manner as described herein. For example, the light field processor can vary the curvature of the wavefront of the image data projected into the eye to correspond to a specific focal length and associated depth plane. The apparent distance from the eye to the depth plane can thus be known. In some cases, the light field processor 70 that controls the projected beam has a variable refractive power that can be selected or computationally adjusted. Thus, the apparent depth plane can be changed or adjusted as desired in such cases. Alternatively, or in addition, the apparent distance to the object can also be changed by placing an actual object in the world in front of the wearer and within the wearer's field of view, as described herein (e.g., via the integrated imaging camera 16), as seen through the display of the light field processor system. Any of these approaches can be used, for example, to determine the wearer's accommodation response to changes in target distance. These approaches can also be used, for example, to determine the near point of the eye. This can be compared to static retinoscopy, which determines, among other things, the far point of the eye.

[0345] The wearable light field processor system may include one or more user interface features configured to allow the wearer or other person to provide input to the device. The user interface features may be integrated with the device. In some implementations, the user interface features are provided by devices or components that are not physically integrated with the device. For example, the user interface features may be provided by a device or system that communicates with the device. This may be a smartphone, computer, tablet, or other computing device that communicates wired or wirelessly with the device. In some embodiments, the user interface features may be provided by a combination of different devices and systems linked to the device, for example, through a wired or wireless communication network, or through components that are physically linked to or integrated with the device. The user interface features may be presented on the device with a touch screen, and interaction with the touch screen provides input to the wearable light field processor system. A voice recognition system as well as virtual touch capabilities may additionally or alternatively be included. Thus, the user interface features may include capacitive features that are sensitive to touch, keyboards, buttons, microphones, photodetectors, cameras or tracking sensors for tracking gestures such as pointing by the wearer, or various software-implemented features provided by a graphical user interface. In various embodiments, a virtual touch screen is provided through an image projected to the user's eyes, and sensors sense the user moving their body, e.g., fingers. In some embodiments, the user interface features include a gesture detection component, allowing the wearer to provide user input through gestures.In some embodiments, the user interface features include an eye gaze detection component, allowing the wearer to provide user input through eye gaze (e.g., this can include selecting a button or other element when the wearer fixates the button for a period of time or when the wearer blinks while fixating on the button). Such user interface systems can be employed for other devices and systems described herein.

[0346] In some implementations, the wearer, clinician, or physician can use the interface features to control aspects of the retinoscopy test. This can be done, for example, to change the characteristics of the image or light being provided and / or the depth plane onto which the light or image is projected. This can be used to modify the light and optical correction being provided to the wearer and determine the appropriate optical prescription for the wearer.

[0347] In some embodiments, the light field processor system may be configured to provide both static and dynamic retinoscopy. Both types of retinoscopy may be performed using the same device, since the focus of the image can be dynamically modified through the light field processor 70. It should be understood that the light field processor system may also provide a static or sweeping path of light to the retina. This may be a light beam projected by a light source, such as the integrated imaging display 62. In some implementations, the light field processor system may include additional components configured to sweep the light across the retina or otherwise move the light on the retina. The light field processor system may be used to perform retinoscopy and objectively determine refractive error, which may be advantageous over other instruments that use subjective feedback from the patient to determine refractive error.

[0348] To provide static retinoscopy, the light field processor system is used when the accommodation of the wearer's eye is relaxed. This can be accomplished, for example, through the use of cycloplegic eye drops in the wearer's eye. A light spot or beam can be provided and moved across the wearer's eye. A computational refractive correction, which can modify the shape of the wavefront, can be applied to neutralize or compensate for vision impairments. In some embodiments, when operating as a static retinoscope, the image provided by the light field processor system for viewing used for fixation can be provided from an apparent depth plane that is far away, for example, virtually at infinity. In some embodiments, when operating as a static retinoscope, the light provided by the light field processor system that is swept across the eye can be provided from a static apparent depth plane. The apparent depth plane, onto which the virtual image is projected, can be located at infinity to about 0.1 m. In some implementations, the light field processor system includes a sprayer or other delivery device to deliver eye drops or spray used to dilate the pupil and / or ease accommodation of the wearer's eye. For example, the light field processor system can be configured to spray cycloplegic eye drops into the wearer's eye.

[0349] To provide dynamic retinoscopy, the light field processor system can be used when the wearer's eyes are allowed to accommodate. An image can be displayed to the wearer or an object can be provided for the wearer to fixate on. The apparent distance of the image or object can be varied to induce accommodation. In some embodiments, the accommodation of the wearer's eyes can be observed and / or measured. Using this technique, lags or advances in accommodation may be measured.

[0350] The light field processor system described herein can be used to switch between static and dynamic retinoscopy. This can be accomplished while the light field processor system is in use. The light field processor system can be configured to provide images for viewing by the wearer from various apparent depth planes, allowing both static and dynamic retinoscopy to be performed. For example, the images provided by the light field processor system can be dynamically modified to switch between static and dynamic retinoscopy.

[0351] As in the previous embodiment, an input may be received by the wearer to determine a diagnosis. In some embodiments, the light field processor system may also include an eye scanning module configured to measure the response of the retina to the swept light. This response may be recorded and analyzed based on a retinoscopy-specific algorithm to provide a diagnosis to the patient. For example, the algorithm may be based at least in part on retinoscopy, where a light is swept across the eye and a reflection is observed and measured, and a refractive error is associated with the observed or measured properties of the reflection. In some implementations, the direction of the reflection movement may be used to determine the refractive error. For example, if the reflection moves in the same direction as the light swept across the eye, i.e., demonstrates a "following" movement, the light field processor system may determine that the wearer's eye is hyperopic. Similarly, if the reflection moves in the opposite direction to the light swept across the eye, i.e., demonstrates a "counter" movement, the light field processor system may determine that the wearer's eye is myopic. If the reflection moves in a direction that is not parallel to the direction in which the light is swept across the eye, the light field processor system may determine that the wearer's eye is astigmatic. Additionally, the direction of movement of the reflection relative to the direction of movement of the light provided to the eye may indicate whether a positive or negative refractive power is required to correct the vision defect. For example, a "follow" movement indicates that a positive refractive power may be required to correct the refractive error, a "counter" movement indicates that a negative refractive power may be required, and a diagonal movement indicating a cylindrical refractive power may be required. As explained above, different optical corrections (e.g., sphere and / or cylinder with variable axis) may be provided to determine the wearer's prescription and the preferred refractive correction.

[0352] In some implementations, the speed of the reflex may also be combined with the virtual working distance of the light field processor system and used to determine a characteristic of the vision impairment. For example, the speed of the reflex may be correlated with the refractive error or refractive error of the eye (e.g., a faster speed indicates a lower refractive error or a smaller refractive error of the eye).

[0353] In some implementations, the width of the reflex may also be used to determine the characteristics of the vision impairment. For example, the width of the reflex can be correlated with the refractive error of the eye (e.g., a wider reflex indicates a lower refractive error or a smaller refractive error of the eye).

[0354] In some implementations, the orientation of the reflection relative to the source light beam may also be used to determine characteristics of the vision defect. For example, a rotation of the orientation of the reflection relative to the source light beam may indicate astigmatism.

[0355] In some implementations, the relative brightness of the reflection may also be used to determine a characteristic of a vision defect, for example, the relative brightness of the reflection can be used to determine the refractive error of the eye (e.g., a brighter reflection indicates a lower refractive error or a smaller refractive error of the eye).

[0356] Any combination of the aforementioned characteristics of the reflection may be used to determine the refractive error as well. Similarly, changes in the aforementioned characteristics of the reflection may be used, alone or in any combination with each other, to determine whether a refractive correction applied by the light field processor system and / or resulting from the addition or subtraction of a refractive optical component or other components that introduce optical corrections, improves or worsens the determined refractive error. In some embodiments, the light field processor system may be configured to determine in real time whether a refractive correction improves or worsens the determined refractive error. The light field processor system may be configured to measure or monitor the accommodation reflex by measuring the accommodation, convergence and / or pupil size and variations in these physical characteristics and assessing whether the wearer is able to view the image with normal visual acuity. For example, the accommodation, convergence and / or pupil size of the eye vary when fixating a stationary target. These variations increase when the eye has difficulty focusing on the image. Thus, the light field processor system can be configured to monitor variations in ocular characteristics and use this biofeedback to assess the quality of the image seen by the wearer (e.g., whether the wearer is viewing an object or image with normal vision).

[0357] 17 illustrates an exemplary method 1700 for measuring a refractive error of a wearer of a light field processor system configured as an ophthalmic device and performing retinoscopy. For ease of explanation, the method 1700 will be described as being performed by a light field processor system such as those described herein. However, some components or sub-portions of various light field processor systems or other similar devices disclosed herein may be used to perform some steps, combinations of steps, or portions of steps in the method 1700.

[0358] At block 1702, the light field processor system initiates a retinoscopy program. A retinoscopy program can be a stored process or sequence of functions provided by the light field processor system. Initiating a retinoscopy program can include determining or retrieving a starting optical prescription, such as for a wearer who has previously undergone a retinoscopy exam or other eye test. In some implementations, the retinoscopy program can integrate information about ocular abnormalities of the wearer's eye, which information about ocular abnormalities can be entered by the wearer or clinician, determined from a previous retinoscopy program, or retrieved from a data store (e.g., a data store that is part of the light field processor system or a networked data store). Initiating a retinoscopy program can include a light beam to be projected onto the wearer. Initiating a retinoscopy program can include determining whether a clinician or doctor will administer an eye test or whether the test will be self-administered by the wearer. In some embodiments, the light field processor system initiates a retinoscopy program in response to input received from the wearer or clinician.

[0359] In block 1704, a beam of light is swept through the wearer's eye. The light can be, for example, a beam (e.g., a spot projected into the eye). The light may be collimated, converging, or diverging. The light can be swept across the eye or otherwise moved around the eye. The beam of light can be provided with an amount of optical correction / refractive power to be tested (e.g., sphere and / or cylinder (with variable axis)) to determine defocus error and astigmatism.

[0360] In block 1706, an eye scanning component of the light field processor system is configured to measure the response of the wearer's eye in response to the swept light (e.g., reflection from the wearer's eye). The eye scanning component can be a camera or other sensor as described herein (e.g., an integrated image display 62 as controlled by the light field processor 70). The eye scanning component can be configured to analyze the measurement of the reflection and determine the refractive error. For example, the component (e.g., a camera) can include an analysis module configured for pattern recognition measurement, response pattern identification, sensor measurement, reflection tracking, brightness measurement, speed tracking, orientation determination, or the like. The retinoscope program may be pre-coded with a pattern recognition algorithm to identify patterns and / or analyze a given pattern. The retinoscope program may be pre-coded with previous images from the wearer to identify changes in historical analysis.

[0361] In block 1708, the wearer's ocular response may be compared to a correlation table that holds corresponding response values ​​for various vision impairments. For example, in block 1708, the light field processor system compares the information measured in block 1706 to a correlation table or other data that corresponds to expected values ​​of the measurements for various vision impairments. The comparison can be used to determine a refractive error based on the characteristics of the reflection measured in block 1706.

[0362] In block 1710, the values ​​are compared to determine any vision defects. Examples of characteristics and their relationship to vision defects are described herein above. For example, the direction, speed, brightness, and / or width of the reflection can be used to determine refractive error. The shape of the reflection can be used to determine other vision defects such as astigmatism. In some embodiments, the test can be initiated if the wearer has difficulty focusing or when problems with their vision arise, as indicated by the dotted line from block 1710 to block 1702.

[0363] In various embodiments, to reduce distraction, the view of the world in front of the wearer's eyes through the light field processor system is blocked or otherwise invisible during retinoscopy. This may occur, for example, when an image is presented to the wearer, but this approach is not required. In some embodiments, eye tracking, head pose (e.g., head orientation and gaze direction) tracking, and / or eye or facial tissue tracking can be employed to monitor whether the wearer is distracted or fatigued. The system can be configured to dynamically filter out distractions based on the results of monitoring the eye tracking system.

[0364] The light field processor system may be a system provided by a doctor or clinician for testing in a medical facility or optometrist's office or elsewhere. In other embodiments, the system may belong to the wearer and may be employed for other purposes, such as entertainment (e.g., games and movies) and / or professional tasks. As explained above, one advantage of implementing retinoscopy on the wearer's system is that the procedures can be conveniently taken multiple times (at least 2, 3, 4, 5, 6, 8, 10, 12, 16, 18, 24, or more) throughout the year. In some embodiments, the frequency or schedule of the procedures can be modified based on the results of the retinoscopy test and / or trends in the results. For example, if the test results indicate that a vision impairment is worsening or if the system detects that the wearer is struggling with their vision (e.g., through analysis of accommodation fluctuations, vergence-divergence fluctuations, etc.), the frequency or schedule of the procedures can be modified to increase the frequency of procedures and / or decrease the time between procedures. Similarly, the procedure can be performed with or without the presence of medical personnel, such as an optometrist, ophthalmologist, nurse, technician, medical assistant, etc.

[0365] The following paragraphs describe various exemplary embodiments of the devices, systems, and methods described herein.

[0366] A wearable ophthalmic device comprising: a head-mounted light field display configured to generate a physical light field consisting of a beam of light and sweep the beam of light across a user's eye, thereby producing a retinal reflex; a head-mounted photodetector array configured to receive the retinal reflex and generate numerical image data; and a light field processor configured to computationally introduce an amount of positive or negative refractive power into the beam of light and determine an optical prescription for the user's eye based on the retinal reflex response to the beam of light.

[0367] The device of the preceding paragraph, wherein the photodetector array is part of a light field camera and the numerical image data comprises numerical light field image data.

[0368] The device of any of the preceding paragraphs, wherein the light field display comprises an integral imaging display.

[0369] The device of any of the preceding paragraphs, wherein the photodetector array is integrated with a light field display.

[0370] The device of the preceding paragraph, wherein the light field display comprises a microlens array that is shared with the light field camera.

[0371] The device of any of the preceding paragraphs, further comprising an outwardly facing, head-mounted light field camera configured to receive light from around the user and generate numerical light field image data, and the light field processor configured to computationally modify the numerical light field image data based on the optical prescription.

[0372] A method for using a wearable ophthalmic device, the method comprising: generating a physical light field consisting of a beam of light; and using a head-mounted light field display to sweep the beam of light across a user's eye, thereby producing a retinal reflex; using a head-mounted photodetector array to receive the retinal reflex and generate numerical image data; and using a light field processor to computationally introduce an amount of positive or negative refractive power into the beam of light and determine an optical prescription for the user's eye based on the retinal reflex response to the beam of light.

[0373] The method of the preceding paragraph, wherein the photodetector array is part of a light field camera and the numerical image data comprises numerical light field image data.

[0374] The method of any of the preceding paragraphs, wherein the light field display comprises an integral imaging display.

[0375] The method of any of the preceding paragraphs, wherein the photodetector array is integrated with a light field display.

[0376] The method of the preceding paragraph, wherein the light field display comprises a microlens array shared with the light field camera.

[0377] The method of any of the preceding paragraphs, further comprising the steps of using an outward-facing, head-mounted light field camera to receive light from around the user and generate numerical light field image data, and computationally modifying the numerical light field image data based on the optical prescription.

[0378] Automatic Refractometer In one or more embodiments, an ophthalmic device described herein (e.g., the light field processor system 600) may be configured to function as an autorefractometer. An autorefractometer provides an objective measurement of a person's refractive error. In contrast to an eye positioner, which may involve a subjective response from the patient, an autorefractometer does not rely on a response from the user. A cycloplegic drug (e.g., eye drops) may be used to keep the ciliary muscles in a relaxed position, resulting in a loss of accommodation for the user. This relaxed position of the eye provides a more consistent view of the retina. The patient may be asked to view an image projected by the autorefractometer device. The image may move in and out of focus across the depth plane as the machine takes a reading, and determine when the image is on the retina. The machine may average the results and determine a prescription.

[0379] To this end, the ophthalmic device may use a display (e.g., an integrated imaging display 62 as controlled by the light field processor 70) to provide one or more images at variable apparent depths, scan through an eye scanning module (e.g., optical sensor 715 in FIG. 14) and capture an image of the retina while the eye is focused on the image at variable apparent depth. As in the case of the previous example, various algorithms may be used to determine when the patient focuses on the image and subsequently determine the optical prescription for the user. The processor of the ophthalmic device may be used to perform several objective refraction tests using visible or infrared light. In one or more embodiments, image quality analysis / contrast peak detection techniques may be used in the analysis. Similarly, Scheinel double pinhole alignment, Shack-Hartmann grid alignment, and / or retinoscopy reflection neutralization may be used as well.

[0380] In some embodiments, the light field processor system 600 in FIG. 14 can be configured as an autorefractor. As previously discussed, the system 600 can be head mounted so as to be aligned with the user's eyes. The autorefractor system can be a binocular autorefractor capable of testing the refraction of both of a patient's eyes simultaneously.

[0381] The integrated imaging display 62 can be used in conjunction with the light field processor 70 to transmit light with variable amounts of wavefront curvature to the eye. For example, the integrated imaging display 62 can transmit light beams with different amounts of divergence to the eye, including light beams with positive or negative divergence as well as collimated beams. The amount of divergence can be controlled by the light field processor 70 based on the wavefront curvature of the image data provided to the integrated imaging display 62. This can be done computationally without requiring a physical lens to change the divergence of the beam. The light provided by the light source 714 can be in the visible or infrared spectrum.

[0382] As explained immediately above, the integrated imaging display 62 can be used to controllably provide beams of light with a range of different divergences to the eye, from a beam with a positive divergence to a collimated beam with a negative divergence. The beams of light output from the integrated imaging display 62 propagate along a visual axis towards the user's eye. In some embodiments, these beams may be transmitted through a beam splitter and provided between the integrated imaging display 62 and the eye. Such a beam splitter may be aligned with the visual axis of the eye and allow a separate camera to view the eye.

[0383] As previously discussed, the autorefractometer system can provide image data to the eye using a beam of light with a variable degree of divergence. As this image data is provided to the eye, a photodetector 715 in the integrated imaging display 62 (or a separate camera viewing the eye through a beam splitter camera) can be used to monitor the retina of the eye. The photodetector 715 can provide the retinal image to the light field processor 70. The light field processor 70 can then perform image processing algorithms on the retinal image to determine when the image data projected by the autorefractometer system is best focused on the retina of the eye. Such image processing algorithms can include, for example, contrast peak detection. (Image data projected onto the retina of the eye will generally have a relatively low contrast when the image data is blurred and a peak contrast when the image data is sharply focused by the eye.) The light field processor can calculate the refractive power of the eye based on the degree of divergence (whether positive, collimated, or negative) required to allow the eye to focus the light on the retina. The light field processor 70 can determine image quality at multiple meridians in order to calculate not only the spherical power of the eye, but also the cylindrical power and axis.

[0384] The light field processor can control the operation of the autorefractometer system. In one exemplary embodiment, the control method can include causing one or more of the light sources 714 to project an image toward the eye using a beam of light having a first divergence value (whether positive, collimated, or negative). The light field processor 70 can then capture an image of the retina of the eye using the camera photodetector 715. The light field processor 70 can analyze the captured retinal image and determine a metric of the quality of the image formed on the retina when using a beam of light having the first divergence value.

[0385] The light field processor can then cause one or more of the light sources 714 to project an image towards the eye using a beam of light having a second divergence value different from the first divergence value. The light field processor 70 can then again use the light detector 715 to capture an image of the retina of the eye, analyze the retinal image and determine a metric of the quality of the image formed on the retina when using a beam of light having the second divergence value. The light field processor 70 can then compare the first image quality metric to the second image quality metric. Based on this comparison, the light field processor 70 can use any of a variety of optimization algorithms to select a third divergence value to use when projecting the image towards the eye. The light field processor 70 can then calculate a third image quality metric indicative of the quality of the image formed on the retina when using a beam of light having the third divergence. This process can be performed iteratively until the image quality metric is maximized or otherwise determined to be sufficient. Finally, the light field processor can calculate the refractive power of the eye based on the divergence value corresponding to this image quality metric. In addition, the light field processor 70 can initiate execution of the eye positioning method described herein if the auto-refractor system identifies a refractive error above a threshold. The eye positioning system can be used as a check on the accuracy of measurements by the auto-refractor system or vice versa. In this manner, the auto-refractor system and the eye positioning system described herein can be used together to characterize a patient's visual acuity.

[0386] FIG. 18A illustrates another exemplary embodiment of an augmented and / or virtual reality system configured as an autorefractometer. The autorefractometer system can include all of the features of the autorefractometer system 600 illustrated in FIG. 14 (except the light field processor 70 and the integrated imaging camera 16 are not shown). In addition, the autorefractometer system illustrated in FIG. 18A can include a Shinel pinhole disk 1860 located along the optical path of the system in front of the eye 58. The Shinel disk 1860 can be located, for example, between the integrated imaging display 62 and the eye 58. It is an opaque disk with two or more small apertures. As illustrated in FIG. 18A, when a collimated beam of light enters the Shinel disk 1860, the beam is blocked from being transmitted to the eye except for the light rays that can pass through the two apertures. In the case of an emmetropic eye, the light rays transmitted through each of the two apertures are focused to a common spot on the retina of the eye 58. Thus, the retinal image captured by the optical sensor 715 will reveal a single spot.

[0387] FIG. 18A illustrates the effect of a collimated beam passing through the Shinel disk 1860 on an emmetropic eye, while FIGS. 18B and 18C show the same effect on a hyperopic eye and a myopic eye, respectively. As seen in FIG. 18B, the refractive power of the hyperopic eye is not strong enough to focus the light beam transmitted through the two apertures of the Shinel disk 1860 to a single spot. Thus, the retinal image taken by the optical sensor 715 will reveal two distinct spots when the collimated beam illuminates the Shinel disk in front of the hyperopic eye. As seen in FIG. 18C, the refractive power of the myopic eye is too strong, which results in the light beam transmitted through the two apertures of the Shinel disk 1860 being focused in front of the retina. This also results in two distinct spots being formed on the retina.

[0388] The autorefractometer system 600 can therefore vary the divergence of the beam of light incident on the Shinel disk until a single spot is formed on the retina of the eye. The refractive power of the eye 58 can be calculated based on the beam divergence required to form a single spot on the retina.

[0389] The light field processor 70 can control the operation of the autorefractometer system 600. In one exemplary embodiment, the control method can include having one or more of the light sources 714 project a beam of light having a first divergence value (whether positive, collimated, or negative) onto the Shinel disk 1860. The light field processor 70 can then capture an image of the retina of the eye 58 using the light sensor 715. The light field processor 70 can analyze the retinal image and determine the number of spots that are evident. If only a single spot is evident, the processor can calculate the refractive power of the eye 58 based on the first divergence value. Alternatively, if multiple spots are evident, the processor 2606 can select a second divergence value that is different from the first divergence value. The light field processor 70 can then project a beam of light having the second divergence value onto the Shinel disk 1860. The light field processor 70 may again use the optical sensor 715 to capture an image of the retina of the eye 58 and analyze the retinal image to determine the number of spots that are evident. If a single spot is evident, the light field processor 70 may calculate the refractive power of the eye 58 based on the second divergence value. If not, a third divergence value may be selected and the process may be repeated iteratively until a single spot is formed on the retina. The light field processor 70 may then calculate the refractive power of the eye 58 based on the divergence value.

[0390] Any of the autorefractometers or other diagnostic methods described herein can be used for real-time adjustments while the user is viewing content to ensure that the content is in focus. In addition, monitoring of the user's refractive error can be performed on a long-term basis (e.g., weeks, months, or years) to provide longitudinal monitoring and analysis of the user's refractive error. The frequency of regularly scheduled tests may be automatically adjusted based on testing trends.

[0391] The following paragraphs describe various exemplary embodiments of the devices, systems, and methods described herein.

[0392] 1. A wearable ophthalmic device comprising: a head-mounted light field display configured to generate a physical light field consisting of a beam of light and direct the beam of light into a user's eye, thereby producing a retinal reflex; a head-mounted photodetector array configured to receive the retinal reflex and generate numerical image data; and a light field processor configured to control the light field display, analyze the retinal reflex using the numerical image data, and determine an optical prescription for the user's eye based on the analysis of the retinal reflex.

[0393] The device of the preceding paragraph, wherein the light field processor is further configured to computationally introduce an amount of positive or negative refractive power into the beam of light based on analysis of the numerical image data.

[0394] The device of any of the preceding paragraphs, wherein the photodetector array is part of a light field camera and the numerical image data comprises numerical light field image data.

[0395] The device of any of the preceding paragraphs, wherein the light field display comprises an integral imaging display.

[0396] The device of any of the preceding paragraphs, wherein the photodetector array is integrated with a light field display.

[0397] The device of any of the preceding paragraphs, further comprising a disk with at least two pinholes, the disk being positioned in front of the eye such that the beam of light passes through the pinholes.

[0398] The device of any of the preceding paragraphs, wherein the light field processor is configured to analyze retinal reflexes and determine a focus of light on the retina.

[0399] The device of any of the preceding paragraphs, further comprising an outward-facing, head-mounted light field camera configured to receive light from around the user and generate numerical light field image data, and the light field processor configured to computationally modify the numerical light field image data based on analysis of retinal reflexes.

[0400] 1. A method of using a wearable ophthalmic device, the method comprising: using a head-mounted light field display to generate a physical light field consisting of a beam of light and directing the beam of light into a user's eye, thereby producing a retinal reflex; using a head-mounted photodetector array to receive the retinal reflex and generate numerical image data; controlling the light field display and using the numerical image data to analyze the retinal reflex; and using a light field processor to determine an optical prescription for the user's eye based on the analysis of the retinal reflex.

[0401] The method of the preceding paragraph, further comprising using a light field processor to computationally introduce an amount of positive or negative optical power into the beam of light based on analysis of the numerical image data.

[0402] The method of any of the preceding paragraphs, wherein the photodetector array is part of a light field camera and the numerical image data comprises numerical light field image data.

[0403] The method of any of the preceding paragraphs, wherein the light field display comprises an integral imaging display.

[0404] The method of any of the preceding paragraphs, wherein the photodetector array is integrated with a light field display.

[0405] The method of any of the preceding paragraphs, further comprising transmitting the beam of light through a disk with at least two pinholes, the disk being positioned in front of the eye.

[0406] The method of any of the preceding paragraphs, further comprising using a light field processor to analyze retinal reflections and determine the focal point of light on the retina.

[0407] The method of any of the preceding paragraphs, further comprising the steps of using an outward-facing, head-mounted light field camera to receive light from around the user and generate numerical light field image data, and computationally modifying the numerical light field image data based on analysis of the retinal reflex.

[0408] Aberrometer In one or more embodiments, the light field processor system 600 may function as an aberrometer. The aberrometer measures intraocular irregularities, potentially including defocus, regular astigmatism, and higher order aberrations (e.g., spherical aberration, coma, trefoil, irregular astigmatism). Even higher order aberrations can have a significant effect on the quality of vision, affecting depth perception, contrast, color vision, night vision, etc. Identification of higher order aberrations in addition to lower order aberrations can help produce more accurate prescription eyewear.

[0409] To identify refractive errors, the aberrometer may send a beam of light into the eye. In some embodiments, the beam of light is infrared light. The light passes through the cornea and lens of the eye and is reflected by the retina. The reflected light may then be measured by the aberrometer to produce data regarding the refractive errors of the eye (including higher order refractive errors). Aberrometers are often used to gather data for performing laser vision correction surgery. The map created by the aberrometer directs the delivery of laser light, which precisely reshapes the cornea.

[0410] In some embodiments, the aberrometer system may include elements of the light field processor system 600. For example, the aberrometer system may include a light field processor 70 and an integrated imaging display 62. In some embodiments, the integrated imaging display is of the type illustrated in FIG. 14, which includes a light sensor 715. The integrated imaging display 62 of the light field processor system 600 may be used to produce light having a desired wavefront. As in the case of the previous embodiment, the response to the applied stimulus may be measured. It should be understood that wavefronts of different frequencies may be applied. Similarly, visible or non-visible light may be projected into the eye. In one or more embodiments, the captured data may be processed to determine any abnormalities.

[0411] The wavefront of light output from the integrated imaging display 62 propagates along the visual axis towards the user's eye. In some embodiments, the integrated imaging display 62 outputs a probe beam of light having a planar wavefront. However, the integrated imaging display 62 can also be controlled by the light field processor to output a probe beam with positive or negative optical power. The probe beam enters the eye 58 and is eventually backscattered by the retina. As the probe beam propagates through the eye, its planar wavefront may be affected by irregularities or imperfections in the optics of the eye 58. Such irregularities or imperfections may cause the wavefront to become irregular as well.

[0412] Once the backscattered probe beam exits the eye, it propagates back towards the integrated imaging display 62. These wavefronts exiting the eye are typically irregular. The specific shape of the aberrated wavefront depends on the irregularities or imperfections in the eye 58. The system may include a relay lens system that relays the wavefront approximately at the pupil plane of the eye to an array of microlenses 712.

[0413] The array of microlenses 712, in conjunction with light sensor 715, can act as a wavefront aberrometer, which can measure and characterize the shape of these wavefronts. Specifically, a Shack-Hartmann type wavefront sensor. The array of lenslets 712 spatially samples the incident wavefront at many different locations. The light sensor 715, which can be, for example, a CCD or CMOS element, can act as a detector. Each lenslet focuses a spot of light onto the detector. The precise location of each spot on the detector depends on the local curvature of the wavefront at the corresponding lenslet location. The detector thus creates an image, consisting of an array of spots. This image can be analyzed by light field processor 70 to determine the precise location of each spot, which in turn indicates the wavefront curvature at the corresponding lenslet location. In this way, light field processor 70 can determine the curvature of the wavefront at each spatial location sampled by the lenslet array. Based on the measured wavefront shape, the processor can calculate the ocular aberrations, including both low and high order aberrations. These aberrations can be expressed numerically, for example, as Zernike coefficients.

[0414] Once the light field processor 70 has determined the aberrations of the eye 58, it can output those measurements, for example, in numerical or graphical form. The measurements can be used to determine a treatment plan for the eye 58, such as a corrective optical prescription. In addition, the measurements of the aberrations of the eye 58 can be used by the light field processor to computationally correct the incoming image data from the integrated imaging camera 16, thus providing a sharper image to the user. For example, in some embodiments, the light field processor 70 can be used to computationally control the shape of the wavefront output by the integrated imaging display 62 as it projects virtual and / or augmented reality image data into the eye 58. In this manner, the image data provided to the user can be specifically corrected based on the high and / or low order aberrations of the user's own eye.

[0415] The following paragraphs describe various exemplary embodiments of the devices, systems, and methods described herein.

[0416] 1. A wearable ophthalmic device comprising: a head-mounted light field display configured to generate a physical light field consisting of a beam of light and direct the beam of light into a user's eye, thereby producing a retinal reflex; a head-mounted light field camera configured to receive the retinal reflex and generate numerical light field image data; and a light field processor configured to control the light field display and use the numerical light field image data to determine high-order aberrations of the eye based on analysis of the retinal reflex.

[0417] The device of the preceding paragraph, wherein the light field processor is further configured to computationally introduce an amount of positive or negative refractive power into the beam of light.

[0418] The device of any of the preceding paragraphs, wherein the light field display comprises an integral imaging display.

[0419] The device of any of the preceding paragraphs, wherein the light field camera is integrated with a light field display.

[0420] The device of the preceding paragraph, wherein the light field display comprises a microlens array that is shared with the light field camera.

[0421] The device of the preceding paragraph, wherein the light field camera comprises a Shack-Hartmann sensor.

[0422] The device of any of the preceding paragraphs, further comprising an outwardly facing head-mounted light field camera configured to receive light from around the user and generate additional numerical light field image data, and the light field processor configured to computationally correct the additional numerical light field image data based on high-order aberrations of the eye.

[0423] 1. A method of using a wearable ophthalmic device, the method comprising: using a head-mounted light field display to generate a physical light field consisting of a beam of light and directing the beam of light into a user's eye, thereby producing a retinal reflex; using a head-mounted light field camera to receive the retinal reflex and generate numerical light field image data; controlling the light field display to analyze the retinal reflex using the numerical light field image data; and using a light field processor to determine high-order aberrations of the eye based on the analysis of the retinal reflex.

[0424] The method of the preceding paragraph, further comprising the step of computationally introducing an amount of positive or negative optical power into the beam of light using a light field processor.

[0425] The method of any of the preceding paragraphs, wherein the light field display comprises an integral imaging display.

[0426] The method of any of the preceding paragraphs, wherein the light field camera is integrated with a light field display.

[0427] The method of the preceding paragraph, wherein the light field display comprises a microlens array shared with the light field camera.

[0428] The method of the preceding paragraph, wherein the light field camera comprises a Shack-Hartmann sensor.

[0429] The method of any of the preceding paragraphs, further comprising using an outward-facing head-mounted light field camera to receive light from around the user and generate additional numerical light field image data, and using a light field processor to computationally correct the additional numerical light field image data based on high-order aberrations of the eye.

[0430] Macular degeneration In one or more embodiments, the light field processor system 600 may be configured to detect, diagnose, and / or compensate for macular disorders. Macular disorders (e.g., holes, cysts, degeneration, etc.) are damages in the macula and foveal tissues in the retina that result in abnormalities, dead spots, or areas of reduced sensitivity to light or lack of sensitivity to light in the user's visual field. Common forms of macular disorders include age-related macular degeneration (AMD), Stargardt's disease, Best's disease, and other degenerative conditions. Age-related macular degeneration includes "dry" AMD, which is characterized by atrophy of the retinal pigment epithelium layer, and "wet" AMD, which is blindness resulting from complications from abnormal blood vessel growth in the retina. Macular disorders can result in abnormalities, dead spots, or areas of reduced sensitivity to light in various parts of the visual field, as well as loss of contrast or color sensitivity. Frequently, the abnormalities, dead spots, or areas of reduced sensitivity occur near the center of the visual field, rather than at the periphery.

[0431] The light field processor system 600 may be configured to detect or diagnose macular disorders by determining the capacity of a portion of the retina and detecting images. In some embodiments, the light field processor system may include a user display device 62, such as the integrated imaging display shown in Figures 6, 7, and 14, which can project light into the wearer's eye and form an image intraocularly. The user display device 62 may include a display lens, screen, etc. 106, which may be mounted to the user's head or eye by a housing or frame 108.

[0432] The system can then detect a response from the wearer. For example, the image may be a small dot that is clearly visible when projected onto a healthy portion of the retina, but likely not visible when projected onto an impaired portion. In some embodiments, the wearer may be prompted, automatically or by another user, such as a medical professional, to indicate whether the image was seen by the wearer. The user may then input the wearer's response through a user interface. In some embodiments, the light field processor system may increase accuracy without requiring conscious input, using an accuracy light sensor 715 in the integrated imaging display device 62, a separate eye tracking camera 24, or a similar detection method, by observing whether an involuntary reaction occurs in response to the projection of the image or continued eye scanning, such as a change in focus or gaze. The eye tracking camera 24 may be an inward-facing (i.e., directed towards the user's eye) camera, as illustrated in FIG. 5. In some embodiments, the light field processor system may prompt the wearer to indicate whether the image was seen by manual input or directly, such as by intentionally directing the wearer's gaze towards the image or the projected virtual button image. The virtual button image may be projected by the display lens 106, and the wearer's selection of the button may be detected by the eye tracking camera 24 or through gesture recognition.

[0433] In some embodiments, the test described above may be repeated with different images at different parts of the retina or at the same part of the retina to detect areas of macular damage. For example, a particular part of the retina where an image is seen by the wearer may be determined to be healthy, while a part of the retina where the same image is not seen may be determined to be damaged. In another example, an image consisting mainly of longer wavelength visible light, such as red light, may be projected first. An image consisting mainly of shorter wavelength visible light, such as blue light, may then be projected onto the same part of the retina, and any difference in visibility to the wearer may indicate a loss of color sensitivity in the wearer. In some embodiments, multiple images differing in contrast, saturation, hue, intensity, periodicity, or spatial frequency, or any other characteristic, may be presented to the wearer at different locations on the wearer's retina to diagnose various sensitivity losses due to macular damage. Images of the wearer's retina may be used to improve the reliability of macular damage diagnosis in addition to the results of the test described above. Such images may be acquired, for example, by the system described with respect to FIG. 14, an ophthalmoscope or fundus microscope, optical coherence tomography, or other imaging techniques.

[0434] The macular disorder test may be performed in discrete tests on demand, or may be performed periodically and / or repeatedly over time. Repeated analysis may allow tracking of progressive macular disorders such as age-related or other macular degeneration. Thus, the macular disorder diagnostic function may be incorporated into a device worn solely for ophthalmic diagnosis, or may be part of a device worn periodically for entertainment, work, or other purposes, etc., such that the test may be performed automatically at regular intervals and / or at various times of the day, week, month, year, etc. In some embodiments, the wearer may be notified before the automatic test is administered, such as by an alert sound and / or a visually displayed message. The results of the macular disorder test may be evaluated in real time at the device, or may be transmitted via a cloud or other network and evaluated remotely for evaluation and / or diagnosis. Remote evaluation and / or diagnosis of anomalies or other macular disorders may be transmitted back to the device to enable treatment or compensation methods described below. Unique characteristics of the wearer's eye may be recorded by the device and used for identification matching, ensuring security and privacy of the transmitted data. For example, the optical sensor 715 may image the iris, and the light field processor 70 may perform pattern recognition to determine whether the identity of the wearer corresponds to the identity of the person to whom the test results correspond. The system may then only display the test results if the wearer is the person to whom the test results correspond.

[0435] The light field processor system can help compensate for macular degeneration by projecting light directly into the retina, specifically targeting healthy cells at the periphery of the macula. By varying where the light is projected, the device can selectively target healthy cells and improve the quality of the user's vision. In one or more embodiments, the light projection source comprises an integrated imaging display with an LED light source configured to project an image into different parts of the user's eye. The system may comprise other types of displays that may be configured to selectively project light onto different parts of the retina. The technique may be utilized to selectively project pixels of an image onto healthy retinal cells, reducing, minimizing, or altering the nature of the light projected onto damaged areas. For example, abnormally projected pixels may be enlarged or brighter. It should also be understood that the technique may also require modifications to the projected image data itself, and the light field processor may alter the nature of the image so that the user does not notice a significant difference when viewing the image.

[0436] The light field processor system may modify the wearer's view of light from the world. The system may detect the light entering the device in real time or near real time and may modify some of the light or project additional light to correct the wearer's macular disorders. For example, the system may image the world using an outward facing camera such as the integrated imaging camera 16. The system may then project the image of the world to the wearer. The projected image data may be altered such that pixels may be selectively projected onto healthy retinal cells, while abnormally projected pixels may be reduced, minimized, magnified, brightened, or otherwise altered in magnification, intensity, hue, saturation, spatial frequency, or other quality. The system may also be used to darken bright rooms and / or brighten nighttime views for wearers with difficulty accommodating to variable light conditions due to macular degeneration in general.

[0437] The light field processor system may provide alerts to the wearer using the outward facing camera. The alerts may be based on detection of a hazardous condition invisible to the wearer due to macular impairment. The system may determine the presence of an invisible hazard based on correlation of images from the outward facing camera with the wearer's known macular impairment data, such as the location of an anomaly on the retina. When a hazard is detected within the blind spot, such as an incoming object, a hole in the ground, or other condition, the system may alert the wearer. The alerts may include visual, auditory, or tactile notifications. In the case of complete blindness, the system may be configured to detect the presence of a desired item (e.g., a chair, table, bed, etc.) and provide proximity information to the wearer, such as by an audible notification.

[0438] In one or more embodiments, the light field processor system may diagnose or evaluate the anatomy of the user's eye and determine the location of macular degeneration. Now referring to FIG. 19, an exemplary process flow 1900 for diagnosing, detecting, and / or identifying any area of ​​macular degeneration is provided. At 1902, a macular degeneration diagnosis / assessment program may be initiated. As in the case of many of the embodiments described above, the program may be pre-coded or downloaded into the light field processor system. At 1904, an image is projected onto a particular portion of the user's eye, for example, through one or more integrated imaging displays 62. For example, an image (e.g., a small dot, a small shape, etc.) is directed to the center of the user's eye (e.g., formed at the center of the retina).

[0439] At 1906, the system may receive input regarding the quality of the image as viewed by the user through any type of user interface. For example, the user may be asked to rate the quality of the image from 1 to 10. Or, in another embodiment, the image may be projected with increasing or decreasing visual stimuli, and the user may be required to identify when the image appears or disappears from the user's vision, the visual stimuli is reduced, and / or moved. In some embodiments, the system may detect the time required for the wearer to respond, and a wearer who takes a long time to respond may have difficulty seeing the stimuli. Similarly, many such techniques such as the Pelli Robson or sine wave grating test may be used. At 1908, based on the received user input, the system may determine the health of that portion of the user's eye.

[0440] At 1910, the system may determine whether other portions of the eye need to be diagnosed and / or evaluated as well. If so, steps 1904-1908 are repeated. After the various other portions of the eye have been tested as well, at 1912, the health results of the various portions of the user's eye may be analyzed and any abnormalities identified.

[0441] In one or more embodiments, the AR system behaves like a visscope, containing a small reticule target for measuring eccentric fixation. A light projection source may project an image onto the patient's retina, and the patient may be asked to look at the center of the target. The location of the foveal reflex relative to the center of the reticule target may indicate whether the patient has eccentric fixation and its extent. Similarly, the direction and degree of eccentric fixation may also be determined through the above process.

[0442] If it is determined that the user has one or more anomalies, the light field processor system may be configured to project a modified image into the user's eye such that the majority of the image is viewed through healthy peripheral retinal cells, and any pixels projected for the anomaly are adjusted. It should be understood that the image to be projected may need to be modified through a predetermined algorithm so that the user views the image through healthy cells, but does not notice any significant changes in the image itself.

[0443] The following paragraphs describe various exemplary embodiments of the devices, systems, and methods described herein.

[0444] 1. A wearable ophthalmic device comprising: a head-mounted light field display configured to generate a physical light field consisting of a beam of light and direct the beam of light to a selected sub-portion of a retina of a user's eye, thereby generating a retinal reflex; a head-mounted photodetector array configured to receive the retinal reflex and generate numerical image data; and a light field processor configured to control the light field display and use the numerical image data to analyze the retinal reflex from the photodetector array and to determine the health of the selected sub-portion of the retina based on the analysis of the retinal reflex.

[0445] The device of the preceding paragraph, further comprising a user interface configured to receive input from a us...

Claims

1. A wearable ophthalmic device, comprising: an outwardly facing head-mounted light field camera configured to receive light from an environment surrounding a user of the device and generate numerical light field image data, the numerical light field image data comprising a plurality of color components; a light field processor configured to computationally modify the numerical light field image data to generate modified numerical light field image data, different ones of the color components being modified differently to, at least in part, compensate for chromatic aberration; a head-mounted light field display configured to generate a physical light field corresponding to the modified numerical light field image data; wherein the light field processor is configured to computationally introduce a first amount of refractive power into a first color component of the numerical light field image data and a second amount of refractive power into a second color component of the numerical light field image data, the second amount of refractive power being different from the first amount of refractive power, and the color components of the numerical light field image data being modified based on an optical prescription for the eye of the user.

2. The device of claim 1 , wherein the chromatic aberration comprises longitudinal chromatic aberration.

3. The device of claim 1 , wherein the first amount of refractive power and the second amount of refractive power reduce a difference between a focal plane of the first color component and a focal plane of the second color component.

4. The device of claim 1 , wherein the chromatic aberration comprises lateral chromatic aberration.

5. 5. The device of claim 4, wherein the light field processor is configured to project the first color component of the numerical light field image data at a first angle and project the second color component of the numerical light field image data at a second angle different from the first angle to compensate for the lateral chromatic aberration.

6. A wearable ophthalmic device, comprising: an outwardly facing head-mounted light field camera configured to receive light from an environment surrounding a user of the device and generate numerical light field image data, the numerical light field image data comprising a plurality of color components; a light field processor configured to computationally modify the numerical light field image data to generate modified numerical light field image data, different ones of the color components being modified differently to, at least in part, compensate for chromatic aberration; a head-mounted light field display configured to generate a physical light field corresponding to the modified numerical light field image data; wherein the light field processor is configured to computationally introduce a first amount of refractive power into a first color component of the numerical light field image data and a second amount of refractive power into a second color component of the numerical light field image data, the second amount of refractive power being different from the first amount of refractive power, and the color components of the numerical light field image data being modified based on a measurement of chromatic aberration of the device.

7. The device of claim 1 , wherein the color components comprise a red color component, a green color component, and a blue color component.

8. 1. A method of operating a wearable ophthalmic device, the wearable ophthalmic device comprising an outwardly facing head mounted light field camera and a light field processor, the method comprising: the outwardly facing head-mounted light field camera receiving light from around a user of the device; the outwardly facing head-mounted light field camera generating numerical light field image data, the numerical light field image data comprising a plurality of color components; and and the light field processor computationally modifies the numerical light field image data to generate modified numerical light field image data, wherein different ones of the color components are differently modified to at least partially compensate for chromatic aberration, and the computationally modifying the numerical light field image data includes: the light field processor computationally introducing a first amount of refractive power into a first color component of the numerical light field image data; the light field processor computationally introducing a second amount of refractive power into a second color component of the numerical light field image data; wherein the second amount of refractive power is different from the first amount of refractive power; and a head mounted light field display generating a physical light field corresponding to the modified numerical light field image data; wherein the color components of the numerical light field image data are modified based on an optical prescription for the user's eye.

9. The method of claim 8 , wherein the chromatic aberration comprises longitudinal chromatic aberration.

10. 9. The method of claim 8, wherein the first amount of refractive power and the second amount of refractive power reduce a difference between a focal plane of the first color component and a focal plane of the second color component.

11. The method of claim 8 , wherein the chromatic aberration comprises lateral chromatic aberration.

12. the light field processor projects the first color component of the numerical light field image data at a first angle; and the light field processor compensates for the lateral chromatic aberration by projecting the second color component of the numerical light field image data at a second angle different from the first angle. The method of claim 11 further comprising:

13. 1. A method of operating a wearable ophthalmic device, the wearable ophthalmic device comprising an outwardly facing head mounted light field camera and a light field processor, the method comprising: the outwardly facing head-mounted light field camera receiving light from around a user of the device; the outwardly facing head-mounted light field camera generating numerical light field image data, the numerical light field image data comprising a plurality of color components; and and the light field processor computationally modifies the numerical light field image data to generate modified numerical light field image data, wherein different ones of the color components are differently modified to at least partially compensate for chromatic aberration, and the computationally modifying the numerical light field image data includes: the light field processor computationally introducing a first amount of refractive power into a first color component of the numerical light field image data; the light field processor computationally introducing a second amount of refractive power into a second color component of the numerical light field image data; wherein the second amount of refractive power is different from the first amount of refractive power; and a head mounted light field display generating a physical light field corresponding to the modified numerical light field image data; wherein the color components of the numerical light field image data are modified based on a measurement of chromatic aberration of the device.

14. The method of claim 8 , wherein the color components comprise a red color component, a green color component, and a blue color component.

15. A non-transitory computer-readable storage medium storing instructions that, when executed by at least one processor, direct the at least one processor to perform actions, the actions including: Receiving light from an environment surrounding a user of the wearable ophthalmic device; generating numerical light field image data using an outward-facing head-mounted light field camera, the numerical light field image data comprising a plurality of color components; and computationally modifying the numerical light field image data using a light field processor to generate modified numerical light field image data, wherein different ones of the color components are differently modified to at least partially compensate for chromatic aberration, and wherein computationally modifying the numerical light field image data includes: using the light field processor to computationally introduce a first amount of refractive power into a first color component of the numerical light field image data; using the light field processor to computationally introduce a second amount of refractive power into a second color component of the numerical light field image data; wherein the second amount of refractive power is different from the first amount of refractive power; and generating a physical light field corresponding to the modified numerical light field image data using a head mounted light field display; wherein the color components of the numerical light field image data are modified based on an optical prescription for the user's eye.

16. 16. The non-transitory computer-readable storage medium of claim 15, wherein the chromatic aberration comprises at least one of longitudinal chromatic aberration or lateral chromatic aberration.

17. A non-transitory computer-readable storage medium storing instructions that, when executed by at least one processor, direct the at least one processor to perform actions, the actions including: Receiving light from an environment surrounding a user of the wearable ophthalmic device; generating numerical light field image data using an outward-facing head-mounted light field camera, the numerical light field image data comprising a plurality of color components; and computationally modifying the numerical light field image data using a light field processor to generate modified numerical light field image data, wherein different ones of the color components are differently modified to at least partially compensate for chromatic aberration, and wherein computationally modifying the numerical light field image data includes: using the light field processor to computationally introduce a first amount of refractive power into a first color component of the numerical light field image data; using the light field processor to computationally introduce a second amount of refractive power into a second color component of the numerical light field image data; wherein the second amount of refractive power is different from the first amount of refractive power; and generating a physical light field corresponding to the modified numerical light field image data using a head mounted light field display; wherein the color components of the numerical light field image data are modified based on a measurement of chromatic aberration of the device.

Images