Digital Therapeutic Corrective Glasses

The wearable eyeglass device with image processing technology dynamically corrects visual impairments, addressing limitations of current methods by enhancing visual acuity and field of view through personalized, real-time adjustments.

JP7592112B2Active Publication Date: 2024-11-29UNIV OF MIAMI
View PDF 6 Cites 0 Cited by

Patent Information

Application Number
JP2023005907
Authority / Receiving Office
JP · JP
Patent Type
Patents
Current Assignee / Owner
Priority Date
2017-09-27
Filing Date
2023-01-18
Publication Date
2024-11-29
Estimated Expiration
2038-09-27

AI Technical Summary

Technical Problem

Current methods for correcting visual impairments such as glaucoma, refractive errors, anisometropia, and anisocoria are inadequate, as they either provide limited correction, cause image distortion, or fail to address dynamic and higher-order aberrations.

Method used

A wearable eyeglass device equipped with image processing technology and sensors to identify visual impairments, track pupil movement, and adjust image display in real-time to compensate for these conditions, enhancing visual acuity and field of view through dynamic correction techniques.

Benefits of technology

The device provides personalized, real-time correction for various visual impairments, improving visual acuity and field of view by compensating for dynamic aberrations and enhancing safety and functionality for users.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 0007592112000004
    Figure 0007592112000004
  • Figure 0007592112000005
    Figure 0007592112000005
  • Figure 0007592112000006
    Figure 0007592112000006
Patent Text Reader

Abstract

The company provides devices to test, identify and compensate for eye diseases affecting patients' vision, as well as digital therapeutic corrective glasses that provide customized visual field correction / enhancement. The device includes wearable glasses with one or more digital monitors that can be used to recreate the entire field of view as a digitized, modified image, or custom reality glasses that can be used to overlay a generated image onto a viewed scene to correct or enhance the subject's field of view.
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS This application claims priority to U.S. Provisional Patent Application No. 62 / 563,770, entitled "Digital Therapeutic Corrective Spectacles," filed September 27, 2017, the entire contents of which are incorporated herein by reference. [Background technology]

[0002] The present disclosure relates to techniques for compensating for vision defects, visual aberrations and visual alignment errors in a user's visual field, and in particular to providing wearable devices and methods for users to correct these vision defects.

[0003] The background discussion provided herein is intended to provide a general overview of the context in which the present disclosure was made. Work by the inventors named in this application is not admitted, expressly or impliedly, as prior art to the present disclosure, as are various aspects of the description herein that would not be admitted as prior art at the time of filing, to the extent that they are described in this Background section.

[0004] Patients with eye diseases such as optic nerve and / or retinal diseases (e.g., glaucoma patients) have varying degrees of localized loss of visual sensitivity in the visual field, meaning that images appear blurrier in some areas of the visual field than in other areas. This blurring in the visual field occurs because a higher intensity of light is required to stimulate the eye in diseased areas compared to non-diseased areas. It is caused by an eye disease. Patients describe this blurring as a clouding or haze in their field of vision. As the disease progresses, affected areas of the field of vision continue to lose vision, and may eventually lead to complete blindness.

[0005] A visual field diagnostic device is used to test the visual field sensitivity of a patient. A low light is first applied and, if the patient does not indicate that he or she sees it, the intensity is gradually increased until the patient indicates that he or she sees the light. The sensitivity of the illuminated area is then recorded. If the patient is unable to see the light even at the highest illumination intensity, the target area of ​​the visual field is identified as blind.

[0006] Refractive errors have a negative effect on vision. They are due to abnormalities in the refractive elements of the eye. Refractive errors also result in blurred vision, but can be corrected to some extent with glasses and contact lenses. This results in some people having good and some having poor vision, and some having good and some not so good Quality of Vision (QOV). Glass glasses and contact lenses are available only in certain increments and only correct normal refractive errors, e.g. regular astigmatism. These normal refractive errors are called low order aberrations. Higher order aberrations are refractive errors that cannot be corrected with glasses or contact lenses. Higher order aberrations are also dynamic and not constant. They change with the size of the pupil, the state of accommodation of the eye and the direction of gaze.

[0007] Current methods to treat presbyopia include monofocal, bifocal and multifocal reading glasses, as well as multifocal contact lenses. With multifocal or bifocal glasses, patients obtain the required correction by looking through a specific area of ​​the glasses. Multifocal contact lenses diffract light to reach multiple foci, improving depth of focus, but at the expense of reduced quality of vision. All of these technologies are not very convenient and limit near vision.

[0008] Diplopia is caused by a deviation in the patient's gaze. Diplopia is dynamic, not static; that is, it increases and decreases in one or more directions. Thus, if a patient has a restriction in the outward movement of their right eye, looking to the right will increase diplopia, and looking to the left will decrease it.

[0009] Anisometropia (different refractive indices in a subject's eyes) is not uncommon, especially after eye surgery or trauma. Anisometropia is one of the indications for cataract surgery, according to Medicare. Corrective glasses cannot correct anisometropia. This is because corrective glasses create two unequal sized images (anisometropia), one for each eye, and the brain cannot fuse these two images for binocular monocular vision. The cause of this problem is simple: eyeglass lenses are either convex, magnifying the image, or concave, reducing the image. The degree of magnification or reduction depends on the amount of correction.

[0010] Glasses lenses are either convex, magnifying the image, or concave, reducing the image. This affects the subject's field of vision. Glasses correct the patient's refractive error, but at the same time distort the image they see.

[0011] Subjects with anisocoria have unequal pupils, which may be congenital, result from eye disease, or occur following surgery or trauma. Patients with anisocoria have light sensitivity in one eye that cannot tolerate the same brightness of light as the unaffected eye.

[0012] There is a need for optical devices that can compensate for the vision impairments mentioned above. Summary of the Invention

[0013] According to example embodiments, the present technology provides an apparatus for screening, identifying, and / or compensating for one or more ocular disorders affecting a patient's vision, including, for example, disorders of the optic nerve, such as glaucoma, optic neuritis, and optic neuropathy, disorders of the retina, such as macular degeneration, retinitis pigmentosa, disorders of the visual pathway, such as capillary disorders and tumors, and other conditions, such as presbyopia, strabismus, higher and lower order aberrations, monocular vision, anisometropia and anisometropia, photosensitivity, anisometropia, refractive errors, and astigmatism. According to some example embodiments, the present technology provides an apparatus for making changes to improve the visual field for a subject, such as changes in the horizontal, vertical, and / or diagonal angle of view, the light provided to one or more regions, the size of objects in one or more regions, and / or the position of objects in one or more regions.

[0014] In example embodiments, the systems and devices described herein may include a wearable eyewear device configured to screen for, identify, compensate for, and / or improve a subject's vision or field of vision. Some such embodiments may be configured to provide personalized, customized vision correction to a user subject. In one example, the eyewear device includes digital therapeutic corrective glasses (also referred to herein as "DTS"). In one example, the eyewear device may also include glasses, sunglasses, and eyewear.

[0015] In one aspect, the vision system may include a wearable eyewear device. The system may further include an image processor having a processor and a memory. The image processor may include instructions stored in the memory that, when executed, cause the processor to perform the inspection mode and / or the vision mode.

[0016] In one example, the system may further include a pupil tracking sensor configured to track a physical state and / or a line of sight of the subject's pupil. In another example, the pupil tracking sensor includes one or more inward-facing image sensors. In this or another example, the system may include a visual field sensor configured to obtain a visual field in a visual mode.

[0017] In any of the above or other examples, the instructions when executed on a processor may cause the processor to perform the following operations: in a test mode, (i) direct the wearable eyewear device to present a plurality of test stimuli to the subject at one or more test locations in a test field of view, (ii) direct the inward-facing image sensor to acquire pupil physical state and / or gaze position indications during presentation of the plurality of test stimuli at the one or more test locations, and (iii) determine one or more lesion areas in the test field of view to determine one or more visual disorders for the subject, the plurality of stimuli having different contrast levels from each other and from a reference contrast level.

[0018] In any of the above or other examples, the instructions, when executed on a processor, may cause the processor to: in a vision mode, modify an image of the visual field to improve the visual field and / or compensate for one or more areas of pathology, and prompt a subject using the wearable eyewear device to display the modified image via the wearable eyewear device.

[0019] In any of the above or other examples, the image processing device may store instructions that, when executed, cause the processor to perform the following operations: instruct a field of view camera to capture an image of the field of view, process the image in response to one or more determined lesion areas in the test field of view, modify the image to compensate for the one or more lesion areas, and display the modified image as a digital image to the subject via the wearable eyewear device in a vision mode; do.

[0020] In any of the above or other examples, the digital glasses may further include a first digital monitor and a second digital monitor, each configured to display one of the plurality of stimuli to each eye of the subject in the test mode. In any of the above or other examples, the field of view camera includes a first field of view camera and a second field of view camera, the first field of view camera corresponding to the first digital monitor and the second field of view camera corresponding to the second digital monitor. In any of the above or other examples, the physical state of the pupil is selected from one or more of: (i) pupillary movement of one or more pupils, (ii) limbus, (iii) line of sight, and / or (iv) visual axis of the subject. In any of the above or other examples, the field of view camera includes at least one field of view camera extending inwardly from an outer surface of the wearable glasses. In any of the above or other examples, the field of view camera includes at least one field of view camera extending outwardly from an outer surface of the wearable glasses. In any of the above or other examples, in the vision mode, the field of view camera captures a plurality of sequential images of the field of view.

[0021] In any of the above or another example, the plurality of test stimuli includes at least one test image of a text or an object. In any of the above or another example, the one or more lesion areas include areas of reduced visual sensitivity or areas of high or low order optical aberrations. In any of the above or another example, the one or more lesion areas include areas of reduced brightness. In any of the above or another example, the plurality of stimuli differ from one another and from a reference contrast level by at least 20 dB. In any of the above or another example, the plurality of stimuli differ from one another and from a reference contrast level by at least 30 dB. In any of the above or another example, the image processing device stores instructions that, when executed, cause the processor to perform the following operations: Instruct the wearable eyewear device to display the plurality of test stimuli to the subject in a test mode with decreasing or increasing contrast.

[0022] According to another aspect, a vision system includes a wearable eyewear device, at least one digital monitor, at least one field of view camera, and an image processor.

[0023] In some examples, the at least one digital monitor is configured to display an image to the subject's eyes. In one example, the at least one field of view camera may be configured to capture a plurality of monocular images of a scene, each of the monocular images being offset from the other monocular images. In one example, the image processor may include a processor and a memory and may be coupled to the at least one digital monitor. The image processor may have instructions stored in the memory that, when executed, cause the processor to: combine the plurality of monocular images to create a composite image having a field of view larger than the field of view of any of the plurality of monocular images. In any of the above or other embodiments, the instructions may cause the processor to display the composite image on the at least one digital monitor to present the subject with an increased field of view of the scene.

[0024] In any of the above or other examples, the image processing device has instructions stored in memory that, when executed, cause the processor to perform the following operations: combine the monocular images to create a composite image by performing a selective field of view shift on at least one monocular image of the plurality of monocular images relative to the other monocular images, and generate a wider peripheral area for the composite image. In any of the above or other examples, the image processing device has instructions stored in memory that, when executed, cause the processor to perform the following operations: A peripherally selective viewing operation is performed on at least one of the plurality of monocular images relative to the other monocular images to combine the plurality of monocular images to create a composite image.

[0025] In any of the above or other examples, the peripherally selective vision operation includes performing a shrink or enlargement on a peripheral or central macular region of the plurality of monocular images. In any of the above or other examples, the image processing device has instructions stored in a memory that, when executed, cause a processor to perform the following operations: combine the plurality of monocular images into a composite image by identifying a defective field of vision in at least one of the plurality of monocular images, imaging the defective field of vision, moving the imaged defective field of vision into a non-defective field of vision, and forming a composite image for display to the subject to include the imaged and moved defective field of vision.

[0026] In any of the above or other examples, the image processing device has instructions stored in the memory that, when executed, cause the processor to: identify a common central region for each of the plurality of monocular images, identify different peripheral regions of the plurality of monocular images to combine the plurality of monocular images to create a composite image, and form the composite image to include a first region corresponding to the common central region and a second region formed by combining the different peripheral regions to form a wider peripheral region surrounding the first region. In any of the above or other examples, the image processing device has instructions stored in the memory that, when executed, cause the processor to: form the composite image such that the second region corrects for visual field defects and aberrations in the subject's eye. In any of the above or other examples, the at least one digital monitor includes a first digital monitor and a second digital monitor each configured to display the composite image to a corresponding eye of the subject.

[0027] In any of the above or other examples, the image processing device may include instructions stored in a memory that, when executed, cause a processor to perform the following operations: perform a fisheye transform on a first region of the plurality of monocular images to correct radial components of the plurality of monocular images according to the following equation: r new =r+αr 3 where α is a constant.

[0028] In any of the above or other examples, the image processing device has instructions stored in a memory that, when executed, cause a processor to perform the following operations: Perform a conformal mapping transformation on the plurality of monocular images to modify a radial component according to the following equation: r new =rβ β is a constant exponent for the radial component, β>1.

[0029] In any of the above or other embodiments, the image processing device may have instructions stored in the memory that, when executed, may cause the processor to: perform a polynomial transform to map points from a wide annulus surrounding a center of the multiple monocular images to a narrower annulus to form a composite image;

[0030] According to yet another aspect, an apparatus may include wearable glasses having a housing. The wearable glasses may include a controllable projector configured to project a patterned image onto a retina of a subject. The apparatus may further include an image processor having a processor, a memory, and an input device. The image processor may be coupled to the controllable projector.

[0031] In some examples, the image processing device is configured to: receiving at an input device a visual score signal indicative of a patterning image perceived by the subject's retina; (B) analyzing the visual score signal and determining the presence or absence of a distortion experienced by the retina based on the visual score signal, and if distortion is present, determining a pattern adjustment to the patterned image based on the visual score signal; (C) Adjusting the patterned image based on the pattern adjustment to form a modified patterned image, projecting the modified patterned image onto the retina, and performing (A) again.

[0032] In any of the above or other examples, the corrective imaging element is an adjusted intensity of a peripheral element relative to a central image region of the viewed scene, or an adjusted intensity of a central element relative to a peripheral image region of the viewed scene. In any of the above or other examples, the image processor is configured to adjust a position and / or configuration of the corrective imaging element in response to detected eye movements of the subject. In any of the above or other examples, the image processor is configured to identify one or more diseased areas in one or both eyes of the subject and determine a corrective imaging element that compensates for the one or more diseased areas.

[0033] According to yet another aspect, the device may include a wearable eyewear device having at least one optical element for transmitting an image of a viewed scene to a subject. The wearable eyewear device may further include at least one digital monitor corresponding to the at least one optical element. The at least one digital monitor is configured to overlay a corrective imaging element onto the image of the viewed scene of the at least one optical element. The device may further include an image processing device having a processor and a memory. The image processing device may be coupled to the at least one digital monitor.

[0034] In one example, the image processing device is configured to generate corrective imaging elements as peripheral elements of an image of a viewed scene to correct for peripheral vision defects, or to generate corrective imaging elements as central elements of an image of a viewed scene to correct for central vision defects. In any of the above or other examples, the image processor may be configured to display to the subject a corrective imaging element overlaid on the viewed scene.

[0035] In any of the above or other examples, the corrective imaging element is an adjusted intensity of a peripheral element relative to a central image region of the viewed scene, or an adjusted intensity of a central element relative to a peripheral image region of the viewed scene. In any of the above or other examples, the image processor is configured to adjust a position and / or configuration of the corrective imaging element in response to detected eye movements of the subject. In any of the above or other examples, the image processor is configured to identify one or more diseased areas in one or both eyes of the subject, and determine a corrective imaging element that compensates for the one or more diseased areas.

[0036] In any of the above or other examples, the image processing device is configured, in a test mode, to (i) direct at least one digital monitor to display a plurality of test stimuli to the subject at one or more test locations in the test field of view, (ii) direct an image sensor of the device to acquire pupil physical state and / or gaze position indications while displaying the plurality of test stimuli at the one or more test locations, and (iii) determine one or more lesion areas in the test field of view and determine one or more visual disorders of the subject. In any of the above or other examples, the plurality of stimuli have different contrast levels from each other and from a reference contrast level.

[0037] In any of the above or other examples, the at least one digital monitor is housed in the at least one layer of optical elements. is an inner or outer layer of at least one optical element. [Brief description of the drawings]

[0038] The figures described below depict various aspects of the systems and methods disclosed herein, and it should be understood that each figure illustrates an example of an aspect of the systems and methods.

[0039] [Figure 1A]1 illustrates an example eyewear device according to various embodiments described herein. [Figure 1B] 1 illustrates an example eyewear device according to various embodiments described herein. [Figure 1C] 1 illustrates an example eyewear device according to various embodiments described herein.

[0040] [Diagram 2] FIG. 1 illustrates a schematic diagram of an example of a vision system according to various embodiments described herein.

[0041] [Diagram 3] FIG. 2 is a schematic diagram illustrating a vision correction framework device implemented in a wearable eyewear device and an image processing device according to various embodiments described herein.

[0042] [Figure 4] FIG. 1 illustrates an example process including an inspection mode and a vision mode according to various embodiments described herein.

[0043] [Diagram 5] FIG. 1 illustrates an example process including an inspection mode and a vision mode according to various embodiments described herein.

[0044] [Figure 6A-6C] FIG. 1 illustrates an example of an evaluation protocol for a test mode process including pupil tracking, according to various embodiments described herein.

[0045] [Figure 7A-7C] FIG. 1 illustrates an example of an evaluation protocol for a test mode process including pupil tracking, according to various embodiments described herein.

[0046] [Figure 8]FIG. 13 is a schematic diagram illustrating a workflow including a testing module that generates and presents a plurality of visual stimuli to a user by a wearable eyewear device according to various embodiments described herein.

[0047] [Figure 9] FIG. 1 illustrates an inspection mode process according to various embodiments described herein.

[0048] [Figure 10] FIG. 13 illustrates a process for an artificial intelligence correction algorithm mode that may be implemented as part of an inspection mode according to various embodiments described herein.

[0049] [Figure 11] 1 illustrates an inspection image according to various embodiments described herein.

[0050] [Figure 12] FIG. 13 illustrates the generation of a simulated visual acuity image, including overlaying a defective visual field onto a test image presented to a subject, according to various embodiments described herein.

[0051] [Figure 13] 1A-1C are diagrams illustrating examples of various correction transformations presented to a subject and applied to an image, according to various embodiments described herein.

[0052] [Figure 14] FIG. 2 illustrates an example of a translation method according to various embodiments described herein.

[0053] [Figure 15] FIG. 1 illustrates a schematic diagram of an example machine learning framework according to various embodiments described herein.

[0054] [Figure 16]FIG. 1 illustrates a process of an AI system of a machine learning framework in accordance with various embodiments described herein.

[0055] [Figure 17] 1A-1C illustrate example transformations of an inspection image according to various embodiments described herein.

[0056] [Figure 18] 1A-1C are diagrams illustrating example translations of an inspection image according to various embodiments described herein.

[0057] [Figure 19] 1 is a graphical user interface illustrating various implementations of an AI system according to various embodiments described herein.

[0058] [Figure 20] FIG. 1 illustrates a schematic framework of an AI system including a feedforward neural network according to various embodiments described herein.

[0059] [Figure 21] FIG. 1 illustrates an example of a testing mode process for an AI system including an AI neural network, according to various embodiments described herein. [Figure 22] FIG. 1 illustrates an example of a testing mode process for an AI system including an AI algorithm optimization process, according to various embodiments described herein.

[0060] [Figure 23] FIG. 1 illustrates an example process for implementing an inspection mode and a vision mode according to various embodiments described herein.

[0061] [Figure 24]FIG. 1 illustrates a wearable eyewear device comprising custom reality wearable glasses that pass images from the external environment through a portion thereof, specifically, pass the peripheral vision of the viewer while blocking the central region, in accordance with various embodiments described herein.

[0062] [Diagram 25] FIG. 1 illustrates a wearable eyewear device comprising custom reality wearable glasses that pass images from the external environment through a portion thereof, specifically, through a central region of the viewer while blocking peripheral vision regions, in accordance with various embodiments described herein.

[0063] [Figure 26] FIG. 1 illustrates a subject's normal binocular vision, showing how a monocular image from the left eye and a monocular image from the right eye are combined to generate a single perceived image that includes a central macular region and the peripheral vision region surrounding the central region.

[0064] [Figure 27] FIG. 1 illustrates tunnel vision, a condition in which peripheral areas are not visible to the subject.

[0065] [Figure 28] 1 illustrates an image shifting technique for correcting tunnel vision or improving vision, according to various embodiments described herein.

[0066] [Figure 29] FIG. 2 is a diagram illustrating an image resizing transformation technique for improving vision or preserving central vision while expanding the field of view, according to various embodiments described herein.

[0067] [Diagram 30] FIG. 1 illustrates a binocular field expansion technique according to various embodiments described herein.

[0068] [Figure 31A]FIG. 1 illustrates a technique for assessing dry eye and corneal abnormalities that involves projecting a pattern onto the corneal surface and imaging the corneal surface reflecting the pattern, according to various embodiments described herein.

[0069] [Figure 31B] FIG. 13 is a schematic diagram illustrating the presentation of a reference image including a grid pattern displayed to a subject or projected onto the subject's cornea or retina by wearable glasses according to various embodiments described herein.

[0070] [Figure 31C] 1A-1C are diagrams illustrating examples of subject-operated grid patterns according to various embodiments described herein.

[0071] [Figure 31D] FIG. 31D illustrates an example of the manipulation of the grid pattern shown in FIG. 31C according to various embodiments described herein.

[0072] [Figure 31E] FIG. 2 illustrates a scene to be sensed by a subject, according to various embodiments described herein.

[0073] [Fig. 31F] FIG. 31C shows an example of a corrected visual field that, when provided to a subject with visual distortion determined by grid pattern technology, would result in the subject perceiving the visual field shown in FIG. 31E, in accordance with various embodiments described herein.

[0074] [Figure 31G] FIG. 13 illustrates a display including a steerable grid pattern for a subject to communicate distortions in their field of view, according to various embodiments described herein.

[0075] [Diagram 32]3A-3C show images of the corneal surface reflecting a pattern projected onto the corneal surface according to various embodiments described herein.

[0076] [Diagram 33] 1A-1C are diagrams illustrating examples of normal pattern reflections according to various embodiments described herein.

[0077] [Diagram 34] 1A-1C are diagrams illustrating examples of anomalous pattern reflections according to various embodiments described herein.

[0078] [Figure 35A] FIG. 13 illustrates a fast thresholding technique for a test mode using a 52 stimulus sequence at predetermined locations to cover a central 40 degree radius and including stimuli at four levels of contrast, according to various embodiments described herein.

[0079] [Figure 35B] FIG. 2 is a timing diagram illustrating five stages (a-e) of a test sequence at one stimulation position according to various embodiments described herein.

[0080] [Diagram 36] FIG. 13 is a diagram illustrating the calculation of pixel width and height that bounds the maximum bright field according to various embodiments described herein.

[0081] [Figure 37] 1A-1C are diagrams illustrating width and height maps according to various embodiments described herein.

[0082] [Figure 38] 1A-1C illustrate test images used to test the four main quadrants of the field of view according to various embodiments described herein.

[0083] [Figure 39A]1A-1C are diagrams illustrating an example of a field of view before remapping, according to various embodiments described herein.

[0084] [Figure 39B] 4A-4C are diagrams illustrating examples of fields of view after remapping according to various embodiments described herein.

[0085] [Figure 40A] FIG. 1 illustrates an example of a custom reality glasses device according to various embodiments described herein. [Figure 40B] FIG. 1 illustrates an example of a custom reality glasses device according to various embodiments described herein. [Figure 40C] FIG. 1 illustrates an example of a custom reality glasses device according to various embodiments described herein. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0086] The present application provides techniques and devices for screening, identifying and compensating for eye diseases that affect a patient's vision, including diseases of the optic nerve, such as glaucoma, optic neuritis and optic neuropathy, diseases of the retina, such as macular degeneration and retinitis pigmentosa, diseases of the visual pathway, such as capillary disorders and tumors, and other conditions, such as presbyopia, strabismus, higher and lower order aberrations, monocularism, anisometropia and anisometropia, photosensitivity, anisocoria, refractive errors and astigmatism.

[0087] The technology described herein provides vision systems, eyewear devices, and related systems and devices that test, improve and / or correct visual field perception or visual acuity.

[0088] One or more devices of the vision system may be configured for use within one or more of the systems described herein or may be configured for separate use. For example, in various embodiments, the vision system comprises an eyewear device. The devices described herein may be considered to comprise one or more systems having one or more devices. In this manner, a device may include one or more associated systems or devices.

[0089] The vision system may include an image processing device (which may also be referred to as an image processor, computing device, etc.) configured to perform image processing for the vision system as described herein. As described herein, the image processing device may be fully or partially integrated with the eyewear device, or may be fully or partially external, e.g., remote, to the eyewear device. Such an external image processing device may be configured to communicate with the eyewear device in a wired or wireless manner.

[0090] Exemplary embodiments of the eyewear device include wearable eyewear devices. Some embodiments of the eyewear device may employ digital methods for imaging, image processing, communication, display, or one or more of the other functions described herein. Various embodiments of the eyewear device may be configured, alone or in combination with other systems or devices, to provide a subject with a customized, visually corrected field of vision. In some examples, the eyewear device may be configured to provide digital therapeutic corrective glasses (see herein). An example eyewear device may include a wearable digital eyeglasses for use by an individual for non-therapeutic correction purposes. For example, the eyewear device may be configured to improve a subject's normal vision, field of vision, or perception of field of vision, for example, by zooming in or out of the field of vision, changing the horizontal, vertical, and / or diagonal angle of view, modifying the light provided to one or more regions, changing the size of an object or region within one or more regions of the field of vision, and / or relocating an object or region to another region of the field of vision. The eyewear devices described herein may be activated by voice activation, remote control (e.g., a mobile phone), or body movement (e.g., a blink or two clear blinks), to name a few examples.

[0091] An embodiment of the vision system or eyewear device may include one or more digital monitors. The vision system or eyewear device may further include one or more image sensors. In some embodiments, the image sensor may include one or more outward-facing image sensors for imaging a visual environment of a subject (which may also be referred to as a user, wearer, or patient). The visual environment of the subject may typically correspond to a field of view obtained by the subject's two eyes, but in some configurations may be a field of view obtained from other origins. The outward-facing image sensor may include, for example, one or more cameras positioned to fully or partially image one or more fields of view. Such fields of view may include a large portion of a field of view for a person. In these or other embodiments, the one or more image sensors may include one or more inward-facing image sensors for imaging the subject from various angles, such as imaging the physical state of the subject's pupils. For example, the eyewear device may include an inward-facing image sensor, such as a camera (e.g., visible light camera, infrared camera, etc.) that captures and tracks the subject's line of sight, limbal and pupil data, the subject's corneal data, retinal images, and images of patterns reflected by the cornea or retina. The line of sight, also known as the visual axis, may also be obtained by tracking the pupil, the limbus (the edge between the cornea and the sclera), or by tracking blood vessels on the surface of the eye or within the eye. Thus, the image sensor may be used to image the limbus, blood vessels, and pupil.

[0092] Some vision systems or eyewear devices may include one or more displays. Such displays may be referred to as digital monitors. Digital monitors may include monitors that generate a display on a screen, which may include projection onto a screen, including a head-up display. Alternatively, digital monitors may include monitors for projecting a display onto one or both eyes of a subject. For example, eyewear devices may include one or more digital monitors that display an image to a subject. These or other vision systems or eyewear devices may include projectors configured to display an image to a subject by projecting an image onto a monitor, e.g., a screen such as glasses, or onto the subject's eyes, such as retinal projection. In some examples, the device includes a headset with two miniature external viewfinder cameras. The headset may include, for example, a wearable eyewear device as described herein. In some examples, the eyewear device may include eyewear devices configured to regenerate the entire field of view as a digitally modified image for an optimized rendition of the field of view. In some examples, the vision system or eyewear device may include eyewear devices including an alternative reality (AR) headset or a virtual reality (VR) headset. In these or other examples, the systems and devices described above may include an eyewear device in which the field of view is what the user sees, but the field of view is modified by introducing a modified image.

[0093] In some examples, the vision system or eyewear device is configured to process and / or display images to correct low-order and / or high-order aberrations and / or refractive errors, thereby providing the subject with improved, customized vision. In some examples, a system or device including an eyewear device may be configured to treat a number of ocular abnormalities, including various types of diagnosable conditions related to one or more of visual field defects, reduced visual acuity, visual field distortion, secondary effects, and diplopia. Ocular abnormalities that may be corrected by treatment of a system or device described herein may include, but are not limited to, one or more of the following conditions: presbyopia, diplopia due to strabismus, glaucoma, age-related macular degeneration, monocular vision, anisometropia and anisometropia, light sensitivity, anisocoria, glaucoma, diseases of the optic nerve such as optic neuritis and optic neuropathy, diseases of the retina such as macular degeneration, retinitis pigmentosa, diseases of the visual pathway such as capillary disorders and tumors, and conditions such as presbyopia, strabismus, high and low order aberrations, refractive errors, and astigmatism.

[0094] In example embodiments, the vision system or eyewear device may be configured to enhance and / or modify an image to be displayed to the subject, either by digitally recreating or by enhancing the field of view. In example embodiments, the eyewear device may include one or more projectors configured to project the digitally regenerated or augmented image onto the subject's eye by projecting onto the retina via retinal projection.

[0095] In example embodiments, the vision system or eyewear device may be configured to modify or improve the subject's field of vision, for example to correct or increase the subject's viewing angle. In some examples, the central and peripheral viewing regions are affected differently to improve the subject's viewing angle (e.g., by zooming in or out of an image projected or displayed to the subject's eye) or to increase the detail perceived by the subject.

[0096] In example embodiments, the vision system or eyewear device may be configured to compensate for local brightness changes in the patient's visual field, for example as determined from the results of a visual field test performed by the eyewear device or separately from the eyewear device. The eyewear device may be configured to compensate by increasing brightness in areas of the visual field with respect to normal sensitivity. In some examples, the eyewear device or associated system is configured to align and track such areas of low sensitivity using the pupillary axis and the visual axis. The eyewear device or associated system described herein employs compensation techniques in these areas of low sensitivity such that the subject perceives a homogenous image. The compensation techniques remove the local haze that the subject has with respect to the areas of low sensitivity, improving vision and increasing the subject's functional visual field.

[0097] In example embodiments, the vision system or eyewear device may have a test mode, for example, to identify and test the visual acuity or functional visual field of a subject. In this or other embodiments, the eyewear device may have a visual mode, for example, to improve or correct the visual acuity or visual field in real time and / or on an individualized basis. In some embodiments, the eyewear device or associated system has both a test mode and a visual mode, and may be configured to utilize follow-up or maintenance test procedures to streamline reprogramming of the visual mode process as the visual acuity of the subject changes. In some embodiments, the eyewear device may include a programming interface configured to receive updates related to the test mode process and / or the visual mode process. For example, the programming interface may include a wired or wireless communication port including a receiver or transceiver. In some embodiments, the eyewear device may be configured to receive updates including test results performed by a test mode of the system or another system or device for incorporation into the visual mode process. In some embodiments, the updates may include data or instructions provided by the subject, for example, via a user interface in signal communication with the programming interface via the communication port. Such data or instructions may be provided in a test mode, including a feedback mode, or similarly in a feedback mode, as described herein. During operation of the visual mode, including the eye tracking mode, the user may provide the visual information by interacting with a user interface, including a tablet, smartphone, computer, or peripheral device. Some embodiments may include a user interface attached to the eyewear device. Examples include switches, touch sensors, capacitive sensors, or other interfaces through which the user may provide or adjust parameters related to the visual acuity or correction profile that the visual mode uses to process and present images to the subject.

[0098] In example embodiments, the vision system or eyewear device may include one or more outward-facing image sensors, e.g., cameras, arranged to capture the subject's visual field and, after image processing, display the image on a monitor, e.g., a display screen or glasses, or project the image onto the eye of the subject wearing the eyewear device. Image processing may include customizing the image to treat and / or correct the above-mentioned conditions or to improve visual acuity or functional vision. As mentioned above, the eyewear device may include or be associated with one or more inward-facing image sensors, e.g., cameras, that observe the subject's eyes, line of sight, pupil size, and / or limbus position for the purpose of aligning and / or adjusting the above-mentioned correction or improvement.

[0099] In example embodiments, the vision system or eyewear device may be configured to dynamically correct low-order and / or high-order visual aberrations. The technique may detect changes in pupil size, accommodation and gaze and modify the visual aberration correction profile accordingly. The high-order and / or low-order aberrations may be obtained using an aberrometer in conjunction with pupil size, accommodation and gaze direction, allowing the eyewear device to generate such a dynamic correction profile. Images projected to the subject using the techniques described herein may be inversely distorted depending on the subject's actual aberrations, so that the subject's own aberrations are inverted again to obtain best vision. In some embodiments, a technique may be implemented to detect the accommodation state by detecting signs of the pupil near reflex, i.e., miosis (pupils becoming smaller) and accommodative convergence (pupils turning inwards). For example, the eyewear device may include a pupil for detecting pupil size and / or an eye-tracking device for detecting gaze direction. These inputs allow the technique to detect the correction profile to be displayed.

[0100] In example embodiments, the techniques of the present invention may be implemented to provide vision correction by automatically auto-focusing images displayed via one or more monitors to provide near vision. To further enhance and improve near vision, an inward-facing image sensor, e.g., a camera, may detect whether a subject is attempting to view a nearby object by detecting signs of pupillary near reflex, miosis (pupils become smaller) and accommodative convergence (eyes move inward), and automatically auto-focus to improve near vision. Near vision correction for reading a newspaper is different than, for example, viewing a computer monitor. Example eyewear devices and / or related systems described herein may be configured to determine how far away an object is by quantifying the subject's pupillary near reflex, and thereby provide a corresponding focus correction.

[0101] In example embodiments, the vision system or eyewear device may be configured to dynamically correct double vision associated with strabismus. For example, pupil and gaze tracking may work in conjunction with an inward-facing image sensor to track the pupil, limbus, or ocular tissues, such as blood vessels, of the subject and the gaze. This tracking may be used to dynamically correct images displayed to the subject, e.g., projected or projected onto one or more monitors, to compensate for the strabismus and suppress double vision in all gaze directions. may be used to dynamically inform the subject of any misalignment of the image displayed or projected onto the subject's eye.

[0102] In example embodiments, the vision system or eyewear device may be configured to improve the safety and vision of a patient with visual field defects, e.g., a glaucoma patient. Such subjects may be missing a portion of their visual field. For example, if a vehicle or person is in the subject's blind spot, the subject cannot see the vehicle or person. The vision system and eyewear device described herein may be implemented to correct such blind spots. For example, the visual field defect may be detected using a visual field testing mode of the vision system or eyewear device. In some examples, the software executed by the example systems and devices described herein may be configured to redistribute an image captured by an outward-facing image sensor, e.g., a camera, to the subject's actual functional visual field. The actual visual field may be dynamically projected with reference to the pupil or line of sight, e.g., using data obtained from pupil and eye gaze tracking. In other words, the technology of the present invention may improve the safety and function of the subject by moving an image of a vehicle or person in the subject's blind spot to a position outside the subject's blind spot.

[0103] For patients with age-related macular degeneration or other conditions affecting the macula of the eye who have a central blind spot, the visual system or eyewear device may be configured to distribute the image or a portion thereof to a peripheral or paracentral portion of the patient's functional field of vision. Techniques according to the present invention may, for example, project a portion of the target image onto a normal portion of the retina and avoid abnormal portions of the retina. In some examples, the visual system or eyewear device may have a test mode that partitions visible and invisible portions of the visual field that is utilized when modifying the image to direct the distribution of the image.

[0104] For patients with monocular vision or low vision in one eye, the vision system or eyeglass device may image the normal binocular field of view and distribute the normal binocular field of view to the actual functional field of view of both eyes to allow the patient to obtain the widest possible field of view. In practice, these eyeglass devices may be implemented to enhance the field of view of normal subjects for military purposes and other applications to enhance the field of view of the subject. For example, eyeglass devices may be implemented to enhance the field of view of subjects such as athletes, physicians, drivers, etc.

[0105] Anisometropia occurs due to the unequal refractive power of each of a subject's eyes. In various embodiments, a vision system or eyewear device may be configured to correct the anisometropia by modifying image size to generate multiple images of uniform size and displaying or projecting the images to both eyes, thereby avoiding visual impairments.

[0106] Unlike eyeglass lenses, which introduce distortions into the field of vision, such as shrinking or magnifying the image of an object, the techniques of the present invention can be used independently of corrective lenses and therefore do not affect the subject's field of vision.

[0107] In some examples, the vision system or eyewear device may be configured to display or project light regardless of the brightness of the surrounding environment. In one example, the displayed or projected light may be adjusted automatically depending on the pupil size detected by the system and / or device, or may be adjusted manually at the patient's request, for example, by a user interface coupled to, e.g., in signal communication with, the eyewear device. Pupils tend to constrict more in bright environments and dilate in less bright environments. As mentioned above, the systems and devices described herein may be configured to detect the degree of constriction / dilation and adjust the brightness accordingly. This may be customized for each individual. Subjects with anisocoria may, for example, For example, the techniques of the present invention may be used to allow separate brightness adjustments for each eye, which in some examples may be performed automatically by a system or device upon detecting pupil size.

[0108] FIG. 1A illustrates an example of an eyewear device 100 forming a wearable device for a subject. In some embodiments, the eyewear device 100 may be part of a vision system as described herein. The eyewear device 100 includes a left eye lens 102 and a right eye lens 104. Each of the eye lenses 102 and 104 may include and / or be associated with a digital monitor configured to display (or project) a reproduced image to the corresponding eye of the subject. In various embodiments, the digital monitor may include a display screen, a projector, and / or hardware for displaying an image on a display screen. It is contemplated that a digital monitor including a projector may be located at other locations to project an image to the subject's eye or to the eye lenses including a screen, glasses, or other image projection surface. In one embodiment, the left eye lens 102 and the right eye lens 104 may be positioned relative to the housing 106 to fit an annular region of the subject so as to be able to collect data and display / project image data. This includes, in another example, displaying / projecting image data to different eyes.

[0109] Each of the eye lenses 102, 104 may further include one or more inward-facing sensors 108, 110, which may be inward-facing image sensors. In one example, the inward-facing sensors 108, 110 may include an infrared camera, receiver, or other infrared sensor configured to track pupil movement and determine and track the subject's visual axis. The inward-facing sensors 108, 110 may include, for example, an infrared camera and may be positioned lower relative to the eye lenses 102, 104 so as not to obstruct the subject's field of view, either the actual field of view or the field of view displayed or projected to the subject. The inward-facing sensors 108, 110 may be oriented toward the expected pupil area to better track the pupil and / or gaze. In some examples, the inward-facing sensors 108, 110 may be embedded in the eye lenses 102, 104 to form a contiguous inner surface.

[0110] FIG. 1B is a front view of the eyewear device 100, showing the front view of the eyepieces 102, 104. The eyepieces 102, 104 are provided with corresponding outward-facing image sensors 112, 114 constituting field-of-view cameras. In other embodiments, more or less outward-facing image sensors 112, 114 may be provided. The outward-facing image sensors 112, 114 may be configured to capture a number of successive images. The eyewear device 100 or a corresponding vision system may further be configured to subsequently correct and / or enhance the images. This may be individualized based on the subject's visual impairment. The eyewear device 100 may further be configured to display the corrected and / or enhanced images to the subject using a monitor in the vision mode. For example, the eyewear device may generate the corrected and / or enhanced images on a display screen associated with the eyepieces or an adjacent area, project the images on a display screen associated with the eyepieces or an adjacent area, or project the images onto one or both eyes of the subject.

[0111] 1C is a diagram showing an example of an eyewear device 100 after construction, with two digital monitor-containing eyepieces 102, 104, along with focusing lenses 116, 118. In this example, only one inward light sensor 110 is provided for pupil and eye tracking, although in other examples, multiple inward light sensors 110 may be provided.

[0112] In an example embodiment, the eyewear device 100 may have a test mode. In one example test mode, the inward facing sensors 108, 110 track pupil movement to perform visual axis (e.g., line of sight) tracking according to a test protocol. In this or another example, the inward facing sensors 108, 110 may be configured to capture reflections of patterns reflected by the cornea and / or retina to detect distortions and abnormalities in the cornea or ocular optics.

[0113] The examination mode may be used to perform vision evaluations to identify eye disorders, such as high and / or low order aberrations, disorders of the optic nerve, such as glaucoma, optic neuritis and optic neuropathy, disorders of the retina, such as macular degeneration, retinitis pigmentosa, disorders of the visual pathway, such as capillary disorders and tumors, and other conditions, such as presbyopia, strabismus, high and low order aberrations, monocular vision, anisometropia and anisometropia, light sensitivity, anisometropia, refractive errors and astigmatism. In the examination mode, data is collected for a particular subject and used to correct captured images before displaying them. Displaying an image may include projecting it onto the subject via a monitor, as described herein.

[0114] In some cases, external sensors may be used to provide additional data for evaluating the subject's visual field. For example, data used to correct the captured image may be obtained from external testing devices such as a visual field tester, an aberrometer, an electro-oculogram or a visual evoked potential device. Data obtained from these devices may be combined with pupil or eye tracking to determine the visual axis to generate a correction profile used to correct the image projected or displayed to the viewer.

[0115] In addition to or instead of the inspection mode, the eyewear device 100 may have a vision mode. In the vision mode, one or more outward-facing image sensors 112, 114 capture images that are transmitted to an image processor for real-time image processing. The image processor may be embedded, for example, within the eyewear device 10, for example, integrated or attached, or may alternatively be external to the eyewear device 100, for example associated with an external image processing device. The image processor may be a component of a vision module and / or may include a scene processing module as described herein.

[0116] The eyewear device 100 may be communicatively coupled to one or more image processors in a wired or wireless manner, for example via a wireless transceiver embedded in the eyewear device 100. The external image processor may include a computer, such as a laptop computer, a tablet, a mobile phone, a network server, or other centralized or distributed computer processing device, and may be characterized by one or more processors and one or more memories. According to the described example, the captured images are processed by the external image processing device. However, in other examples, the captured images may be processed by an image processor embedded in the digital glasses. The processed images, for example, enhanced to improve the functional visual field or other visual acuity and / or enhanced to correct the subject's visual field disease, are then transmitted to the eyewear device 100 and displayed by a monitor for the subject to view.

[0117] In one example of the operation of a vision system including an eyewear device, real-time image processing of captured images may be performed by an image processor using, for example, customized MATLAB® (Mathworks, Natick, Massachusetts, USA) code running on a small computer embedded in the eyewear device. In another example, the code may be executed by an external image processing device or by another computer that is wirelessly networked to communicate with the eyewear device. In one embodiment, the vision system includes a 3D image processing system. The vision system includes an eyewear device, an image processor, and corresponding instructions for performing a vision mode and / or an inspection mode, which may be embodied in the eyewear device alone or in combination with one or more external devices, such as a laptop computer. Such a vision system may operate in two modes: a vision mode and another inspection mode.

[0118] FIG. 2 illustrates an example of a vision system 200 including an eyewear device 202 communicatively coupled to a network 204 for communication with a server 206, a mobile phone 208, or a personal computer 210. The server 206, the mobile phone 208, or the personal computer 210 may each include a vision correction framework 212 for implementing the processing techniques described herein, such as image processing techniques, which may include techniques related to an examination mode and / or a visual mode. In the illustrated example, the vision correction framework 212 includes a processor and a memory for storing an operating system and applications for implementing the techniques described herein, and further includes a transceiver for communicating with the eyewear device 202 over the network 204. The framework 212 includes an examination module 214, which in this example includes a machine learning framework. The machine learning framework may be used in conjunction with an examination protocol performed by the examination module, either in a supervised or unsupervised manner, to adaptively adjust the examination mode to more accurately assess eye diseases. The results of the processing by the examination module may include the generation of a vision correction model 216 customized for the subject 218. The vision module 220, which in some embodiments may also include a machine learning framework with accessed customized vision correction models, generates corrected vision images for display by the eyewear device 202. The vision correction framework 212 may further include a scene processing module that processes images for use in inspection and / or vision mode processing and includes processing described herein with respect to the processing module. As described herein, in some embodiments, the eyewear device 202 may include some or all of the vision correction framework 212.

[0119] In the test mode, the eyewear device 100 or 202, and in particular one or more inward-facing image sensors including tracking cameras positioned along the inner surface of the eyewear device 100 or 202, may be used to acquire pupil and visual axis tracking data that is used to precisely align the processed images with the subject's pupil and visual axis.

[0120] 3 illustrates a vision system 300 that includes a vision correction framework 302. The vision correction framework 302 may be implemented in an image processor 304 and an eyewear device 306 that is placed on a subject. The image processor 304 may be contained entirely in an external image processor or other computer, or in other examples, some or all of the image processor 304 may be implemented within the eyewear device 306.

[0121] The image processing device 304 may have a memory 308 storing instructions 310 for performing the test mode and / or the vision mode described herein. Such instructions may include instructions for collecting high resolution images of the subject from the eyewear device 306. In the vision mode, the eyewear device 306 may acquire real-time visual field image data as raw data, processed data, or pre-processed data. In the test mode, the eyewear device may project test images (such as "text" or images of vehicles or other objects) to test various aspects of the subject's visual field.

[0122] The eyewear device 306 may be communicatively coupled to the image processing device 304 via a wired or wireless link. This link may be any suitable protocol, such as Universal Serial Bus (USB), IEEE 1394 (Firewire), Ethernet, or any other wired communication protocol. The wireless connection may be via a wireless device. The wireless connection may be via any suitable wireless communication protocol, such as WiFi, NFC, iBeacon, Bluetooth, Bluetooth Low Energy, etc.

[0123] In various embodiments, the image processing device 304 may have a controller operatively connected to a database via a link connected to input / output (I / O) circuitry. Additional databases may be connected to the controller in a known manner. The controller may include a program memory, a processor (also called a microcontroller or microprocessor), a random access memory (RAM), and input / output (I / O) circuitry, all interconnected via an address / data bus. Although only one microprocessor is described, it is contemplated that the controller may include multiple microprocessors. Similarly, the memory of the controller may include multiple RAMs and multiple program memories. The RAM(s) and program memory may be implemented as, for example, semiconductor memory, magnetically readable memory, and / or optically readable memory. A link may operatively connect the controller to the capture device via the I / O circuitry.

[0124] The program memory and / or RAM may store various applications (i.e., machine-readable instructions) executed by the microprocessor. For example, an operating system may generally control the operation of the vision system 300, such as the eyewear device 306 and / or the image processing device 304, and in some embodiments may provide a user interface to the device for implementing the processes described herein. The program memory and / or RAM may also store various subroutines for accessing specific functions of the image processing device described herein. By way of example and without limitation, the subroutines may include, among others, obtaining a high resolution image of the field of view from the eyewear device, enhancing and / or modifying the image, and providing the enhanced and / or modified image for display to the subject by the eyewear device 306.

[0125] In addition to the above, the image processing device 304 may include other hardware resources. The device may further include various types of input / output hardware, such as a visual display and input device(s) (e.g., keypad, keyboard, etc.). In one embodiment, the display is touch-sensitive and may work with a software keyboard routine as one of the software routines to accept user input. It may be beneficial for the image processing device to communicate with a wider network (not shown) using any of a number of known network devices and network technologies (e.g., via a computer network such as an intranet, the Internet, etc.). For example, the device may be connected to a database of aberration data.

[0126] <Example> "Text" inspection mode In the implementation of the vision system, tests were conducted on four subjects. The test protocol included displaying a string of characters at different positions on one or more display monitors of the eyeglass device. To assess the area of ​​visual field deficit of the subjects, the word "text" was displayed on the eyeglass monitor of each eye and the subjects were asked to identify "text". First, the operator intentionally placed the "xt" part of the word "text" in the subject's blind spot. All four subjects answered that they could only see the "te" part of the word. After this, the characters were moved using the software controlling the display. Specifically, the string "text" was moved away from the subject's blind spot and the subject was asked to read the word again. The subject was able to read "text" and was now able to identify the "x" part of the word. He said the "t" part became visible.

[0127] An example of this assessment protocol in test mode is shown in Figures 6A-6C. As shown in Figures 6A and 6B, the code automatically detects blind spots in the Humphrey visual field. The word "text" 600 is projected so that the "xt" part of the word is located in the blind spot 602 (Figure 6A). The subject is asked to read the word. The word "text" 600 is then moved away from the blind spot 602 (Figure 6B) and the subject is asked to read it again. The word "text" 600 can be displayed at several different coordinates in the subject's visual field, which in the illustrated example is divided into four coordinates. This protocol allows the identification of several blind spots, including the peripheral blind spot 604. A string of characters may be moved across the subject's visual field and the subject may be asked to identify when the string is partially or completely invisible, partially visible, or visible at a reduced intensity.

[0128] Pupil tracking functions as described herein may include the physical state of the pupil (e.g., visual axis, pupil size and / or limbus), alignment, dilation and / or line of sight. Line of sight, also known as the visual axis, may be obtained by tracking one or more of the pupil, the limbus (the edge between the cornea and the sclera), or by tracking blood vessels on the surface of the eye or within the eye. Thus, pupil tracking may include tracking of the limbus or blood vessels as well. Pupil tracking may be performed using one or more inward-facing image sensors as described herein.

[0129] In various embodiments, pupil tracking may be used to determine parameters for aligning the projected image with the subject's field of view (FIG. 6C).

[0130] A display of the GUI 606 may be displayed to the operator. The GUI 606 may provide information regarding the test. For example, the GUI 606 may indicate the measured visual field defect and indicate the relative location of the image to the defect. The GUI 606 may be configured to automatically distribute the images to functional portions of the visual field, but may include a button to allow the operator to override the automatic mode. The external image processor may be configured to determine where the evaluation string should be displayed, and may wirelessly communicate instructions to the digital glasses to display the string at various locations in the test mode.

[0131] <Example> "Image" inspection mode 7A-7C show another example of the test mode operation. In this example, instead of using "text", the subject was tested to determine whether they could see a vehicle 700 placed in different parts of their visual field in order to track their pupils and determine the lesion area. The pupil tracking function allows the vision system to align the projected image with the subject's visual field.

[0132] FIG. 4 illustrates a process 400 for illustrating an example implementation of both the test mode and the subsequent vision mode. In block 402, the test mode acquires data from diagnostic devices such as eyeglass devices and other user input devices, e.g., image sensors embedded inside a mobile phone or tablet PC. In block 404, test mode diagnostics may be performed to detect and measure ocular abnormalities from the received data, such as visual field defects, eye misalignment, pupil movement and size, and images of patterns reflected from the corneal or retinal surfaces. In one example, the control program and algorithms were implemented using MATLAB® R2017b (MathWorks, Inc., Natick, Massachusetts, USA). In various embodiments, the subject or examiner may be asked to test each eye individually or to test both eyes sequentially in one go. In some embodiments, the test mode may include an adapted fast thresholding scheme that utilizes a stimulus sequence at a predetermined location and includes contrast stepped stimuli covering a central 20 degree radius or greater. For example, the test mode may include an adapted fast thresholding scheme that utilizes a 52 stimulus sequence at a predetermined location and includes stimuli that are 4 levels of contrast covering a central 40 degree radius, as further described below with reference to Figures 35A and 35B.

[0133] In block 406, the determined diagnostic data may be compared to a database or data set storing correction profiles to compensate for identifiable eye diseases (see, e.g., FIG. 16 and associated discussion).

[0134] The identified correction profile may then be tailored to the individual to compensate for, for example, visual axis difference, visual field defects, light sensitivity, diplopia, image size changes between the eyes, image distortion, and visual acuity loss.

[0135] The personalized profile may be used by block 408 together with real-time data to process the image using, for example, an image processor, a scene processing module, and / or a vision module. The real-time data may include data detected by one or more inward-facing image sensors 410 providing pupil tracking data and / or data from one or more outward-facing image sensors including one or more field of view cameras 412 arranged to image a field of view screen. In block 414, real-time image correction may be performed and the image may be displayed on the eyewear device (block 416) as a regenerated digital image, or as an augmented reality image with the corrected portion superimposed and passing through the eyewear device, or as an image projected onto the subject's retina. In some examples, the processing of block 414 is performed in combination with a calibration mode 418. In the calibration mode 418, a user may use a user interface to adjust the image correction. The user interface may be an input device or the like that allows the user to control the image and the correction profile. For example, a user may shift one eye's image laterally, upwards, and downwards, or cycloterted to alleviate double vision. In these or other examples, a user may tweak the degree of field of view transformation (e.g., fisheye, polynomial, or conformal mapping) or translation, tweak brightness and contrast, or invert colors to increase the field of view without adversely affecting visual function or introducing unacceptable distortion.

[0136] FIG. 5 illustrates another example process 500. Process 500 provides an examination mode and a vision mode similar to the example process 400. Block 502 collects data on high-order and low-order aberrations as a function of pupil size, degree of accommodation, and gaze. In some embodiments, all or part of this data may be collected from an aberrometer or by taking images of a pattern or grid pattern projected onto the cornea and / or retina and comparing them with a reference image to detect aberrations of the cornea or the entire ocular optics, for example. The collected data may be sent to a vision correction framework. In block 504, the vision correction framework may determine an individualized correction profile similar to block 406 described above. Blocks 508 to 518 perform similar functions to corresponding blocks 408 to 418 of process 400.

[0137] 8 is a diagram illustrating a workflow 800 illustrating a test module 802 generating and presenting a plurality of visual stimuli 804 to a user 806 using an eyewear device with which the user interacts to provide input responses to the test stimuli. 808. In some examples, the user device 808 may include a joystick, electronic tally, keyboard, mouse, gesture detector / motion sensor, computer, phone such as a smartphone, dedicated device, and / or tablet PC that the user uses to interact to provide input responses to the test stimuli. The user device 808 may further include a processor and memory storing instructions. The instructions, when executed by the processor, generate a display of a GUI for the user to interact with. The user device 808 may include a memory, a transceiver (XVR) for transmitting and receiving signals, and an input / output interface for wired or wireless connection with the vision correction framework 810. The vision correction framework 810 may be stored in an image processing device. The vision correction framework 810 may be stored in a glasses device, a user device, etc., but in the illustrated example, the framework 810 is stored in an external image processing device. The framework 810 receives test mode information from the test module 802 and user input data from the user device 808.

[0138] FIG. 9 illustrates a test mode process 900 performed by the workflow 800. At block 902, a plurality of test stimuli are presented to the subject according to a test mode protocol. The stimuli may include images of text, images of objects, patterns such as blinking lights, grid patterns, etc. The stimuli may be displayed to the subject or projected onto the subject's retina and / or cornea. At block 904, the vision correction framework may receive detection data from one or more inward-facing image sensors, such as data corresponding to a physical state of the pupil (e.g., visual axis, pupil size, and / or limbus). Block 904 may further include receiving user response data collected from a user in response to the stimuli. At block 906, a pupil position state may be determined at a plurality of different stimuli, such as by measuring the difference in position and misalignment between one stimulus and another different stimulus.

[0139] At block 908, a determination of astigmatism may be made across the visual field, which may include analyzing pupil displacement data and / or the aberrations of the eye (e.g., projecting a reference image onto the retina and cornea and comparing the reflected image from the retinal or corneal surface to the reference image).

[0140] In block 910, the total aberration of the eye may be determined, for example, by projecting a reference image onto the retina and / or cornea and then comparing a reflected image from the retinal or corneal surface to the reference image (see, e.g., Figures 31A, 32-34 and associated discussion).

[0141] In block 912, visual distortions, such as optical distortions such as coma, astigmatism or spherical aberration, or visual distortions due to retinal disease, may be measured across the entire field of view.

[0142] At block 914, visual field sensitivity may be measured across the visual field.

[0143] In various embodiments of the process of FIG. 9, one or more of blocks 904 through 914 may be optional.

[0144] In some examples, the vision systems described herein can evaluate data obtained in the inspection mode to determine the type of ocular anomaly and the type of correction required. For example, FIG. 10 illustrates a process 1000 that includes an artificial intelligence correction algorithm mode that can be implemented as part of the inspection mode. A machine learning framework is loaded at block 1002. Example frameworks include, for example, dimensionality reduction, ensemble learning, meta-learning, These may include reinforcement learning, supervised learning, Bayesian, decision tree algorithms, linear classifiers, unsupervised learning, artificial neural networks, association rule learning, hierarchical cluster analysis, cluster analysis, deep learning, and semi-supervised learning.

[0145] In block 1004, the type of visual field defect is determined. Three examples of visual field defects are illustrated: uncompensated blind spot field 1006, low sensitivity partial blind spot 1008, and normal visual field 1010. After determining the visual field defect, block 1004 applies the appropriate correction protocol for the vision mode. For example, for the uncompensated blind spot field 1006, in block 1012, the vision correction framework tracks the vision using, for example, pupil tracking with an inward-facing image sensor, and performs video tracking of moving objects in the visual field using, for example, an outward-facing image sensor, for example, an external camera. In the illustrated example, in block 1014, a safety hazard in the blind spot area or a safety hazard moving into the blind spot area is detected, for example, by comparing the location of the safety hazard with the mapped visual field including the defect measured in the inspection mode. In block 1016, the object of interest may be monitored at various locations, including central and peripheral locations.

[0146] In the partial blind spot 1008 example, an enhanced vision mode may be initiated at block 1018. From this block, objects within the field of view may be monitored by tracking a central portion of the field of view. At block 1020, an image segmentation algorithm may be employed to separate the object from the field of view. Further, an enhanced contour may be applied to the object and displayed to the user, where the contour coincides with the identified outer edge of the segmented object.

[0147] For a normal visual field 1010, customized correction algorithms may be applied in block 1022 to correct for aberrations, field defects, esotropia, and / or visual distortions.

[0148] In example embodiments, artificial intelligence (AI) may be utilized in the examination and / or vision modes. For example, such technology may be built on the recognition that image warping (translation, translation, and resizing) methods to improve visual field can generate hundreds of different correction profiles. Much like fingerprints, different patients have different visual field defects. It has been found that some visual field defects can tolerate certain image warping in some patients but not others. Some image warping improves visual field but reduces central vision (e.g., central shrinkage). Thus, AI algorithms have been developed to address a wide range of conditions.

[0149] In one example, a vision correction framework having a machine learning framework including AI algorithms may be used to create an automated personalized correction profile by applying transformation, translation and resizing of the visual field to better match the remaining functional visual field. The machine learning framework may include one or more of data collection, visual field classification and / or regression models. A graphical user interface (GUI) and data collection program may be utilized to facilitate recording of participant responses, quantitative scores and feedback.

[0150] With respect to transformations applied to images in visual mode, example transformations of the machine learning framework may include one or more of: 1) conformal mapping transformation; 2) fisheye transformation; 3) customized fourth-order polynomial transformation; 4) polar polynomial transformation (using polar coordinates); or 5) rectangular polynomial transformation (using rectangular coordinates) (see, e.g., FIG. 13).

[0151] With respect to the translations applied to the images in the visual mode, examples may include one or more of the following: For example, the closest point may be determined by finding the point closest to the center location. The best center may be determined by one or more of the following: 1) the centroid of the largest component; 2) the center of the largest inscribed circle, the largest inscribed square, the largest inscribed rhombus, and / or the largest inscribed rectangle; or 3) the center of the locally largest inscribed circle, the largest inscribed square, the largest inscribed rhombus, and / or the largest inscribed rectangle (see, e.g., FIG. 14). For example, instead of searching for the largest shape, the framework may employ a weighted average of the closest points from multiple methods to avoid straying from the macular vision region.

[0152] In various embodiments, the AI ​​algorithm may be initially trained using simulated visual field defects. For example, a dataset of visual field defects may be collected to train the AI ​​algorithm. For example, in one experimental protocol, a dataset of 400 visual field defects was obtained from glaucoma patients. This dataset may be used to simulate visual field defects on virtual reality glasses to be presented to normal subjects for grading. Feedback from this grading may then be used to train the algorithm.

[0153] For example, an AI algorithm may be used to automatically fit the input image into regions that correspond to normal visual field patterns for each patient. In various embodiments, the algorithm may have at least three degrees of freedom in remapping the image, but may use more or fewer degrees of freedom. In one example, the degrees of freedom include translation, shifting, and resizing. Additional image transformations may be applied to preserve the quality of the central region of the image, which corresponds to central vision, where vision is highest, while concentrating peripheral regions of moderate quality. This may be applied so that the generated image is fully perceptible to the patient.

[0154] The image transformations included in the AI ​​algorithm may include one or more of a conformal mapping transformation, a polynomial transformation, or a fisheye transformation. In some embodiments, other transformations may be used. The machine learning technique may be trained on a sign dataset before actually performing the task. In one example, the AI ​​algorithm may be trained on a visual field dataset that includes various types of peripheral defects. For example, in one experiment, the dataset included 400 visual field defect patterns. After this training phase, normal participants assigned a quantitative score to the remapped images created by the AI ​​algorithm.

[0155] 11 illustrates an example test image (stimuli) image 1100. The test image 1100 may be designed to measure visual acuity, paracentral visual acuity, and / or peripheral visual acuity. The test image shown displays five letters in the central region, four inner diamonds 1102 in the paracentral region, and eight outer diamonds 1104 in the peripheral region, as illustrated in FIG. 11.

[0156] As mentioned above, a large amount of data is needed to be able to train an AI system. As a first step, a patient's binocular vision may be simulated using a binocular field including defects, as shown in Figure 12. The simulated visual acuity may then be presented to the subject using a glasses device. In this way, the input image can be manipulated using various image manipulations and then presented again to the subject to rate the modified appearance. The rectification process may continue, with the rectified images being further modified and presented to the subject until an optimal rectified image is determined. Figure 13 shows examples of different rectification transformations applied to the image to be presented to the user. Figure 14 shows examples of different translation methods (shifting the image to fit into the normal field of view). The normal field is shown in white. , the blind spot field is black.

[0157] The AI ​​system may be designed using machine learning models such as artificial neural networks and support vector machines (SVMs). In some examples, the AI ​​system is designed to generate an output including an estimate of the best image manipulation (i.e., geometric transformation and translation) by an optimization AI system. The vision system may present images manipulated according to the output image manipulation to the patient using a headset in a vision mode, such that the patient has the best possible vision based on the patient's field of vision, including the deficiency. The machine learning framework of the vision correction framework (also referred to herein as the "AI system") may be trained using collected data, for example, as described herein. A block diagram of an example AI system 1500 is shown in FIG. 15.

[0158] The process 1600 of the AI ​​system 1500 is shown in FIG. 16. The inputs to the system 150 include the test image and the visual field image. The AI ​​system 1500 estimates which geometric transformation is best performed on the test image to allow more details to be presented across the visual field. The AI ​​system 1500 then estimates which translation is best for the test image so that the displayed image covers most of the visual field. After this, the test image is transformed and translated as shown in FIG. 17 and FIG. 18, respectively. Finally, this image is combined again with the visual field for training purposes only for simulation, but is displayed as is to the patient during the testing phase. A screenshot of a graphical user interface presenting a summary of the visual field analysis, including an example of the final implementation of the visual field AI system, including the parameters of the image transformation and translation to be applied to the image, is shown in FIG. 19.

[0159] In an example implementation, a machine learning framework ("AI system") is implemented in a vision correction framework using an artificial neural network model. The AI ​​system acquires as a visual field image converted into a vector. As an output, the AI ​​system provides a prediction of image transformation and translation parameters to be applied to the scene image. These parameters are then used to manipulate the scene image. The AI ​​system has two hidden layers. Each hidden layer has three neurons (i.e., units) and an output layer. An example of such an AI system model is shown in FIG. 20. In another example, the AI ​​system may be extended to a convolutional neural network model to obtain even more accurate results. FIG. 21 and FIG. 22 respectively show, as an example, a process 2100 that is a result of applying an AI neural network to an inspection mode and a process 2200 that is an AI algorithm optimization process using an AI neural network.

[0160] In various embodiments, the vision system includes an eyewear device and / or an image processor. An embodiment of the vision system may include only an image processor. The image processor and its functions, such as those associated with the vision correction framework described herein, may be configured to be used with the eyewear device described herein, or with other devices, or may be used for processing and / or diagnosing conditions for real-time augmented display. The former may or may not be associated with displaying processed image data. For example, in one embodiment, the image processor may be configured to process image data to enhance the pilot's field of vision. The enhanced field of vision may be provided to the pilot using an eyewear device described herein. Such eyewear device may be incorporated, for example, in a helmet visor that includes one or more displays of the enhanced field of vision to the pilot. In some examples, the eyewear device is a goggle. The enhanced field of vision may also be displayed as a display screen or monitor across the windshield or canopy of the aircraft. Such a display screen or monitor may be configured to display the enhanced field of vision to the pilot using a display screen or monitor. The data may include glass, films and / or layers, the transparency of which may be controlled as described herein.

[0161] In any of the above or other examples, the image processing device may be configured to process images for the inspection mode and / or the vision mode as described herein (see, e.g., FIGS. 4, 5, 9, 10, 15, 16, 20-23). ​​In some examples, the image processing device may include a vision correction framework configured to perform one or more operations for the inspection mode and / or the vision mode (see, e.g., FIGS. 2, 3, 8). In any of the above or other examples, the vision correction framework includes a machine learning framework including an AI correction algorithm (see, e.g., FIGS. 2, 10-23). ​​The vision system may include any hardware, software, and / or network configurations described herein (see, e.g., FIGS. 1A-3, 8, 21, 22, 24, 25).

[0162] In any of the above or other examples, the image processing device may be integrated with the eyewear device. The integration may be total or partial. The image processing device may also be external to the eyewear device. This arrangement may also be total or partial. In one example, the image processing device and / or the vision correction framework may be distributed, for example via a network or communication protocol. For example, the image processing device and / or the vision correction framework and its functions may be distributed in two or more of a user device, such as a smartphone, laptop, tablet or dedicated device, the eyewear device, such as an embedded processing system, and an external processing system, such as a computer, PC, laptop or server.

[0163] As mentioned above, the vision system may include a eyewear device and an image processing device, and some embodiments may include only a eyewear device or only an image processing device, but may also have other systems and devices associated with them.

[0164] In any of the above examples or other examples, the eyewear device may be configured to selectively control the transparency of a display area of ​​a monitor, such as a screen, glass, film, and / or multi-layer media. For example, the techniques described herein may be implemented in an augmented reality (also referred to herein as "custom reality") eyewear device. FIG. 23 illustrates an example process 2300 for implementing a test mode and a viewing mode. In one example, the custom reality eyewear device mechanism may be based on steering the vision between the macular (central) portion and the peripheral portion relative to each other.

[0165] Some examples of custom reality glasses devices (see, e.g., Figures 40A-40C) include transparent glasses that overlay the modified image on the viewed scene. The glasses may comprise a monitor that includes a screen, the screen having a controllable transparency for projecting the image to be displayed. In one example, such a display includes a head-up display. In various embodiments, the custom reality glasses device includes glasses with multiple controllable layers that overlay the modified image on the viewed scene through the glasses. The layers may be made of glass, ceramic, polymer, film, and / or other transparent materials and may have a multi-layer configuration. The controllable layers may include, for example, one or more electrically controllable layers that can adjust the transparency in one or more portions of the field of view while addressing the pixels. In one embodiment, the controllable layers may include pixels or cells that are individually addressable, for example, using an electric current, an electric field, or light. The controllable layers may control the contrast of one or more portions of the field of view, the color filter of each portion, zooming in / out of each portion, the focus of each portion, the visibility of each portion from the outside environment, and the like. The layer may be controlled to adjust the transparency of the surface of the eyewear device that displays the image to block or pass light coming from the retina at a particular location in the visual field. If a portion of the visual field is manipulated to enhance the subject's vision (e.g., peripheral or macular vision or partly macular and partly peripheral), the transparency of that portion of the glasses may be reduced to block the outside environment from being seen through that portion of the glasses and allow the patient to more clearly see the manipulated image displayed in that portion of the glasses. In various embodiments, the vision system or custom reality glasses device may dynamically control the transparency of each region to provide a natural view of the outside environment to the subject as eye movement changes eye direction in addition to head movement. For example, pupil tracking data, e.g., pupil and / or eye gaze tracking, may be used to change the less transparent portions of the glasses such that the less transparent regions translate relative to the subject's eyes.

[0166] For example, the transparency of the glasses included in the eyewear device constituting the custom reality glasses may be adjusted, but the adjustment may be controllable to block light from the portion of the field of view where the image modification is performed, e.g., the central or peripheral area. Without such adjustment, the subject would see the manipulated image and would perceive the actual field of view in this area overlaid on the image. To block the light, the eyewear device may include a photochromic glass layer. Furthermore, the eyewear device may vary the location of the area of ​​the glasses where the transparency is reduced by measuring the movement of the eye (pupil) using an inward-facing image sensor and compensating based on this movement by performing processing on the vision correction frame. In one example, the display screen of a monitor includes pixels or cells that include electric ink technology. These pixels or cells may be individually addressable to generate an electric field that changes the configuration of the ink in the cells to change the transparency and / or generate a pixel of the display. In an example implementation, FIG. 40A illustrates a custom reality glasses 4000 that is comprised of a frame 4002 and two transparent glasses assemblies 4004. As shown in Figures 40B and 40C, the transparent eyeglass assembly 4004 has embedded therein an electronically controllable correction layer 4006. The correction layer 4006 may be controllable from completely transparent to completely opaque and may be a digital layer capable of generating a corrective image to overlay or replace a portion of the field of view of the eyeglasses 4004. The correction layer 4006 may be connected via electrical connections 4008 to an image processor 4010 on the frame 4002.

[0167] With specific reference to process 2300 of FIG. 23, in block 2302, inspection mode data may be received in a vision correction framework, and in block 2304, visual field distortions, defects, aberrations and / or other ocular abnormalities may be determined along with their location.

[0168] For a diagnosed central visual field abnormality 2306, in block 2308, the custom reality glasses device may pass images from the external environment through the glasses to the viewer's peripheral vision, for example as shown in FIG. 24. As shown, the custom reality glasses device 2400 may include a multi-layered glasses viewfinder 2402. A peripheral region 2404 may be transparent to allow light to pass through, allowing the subject to see the actual unmodified external environment. In block 2312, a central region 2406 of the external environment may be occluded by the glasses device 2400, and a modified rendition of the central region may be presented to the user via the display, for example using modifications as shown in FIGS. 13, 14, 17, and 18.

[0169] For diagnosed peripheral vision abnormalities 2308, in block 2314, a central region 2406' (see FIG. 25) of the external environment is allowed to pass through the transparent portion of the eyewear device 2400, and may be corrected, for example, by utilizing the corrective transformations described herein. The transparency of the peripheral region 2404' is changed to block out light so that the modified peripheral region image is displayed within the peripheral region 2404'.

[0170] In other examples, the techniques described herein may be used to obtain and enhance a binocular field of view that may be provided to both eyes to provide a corrected (or, in some examples, enhanced) field of view to the subject. Figures 26-30 show examples of binocular field enhancement techniques.

[0171] FIG 26 illustrates normal binocular vision of a subject, where a monocular image from the left eye 2602 and a monocular image from the right eye 2604 are combined to form a single sensed image 2606 that includes a central macular region 2608 and a peripheral vision region 2610 surrounding the central region 2608. However, in some cases, the subject may have a condition called tunnel vision, in which case the peripheral region 2610 is not visible to the subject, as shown in FIG 27. As shown, in such cases, one or more objects do not appear in the field of view, and a peripheral defect 2612 is seen in region 2610, and objects within region 2610 are not visible to the patient.

[0172] In some examples, the defects shown in FIG. 27 may be corrected using a shift image correction technique. As shown in FIG. 28, each field camera captures a monocular image 2702 and 2704, which are different because they capture the viewed scene from a slightly different (shifted) position. The captured monocular images 2702, 2704 are then shifted closer to each other by a vision correction framework to obtain images 2702' and 2704'. These two shifted images are then combined to generate a binocular image 2706 that captures the entire periphery of the viewed scene. For eyewear devices with monitor displays, each display may display the corrected binocular image 2706 to the subject. In one example, as described, this shift transformation may increase the subject's field of view by 5%, 10%, 15%, or 20% without causing the patient to experience diplopia.

[0173] FIG. 29 illustrates another binocular vision correction process. In this example, the captured monocular images 2902 and 2904 are resized, for example, only in the peripheral regions while leaving the central macular region (the central 20 degrees) intact to generate corrected images 2902', 2904'. Such resizing transformation maintains central vision while expanding the field of view. The combined binocular image 2906 can be seen to capture previously unseen peripheral objects while maintaining the detail of the central macular region. Peripheral objects are clearly perceived by the subject even after resizing. This is because peripheral vision is not as sensitive as central vision. In one example, it has been shown that a reduction in image size of up to 20% can be achieved without causing diplopia in the patient. In various embodiments, the system may resize the peripheral region in addition to or instead of resizing the central region. For example, the peripheral region may be reduced in size while maintaining the size of the central macular region, for example, for glaucoma patients.

[0174] In the case of macular degeneration, the opposite can be done: central vision is expanded without changing peripheral vision.

[0175] Fig. 30 illustrates another binocular visual field correction process. For a patient with a peripheral defect well outside one eye, an object 3002 that is not visible in the affected eye's visual field 3004 can be digitally moved to the mid-peripheral region 3006 of the visual field 3004, while the other normal eye's visual field 3008 covers this region. In other words, the previously invisible object 3002 appears in the normal visual field in the combined binocular image 3010. The subject will notice a visual disruption in this region, but based on adaptation, they will be able to isolate information from this region of the visual field in response to the movement of the object or the change in the environment. It is.

[0176] In various examples of the test mode, the projection wearable glasses may be used to project a pattern onto the retina. The pattern may be used to determine defects directly on the retina as well as defects affecting the cornea. In one example, the projected pattern may be used to evaluate and correct dysmorphopsia in age-related macular degeneration and other retinal diseases. As shown in FIG. 31A, a digital projection of a pattern 3100 may be projected onto the subject's eye 3102. The pattern may be digitally generated by a projector located inside the eyewear device. A digital camera 3104, such as an inward-facing image sensor, may also be located inside the eyewear device to capture an image of the pattern 3100 reflected by the eye 3102. The image may be captured from the corneal surface of the eye, for example, as shown in FIG. 32. By capturing an image of the pattern 3100', the vision correction framework may determine whether the pattern appears normal, for example, as shown in FIG. 33, or whether there is an abnormality, for example, as shown in FIG. 34 (3101). Anomalies may be evaluated and corrected using one of the techniques described herein.

[0177] In some examples, the pattern 3100 may be a grid diagram, such as an Amsler grid, or any known fiducial shape designed to detect the transformation required to treat one or more ocular anomalies. The transformation may then be used to inversely distort the image in real time to improve vision. For example, this technique may be used with a virtual reality model or an augmented reality model. In the implementation of FIG. 8, the vision system 800 may include a test module 802. The test module 802 may be associated with wearable glasses or may be executed in combination with an external device as described herein. The test module 802 may present a test stimulus to a subject 806, including an Amsler grid. The subject may manipulate the grid diagram image via a user device 808 or other input device to improve the distortion. The vision correction framework 810 may present the Amsler grid for further correction by the subject. Once the subject has completed the manual corrections, the vision correction framework 810 may generate a correction profile for the subject to be applied to the scene as viewed when using the eyewear device. The workflow described for vision system 800 may be similarly applicable to the processing of the other inspection modes described herein.

[0178] FIG. 31B is a schematic diagram illustrating an Amsler grid 3100 (i.e., an example of a reference image) being presented as an image on a wearable pair of glasses (e.g., a VR or AR headset). The Amsler grid 3100 may be displayed or projected onto the subject's cornea and / or retina. An example of a standard grid diagram 3100 is shown in FIG. 31C. The same grid pattern may be displayed on a user device. The subject may manipulate the lines of the grid pattern, particularly the lines that appear curved, using a keyboard, mouse, touch screen, or other input of the user device including a user interface. The subject may identify a reference point 3110 from which to begin manipulating the image. After identifying the reference point, the subject may adjust the identified lines using the user device (e.g., arrow keys) to correct the distortions perceived due to the abnormal macula. This procedure may be performed separately for each eye, resulting in two corrected grid diagrams.

[0179] Once the subject has completed modifying the line so that it appears straight, the vision correction framework uses the new grid diagram to generate a mesh of vertices that corresponds to the added distortion. Such a mesh generated by the test mode can be applied to any image to compensate for the patient's anomalies. For example, as part of the validation of the test mode, the modified image corresponding to the appropriate mesh can be A user device may display a corrected image of the subject's eyes, one for each eye. The subject may then indicate on the user device whether there are any anomalies in the corrected image, and if there are no anomalies, the correction is successful. For example, FIG. 31E illustrates an actual scene as the user should perceive it. FIG. 31F illustrates a corrected field of view, which, when presented to a subject with visual distortions as determined by the Amsler grid method, will result in the subject viewing the field of view of FIG. 31F perceiving the actual field of view of FIG. 31E.

[0180] Such corrections may be performed in real time on a live image, providing a continuously corrected version of the viewed scene for presentation to the subject. Whether the eyewear device includes a display that generates an image field or not, or whether the eyewear device utilizes a correction layer for adjusting distortions in a custom reality manner or not, the corrections may be performed in real time, since in both cases a determined correction mesh may be used.

[0181] In some examples, the reference image, such as the Amsler pattern, may be displayed directly on a touch screen or tablet PC, such as 3150 (e.g., tablet PC) shown in FIG. 31G. The Amsler pattern is presented on the display of the device 3150, and the subject may manipulate the lines that appear to be curved with a stylus 3152 to draw the correction that should be applied to the lines so that they appear straight. During the test mode, after each change is made, the grid diagram may be redrawn to reflect the latest edits. This procedure may be performed separately for each eye, resulting in two modified grid diagrams. After the subject completes the changes in the test mode, the tablet PC runs an application to create and send mesh data to a companion application on the eyewear device to process the image to which the determined mesh is applied.

[0182] The eyewear device may receive the modifications in the test mode and apply them to any image to compensate for the subject's anomalies. The eyewear device may then display the images resulting from the correction, which may be displayed via a virtual reality or augmented reality headset. In one example, the display may utilize the headset to present images to the user in a holographic fashion. Each displayed image may correspond to a mesh generated for each eye. If the patient does not see the modified images, the correction may be deemed successful and may be retained for further image processing. In some embodiments of the test mode, instead of or in addition to presenting a single image modified according to the modified grid diagram, a video incorporating the modifications may be presented. In one example, the video includes a live video stream from the modified camera and is displayed to the subject.

[0183] The technology disclosed herein may be used in any number of applications, including with otherwise healthy subjects, such as military personnel and veterans, who are frequently subject to the sudden onset of vision-related disorders. Visual field defects impair the essential abilities of military personnel, veterans, and other patients, as well as their ability to perform daily activities. Such visual impairments reduce independence, safety, productivity, and quality of life, leading to low self-esteem and depression. Despite recent scientific advances, once damage has occurred to the retina, optic nerve, or visual cortex, there are limited treatment options to reverse the damage. As such, treatments mostly involve providing patients with visual aids to maximize function. Current visual aids are inadequate to achieve the above goals. As such, there is an increasing need for better visual aids to improve visual function, quality of life, and safety. The technology described herein, incorporated into eyewear devices, is capable of diagnosing and mitigating common eye injuries, such as military-related eye injuries and disorders, that are sudden onset and result in visual field defects in general as well as harsh or isolated environments. The technology described herein is capable of diagnosing and quantifying visual field defects. The device uses this data to process the patient's visual field in real time, leaving a corrected image. The glasses device projects the image onto the patient's functioning visual field, thereby minimizing the adverse effect of blind spots (or areas of reduced vision) in the visual field on the patient's visual function. Additionally, the glasses device does not utilize a separate medical device to diagnose the visual field defect, making it particularly useful in harsh or isolated environments. Similarly, the techniques described herein may be used to enhance the visual field of a normal subject to achieve better than normal visual field or visual acuity.

[0184] The techniques described herein may dynamically correct low-order and / or high-order visual aberrations. The techniques described herein may detect changes in pupil size, accommodation and gaze and process the visual image displayed or projected to the user's eye with a corresponding visual aberration correction profile. The high-order and / or low-order aberrations may be obtained using an aberrometer in relation to pupil size, accommodation and gaze direction, allowing the glasses to generate such a dynamic correction profile. The image projected to the subject using the techniques described herein may be inversely distorted according to the subject's actual aberrations, so that the subject's own aberrations are reversed again to optimize vision (see, for example, Figures 31B to 31F). The techniques described herein may detect the accommodation state by detecting the pupil near reflex, i.e., pupil constriction (pupils become smaller) and accommodative convergence (eyes move inward). The pupil tracker may include a pupil tracker for tracking the pupil and gaze to detect gaze direction. Such an input, as well as other inputs described herein, may enable the techniques described herein to detect the correction profile to be displayed.

[0185] The techniques described herein may automatically autofocus the image to be displayed to achieve near vision. To further enhance and improve near vision, the techniques described herein may utilize an inward-facing image sensor, e.g., a camera, to detect whether a subject is attempting to view a nearby object by detecting signs of pupillary near reflex, i.e., miosis (pupils become smaller) and accommodative convergence (eyes move inward), and automatically improve near vision. The techniques described herein may further obtain an appropriate correction for the pupillary near reflex by determining how far away an object is by quantifying the amount of the subject's pupillary near reflex.

[0186] The techniques described herein may dynamically correct ocular misalignment, and secondarily diplopia: they track the subject's pupils and line of sight or visual axis, and may shift the image in real time to continuously compensate for ocular misalignment, thereby suppressing diplopia at all fixations.

[0187] The techniques described herein may include software that redistributes the images captured by the DTS field of view camera to the subject's actual functional field of view, which may be dynamically projected with respect to the pupil, line of sight, or visual axis.

[0188] For patients with age-related macular degeneration or other diseases of the human macula who have a central blind spot, the techniques described herein may be used to distribute the image to the peripheral or paracentral portions of the subject's functional visual field. The techniques of the present invention may project a portion of the target image onto normal portions of the retina and avoid abnormal portions of the retina.

[0189] The techniques described herein may capture the normal binocular visual field and distribute it between the actual functional visual fields of both eyes to give the subject the widest possible field of view.

[0190] Anisometropia, which is caused by non-uniform refractive powers in a subject's eyes, can be corrected by the techniques described herein, for example, by generating and displaying or projecting a uniformly sized image to both eyes. This may be corrected to reduce visual disturbance.

[0191] Unlike eyeglass lenses, which introduce distortions into the field of view, such as shrinking or enlarging the image of an object, the technology of the present invention can be utilized without affecting the subject's field of view, since the displayed or projected field of view is independent of the corrective lenses.

[0192] The technology described herein may display or project light independent of the brightness of the surrounding environment and may be automatically adjusted according to the pupil size detected by the technology described herein or manually requested by the patient. The technology described herein may detect pupil size and adjust the brightness in a customized manner for each individual. Subjects with anisocoria may utilize the technology of the present invention to allow separate brightness adjustments for each eye, again automatically performed by the technology described herein upon pupil size detection.

[0193] <Example> An example of the application of the techniques described herein to a visual field testing protocol is described. The test mode used 4 contrast stimuli covering a central 40 degree radius using a 52 stimulus sequence at a given location, applying a fast thresholding scheme. This is illustrated in FIG. 35A. Other examples may have different numbers of contrast stimuli, coverage areas, and stimulus locations. In this example, stimuli were placed at the center of each cell illustrated in FIG. 35A. The 12 corner cells were not tested because the stimuli were not visible due to the circular shape of the display lens. The spacing between the stimulus locations was approximately 10 degrees. Each stimulus sequence included four consecutive stimuli with different contrast levels against the background. The contrast levels of the stimuli varied in descending order between 33 dB and 24 dB in 3 dB steps. The threshold was recorded for the last stimulus shown. If the patient did not see any of the stimulus contrasts at a particular location, this location was considered invisible and given a value of 0 dB.

[0194] The background was bright (100 lux) while the stimuli were dark spots with different contrast levels. Thus, the test was photopic rather than mesopic. In some embodiments, the background may be dark and the stimuli may be bright spots. Each stimulus was presented for a period of about 250 ms, followed by a response latency of about 300 ms. These periods could be adjusted by the control program according to the subject's response speed. For example, they could be adjusted before the test based on pre-test confirmation or dynamically during the test. In general, a stimulus size of 0.44 degrees was used with a central 24 degree radius. This corresponds to a standard Goldmann stimulus size III. The stimulus size in the periphery (24 to 40 degree radius) was doubled to 0.88 degrees. The doubling of the stimulus size in peripheral vision was intended to compensate for the degradation of the display lens performance in the periphery. This lens degradation was significant since normal human vision also deteriorates in the peripheral region. The testing program also allowed for varying stimulus size to suit different patient cases.

[0195] A fixation target (pattern) in Figure 35A was placed in the center of the screen for each eye tested. The target was constructed as a multicolored dot, rather than a single-colored fixation point as is typically used in the traditional Humphrey test. This change in color was intended to help catch the subject's attention and help them focus on the target. The frequency of color changes was not synchronized with the frequency with which the stimuli were visible, so that subjects would not associate the two and respond incorrectly. This test protocol also allowed for the fixation target size to be varied to suit the patient's condition. Additionally, an eye / pupil tracking system may be used to monitor the subject's eye fixation at various time intervals. The eye tracking system may incorporate a gaze vector into the test program. The lens orientation is sent to inform the program whether the subject is properly centered or not.

[0196] Fixation checks were performed using pupil / gaze data for each eye separately. Pupil / gaze data were acquired at various times to determine if the gaze direction vector was approximately 0 degrees, the subject was focused on the central target, otherwise the program paused and waited for the subject to fixate again. If the subject disengaged from fixation, no stimuli were presented and the test stopped until the participant refixated. A margin of error was allowed for deviations due to small eye movements at the fixation target. Fixation checks were performed at two main times for each stimulus location: before presenting each stimulus in the stimulus sequence (i.e., before each stimulus contrast level out of the four levels mentioned above) and before recording a response, whether the response was positive (if the patient was looking at the stimulus) or negative (if the patient was not looking at the stimulus). Negative responses were recorded at the end of the stimulus sequence period in addition to the allowed response time. Fixation checks before presenting the stimulus sequence were performed to ensure that the subject was focused on the fixation target. If the subject disengaged from fixation, no stimuli were presented and the test stopped until the participant refixated.

[0197] FIG. 35B is a timing diagram showing five stages (a-e) of a test sequence at one stimulation location.

[0198] In one example, the pupil tracker, which may be separate from or a component of the vision system or device of the vision system, may include an inward-facing image sensor and may be configured to provide data to a visual display device, e.g., a monitor. The visual display device may include a projector to change the location of the projected stimuli according to eye movements. In this way, even if the subject is looking around and not fixating, the stimuli may move with the subject's eyes and continue to test the desired location in the visual field. Thus, rather than stopping the stimulus sequence when it is determined that the subject is focusing on something other than the fixation target, the stimulus sequence may continue to change the stimuli to correspond to the intended location in the visual field of the subject in the sequence, so that they are repositioned based on determining the subject's current fixation point.

[0199] For each subject, the visual field test started with explaining to the subject how to proceed. The eyeglass device was fitted to the patient so that the subject could clearly see the fixation target, and if necessary, the target size was adjusted accordingly. A calibration of the eye tracker was performed at a single point, the fixation target. After this, the subject was presented with a demonstration mode, which involves the same sequence as the main test, but with a reduced number of positions, in this example seven positions, and without recording any responses. The purpose of this mode was to provide training to the subject on the test. This training mode also allowed the program administrator to check the accuracy of the eye tracker system, the patient's speed of response, and the position of the patient's eyes relative to the worn headset, to avoid errors and deviations in the main test.

[0200] The scan then searched for the normal blind spot by presenting suprathreshold stimuli positioned at four different locations, 1 degree apart, within a close 15-degree range -- a useful step in avoiding rotational misalignment between the headset and the subjects' eyes.

[0201] The 52-stimulus sequence was then presented to the patient in random order at pre-specified locations. Subjects responded to the stimuli by activating an electronic clicker or making a gesture. After recording the subjects' responses at all locations, the subjects were asked to select the "blind" points. The positions were temporarily stored. A search algorithm was then used to find the positions of all the "seen" points on the boundary of the "missing" points' positions. These two sets of points were then examined again to exclude random response errors of the participants and to ensure that the visual field was continuous. False positive responses, false negative responses, and fixation losses (if any) were calculated and reported by the end of the test. All 52 responses were then interpolated using a cubic method to generate a continuous visual field plot for the tested participant.

[0202] Visual field tests were conducted on 20 volunteer subjects using a simulated visual field defect created by covering a portion of the inner display lens of the eyeglass device. Results were assessed using a point-by-point comparison using an image showing the covered area inside the display. As a measure of test accuracy, 52 responses were compared with their approximate corresponding location in the display image of the covered headset. The calculated errors are summarized in Table 1. [Table 1]

[0203] Meanwhile, visual field tests of 23 clinical patients were compared with recent results of the Humphrey Field Analyzer (HFA) test that subjects usually undergo during their consultation. The two visual field test devices were compared by matching a common central 24° area. Again, comparisons and relative error calculations were performed point-by-point in the common central 24° area, and areas outside this area were judged by their continuity with the central area and the absence of isolated response points. The calculated errors are summarized in Table 2. [Table 2]

[0204] This was followed by an image remapping process, which involved identifying new dimensions and centering for the displayed image shown to the patient. The output image was fitted into the light field of the subject's eye by resizing and shifting the original input image.

[0205] The visual field was binarized by setting all "seen" patient responses to 1 and leaving "not seen" responses at zero. This resulted in a small binary image of size 8x8. In other embodiments, larger or smaller binary images may be used. Subregions containing up to 4 connected pixels were removed from the binary visual field image. Four connected pixels represented a predefined threshold for determining a subregion, although in some embodiments the threshold may be larger or smaller. Such subregions were not considered in the image fitting process. The ignored subregions represent either normal blind spots, non-significant defects, or any random error responses that may occur during the subject's visual field testing.

[0206] Based on the binary field image thus interpolated, properties of the bright field region were calculated. The properties calculated for the bright region included: 1) Bright areas in pixels 2) A bounding box for the region 3) The center of gravity of the weighted region 4) A list of all pixels that make up the bright areas of the visual field The bounding box was the smallest rectangle that enclosed all the pixels that made up the bright region. The region's centroid was the center of mass of the region calculated in terms of horizontal and vertical coordinates. The value of this property corresponds to the new center of the output image and the amount of image shift required for the mapping.

[0207] Using the list of pixels that make up the largest bright field, we calculated the width and height of all pixels that bound the bright field, as shown in Figure 36. For each bright field row, we identified two border pixels and subtracted their vertical coordinates to determine the width BF of the bright field in that row. width Such a width calculation is done by BF widths A similar iteration was performed for all the rows that make up the bright field to calculate BF heights Then, one of two scaling equations can be used to calculate the size of the mapped output image, Width map and Height map may be newly determined as shown in FIG.

[0208] Width map may be calculated using the following resizing equation: Width map1 = Median(BF widths ) Height map1 = Median(BF heights ) BF widths and B.F. heights are the pixel width and height, respectively, that bound the calculated bright field. In this scaling method, the median bright field size in each direction is calculated as the new output image size, as specified above, and the new image is centered around it. This method of determining the median was used instead of the average value to avoid resizing distortions associated with bright field dimensions that are too large or too small. The mapping behavior of this method is to fit the image within the largest possible bright area, but this method does not maintain the aspect ratio, which may result in image stretching or shrinking.

[0209] Height map may be calculated using the following resizing equation:

number

[0210] In one embodiment, the AI ​​system computes these two equations and a number, if not hundreds, of Ten difference equations may be used in an optimization process to determine which allows a larger portion of the visible field to fit into the image, and based on operator feedback, the system may learn to prefer one equation over others based on the particular field to be corrected.

[0211] These remapping techniques were used in a hazard identification test. The remapping method was tested on 23 subjects using a test image containing a safety hazard, in this case a vehicle. The test image was selected to test the four main quadrants of the visual field, as shown in Figure 38. An example of the visual field was used to remap the test image shown to the subjects. The subjects were tested by showing an image of an oncoming vehicle. The subjects could not see the vehicle until they were shown the remapped image. Figure 39A shows the image the subjects see without remapping, and Figure 39B shows the image seen after remapping. Our preliminary study showed that 78% of subjects (18 out of 23) were able to identify a safety hazard that would not have been possible without the aid of the aid. Some subjects were tested separately with both eyes, so 33 eyes were tested. It was found that the visual aid was effective in helping subjects identify a simulated oncoming hazard in 23 out of 33 eyes (P=0.023).

[0212] Components, operations, or structures described herein as singular may be performed in plural. Although individual operations of one or more methods are illustrated and described as separate operations, one or more of the individual operations may be performed simultaneously, and the operations need not be performed in the order shown. Structures and functions presented in example configurations as separate components may be combined and implemented as one structure or component. Similarly, structures and functions presented as a single component may be implemented as multiple separate components. These and other variations, modifications, additions, and improvements are within the scope of the subject matter of this specification.

[0213] Additionally, certain embodiments are described herein as including logic or certain routines, subroutines, applications, or instructions, which may constitute either software (e.g., code embodied in a machine-readable medium or transmission signal) or hardware, where the routines, etc., are tangible components capable of performing certain operations and may be configured or arranged in a particular way. In example embodiments, one or more computer systems (e.g., stand-alone, client, or server computer systems) or one or more hardware modules of a computer system (e.g., a processor or a collection of processors) may be configured as hardware modules that operate with software (e.g., an application or application portions) to perform certain operations described herein.

[0214] In various embodiments, a hardware module may be implemented mechanically or electronically. For example, a hardware module may comprise dedicated circuitry or logic that is permanently configured (e.g., as an application-specific processor, such as a field programmable gate array (FPGA) or application-specific integrated circuit (ASIC)) to perform specific operations. A hardware module may also comprise programmable logic or circuitry (e.g., contained within a general-purpose processor or other programmable processor) that is temporarily configured by software to perform specific operations. It should be understood that whether a hardware module is mechanically implemented in dedicated, permanently configured circuitry or in temporarily configured (e.g., configured by software) circuitry may be determined by cost and time considerations.

[0215] Thus, the term "hardware module" should be understood to include a tangible entity that is physically constructed, permanently configured (e.g., hardwired), or temporarily configured (e.g., programmed) to operate in a particular manner or to perform particular operations described herein. Considering embodiments in which the hardware modules are temporarily configured (e.g., programmed), it is not necessary that each of the hardware modules be configured or instantiated at one time. For example, if the hardware modules include a general-purpose processor that is configured with software, the general-purpose processor may be configured as different hardware modules at different times. Thus, the software may configure the processor, for example, to configure one hardware module at one time and another hardware module at another time.

[0216] A hardware module can provide information to and receive information from other hardware modules. Thus, the described hardware modules may be considered to be communicatively coupled. When multiple such hardware modules are present simultaneously, communication may be achieved by signaling (e.g., signaling over appropriate circuits and buses) connecting the hardware modules. In embodiments in which multiple hardware modules are configured or instantiated at different times, communication between such hardware modules may be achieved, for example, by storing and retrieving information in memory structures to which the multiple hardware modules have access. For example, a hardware module may perform processing and store the output of that processing in a communicatively coupled memory device, which another hardware module may later access to retrieve and process the stored output. Hardware modules may also initiate communication with input or output devices and perform operations on resources (e.g., collections of information).

[0217] The various operations of the example methods described herein may be performed, at least in part, by one or more processors that are temporarily or permanently configured (e.g., by software) to perform the associated operations. Such processors, whether temporarily or permanently configured, may constitute processor-implemented modules that operate to perform one or more operations or functions. References herein to modules may, in some example embodiments, include processor-implemented modules.

[0218] Similarly, the methods or routines described herein may be implemented, at least in part, on a processor. For example, at least some of the operations of a method may be performed by one or more processors or processor-implemented hardware modules. The execution of certain operations may be distributed across one or more processors, and may be located on multiple machines as well as a single machine. In some example embodiments, one or more processors may be located at a single location (e.g., in a home environment, an office environment, or as a server farm), while in other embodiments, the processors may be distributed across multiple locations.

[0219] Execution of certain of the processes may be distributed across one or more processors, and may reside on multiple machines, as well as a single machine. In some example embodiments, one or more processors or processor-implemented modules may be located in a single geographic location (e.g., in a home environment, in an office environment, or in a server farm). In other example embodiments, one or more processors or processor-implemented modules may be located in multiple locations, such as a single computer, a single network, or a single network. The rules may be distributed across multiple geographic locations.

[0220] Unless otherwise indicated, descriptions herein using terms such as "processing," "computing," "calculating," "determining," "presenting," "displaying," and the like may refer to the operation or process of a machine (e.g., a computer) that manipulates or transforms data represented as physical (e.g., electronic, magnetic, or optical) quantities in one or more "memories" (e.g., volatile memory, non-volatile memory, or combinations thereof), registers, or other machine components that receive, store, transmit, or display information. In some embodiments, the memory or computer-readable storage medium of the memory stores programs, modules, and data structures, or portions thereof, for causing a processor to control and execute various systems and methods disclosed herein. In one embodiment, a non-transitory computer-readable storage medium stores computer-executable instructions that, when executed by a processor, perform one or more of the methods disclosed herein.

[0221] Whenever "one embodiment" or "an embodiment" is referred to herein, it means that a particular element, feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment. The appearances of the phrase "in one embodiment" in various places in the specification are not necessarily all referring to the same embodiment.

[0222] Some embodiments may be described using the terms "coupled" and "connected." For example, some embodiments may be described using the term "coupled" to indicate that two or more elements are in direct physical or electrical contact. However, the term "coupled" can also mean that two or more elements are not in direct contact with each other, but still cooperate or interact with each other. Embodiments are not limited in this sense.

[0223] As used herein, the terms "comprises," "comprising," "includes," "including," "has," "having," or variations thereof, are intended to be non-exclusive inclusive. For example, a process, method, article, or apparatus that comprises each listed element is not necessarily limited to those elements, but may include other elements not expressly listed or that are inherent in such process, method, article, or apparatus. Also, unless expressly stated otherwise, "or" refers to an inclusive "or" rather than an exclusive "or." For example, a condition "A or B" is satisfied by either of the following: A is true (i.e., exists) and B is false (i.e., does not exist), or A is false (i.e., does not exist) and B is true (i.e., exists), then both A and B are true (i.e., exists).

[0224] Additionally, "a" or "an" are used to describe elements and components of the embodiments described herein. This is done merely for convenience and to allow the description to have a general sense. In this description and the claims that follow, the term "a" or "an" should be interpreted to include one or at least one, and the singular also includes the plural unless it is clear that this is meant otherwise.

[0225] This detailed description should be construed as exemplary only and does not describe every possible embodiment, as describing every possible embodiment would be impractical, if not impossible, and many embodiments could alternatively be implemented using current technology or technology available after the filing date of this application.

Claims

1. A method implemented by one or more processors executing computer program instructions, the execution of which causes the one or more processors to perform steps of the method, the method comprising: Obtaining multiple views of a scene via a wearable device; determining a first region of the scene that is (i) common to the views and (ii) relative to a user's central vision; determining, for each of the views, a second region of the scene that (i) is represented in the view and (ii) is relevant to the user's peripheral vision; generating a modified view based on (i) a portion of at least one of the views corresponding to the common first region, (ii) a portion of at least one of the views corresponding to a peripheral region common to the views, and (iii) a portion of the view corresponding to the second region, such that the modified view includes (i) a first portion including a representation of the common first region and a representation of the common peripheral region, and (ii) a second portion including a representation of the second region and surrounding the first portion; displaying the modified view via the wearable device; and A method comprising:

2. performing a shift of each view of the plurality of views such that a size of the common first region is decreased and a size of at least one of the second regions is increased; 2. The method of claim 1, wherein generating the modified view includes generating the modified view after the performing of the shift such that (i) the common first region represented in the first portion of the modified view reflects the decreased size, and (ii) at least one of the second regions represented in the second portion of the modified view reflects the increased size.

3. performing resizing of one or more regions of the plurality of views such that a degree of any resizing of the common first region differs from a degree of any resizing of at least one of the second regions; The method of claim 1 , wherein generating the modified view comprises generating the modified view after the performing of the resizing.

4. 4. The method of claim 3, wherein performing the resizing includes performing the resizing of one or more regions of the multiple views such that a rate of change of a size of the common first region represented in the first portion of the modified views is greater than or less than a rate of change of a size of at least one of the second regions represented in the second portion of the modified views.

5. The method of claim 4 , wherein the rate of change of size of at least one of the second regions is zero and the rate of change of size of the common first region is greater than zero.

6. The method of claim 4 , wherein the rate of change of size of at least one of the second regions is greater than zero and the rate of change of size of the common first region is zero.

7. 2. The method of claim 1 , wherein the wearable device includes a first and a second camera, and obtaining the multiple views includes obtaining at least one of the multiple views via the first camera of the wearable device and obtaining at least another of the multiple views via the second camera of the wearable device.

8. 2. The method of claim 1 , wherein the wearable device includes a first and a second monitor, and causing the altered view to be displayed includes causing the altered view to be displayed via the first and second monitors.

9. 2. The method of claim 1, wherein a computer system comprises a wearable computer system including the one or more processors executing the computer program instructions, which when executed cause the wearable computer system to perform all of the actions described above.

10. The method of claim 1 , wherein the wearable device comprises a wearable eyewear device.

11. The method of claim 10 , wherein the wearable eyewear device includes the one or more processors that execute the computer program instructions that, when executed, cause the wearable eyewear device to perform all of the actions described above.

12. A system including a memory for storing the computer program instructions, and one or more processors for executing the computer program instructions, the execution of the computer program instructions causing the one or more processors to perform each of the steps of the method according to any one of claims 1 to 11.

13. A computer readable storage medium storing computer program instructions which, when executed by one or more processors, cause the one or more processors to perform each of the steps of the method according to any one of claims 1 to 11.

Citation Information

Patent Citations

  • Apparatus and method for enhancing field of vision of the visually impaired

    JP2010162346A

  • Method, software and apparatus for examining a patient's visual field

    JP2017529964A

  • Multi-functional optometric-ophthalmic system for testing diagnosing, or treating, vision or eyes of a subject, and methodologies thereof

    US20090153796A1

  • System, method, and non-transitory computer-readable storage media related to correction of vision defects using a visual display

    US20160270648A1

  • Methods and Devices for Providing Enhanced Visual Acuity

    US20170092007A1