Vision screening device including color imaging

Abstract

This document describes a vision screening device for performing vision screening tests on patients to determine the presence of diseases and / or abnormalities in the patient's (multiple) eyes. The vision screening device may include relevant methods and systems configured to perform vision screening tests. The device may include a radiation source configured to generate near-infrared (NIR) radiation, a sensor configured to capture a grayscale image representing the radiation reflected by the patient's (multiple) eyes, a white light source, and a camera configured to capture color images of the patient's eyes. The device may also be configured to generate a composite image based on the grayscale and / or color images, determine a difference between an eye-related value and a desired value, and generate an output indicating an eye-related condition based on the difference.

Description

Technical Field

[0001] This application relates to medical devices. In particular, this application relates to a vision screening device, and related systems and methods for detecting and assessing eye diseases and conditions. Background Technology

[0002] Vision screening often includes eye disease screening. Such screenings can include transillumination tests, such as the Brückner red reflex test. During the red reflex test, a clinician uses an ophthalmoscope to illuminate the patient's eye with visible light and examines the color and other properties of the light reflected back from the choroid and retinal surfaces. This test can detect a variety of eye diseases and abnormalities, such as corneal or medial opacities, cataracts, and retinal abnormalities, including tumors and retinoblastomas. Vision screening for eye diseases is recommended for all age groups. For example, newborns can be screened for congenital eye diseases, while older adults can be screened for age-related degenerative diseases such as cataracts and retinal diseases. Vision screening under visible light can also detect the presence of foreign bodies in the eye.

[0003] In addition, vision screening typically includes one or more tests to identify various defects related to a patient's eyes. Such vision tests may include, for example, refractive error tests, accommodation tests, visual acuity tests, color vision screening, etc. Some vision screening tests require the use of infrared or near-infrared imaging, while others may require imaging in visible light and / or a display screen to show the content to the patient. However, ophthalmic testing devices, such as integrated refractive instruments, automated refractometers, and photorefractive meters, may only offer the ability to perform a limited range of tests. Being able to screen for most vision problems and diseases using a single integrated device would be advantageous.

[0004] The various examples disclosed herein are intended to overcome one or more of the aforementioned defects. Summary of the Invention

[0005] In an example of this disclosure, a vision screening device includes a radiation source configured to emit radiation of a first wavelength (e.g., in the near-infrared band), a sensor configured to capture radiation reflected by a patient's eyes, a white light source, and a camera configured to capture a color image of the patient's eyes. The vision screening device also includes a processor operatively connected to the radiation source, the sensor, the white light source, and the camera, and a memory storing instructions executable by the processor. When executed, the instructions cause the processor to: cause the radiation source to emit radiation of the first wavelength during a first time period; cause the sensor to capture a portion of the radiation reflected by the patient's eyes during the first time period; cause the white light source to illuminate the patient's eyes during a second time period following the first time period; and cause the camera to capture a color image of the patient's eyes during the second time period. When executed, the instructions also cause the processor to: generate a synthetic image of the eye, wherein the synthetic image includes a first plurality of pixels representing a grayscale image indicating the captured portion of the radiation and a second plurality of pixels representing the color image; determine, based on the synthetic image, a difference between a value associated with the eye and a desired value; and generate an output indicating a condition associated with the eye, at least in part, based on the difference.

[0006] In another example of this disclosure, a method includes: illuminating a patient's eye with a radiation source during a first time period; capturing a grayscale image of the eye with a sensor during the first time period; illuminating the eye with a white light source during a second time period separated from the first time period; and capturing a color image of the eye with a camera during the second time period. The method further includes: generating a composite image of the eye, wherein the composite image derives a first plurality of pixel values ​​from the grayscale image and a second plurality of pixel values ​​from the color image; and determining an output relevant to the patient based on the properties of the eye independently revealed by NIR and visible light, at least in part based on at least one of the color image and the composite image, according to the analysis.

[0007] In another example of this disclosure, the system includes a memory, a processor, and computer-executable instructions stored in the memory and executable by the processor. When executed, the instructions cause the processor to perform operations including: causing a radiation source to emit near-infrared (NIR) radiation during a first time period; causing a sensor to capture a portion of the NIR radiation reflected by the patient's eye during the first time period; causing a white light source to illuminate the eye during a second time period separated from the first time period; and causing a camera to capture a color image of the eye during the second time period. The instructions, when executed, also cause the processor to: determine a difference between a value related to the eye and a desired value based on the color image and the portion of the NIR radiation; determine that the difference is equal to or greater than a threshold; and generate an output indicating the condition of the eye based at least in part on the determination that the difference is equal to or greater than the threshold. Attached Figure Description

[0008] The features, nature, and various advantages of this disclosure will become more apparent from the following detailed description taken in conjunction with the accompanying drawings.

[0009] Figure 1 Examples of vision screening devices and vision screening systems of this disclosure are illustrated. In some embodiments, Figure 1 The components of the example system shown can be used to perform one or more screening tests related to vision screening and / or the detection of eye diseases or abnormalities.

[0010] Figure 2 An example vision screening device of this disclosure is illustrated.

[0011] Figure 3A Another example vision screening device of this disclosure is illustrated.

[0012] Figure 3B The arrangement of the radiation source in the example vision screening device of this disclosure is illustrated.

[0013] Figure 3C A schematic illustration of an example vision screening system of this disclosure is provided.

[0014] Figure 3D Another schematic illustration of an example visual screening device of this disclosure is provided.

[0015] Figures 4A-4D Examples of features according to this disclosure that can be used by a vision screening device to determine eye diseases and abnormalities based on an image of the eye are illustrated.

[0016] Figure 5A and Figure 5B An example visualization generated by the vision screening device of this disclosure is shown.

[0017] Figure 6 A first flowchart illustrating an example method of this disclosure is provided.

[0018] Figure 7 A second flowchart illustrating the example methods of this disclosure is provided.

[0019] Figure 8 A third flowchart illustrating example methods of this disclosure is provided.

[0020] In the figures, the leftmost number of the reference numerals indicates the figure in which that reference numeral first appears. The same reference numerals are used in different figures to indicate similar or identical items or features. The figures are not drawn to scale. Detailed Implementation

[0021] This disclosure relates in part to vision screening devices and methods. Such example vision screening devices can be configured to perform one or more vision screening tests on a patient and output the results of the vision screening tests to an operator of the device, such as a clinician or a physician's assistant. Specifically, this disclosure relates to devices and methods for screening for eye diseases and abnormalities. For example, a vision screening device can capture one or more images of an eye irradiated by radiation of different wavelength ranges of the electromagnetic spectrum (e.g., infrared, near-infrared, and visible light). The device can determine one or more eye diseases and / or abnormalities associated with one or both eyes of a patient based on analysis of the captured images, such as cataracts, tumors, refractive errors, intraocular foreign bodies, corneal abrasions, retinal detachment or lesions, congenital conditions, etc.

[0022] Based at least in part on the analysis of the captured images, the device can generate output including at least one of a patient-related suggestion or diagnosis. Such output (e.g., suggestion and / or diagnosis) may indicate a detected disease or abnormality, an indication that the patient needs further screening, or an indication that the screening was normal (e.g., no disease or abnormality is indicated). For example, the device can determine the difference between images of the patient's left and right eyes and compare that difference with standard test data corresponding to a normal eye to provide a suggestion and / or diagnosis. Specifically, the standard test data may provide one or more thresholds or value ranges, and the output generated by the device may be based on a difference less than a threshold or within a value range. The device can also generate a visualization of the captured images for display to a clinician or operator of the vision screening device to assist in determining a diagnosis. Thus, the methods described herein can provide automated diagnoses based on the analysis of images captured by the vision screening device. The methods described herein can also provide automated suggestions based on and / or indicating such diagnoses.

[0023] If at least refer to Figure 1The example vision screening device associated with screening for eye diseases and abnormalities may include components for capturing images of a patient's (multiple) eyes under near-infrared and visible light radiation. The device may include components for controlling the emission of near-infrared and visible light radiation and the corresponding capture of radiation reflected from the eyes during screening. In the example, near-infrared images may be captured before visible light image capture begins, such that the pupils of the patient's (multiple) eyes do not constrict or focus in response to visible light screening, and screening can be completed without requiring eye dilation. The device may also include components for analyzing the captured images to determine disease conditions and / or abnormalities in the patient's (multiple) eyes, and components for determining and reporting multiple outputs indicating disease conditions or abnormalities detected during screening.

[0024] The following is for reference. Figures 1-7 Further details relating to the above-described apparatus and techniques are described. It should be understood that although these figures depict apparatuses and systems from which the claimed methods can be utilized, the methods, processes, functions, operations and / or techniques described herein can be equally applied to other apparatuses, systems, etc.

[0025] Figure 1 Example environment 100 for implementing vision screening tests, particularly for detecting eye diseases and / or abnormalities, is illustrated according to some implementation methods. For example... Figure 1 As shown, in some examples, operator 102 can perform vision screening tests on patient 106 using vision screening device 104 to determine the eye health of patient 106. As described herein, vision screening device 104 can perform one or more vision screening tests, which include screening for eye diseases and / or abnormalities when the eye is illuminated by visible light. Furthermore, vision screening device 104 can also be configured to perform other vision screening tests, such as visual acuity tests, refractive error tests, focusing tests, dynamic eye-tracking tests, color vision screening tests, and / or any other vision screening tests, configured to assess and / or diagnose the vision health of patient 106. In the examples, vision screening device 104 may include a portable device configured to perform one or more vision screening tests. Due to its portability, vision screening device 104 can perform vision screening tests in any location, from routine screening environments such as schools and medical clinics to doctor's offices, hospitals, eye care facilities, and / or other remote and / or mobile locations. It is also envisioned that vision screening device 104 can be used to perform vision screening tests on all age groups, including newborns and young children as well as elderly patients.

[0026] As described herein, the vision screening device 104 can be configured to perform one or more vision screening tests on a patient 106. In examples, one or more vision screening tests may include illuminating the patient 106's eye with infrared or near-infrared (NIR) radiation and capturing the reflected radiation from the patient 106's eye. For example, U.S. Patent No. 9,237,846 (the entire disclosure of which is incorporated herein by reference) describes a system and method for determining refractive errors based on photorefractive data using pupil images captured under different illumination modes produced by a near-infrared (NIR) radiation source. In other examples, vision screening tests, such as red light reflectance tests, may include illuminating the patient 106's eye with visible light and capturing a color image of the eye under visible light illumination. The vision screening device 104 may acquire data including color images and / or video data of the eye under visible light illumination and detect the pupil, retina, and / or lens of the patient 106's eye. This data may be used to determine the difference between the left and right eyes, compare the captured images with standard images, or generate visualizations to assist operator 102 or clinician in diagnosing eye diseases and abnormalities in the patient. The vision screening device 104 can transmit data via network 108 to the vision screening system 110 for analysis to determine an output 112 relevant to the patient 106. Alternatively or additionally, the vision screening device 104 can perform some or all of the analysis locally to determine the output 112. In fact, in any of the examples described herein, some or all of the disclosed methods can be performed independently by the vision screening device 104 (e.g., without the vision screening system 110 or its components) or independently by the vision screening system 104 (e.g., without the vision screening device 104 or its components). For example, in some examples, the vision screening device 104 can be configured to perform any of the vision screening tests and / or other methods described herein without being connected to or otherwise communicating with the vision screening system 110 via network 108. In other examples, vision screening system 110 may include one or more components similar to and / or the same as those included in vision screening device 104, so that vision screening system 110 may be configured to perform any of the vision screening tests and / or other methods described herein without being connected to or otherwise communicating with vision screening device 104.

[0027] like Figure 1As schematically shown, the vision screening device 104 may include one or more radiation sources 114 configured to perform functions related to performing one or more vision screening tests. Radiation sources 114 may include individual radiation emitters, such as light-emitting diodes (LEDs), which may be patterned to form an LED array. In this example, radiation source 114 may include a near-infrared (NIR) radiation emitter, such as an NIR LED, for measuring refractive errors in the eye of patient 106 using photorefractive methods. The NIR radiation emitter of radiation source 114 may also be used to measure the gaze angle or gaze direction of patient 106's eye. Furthermore, radiation source 114 may also include colored LEDs for generating colored stimuli to be displayed to patient 106 during a color vision screening test.

[0028] The vision screening device 104 may also include one or more radiation sensors 116, such as infrared cameras, configured to capture reflected radiation from a patient's eyes during a vision screening test. For example, the vision screening device 104 may emit one or more beams of radiation via radiation sources 114 and may be configured to direct these beams towards the eyes of the patient 106. The vision screening device 104 may then capture the corresponding radiation reflected back (e.g., from the pupil of the eye) via the radiation sensors 116. In this example, the radiation sensors 116 may include multiple NIR radiation sensors to capture reflected NIR radiation when the patient 106's eyes are illuminated by the NIR radiation sources 114. Data captured by the NIR radiation sensors 116 can be used to measure refractive errors and / or multiple fixation angles of the patient 106's eyes. The data may include images and / or videos of the pupil, retina, and / or lens of the patient 106's eyes. In some examples, the images and / or videos may be grayscale images (e.g., with values ​​from 0 to 128, or 0 to 256). Data can be acquired intermittently during a specific time period of the vision screening test or throughout the entire duration of the test. Furthermore, the vision screening device 104 can process multiple image and / or video data to determine multiple variations in refractive errors and / or fixation angles of the patient 106's eyes. Grayscale images of the eye captured under NIR illumination can also be used to screen for eye diseases and abnormalities such as refractive errors, strabismus, and occlusion.

[0029] In the example, the vision screening device 104 may further include multiple visible white light sources 118 and a camera 120, which are configured to capture color images and / or videos of the eye under illumination by the white light source 118. The multiple white light sources 118 may include light-emitting diodes (LEDs), such as an array of LEDs configured to produce white light, for example, blue LEDs with a phosphor coating to convert blue light to white light, or combinations of red, blue, and green LEDs configured to produce white light by varying the intensity of activation of individual red, blue, and green LEDs. The individual LEDs in the LED array may be arranged in a pattern configured to operate individually to provide illumination from different angles during multiple vision screening tests. The multiple white light sources 118 may also be configured to produce white light of different intensity levels. The camera 120 may be configured to capture white light reflected from the patient's eye to produce digital color images and / or videos. See reference... Figure 2 As described in further detail, camera 120 may include a high-resolution autofocus digital camera with custom optics for imaging the eye in clinical applications. Color images and / or videos captured by camera 120 can be stored in various formats, such as JPEG, BITMAP, TIFF, etc. (for images) and MP4, MOV, WMV, AVI, etc. (for videos). In some examples, pixel values ​​in the color images and / or videos can be in the RGB (red, green, blue) color space. Color images and / or videos of the eye captured under white light illumination can be used to screen for eye diseases and abnormalities, such as cataracts, medial opacities in the aqueous humor and vitreous humor, tumors, retinal carcinoma, and retinal detachment. Furthermore, color images and / or videos can be combined with grayscale images captured under NIR illumination to generate visualizations to aid in the detection of various eye conditions.

[0030] The vision screening device 104 may also include one or more displays, such as display 122 and display 124, which may be color LCD (liquid crystal display) or OLED (organic light-emitting diode) displays. Display 122 may be an operator display facing toward operator 102 and is configured to provide operator 102 with information related to vision screening tests. In any example described herein, the operator-facing display 122 may be configured to display and / or otherwise provide output 112 generated by vision screening device 104 and / or by vision screening system 110. Output 112 may include test parameters, the current status and progress of the screening test(s), the measurements(s) determined during the test(s), images captured or generated during the screening test, diagnoses determined based on one or more tests, and / or diagnostic recommendations. The operator-facing display 122 may also display patient-related or patient-specific information and the patient's medical history.

[0031] In some examples, the vision screening device 104 may also include a display screen 124 facing toward the patient 106, the display screen 124 being configured to display content to the patient 106. The content may include attention-grabbing images and / or videos to attract the patient's attention and maintain the patient's gaze on the vision screening device 104. Content corresponding to various vision screening tests may also be presented to the patient 106 on the display screen 124. For example, the display screen 124 may display color stimuli to the patient 106 during a color vision screening test, or a Snellen visual acuity chart during a visual acuity screening test. The display screens 122 and 124 may be integrated with the vision screening device 104 or may be external to the device and under the control of a computer program of the device 104.

[0032] The vision screening device 104 can use its multiple network interfaces 126 to transmit data captured by the multiple radiation sensors 116 and camera 120 via network 108. Furthermore, the vision screening device 104 can similarly transmit other test data (e.g., test type, test duration, patient identity, etc.) related to the multiple vision screening tests being performed. The multiple network interfaces 126 of the vision screening device 104 can be operatively connected to one or more processors 128 of the vision screening device 104 and can enable wired and / or wireless communication between the vision screening device 104 and one or more components of the vision screening system 110, as well as with one or more other remote systems and / or other networked devices. For example, the multiple network interfaces 126 may include a personal area network component for enabling communication over one or more short-range wireless communication channels and / or a wide area network component for enabling communication over a wide area network. In any of the examples described herein, network interfaces 126 may enable communication between, for example, the processor 128 of the vision screening device 104 and the vision screening system 110 via network 108. Figure 1 The network 108 shown can be any type of wireless network or other communication network known in the art. Examples of network 108 include the Internet, intranet, wide area network (WAN), local area network (LAN) and virtual private network (VPN), cellular network connection, and connection using protocols such as 802.11a, b, g, n and / or ac.

[0033] Vision screening system 110 can be configured to receive data collected during the performance of multiple vision screening tests from vision screening device 104 via network 108. In some examples, based at least in part on processing the data, vision screening system 110 can determine output 112 relevant to patient 106. For example, output 112 may include recommendations and / or diagnoses related to the eye health of patient 106, based on analysis of color image data and / or NIR image data indicating diseases and / or abnormalities related to the eye of patient 106. Vision screening system 110 can transmit output 112 via network 108 to processors 128 of vision screening device 104. As described above, alternatively or additionally, vision screening device 104 may generate one or more such recommendations, diagnoses, or other outputs.

[0034] As described herein, a processor (such as processor(s) 128) may be a single processing unit or multiple processing units, and may include single or multiple computing units or multiple processing cores. Processor(s) 128 may be implemented as one or more microprocessors, microcomputers, microcontrollers, digital signal processors, central processing units, state machines, logic circuits, and / or any means of manipulating signals based on operating instructions. For example, processor(s) 128 may be one or more hardware processors and / or any suitable type of logic circuitry specifically programmed or configured to perform the algorithms and processes described herein. Figure 1 As schematically illustrated, the vision screening device 104 may further include a computer-readable medium 130 operatively connected to processor(s) 128. The processor(s) 128 may be configured to acquire and execute computer-readable instructions stored in the computer-readable medium 130, which may program the processor(s) 128 to perform the functions described herein.

[0035] Computer-readable medium 130 may include volatile and non-volatile memory and / or removable and non-removable media implemented in any type of technology for storing information, such as computer-readable instructions, data structures, program modules, or other data. Such computer-readable medium 130 may include, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technologies, optical storage, solid-state storage, magnetic tape, disk storage, RAID storage systems, storage arrays, network-connected storage, storage area networks, cloud storage, disk storage, etc., or any other medium that can be used to store desired information and is accessible by a computing device. Computer-readable medium 130 may be a type of computer-readable storage medium and / or may be a tangible non-transitory medium within the scope of: when referred to, non-transitory computer-readable medium excludes media such as energy, carrier signals, electromagnetic waves, and the signal itself.

[0036] Computer-readable medium 130 may be used to store any number of functional components executable by processor(s) 128. In the example, these functional components include instructions or programs executable by processor(s) 128, and when executed, these instructions or programs specifically configure one or more processors 128 to perform one or more actions related to a vision screening test for the detection and diagnosis of diseases and abnormalities in (multiple) eyes. For example, computer-readable medium 130 may store one or more functional components for performing the vision screening test, such as patient screening component 132, image capture control component 134, data analysis and visualization component 136, and / or output generation component 138, such as... Figure 1 As illustrated below, at least some of the functional components of the vision screening device 104 will be described in detail below.

[0037] In this example, the patient screening component 132 can be configured to store and / or access patient data 140 associated with patient 106. For example, patient data 140 may include demographic information such as name, age, and ethnicity. When the vision screening device 104 and / or the vision screening system 110 initiates a vision screening test, patient 106 can provide patient data 140 regarding the patient's demographics, medical information, preferences, etc., or operator 102 can request patient data 140 regarding the patient's demographics, medical information, preferences, etc., from patient 106 or patient 106's guardian. In such examples, operator 102 can request the data during screening or before screening begins. In some examples, operator 102 may be provided with a predetermined category associated with patient 106, such as a predetermined age range (e.g., newborn to six months, six to twelve months, one to five years, etc.), and patient data 140 may be requested to select the appropriate category associated with patient 106. In other examples, operator 102 may be provided with free-form input associated with patient data 140. In some other examples, the input element can be provided directly to the patient 106.

[0038] Alternatively or additionally, the vision screening device 104 and / or vision screening system 110 may determine and / or detect patient data 140 during a vision screening test. For example, the vision screening device 104 may include one or more digital cameras, motion sensors, proximity sensors, or other image capture devices configured to collect image and / or video data of the patient 106, and one or more processors of the vision screening device 104 may analyze the data to determine patient data 140, such as the age category of the patient 106 or the distance between the patient 106 and the screening device. For example, the vision screening device 104 may be equipped with a rangefinder, such as an ultrasonic rangefinder, an infrared rangefinder, and / or any other proximity sensor capable of determining the distance between the patient 106 and the screening device.

[0039] Alternatively or additionally, the vision screening device 104 may be configured to transmit image / video data via network 108 to the vision screening system 110 for analysis to determine patient data 140. Furthermore, the patient screening component 132 may be configured to receive, access, and / or store patient data 140 associated with patient 106 and / or other patients. For example, the patient screening component 132 may store previous patient information associated with patient 106 and / or other patients. For example, the patient screening component 132 may store the previous screening history of patient 106, including data from previous screenings, such as color images, NIR images, and / or videos of patient 106's (multiple) eyes. The patient screening component 132 may receive patient data 140 via network 108 and / or may access such information. For example, the patient screening component 132 may access an external database, such as a screening database 144, that stores data associated with patient 106 and / or other patients. The screening database 144 may be configured to store patient data 140 associated with patient IDs. When operator 102 and / or patient 106 enters a patient ID, patient screening component 132 can access or receive patient data 140 stored in association with the patient ID of patient 106.

[0040] In this example, patient screening component 132 can be configured to determine the vision screening tests to be performed on patient 106, at least in part, based on patient data 140. For example, patient screening component 132 can utilize patient data 140 to determine the test category to which patient 106 belongs (e.g., a test category based on age, medical history, etc.). Patient screening component 132 can determine the vision screening(s) to be performed based on the test category. For example, if patient data 140 indicates that the patient is a newborn, the selected vision screening(s) may include screening for congenital eye conditions such as congenital cataracts, retinoblastoma, corneal opacity, strabismus, etc. Furthermore, eye abnormalities may be associated with systemic genetic diseases such as Marfan syndrome and Tay-Sachs disease. For example, a screening test for a characteristic erythema in the eye may indicate Tay-Sachs disease. As another example, if patient data 140 indicates that the patient is over 50 years old, then patient screening component 132 can determine multiple vision screening tests, including screening for the onset of cataracts, macular degeneration, and other age-related eye diseases.

[0041] The patient screening component 132 can also determine multiple vision screening tests based on the patient's medical history. For example, the screening database 144 can store medical history related to the patient 106's previous vision screening tests in the patient data 140, including test results, images of the (multiple) eyes, measurements, recommendations, etc. The patient screening component 132 can access the patient data 140, which includes medical history, from the screening database 144 and determine the vision screening tests to be performed to monitor the status and changes of previously detected vision health problems. For example, if a progressive eye disease, such as the onset of cataracts or macular degeneration, is detected in a previous screening, further screening can be performed to track the development of the disease. As another example, if the patient 106 has undergone surgery to remove tumors from the (multiple) eyes, the multiple vision screening tests can include screening for additional tumors or scars in the (multiple) eyes. The patient screening component 132 can determine a list of vision screening tests to be performed on the patient 106 during the vision screening period, and keep track of the vision screening tests that have been performed during the vision screening period, as well as the remaining vision screening tests on the list of vision screening tests to be performed.

[0042] In some examples, the computer-readable medium 130 may additionally store an image capture control unit 134. The image capture control unit 134 may be configured to operate the vision screening device 104's plurality of radiation sources 114, plurality of radiation sensors 116, plurality of white light sources 118, and camera 120 to capture images of the plurality of eyes under each desired specific illumination condition in a plurality of specific vision screening tests. As discussed, the plurality of radiation sources 114 may include a plurality of NIR LEDs for illuminating the plurality of eyes during grayscale image capture to measure refractive errors and / or fixation angles of the plurality of eyes of patient 106, and the plurality of white light sources 118 may include a plurality of white LEDs for illuminating the eyes during color image capture by camera 120. In the examples, the image capture control unit 134 may generate commands to operate and control the individual radiation sources, such as the LEDs of the NIR LEDs and the LEDs of the white light sources 118. Control parameters for the LEDs may include intensity, duration, mode, and cycle time. For example, commands can selectively activate and deactivate individual LEDs of radiation source 114 and white light source 118 to generate illumination from different angles as required by the vision screening test indicated by patient screening component 132. Image capture control component 134 can activate the NIR LEDs of radiation source 114, used to measure refractive errors and / or fixation angles of patient 106's (multiple eyes), synchronously with images captured by radiation sensor 116 during the vision screening test. Similarly, image capture control component 134 can activate the LEDs of white light source 118 synchronously with color images of the eyes captured by camera 120.

[0043] Based on control parameters stored in the computer-readable medium 130, the image capture control unit 134 can control individual radiation sources, such as LEDs, of the plurality of radiation sources 114 or the plurality of white light sources 118. For example, the control parameters may include the intensity, duration, mode, cycle time, etc., of the NIR LEDs of the plurality of radiation sources 114 and / or the LEDs that generate white light from the plurality of white light sources 118. For example, the image capture control unit 134 can use the control parameters to determine the duration of radiation emitted by each LED of the plurality of radiation sources 114, 118 (e.g., 50 milliseconds, 100 milliseconds, 200 milliseconds, etc.). Furthermore, the image capture controller 134 can utilize the control parameters to change the intensity and display mode of the NIR LEDs of the plurality of radiation sources 114 to determine refractive errors of the eyes based on the photorefractive properties and / or fixation angle of the eyes. Regarding intensity, the image capture control unit 134 can control parameters to instruct the LEDs of the multiple white light sources 118 to emit light at an intensity sufficient to capture a color image of the eyes using the camera 120, while also limiting the brightness to avoid or reduce pupil constriction or focusing. The image capture control unit 134 can also control the intensity of the multiple white light sources 118 to gradually increase the intensity at a certain rate, while activating the camera 120 to capture images and / or video of the eyes to record the pupil's response to the increased illumination intensity.

[0044] Furthermore, the image capture control unit 134 can cause radiation to be emitted sequentially from the plurality of light sources 114, 118, thereby activating the NIR LEDs and capturing images of the plurality of eyes under NIR radiation before activating the LEDs of the white light source 118. In some examples, this sequencing can prevent pupil constriction in response to white light shining on the plurality of eyes, and / or can allow images of the internal structures of the plurality of eyes to be captured without requiring pupil dilation of the patient 106. In some examples, during vision screening, the image capture control unit 134 can additionally control the plurality of radiation sources 114, 118 to generate patterns such as circular patterns, alternating light patterns, flashing patterns, or shape patterns such as circular or rectangular patterns to attract the attention of the patient 106, and / or control the colored LEDs of the plurality of radiation sources 114, 118 to display colored stimuli such as colored dot patterns to the patient 106.

[0045] Image capture control unit 134 can also control multiple radiation sensors 116 and camera 120 to capture images and / or videos of multiple eyes of patient 106 during the implementation of multiple vision screening tests. For example, multiple radiation sensors 116 can capture data indicating radiation reflected from multiple eyes of patient 106 during activation of one or more radiation sources 114. The data may include grayscale image data and / or video data of multiple eyes. Image capture control unit 134 can synchronize camera 120, which captures multiple color images and / or video data of multiple eyes, with activation of multiple white light sources 118, such that multiple eyes are illuminated by white light radiation during the capture of color images and / or video data. In some examples, images of the left and right eyes can be captured under different illumination conditions (e.g., from different individual light sources) such that the relative angle of illumination to the visual axis of a particular eye is the same for both the left and right eyes. In other examples, images of both eyes can be captured simultaneously under the same illumination. As described herein, the image capture control unit 134 of the vision screening device 104 can generate grayscale images of (multiple) eyes illuminated under NIR radiation, as well as color images of (multiple) eyes illuminated under white light. Capturing both grayscale and color images enables the detection of a wider range of eye diseases and abnormalities.

[0046] In some examples, the computer-readable medium 130 may also store a data analysis and visualization component 136. The data analysis and visualization component 136 may be configured to analyze image and / or video data collected, detected, and / or otherwise captured by components of the vision screening device 104 (e.g., by the multiple radiation sensors 116 and camera 120) during one or more vision screening tests. For example, the data analysis and visualization component 136 may analyze the data to determine the location of the pupils of (multiple) eyes in the images and identify portions of (multiple) images corresponding to the pupils (e.g., (multiple) pupil images). The data analysis and visualization component 136 may analyze (multiple) pupil images to determine a characterization of the appearance of (multiple) pupils in the (multiple) pupil images. For example, in an example of (multiple) color images captured by camera 120, the characterization may include values ​​corresponding to average color, color variance, measurements of uniformity, the presence of inclusions, etc. In instances of infrared images captured by the radiation sensors 116, characterization may include, in addition to measurements of uniformity and the presence of inclusions, average gray values ​​and the variance of gray values ​​(rather than color). The data analysis and visualization component 136 may also compare the left pupil images and the right pupil images to determine appearance differences between the left and right pupils. For example, the difference may correspond to a difference in average color value, average gray value, or uniformity between the left and right pupil images. In a normal eye, the expected value of a characteristic associated with one pupil image (e.g., average color value, gray value, uniformity measurement, etc.) may be approximately the same as the value of the characteristic in another pupil image. The data analysis and visualization component 132 may also compare the pupil images to the standard pupil images and / or to the pupil images of patient 106 captured during previous vision screenings to determine appearance differences, such as differences in average color value or gray value, differences in uniformity measurements, differences in detected inclusions, etc. In such examples, the expected value of the characteristics of (multiple) pupil images may correspond to the characteristic values ​​in (multiple) standard pupil images or (multiple) previously captured pupil images of patient 106. In any of the above examples, the difference may be determined using all (multiple) captured images or a subset of captured grayscale and / or color images. In some examples, the difference may not be used, but may be determined based on (multiple) color images. It should be noted that pixels in a grayscale image can also be considered to have color values, determined by using the same grayscale value for each of the three color channels (e.g., RGB). For example, a pixel with a grayscale value of 128 may be determined to have a color value of (128, 128, 128) in the RGB color space.The data analysis and visualization component 136 can also apply additional image processing steps to (multiple) grayscale images and / or (multiple) color images, which can improve the detection of disease states. For example, images can be sharpened, specific colors can be enhanced or attenuated, the colors or brightness of images can be balanced, and so on. Further details related to the above-described data analysis used to determine the differences are described below with reference to Figure 4.

[0047] Furthermore, the data analysis and visualization component 136 can be configured to receive, access, and / or analyze standard data related to vision screening. For example, the data analysis and visualization component 136 can be configured to access or receive data from one or more additional databases (e.g., screening database 144, third-party databases, etc.), which store test data, measurements, and / or values ​​indicating various thresholds or ranges where the measured values ​​should fall. Such thresholds or ranges can be relevant to a patient with normal visual health and can be learned from standard tests or otherwise determined. The data analysis and visualization component 136 can utilize the standard data to compare with averages and differences determined as described above during the vision screening tests(s). For example, the standard data can indicate a threshold or range for the difference between color values ​​of the left and right pupil images, with differences greater than the threshold or outside the range corresponding to abnormalities in the patient's(s) eyes(s). Alternatively or additionally, the data analysis and visualization component 136 can access the patient 106's previous vision screenings and compare those values ​​and differences with corresponding data from the(s) previous screenings(s). For example, the average color value of the pupils can be compared with the average color value from previous screenings to determine the difference. This difference can then be compared with a standard threshold or range, as described above, to determine the presence of an abnormality. For different types of diseases and abnormalities, individual thresholds and / or ranges can be indicated in the standard data. Furthermore, the thresholds and / or ranges associated with the vision screening tests can also be based on the test category of patient 106 (e.g., patient 106's age group or medical history), and the thresholds and / or ranges can differ for different test categories. The data analysis and visualization component 136 can store the images and / or videos captured or generated during the vision screening tests, measurements associated with the vision screening tests, test results, and other data in a database (e.g., in screening database 144) as part of patient data 140 for comparing data over time to monitor vision health status and changes in vision health. In some examples, the stored images may include images of the patient 106's face or parts of the face (e.g., parts of the eyes and nose).

[0048] Based on comparisons with the aforementioned thresholds and / or ranges, the data analysis and visualization component 136 can generate a normal / abnormal or pass / refer determination for each eye of patient 106. For example, if all measured values ​​and differences are less than or equal to the corresponding threshold(s) or fall within the corresponding range(s) of standard data, the data analysis and visualization component 136 can make a "normal" or "pass" determination; otherwise, it will make an "abnormal" or "reject" determination to indicate referral for further screening. Alternatively or additionally, the data analysis and visualization component 136 can generate a normal / abnormal determination for each disease and / or abnormality screened during the vision screening period.

[0049] In the example, the data analysis and visualization component 136 can utilize one or more machine learning techniques to generate diagnoses for specific disease and / or abnormality types. For example, a machine learning (ML) model can be trained using images of normal eyes and images of eyes labeled to show various disease conditions and abnormalities. When provided as input, eye images captured during a vision screening of patient 106, the trained ML model(s) can then generate output indicating a disease or abnormality diagnosis. In such examples, the data analysis and visualization component 136 can directly generate output by providing the eye image as input to the trained ML model(s) without calculating differences between pupil images or applying comparisons with thresholds and / or ranges. In some examples, multiple trained ML models(s) can be used, each trained to detect a specific disease or abnormality. In such examples, each ML model outputs a binary presence / absence indication to indicate whether the input image shows a disease or abnormality that the ML model was trained to detect. The data analysis and visualization component 136 can provide the eye image as input to each of the multiple trained ML models for detecting one or more of a variety of diseases and abnormalities. In the example, the ML model can be a neural network, including convolutional neural networks (CNNs). In other examples, the ML model can also include regression algorithms, decision tree algorithms, Bayesian classification algorithms, clustering algorithms, support vector machines (SVMs), etc.

[0050] The data analysis and visualization component 136 can also generate visualizations of (multiple) eyes using image and / or video data captured by (multiple) radiation sensors 116 and / or camera 120. For example, the first visualization may include a composite image of (multiple) eyes that combines color information from (multiple) color images and grayscale information from (multiple) grayscale images captured by (multiple) radiation sensors 116 under NIR illumination. Generating the first visualization may include detecting and identifying structures of (multiple) eyes, such as the pupil and / or lens, and then registering (multiple) grayscale images and (multiple) color images such that the pupil is located in the same position in both types of images. The composite image can then be generated by using grayscale pixel values ​​from (multiple) grayscale images in certain portions of the composite image and color pixel values ​​from (multiple) color images in other portions of the composite image. The portions of the composite image using grayscale pixel values ​​and the portions using color pixel values ​​may correspond to regions depicting different structures of (multiple) eyes (e.g., fovea, retina, cornea, etc.). Synthetic images can more clearly depict the structure of (multiple) eyes, thereby improving the detection and assessment of (multiple) eye diseases and / or abnormalities.

[0051] In another example, the second visualization may include a sequence of still images, or an animated video containing a sequence of still images. In some cases, the image sequence may be captured by multiple radiation sensors 116 or cameras 120 when the eyes are illuminated by multiple radiation sources 114 or multiple white light sources 118 at a series of different angles (including angles approximately zero degrees relative to the visual axis) along different axes relative to the visual axis. For example, in some examples, the radiation emitted by multiple radiation sources 114 or multiple white light sources 118 may be emitted substantially parallel to and / or substantially along the visual axis. In such examples, the emitted radiation may be coaxial with or nearly coaxial with the visual axis. In some examples, the visualization may include graphical and / or color coding indicating areas of abnormality in the eye image. (See below for reference.) Figure 5A and Figure 5B Further examples of visualizations are described, including additional details relating to the first and second visualizations. As described herein, the data analysis and visualization component 136 of the vision screening device 104 can process grayscale and color images of (multiple) eyes captured during the implementation of multiple vision screening tests to identify diseases and / or abnormalities associated with (multiple) eyes of a patient. Furthermore, the data analysis and visualization component 136 can generate visualizations of (multiple) eyes based on the grayscale and color images, which assist the clinician or operator of the vision screening device 104 in identifying diseases and / or abnormalities in (multiple) eyes.

[0052] Computer-readable medium 130 may additionally store output generation component 138. Output generation component 138 may be configured to receive, access, and / or analyze data from data analysis and visualization component 136 and generate output 112. For example, output generation component 138 may use the normal / abnormal determination of data analysis and visualization component 136 to generate recommendations in output 112. Recommendations may indicate whether the screening results of patient 106 indicate normal eye health, or whether further screening is required based on one or more screening tests that led to the "abnormal" finding. Furthermore, output generation component 138 may incorporate all or a subset of visualizations generated by data analysis and visualization component 136 into output 112 to aid in diagnosing the condition of the eye. Parts of images and / or videos captured by radiation sensors 116 or camera 120 may also be included in output 112. Additionally, if an abnormality is determined, output generation component 138 may incorporate possible diagnoses into output 112 based on the analysis of data analysis and visualization component 136. Output 112 can be presented to the operator of the device via the device's interface (e.g., on the display screen 122 of the vision screening device 104). In this example, the operator's display screen may be invisible to the patient; for example, the operator's display screen may be facing away from the patient. Output generation component 138 can also store output 112 (which may include suggestions, diagnoses, measurements, captured images / videos, and / or generated visualizations) in a database (e.g., screening database 144) for clinician evaluation or for access during subsequent vision screenings(s) of patient 106. Screening database 144 can provide access to authorized medical professionals to print reports or further evaluate data related to patient 106's screenings.

[0053] although Figure 1Example processor(s) 128 and computer-readable medium 130 are illustrated, storing patient screening component 132, image capture control component 134, data analysis and visualization component 136, output generation component 138, and / or other components and / or other items as components of vision screening device 104. However, in any example described herein, vision screening system 110 may include similar and / or identical components. In such examples, vision screening system 110 may include processor(s) 146 and computer-readable memory 148, configured to perform the functions of some or all of the components in computer-readable memory 130 of vision screening device 104. For example, one or more components of computer-readable memory 130 may be included in analysis component(s) 150 of computer-readable memory 148 and may be executed by processor(s) 146. In such examples, vision screening system 110 may use multiple network interfaces 152 and communicate with vision screening device 104 via network 108 to receive data from vision screening device 104 and send results (e.g., output 112) back to vision screening device 103. Vision screening system 110 may be implemented on a computer near vision screening device 104 or may be located remotely. For example, vision screening system 110 may be implemented as a cloud service on a remote cloud server.

[0054] Multiple network interfaces 152 can enable wired and / or wireless communication between components and / or devices shown in system 100 and / or with one or more other remote systems and other networked devices. For example, at least some of the multiple network interfaces 152 may include personal area network (PAN) components to enable communication over one or more short-range wireless communication channels. Furthermore, at least some of the multiple network interfaces 152 may include wide area network (WAN) components to enable communication over a wide area network. Such multiple network interfaces 152 can enable communication, for example, between vision screening system 110 and vision screening device 104 and / or other components of system 100 via network 108. For example, multiple network interfaces 152 may be configured to connect to an external database (e.g., screening database 144) to receive, access, and / or transmit screening data using a wireless connection. The wireless connection may include a cellular network connection and a connection using protocols such as 802.11a, b, g, and / or ac. In other examples, one or more wireless protocols (such as Bluetooth, Wi-Fi Direct, RFID, infrared signals, and / or Zigbee) can be used to establish a direct wireless connection between the vision screening device 104 and an external system. Other configurations are also possible. Communication with external databases can enable the printing of reports or further evaluation of the patient's vision test data. For example, the collected data and corresponding test results can be wirelessly transmitted and stored in a remote database accessible to authorized medical professionals.

[0055] It should be understood that, although Figure 1 System 100 is described as including a single vision screening system 110, but in other examples, system 100 may include any number of local or remote vision screening systems that are substantially similar to vision screening system 100, configured to operate independently and / or in combination, and configured to communicate via network 108.

[0056] As discussed in this article, Figure 1 An example vision screening device 104 is depicted, which includes components for performing vision screening tests on a patient. In some examples, one or more components may be implemented on a remote vision screening system 110 that communicates with the vision screening device 104 via a network 108. The vision screening device 104 and its components will be described in detail with reference to the remaining figures.

[0057] Figure 2Examples of vision screening devices 200 according to some implementations are illustrated. Example vision screening devices 200 may include one or more of the same components included in vision screening device 104 of system 100. In some other examples, vision screening device 200 may include different components that provide similar functionality to vision screening device 104.

[0058] The vision screening device 200 may be a similar tablet device, which may include one or more processors, computer-readable media, and associated network interfaces (not shown) within a housing 202. The housing 202 may include a front surface 204 and a rear surface 206, the front surface 204 being configured to face a patient (e.g., patient 106) during use of the vision screening device 200, and the rear surface 206, opposite the front surface 204, being configured to face the operator of the vision screening device 200 (e.g., operator 102) during use of the vision screening device 200. The front surface 204 may include: a display 208 that may be substantially similar to or identical to display 124; a plurality of radiation sources 210 that may be substantially similar to or identical to radiation sources 114; a plurality of radiation sensors 212 that may be substantially similar to or identical to radiation sensors 116; a plurality of white light sources 214 that may be substantially similar to or identical to white light sources 118; and / or a camera 216 that may be substantially similar to or identical to camera 120.

[0059] Multiple radiation sources 210 can be configured to emit radiation in the infrared and / or near-infrared (NIR) bands. For example, the multiple radiation sources 210 may include an arrangement of NIR LEDs configured to determine refractive errors associated with one or more of a patient's eyes. The NIR LEDs of the multiple radiation sources 210 may be arranged radially around a central axis 211 of the vision screening device 200, and multiple radiation sensors 212 are arranged substantially along the central axis 211. The NIR LEDs can be used to provide off-center illumination of the patient's eyes during multiple vision screening tests by aligning the central axis 211 with the visual axis of the eye (e.g., for measuring refractive errors using photorefractive techniques). (See reference...) Figure 3B The arrangement of NIR LEDs is described in further detail.

[0060] The vision screening device 200 may also include a white light source 214 and a visible light camera 216, the visible light camera 216 being configured to capture color images and / or video of the patient's eyes. In some examples, the white light source 214 and camera 216 may be included in an image capture module 218. The camera 216 of the image capture module 218 may include a high-resolution lens with a narrow field of view, suitable for imaging the eyes during vision screening tests. Such lenses may incorporate folded prism thin lens technology, which allows for telephoto zoom while maintaining a low profile. The optical system used in folded prism lenses bends and focuses light as it is reflected back and forth within an optical prism, thereby reducing the thickness of the lens and allowing for a substantially low profile. (See above reference...) Figure 1 As discussed, some vision screening tests may require color images of the patient's pupil and / or lens to determine the presence of disease and / or abnormality. In some examples, camera 216 may be equipped with high-resolution zoom capability to capture close-up images of the patient's eyes, from which the pupil and / or lens can be located. In other examples, camera 216 may use a fixed-focus lens, adjusting the eye's placement point to obtain a focused image. A white light source 214, more commonly referred to as a flash, may include one or more intensity-adjustable visible light LEDs. The intensity level of the white light source 214 may be controlled by one or more processors of the vision screening device 200. One or more processors of the vision screening device 200 may also synchronize the activation timing of the white light source 214 with the images captured by camera 216.

[0061] The vision screening device 200 may also include a display screen 220 disposed on the rear surface 206 of the housing 202, which is substantially oriented towards the operator (e.g., operator 102) during operation of the vision screening device 100. The input display screen 220 may be touch-sensitive to receive input from the operator and may display a graphical user interface configured to display information to and / or receive input from the operator during vision screening tests. For example, the operator may use the display screen 220 to input information about the patient or the vision screening test(s) being performed. Furthermore, the display screen 220 may be configured to display information to the operator about the vision screening test being performed (e.g., parameter settings, screening progress, options for sending data from the vision screening device 200, one or more measurements, and / or images or visualizations generated during vision screening, etc.). The displays 208 and 220 may include, for example, liquid crystal displays (LCDs) or active-matrix organic light-emitting diode (AMOLEDs).

[0062] In some examples, the vision screening device 200 may include handles 222a and 222b for maintaining the stability of the vision screening device during vision screening tests. As discussed herein, Figure 2 An exemplary vision screening device 200 is depicted, comprising components for administering one or more vision screening tests to a patient. The vision screening device 200 is designed to perform a complete vision screening, which may include a variety of different vision screening tests, including screening for various diseases, abnormalities, and conditions of the patient's eyes. As shown, the vision screening device 200, exemplified by handles 222a and 222b for the operator's right and left hands respectively, is also lightweight enough to be easily held, allowing for portability and ease of use in young patients, such as newborns. The vision screening device 200 achieves a lightweight and portable form factor by providing the radiation source and image capture sensor required for NIR imaging, as well as color imaging under white light illumination as required by one or more vision screening tests, in a compact and substantially planar arrangement.

[0063] Figure 3A Another embodiment of a vision screening device 300 according to some implementations is illustrated. The example vision screening device 300 may include one or more of the same components included in vision screening devices 104, 200. In some other examples, the vision screening device 300 may include different components that provide similar functionality to vision screening devices 104, 200.

[0064] In the example shown, the vision screening device 300 includes a housing 302 having a transparent display screen 304 (such as a transparent organic light-emitting display (OLED)) facing a first end 306 of the vision screening device 100, which faces a patient (e.g., patient 106). The display screen 304 may cover optical components of the vision screening device 300, such as an LED array 308 that may be substantially similar to or the same as the radiation source(s) 114; a plurality of radiation sensors 310 that may be substantially similar to or the same as the radiation sensor(s) 116; and an image capture module 312 that includes a white light source 312a and a digital camera 312b that may be substantially similar to or the same as the white light source(s) 118 and camera 120. Although the white light source 312a is shown close to the digital camera 312b, in some examples, the white light source 312a and / or other white light sources(s) may be positioned at other locations on the housing 302 (e.g., the white light source 312a and / or other white light sources(s) may be positioned along or near one or more sides or edges of the housing 302, at one or more corners 315 of the housing 302, and / or at any other location). Since the display screen 304 is transparent, radiation from the radiation sources(s) 308 and / or white light from the white light source 312a of the image capture module 312 can reach the patient's(s) eyes, and reflected radiation from the patient's(s) eyes can be received by the radiation sensors(s) 310 and / or the camera 312b of the image capture module 312 without attenuation or change of direction as it travels through the display screen 304. Array 308 may include individual NIR LEDs (e.g., NIR LEDs 308a, 308b, 308c, 308d) patterned around radiation sensors 310, as shown. Also as shown, NIR LEDs 308a-308d may be arranged along different axes, such as axes A-A', B-B', and C-C', which will refer to... Figure 3B Further detailed description. Multiple radiation sensors 310 are positioned substantially along the central axis 314 of the vision screening device 300, which is analogous to the central axis 211 of the vision screening device 200. Although the individual NIR LEDs of the array 308 are shown radiating outwards from the centrally positioned radiation sensors 310, other arrangements of the NIR LEDs in the array 308 with more or fewer individual NIR LEDs are also conceivable. See reference... Figure 2 The arrangement of the NIR LEDs in array 308 described herein can provide the off-center illumination required for measuring refractive errors using photorefractive techniques.

[0065] Figure 3BAn enlarged view of an array 308 of NIR LEDs is shown. In the example, array 308 may include more or fewer individual LEDs than those shown in the example. Figure 3B The example arrangement of the individual NIR LEDs in the array 308 shown includes rows 316 of NIR LEDs extending substantially coaxially through the array 308 along a first axis M1, which may correspond to... Figure 3A The axis is A-A'. As shown in the figure, array 308 may also include rows 318 along the second axis M2 and rows 320 along the third axis M3, the second axis M2 corresponding to... Figure 3A The first axis M1 is the axis B-B' and forms an angle θ with respect to row 316. The second axis M2 can correspond to the axis C-C' and form an angle α with respect to row 318. Angles θ and α can be any acute angle (e.g., 60°). In the example, axis M1 can also be referred to as the first meridian of array 308, axis M2 as the second meridian of array 308, and axis M3 as the third meridian of array 308, and the three axes M1, M2, and M3 can intersect at the central axis 314 of the visual screening device 300. In some examples, additional LEDs can be positioned spaced apart from array 308 along one or more axes M1, M2, and M3.

[0066] For reference Figure 3C In a further detailed description, the individual LEDs of array 308 can be activated sequentially (e.g., by image capture control unit 134) to produce illumination at different angles or eccentricities. For example, LED 322 can be activated sequentially and progressively along axis M2, first LED 322, then the adjacent LED 324, and so on, up to LED 326. The activation sequence can be moved along a first axis M1, then along a second axis M2 and a third axis M3 through the individual LEDs. Furthermore, LEDs 322 and 324 can also be activated substantially simultaneously, thereby simulating the source location of the combined radiation using a diffuser (not shown). In such examples, the amount of current applied to LEDs 322 and 324 can also be controlled (e.g., by image capture control unit 134) to achieve the desired simulated source location of the combined radiation, thereby allowing illumination to be produced at a different angle or eccentricity without having to mechanically move array 308.

[0067] Figure 3C This is a schematic illustration of an example vision screening system 301 according to an example of this disclosure, which uses a vision screening device 300 to perform a vision screening test on a patient 106 by an operator 102. In the example, the central axis 314 of the vision screening device 300 may be substantially aligned with or collinear with the visual axis 328 of the patient's (multiple) eyes, as shown. (Refer to the above...) Figure 1The image capture control unit 134 of the vision screening device under discussion can control individual radiation sources, such as LEDs 308a-308d or 322, 324 of array 308, to emit radiation. The radiation emitted by each LED can be directed at different angles relative to the visual axis 328 onto the eyes of patient 106. For example, the radiation beam 330A emitted by LED 308b can be at an angle 332A to the visual axis 328, while the radiation beam 330B emitted by LED 308c can be at an angle 332B different from angle 332A. Therefore, individual or group activation of the LEDs of array 308 can be used to generate radiation directed at different angles onto the (multiple) eyes of patient 106. In some examples, angles 332A, 332B can be approximately zero degrees relative to, for example, the visual axis 328 (e.g., the illumination can be coaxial or nearly coaxial with the central axis 314). For each illumination angle, reflected radiation from the (multiple) eyes of the patient 106 traveling along the visual axis 328 can be captured by a radiation sensor 310 positioned along a central axis 314 aligned with the visual axis 328 to generate an image of the (multiple) eyes under illumination at each angle relative to the visual axis 328. Although described herein with reference to an array 308 of NIR LEDs, similar NIR LEDs to the array 308 described above are referenced below. Figure 3D Further description suggests that an array or series of individual white LEDs arranged in a two-dimensional array or linear pattern can be used to generate illumination from different angles relative to the viewing axis 328 by multiple white light sources 118, 214, 312a.

[0068] The vision screening device 300 may also include an additional display screen 334, which is mounted on the housing 302 of the vision screening device 100 on the side 336 opposite to the front side 306. The display screen 334 may face the operator 102 and functions similarly to a reference. Figure 1 The described display screen 122 is configured to provide operator 102 with information related to multiple vision screening tests. In some examples, display screen 334 may be detached from vision screening device 300 (e.g., not attached to housing 302), but may be operatively coupled to and controlled by vision screening device 300.

[0069] Figure 3D An example system 303 is illustrated, including components of a vision screening system 301 according to the present disclosure. Example system 303 illustrates a camera 338, a white LED array 340 including multiple white light sources 118, a diffuser 342, and a partial reflector 344. For clarity, other components of the vision screening system 301 are omitted from example system 303; however, it should be understood that, for example, any component of the vision screening device 300 or vision screening system 301 described above may be included. Figure 3D The example system shown.

[0070] In the example, radiation 346 (e.g., light) emitted by one or more LEDs of the LED array 340 passes through the diffuser 342 and illuminates a partial reflector 344. In the example, the partial reflector 344 may be a beam splitter, a mirror arrangement, a prism, or any other optical component configured to reflect a first portion of the radiation illuminating it while simultaneously transmitting a second portion of the radiation. In some examples, the partial reflector 344 is positioned at an angle of approximately 45 degrees relative to the central axis 314 of the vision screening system 301, 303, which may be substantially aligned with or coaxial with the visual axis of the patient 106's eye, as shown in the reference. Figure 3C As discussed, the central axis 314 can also be substantially aligned with or coaxial with the lens of the camera 338. The diffuser 342 can be used as a blur-smoothing filter for the radiation 346 emitted by the LEDs in the LED array 340. In some examples, the lens 348 can be configured to focus the radiation 346 emitted by the LED array 340 onto the partial reflector 344. However, one or more additional optical components may be included to modify the radiation 346 reaching the partial reflector 344. Figure 3D In the example shown, at least a portion 350 of radiation 346 can be reflected away from partial reflector 344 and can be directed to one or both eyes of patient 106. When a portion 350 of radiation 346 is directed to the eyes of patient 106, camera 338 can capture one or more images and / or videos of the eyes(s) of patient 106. In the example, the images and / or videos(s) can depict radiation reflected by the pupils(s) of the eyes(s) of patient 106.

[0071] In various examples, such as those referenced in this article Figure 2 and Figures 3A-3DThe vision screening devices 200 and 300 may include multiple NIR radiation sources and multiple sensors for capturing NIR images of multiple eyes of a patient, and multiple white light sources and a color camera for capturing color images of multiple eyes of a patient. The vision screening devices 200 and 300 may also capture images of multiple eyes under illumination from radiation sources at different angles relative to the visual axes of the multiple eyes. While the multiple radiation sources 114, 210, and 308 have been described as including multiple infrared or near-infrared (NIR) radiation sources, in other examples, the multiple radiation sources 114, 210, and 308 may include LEDs emitting radiation at different wavelengths, and the multiple radiation sensors 116, 212, and 310 may capture images of the eyes when illuminated by radiation at different wavelengths and / or different bands (e.g., infrared, NIR, visible light, ultraviolet, etc.). Radiation at different wavelengths in the visible spectrum may include wavelengths corresponding to specific colors. In such examples, vision screening devices 104, 200, and 300 may be able to detect (multiple) eye diseases and / or abnormalities that may be more apparent in images captured under illumination at a specific wavelength. Furthermore, since colored light and white light are also forms of electromagnetic radiation, the term "radiation source" as used herein may refer to visible light emitters as well as radiation emitters in the infrared / NIR and ultraviolet ranges of the electromagnetic spectrum.

[0072] Figures 4A-4D Images of eyes captured by the (multiple) radiation sensors 116, 212, 310 or cameras 120, 216, 312b of vision screening devices 104, 200, 300 are illustrated. (References) Figures 4A-4D The analysis of image data captured by vision screening devices 104, 200 or 300 was discussed to detect various abnormalities and / or diseases in (multiple) eyes. Figure 4A Image 402 illustrates a patient's eye, which is healthy and without detectable disease or abnormalities. Image 402 includes the patient's right eye 404a and left eye 404b. As shown, in patients exhibiting normal, healthy eyes, the iris 406a and pupil 408a of the right eye 404a appear substantially similar to the corresponding iris 406b and pupil 408b of the left eye 404b. (See reference...) Figure 1The data analysis and visualization component 136 can process the captured images to determine the location of the pupils of (multiple) eyes and generate images of the pupils (e.g., pupil images). Because the pupil allows radiation to enter the eye and allows reflected radiation to return after interacting with different layers of the eye, the pupil image captures the appearance of the layers of the eye illuminated by radiation shining on the eye, such as the cornea, lens, aqueous humor, vitreous humor, and retina. U.S. Patent Application No. 17 / 347,079, filed June 14, 2021 (the entire disclosure of which is incorporated herein by reference), describes an example system and method for detecting pupil images captured under different illumination modes generated by a near-infrared (NIR) radiation source for determining refractive errors based on photorefractive analysis.

[0073] Figure 4B An example image 410 is shown illustrating how a disease condition can be detected by comparing a pupil image 412a of one eye 414a with a pupil image 412b of the other eye 414b. Image 410 may be a grayscale image captured under NIR illumination by (multiple) radiation sensors 116, 212, 310, and an example of the grayscale difference between the left and right eye images is illustrated due to typical lens opacity caused by medial turbidity or cataract development. (See reference...) Figure 1 The data analysis and visualization component 136 can compare pupil images of the left and right eyes to determine the difference in grayscale values ​​between the eyes. For example, image 410 may have grayscale values ​​from 0 to 128, with the average grayscale value of the pupil portion of the image in one eye being 24, while for the other eye it may be 80. The calculated difference can be compared to one or more thresholds and / or ranges in standard test data corresponding to a normal eye to determine if an abnormality is present. However, grayscale images such as image 410 may fail to capture differences between eyes that correspond to some diseases and abnormalities that can be easily identified from color images. For example, a tumor in the retina or cornea of ​​the eye may appear as a uniform gray area, similar to the appearance of a normal retina in a grayscale image captured under NIR illumination.

[0074] Figure 4CAn example is illustrated of a color image 416 captured by multiple cameras 120, 216, 312b of a vision screening device 104, 200, 300 under white light illumination (e.g., from multiple white light sources 118, 214, 312a). Since the pupil images 420a, 420b of image 416 are generated from white light reflected back from the retina of the eye and passed through the cornea, diseases and abnormalities of the retina and cornea are visible in such images. For example, because the retina is densely vascularized, the reflected light may appear orange-red in a healthy eye, but may appear white or pale yellow in an eye with a retinal or corneal tumor. Although the color appearance of pupil images 420a, 420b may vary due to different pigment deposition in the retina among patients of different ethnicities, comparison between the two pupil images 420b and 420a of the same patient can reliably produce differences in color values ​​when only one eye has a disease or abnormality. See reference... Figure 1 The data analysis and visualization component 136 can compare pupil images of the left and right eyes to determine the difference between color values. The data analysis and visualization component 136 can also compare the color value of each pupil image with a standard image of the pupil of a healthy eye. The image 416 captured by the cameras(multiple) 120, 216, 312b can be a typical digital color image, where each pixel indicates an RGB (red, green, blue) value in the range of 0-256 for each of the three color channels. As is known in the art, the RGB color space is generally unsuitable for applications requiring the determination of differences between colors due to its sensitivity to changes in light and the correlation between the distance between colors and the perceived difference between colors. In this example, the color image 416 can be transformed into a color space more suitable for determining differences between colors (e.g., CIE L*a*b*, CIE L*u*v*, CIE 1931 model, HSI (hue, saturation, intensity), etc.). The difference between the average color value of pupil image 420a and the average color value of pupil image 420b can be determined in the transformed color space. In some examples, measurements of refractive errors in (multiple) eyes can be used to further adjust for the difference in color values ​​between the patient's eyes to eliminate the effects of eye refractive errors, which may also cause differences in pupil appearance. Differences greater than a threshold and / or outside the standard data range corresponding to a normal healthy eye can be flagged as detected abnormalities. For example, the color difference illustrated in image 416 could be the result of a tumor (such as retinoblastoma) in one of the eyes.

[0075] Figure 4DFurther examples include images 422 of eyes 424a and 424b, including pupil images 426a and 426b. Image 422 may be a grayscale image captured by (multiple) radiation sensors 116, 212, and 310 under NIR illumination, or a color image captured by (multiple) cameras 120, 216, and 312b under white light illumination. As shown, even though the average grayscale values ​​or average color values ​​in the pupil images 426a and 426b may be similar, other types of differences may exist between them, such as inhomogeneities, inclusions, or other structures, which may indicate disease conditions and / or abnormalities. For example, minute inhomogeneities or inclusions may indicate an early stage of cataract formation, the presence of a foreign body in the ocular media, scratches in the cornea or lens, etc. Data analysis and visualization component 136 can determine these types of differences between pupil images 426a and 426b using various methods. For example, aligning pupil images 426a and 426b and then subtracting the pixel values ​​of the pixels at the corresponding locations will produce a difference image that primarily indicates the areas of difference. The sum of the pixel values ​​in the difference image can be compared to a threshold to determine if the difference is higher than the threshold (e.g., in an anomalous situation). The data analysis and visualization component 136 can also determine the difference by calculating the variance of grayscale or color values ​​within pupil image 426b. If the variance is higher than the threshold, or outside the expected range in a healthy eye, an anomalous condition can be identified.

[0076] In some examples, the data analysis and visualization component 136 can determine certain conditions of the eye by evaluating each pupil image taken individually for the uniformity of characteristics in the pupil image. These characteristics may include color, brightness, texture, etc. For example, the data analysis and visualization component 136 can determine the standard deviation (or variance) of the characteristics within the pupil image, and if the standard deviation is higher than a threshold, or outside the range expected in a healthy eye, an abnormal condition can be identified.

[0077] although Figures 4A-4D Examples of some eye conditions that can be determined using the techniques discussed herein are shown, but it should be understood that other conditions can also be determined. Furthermore, the data analysis and visualization component 136 can analyze color images under white light illumination, grayscale images under NIR illumination, and / or composite images (as shown in the reference). Figure 1 (as described above) to determine the condition of the eye. For example, the presence of a cataract in the eye can be determined based on a synthetic image, while the presence of a blastoma can be determined primarily based on a color image. In some examples, color and / or grayscale images can be extracted from color and / or grayscale video of the eye (e.g., one or more frames of a video).

[0078] As described in this article, Figures 4A-4DExamples of processing of images captured by radiation sensors(s) 116, 212, 310 and / or cameras(s) 120, 216, 312b are illustrated. This processing can be performed by the data analysis and visualization component 136 of the vision screening device 104 to determine differences between pupil images indicating disease conditions and / or abnormalities in the patient's(s) eyes. Other examples of processing tailored for detecting specific disease conditions and abnormalities are also envisioned. For example, images captured under radiation at different wavelengths could be used to detect characteristic differences in grayscale values, color values, or structures of images indicating specific disease conditions.

[0079] Figure 5A and Figure 5B Example visualizations of pupil images that can be generated by vision screening devices 104, 200, or 300 are shown. (See reference...) Figure 1 As described, the data analysis and visualization component 136 can process the captured pupil images and generate one or more visualizations that can help clinicians or operators of the vision screening device diagnose abnormalities or diseases in the eyes of a patient (e.g., patient 106). As described above, the first visualization can generate a composite image 502 that combines information from grayscale images captured under NIR illumination by (multiple) radiation sensors 116, 212, 310 and color images captured under white light illumination by (multiple) cameras 120, 216, 312b. The data analysis and visualization component 136 can align (multiple) grayscale images and (multiple) color images, for example, by performing an image registration step such that the pupils 504 of the eyes in the images overlap each other. The data analysis and visualization component 136 can then perform a more precise image registration step based on specific features of the eye (e.g., the optic disc 506 and / or the fovea 508), such that (multiple) grayscale images and (multiple) color images are precisely aligned, for example, when the features of the eye overlap each other. In some examples, the registration step can be performed even when the vision screening device does not move between image captures, because small movements of the patient or the patient's eye movements may cause misalignment between images.

[0080] In the example, the composite image 502 can be generated, for example, by the data analysis and visualization component 136 from aligned grayscale and color images to include pixel values ​​of the grayscale images in some portions 510 and pixel values ​​of the color images in the remaining portions of the pupil region 504. The selection of images to be merged into different portions of the composite image 502 can be based on whether features in that portion are more easily distinguishable under NIR illumination or white light illumination. For example, portion 510 of the eye may include features or conditions that are better distinguishable under NIR illumination, while vascular structures in the remaining portions of the pupil image 504 may be clearer in a color image captured under white light illumination. In such an example, the data analysis and visualization component 136 can use grayscale values ​​from multiple grayscale images in region 510 of the composite image 502 and color values ​​from multiple color images in the remaining portions of the pupil image 504 to generate the composite image 502. In cases where multiple grayscale and color images have been captured, the data analysis and visualization component 136 can use the average grayscale value at each pixel location in the grayscale image and the average color value at each pixel location in the color image to generate a merged grayscale image and a merged color image. In other examples, a single image can be selected from the multiple images for use in the composite image 502 based on factors such as image quality (e.g., sharpness, illumination angle, or visibility of specific eye features). As described above, the composite image derives a first plurality of pixel values ​​from the grayscale image(s) captured under NIR illumination and a second plurality of pixel values ​​from the color image(s) captured under white light illumination. Therefore, the composite image includes eye characteristics independently revealed by NIR and visible light illumination.

[0081] In some examples, in addition to including the eyes, the synthesized image may include portions of the patient's face (e.g., nose, forehead), or even the entire face. In some examples, additionally or alternatively, the data analysis and visualization component 136 may add graphics 512 (e.g., pseudo-color) to the synthesized image 502 to highlight portions or regions of the synthesized image 502 in which differences are detected between the left and right pupil images or between the captured image and a standard image. In some examples, the colors used for highlighting (e.g., within graphics 512) may be based on a heatmap visualization scheme, where cooler or bluer hues may indicate smaller differences, while warmer or redder hues may indicate larger differences. Portions of the synthesized image 502 where the difference exceeds a standard threshold or is outside the standard range may be assigned pseudo-colors starting from the highest point of the heatmap. Other features of the eyes may also be highlighted using different graphics or different color legends. In such examples, clinicians or operators may choose to turn highlighting and / or graphics on or off, or switch between the two, to aid in diagnosing eye conditions. The composite image 502 can be stored in a database (e.g., screening database 144) as part of the patient data 140.

[0082] Figure 5B Visualization 514 illustrates a sequence of still images or an animated video containing a sequence of still images. The visualization may correspond to, as in the reference... Figure 1 The second visualization diagram described in data analysis and visualization component 136. Radiation source 516 (1-7) can correspond, for example, along... Figure 3B The NIR LEDs along the axis or meridian 316, 318, or 320. Alternatively or additionally, radiation source 516 (1-7) may include an array of white LEDs of (multiple) white light sources 118, 214, 312a. Pupil image 518 (1-7) indicates an image captured by (multiple) radiation sensors 116, 212, 310 or by cameras 120, 216, 312b under illumination from the respective radiation source 516 (1-7). For example, image 518 (1) may be captured when radiation source 516 (1) is enabled, image 518 (3) may be captured when radiation source 516 (3) is enabled, and so on. It should be noted that sequence 514 may not include an image corresponding to each radiation source (e.g., 516 (2, 4, 6)). In the example, any number of images corresponding to (multiple) radiation sources 516 (1-7) may be used when visualization 514 is generated. In addition, more or fewer radiation sources can be envisioned 516.

[0083] As referenced above Figure 5AThe images captured under illumination from various radiation sources (e.g., 516(1, 3, 5, 7)) are aligned such that the pupil appears in the same position relative to the overall image in each image. This alignment step or registration prevents jitter when the images are presented in sequence (e.g., on displays 122, 220). Visualization 514 may include the sequential presentation of images 518(1-7) on display screens 122, 220 from left to right (e.g., as image sequences 518(1), 518(3), 518(5), and 518(7)) and / or from right to left (e.g., as image sequences 518(7), 518(5), 518(3), and 518(1)). Furthermore, the data analysis and visualization component 136 may generate an animated video, each frame of which includes a single image from the sequence. The animated video may include repeatedly displaying a sequence from left to right and then from right to left to create the phenomenon of illumination sources moving back and forth from beginning to end. The vision screening device can (e.g., on displays 122, 220) present a graphical user interface that allows the clinician or operator of the vision screening device to pause the video at any frame and / or zoom in or out. Furthermore, the visualization 514 can use a grayscale image captured under NIR illumination, a color image captured under white light illumination, or the above references. Figure 5A The described composite images are 518 images (1, 3, 5, 7). The graphical user interface provides options to view grayscale images, color images, or composite images.

[0084] In various examples, such as those referenced in this article Figure 5A and Figure 5B The vision screening device 300 can provide the operator with visualizations to aid in the diagnosis of eye diseases and / or abnormalities in patients. Furthermore, these visualizations can be stored as part of patient data 140 (e.g., in a screening database 144), allowing clinicians to access them for review or comparison during future vision screening tests of the same patient.

[0085] Figures 6-8 Flowcharts illustrating example methods for visual screening as described in this article are provided. Figures 6-8The methods described are illustrated as a collection of blocks in a logic flowchart, which represents a sequence of operations that can be implemented in hardware, software, or a combination thereof. In the context of software, a block represents a computer-executable instruction stored on one or more computer-readable storage media that, when executed by the processor(s), performs the described operations. Typically, computer-executable instructions include routines, programs, objects, components, data structures, etc., that perform a specific function or implement a specific abstract data type. The order in which the operations are described is not intended to be construed as limiting, and any number of the described blocks can be combined in any order and / or in parallel to implement the operation. Figures 6-8 The method shown in the figure. In some embodiments, it can be completely omitted. Figures 6-8 One or more blocks of the illustrated method.

[0086] The following text is about Figures 6-8 The operations described by the illustrated methods can be performed by any of the apparatus or system 104, 200, and 300 described herein and / or by its various components. Unless otherwise specified, and for ease of description, reference will be made below. Figure 1 The system 100 shown and Figure 1 - Figure 3 describes the vision screening devices 104, 200, and 300. Figures 6-8 The methods illustrated in the text. Specifically, regarding... Figures 6-8 Any operation described in the methods illustrated herein may be performed, individually or in combination, by the image capture control unit 134, the data analysis and visualization unit 136 and / or the output generation unit 138 executed by the processor(s) 128 of the vision screening device 104, and / or by the analysis unit 150 executed by the processor(s) 146 of the vision screening system 110.

[0087] refer to Figure 6 In the example method 600 illustrated herein, at operation 602, the image capture control unit 134 and / or one or more processors associated therewith can cause a radiation source to emit radiation (e.g., near-infrared (NIR) radiation). For example, the radiation source may include NIR LEDs of radiation sources 114(s) of the vision screening device 104, the NIR LEDs being configured to emit NIR radiation during a time period at least partially corresponding to a vision screening test performed on a patient as instructed by the patient screening unit 132. In some examples, the image capture control unit 134 may cause the emission of radiation of different wavelengths; for example, a first radiation source may emit radiation of a first wavelength, while a second radiation source may emit radiation of a second wavelength. See reference... Figure 3B As described, the image capture control unit 134 can activate the LEDs of the radiation source(s) 114 individually or in groups to generate radiation that illuminates the eye at different angles relative to the visual axis of the eye. For example, as referenced above... Figure 3BAs described, the image capture control unit 134 can set the activation mode of the NIR LED along axes 316, 318, and 320. Furthermore, the image capture control unit 134 can activate radiation of different wavelengths in conjunction with different eye incidence angles. In some examples, the arrangement of the NIR LED activation modes allows different illumination patterns to be presented to the (multiple) eyes of the patient 106, and refractive errors of the (multiple) eyes can be accurately measured based on images captured under the selected illumination modes. Further details regarding the illumination patterns used in examination protocols for determining refractive errors can be found in U.S. Patent Application No. 9,237,846, which is cited above and incorporated herein by reference.

[0088] In operation 604, image capture control unit 134 can cause sensors of the vision screening device (e.g., multiple radiation sensors 116 of vision screening device 104) to capture radiation reflected from the patient's multiple eyes under illumination from multiple radiation sources 114. Image capture control unit 134 can receive data indicating the radiation captured by the multiple radiation sensors 116. The data may include multiple grayscale images and / or videos of the eyes illuminated by radiation from different angles, as described above in operation 602. For example, image capture control unit 134 can cause the sensors to capture a first image under illumination from a first set of NIR LEDs and a second image under illumination from a second set of NIR LEDs. In some examples, near-infrared may be a first band emitted by the multiple first radiation sources, and image capture control unit 134 can also activate a second radiation source emitting radiation in a second band and cause the sensors to capture a third image under illumination from the second radiation source. Furthermore, image capture control unit 134 can cause the multiple sensors to capture images of both eyes simultaneously or one eye at a time. For example, the image capture control unit 134 can change the activation of (e.g., activate different LEDs of (multiple) radiation sources 114, 118) after capturing the left-eye image and before capturing the right-eye image, thereby illuminating both the left and right eyes from the same angle relative to the eyes during image capture. In some examples, the image capture control unit 134 can cause the sensor to capture multiple images of the eyes while guiding the patient 106 to look in different directions, for example, the patient 106's gaze direction may be left, right, up, and / or down relative to the visual axis of the vision screening device 104.

[0089] In operation 606, the image capture control unit 134 and / or one or more processors associated therewith may cause a white light source (e.g., multiple white light sources 118 of the vision screening device 104) to emit white light to illuminate the patient for a period of time after operations 602 and 604 have been completed and during at least a portion of the implementation of the vision screening test. Similar to the radiation sources described in operation 602, individual white light sources of the multiple white light sources 118 may also be activated to illuminate the eye from different angles relative to the visual axis. In some examples, the illumination from the multiple white light sources 118 may be coaxial with or nearly coaxial with the visual axis; for example, the angle may be substantially zero degrees. In some examples, as described above, multiple images of the eye may be captured corresponding to different gaze directions of the patient 106; for example, a first color image of the eye may be captured corresponding to the patient's first gaze direction, and a second color image of the eye may be captured corresponding to the patient's second gaze direction. The image capture control unit 134 may store the capture time at which the image is captured, as well as the patient's illumination angle and / or gaze direction, as image-related metadata.

[0090] In operation 608, image capture control unit 134 can cause a camera (e.g., camera 120 of vision screening device 104) to capture multiple color images of the patient's eyes under white light illumination. In some examples, image capture control unit 134 can also cause the camera to capture video data. For example, video data can be captured during a first time period before the white light illumination begins and can continue during a second time period after the white light illumination begins. For example, the video data can be used to determine the patient's pupillary response (e.g., pupil size) and / or pupillary accommodation to changing illumination levels and / or abrupt changes in illumination (e.g., caused by the onset of white light illumination). Image capture control unit 134 can store multiple color images and / or videos in a database for clinician review. Furthermore, image capture control unit 134 can cause the camera to capture a color image of the patient's face and store this image in patient data 140 as a photographic identifier for the patient. Data analysis and visualization unit 136 can utilize the multiple color images to generate a synthetic color image and determine the difference in operations 610 and 612, as described below. In some examples, a synthetic color image can be generated by combining color images captured in different gaze directions of the patient to illustrate the retina of (multiple) eyes.

[0091] In operation 610, the data analysis and visualization component 136 can generate a composite image of (multiple) eyes by combining information from (multiple) grayscale images captured in operation 604 and (multiple) color images captured in operation 608. (See above reference.) Figure 5AThe data analysis and visualization component 136 can detect pupil images corresponding to the pupils of the eye in grayscale and color images, and align the grayscale and color pupil images so that the eye structures overlap. The data analysis and visualization component 136 can also generate a composite image, a portion of which contains grayscale values ​​from multiple grayscale images and the remainder of which contains color values ​​from multiple color images. For example, the data analysis and visualization component 136 can also annotate (e.g., using graphics and / or pseudo-color values) portions of the composite image to indicate areas of interest.

[0092] In operation 612, the data analysis and visualization component 136 or the output generation component 138 can determine one or more differences between pixel values ​​and desired pixel values ​​in grayscale, color, and / or composite images. For example, the data analysis and visualization component 136 or the output generation component 138 can calculate a first difference as the average difference between pixel values ​​at corresponding pixel locations in the patient's left pupil image and right pupil image. As another example, the data analysis and visualization component 136 or the output generation component 138 can calculate a second difference between average pixel values ​​in a first region of the pupil image and average pixel values ​​in a second region of the same pupil image. In yet another example, the data analysis and visualization component 136 or the output generation component 138 can calculate a third difference between the average pixel values ​​in the pupil image and standard values ​​from a normal healthy eye, which can be stored in a database (e.g., screening database 144). In addition, the data analysis and visualization component 136 or the output generation component 138 can calculate a fourth difference between the pixel values ​​in the pupil image captured during a vision screening test and the pixel values ​​in the pupil image of the same patient captured during a previously performed vision screening test(s).

[0093] In operation 614, output generation unit 138 compares the difference in pixel values ​​obtained in operation 612 with multiple thresholds and / or multiple ranges to determine an output indicating a condition related to multiple eyes, which may include a diagnosis or recommendation. For example, if the difference is less than a threshold (operation 614 - Yes), output generation unit 138 may generate a first patient-related output in operation 616, while if the difference is equal to or greater than a threshold (operation 618 - No), output generation unit 134 may generate a second output in operation 618. The multiple thresholds and / or multiple ranges may be predetermined and may be obtained as part of standard data, which may be stored in screening database 144 or computer-readable media 130, 148. The standard data may include different thresholds and ranges for each type of difference described above, including separate thresholds and ranges for grayscale values ​​and color values. The multiple thresholds and multiple ranges may also vary based on the patient's test category (e.g., the patient's age group or medical history).

[0094] In operation 616, output generation unit 138 may generate a first output as described above (operation 614 - Yes). The first output may correspond to an indication that the patient has passed a vision screening or that the patient's (multiple) eyes appear normal and healthy. The first output may also include a link to a stored image, including captured grayscale and color images and / or a generated visualization. In operation 618, output generation unit 138 may generate a second output (when operation 614 - No). The second output may correspond to an indication that a disease or abnormality has been detected, a recommendation for additional screening, and / or a diagnosis of the detected disease or abnormality.

[0095] As discussed, example method 600 can be performed by components of vision screening device 104 executed by processor(s)128 of device 104. Example method 600 illustrates operations performed during at least a portion of a vision screening test performed on a patient (e.g., patient 106) to determine diseases and / or abnormalities associated with the patient's(s) eyes based on images captured under illumination at different wavelengths. In an alternative example, some or all of the operations of method 600 can be performed by processor(s)146 of vision screening system 110, which is connected to vision screening device 104 via network 108.

[0096] Figure 7An example method 700 for vision screening according to some implementations of this disclosure is illustrated. As discussed, the operation of method 700 will be described as being performed by processor(s) 128 of vision screening device 104, but said operation may alternatively or additionally be performed by processor(s) 146 of remote vision screening system 110.

[0097] In operation 702, the data analysis and visualization component 136 can determine the pupil image from the captured image. As discussed, the captured image may include a grayscale image captured under NIR illumination by (multiple) radiation sensors 116, 212, 310 or a color image captured under white light illumination by (multiple) cameras 120, 216, 312b. The pupil image can be determined from the image captured under NIR radiation illumination using techniques described above and in U.S. Patent No. 9,237,846, which is incorporated herein by reference. The data analysis and visualization component 136 can also use various techniques, such as detecting edges using image processing techniques, then fitting circular arcs, and comparing the detected edge arcs with a model edge map of the eye image to determine the pupil image from the color image. The pupil image can also be determined from the color image using the pupil color to segment the pupil region of the (multiple) eye images. A combination of edge and color-based segmentation can also be used.

[0098] In operation 704, the data analysis and visualization component 136 can determine multiple differences between the patient's left pupil image and right pupil image. These differences can be determined using the grayscale values ​​of multiple grayscale images and / or the color values ​​of multiple color images. (See reference...) Figure 6 As described in operation 612, the difference can be calculated as the average difference between pixel values ​​at corresponding pixel locations in the patient's left pupil image and right pupil image, or the average difference between pixel values ​​at corresponding locations in the patient's left pupil image and right pupil image. In other examples, the data analysis and visualization component 136 can determine the difference by summing the pixel values ​​obtained by subtracting the left pupil image from the right pixel image, or vice versa.

[0099] In operation 706, the data analysis and visualization component 136 can compare the difference obtained in operation 704 with a first threshold to determine whether the difference is less than the first threshold. For example, the first threshold may be predetermined and may be obtained as part of standard data (which may be stored in the screening database 144), indicating the maximum expected difference between two pupil images of the same patient when the patient exhibits normal, healthy eyes. If the difference is equal to or greater than the first threshold (operation 706 - Yes), the output generation component 138 can generate an output reporting an abnormality in operation 716, as described in more detail below.

[0100] In operation 708 (operation 706 - No), the data analysis and visualization component 136 can determine the difference between a patient's left pupil image or right pupil image and (multiple) standard images of a normal, healthy eye. The left and right pupil images can be captured simultaneously or at different times during the vision screening test. (Multiple) standard images can be provided as part of standard data, which can be stored in a screening database 144 or computer-readable media 130, 148.

[0101] In operation 710, the data analysis and visualization unit 136 can compare the difference obtained in operation 708 with a second threshold to determine that the difference is less than the second threshold. For example, the second threshold may also be predetermined and may be obtained as part of standard data stored in the screening database 144, and indicates the maximum expected difference between the pupil image and a standard image of a normal healthy eye. If the difference is equal to or greater than the second threshold (operation 710 - Yes), the output generation unit 138 can generate an output reporting an anomaly in operation 716, as described in further detail below.

[0102] In operation 712 (operation 710 - No), the data analysis and visualization component 136 can determine the difference between the patient's pupil image and pupil images of the same eye captured during previous vision screening tests(s). The patient's pupil images captured during previous vision screening tests(s) (e.g., annual screening tests from previous years) can be stored in a database (e.g., screening database 144) as part of patient data 140. The data analysis and visualization component 136 can access pupil images from previous screening tests(s) from the database before the current vision screening test begins, and / or load the images into computer-readable medium 130.

[0103] In operation 714 (operation 712 - No), the data analysis and visualization component 136 can compare the difference obtained in operation 712 with a third threshold to determine that the difference is less than the third threshold. For example, the third threshold may be predetermined and may be obtained as part of standard data (which may be stored in the screening database 144), indicating the maximum expected difference between pupil images of the same eye captured after a period of time. If the difference is equal to or greater than the third threshold (operation 714 - Yes), the output generation component 138 can generate an output reporting an abnormality in operation 716. For example, the output may indicate changes in the patient's eye, which may be due to progressive eye diseases such as cataracts or macular degeneration, or new disease conditions not present during previous screening tests(s).

[0104] In operation 716, output generation component 138 can generate an output reporting an abnormality. As described above, if any of the differences(s) determined in operations 704, 708, 712 are greater than or equal to their respective thresholds, data analysis and visualization component 136 or output generation component 138 can determine that an abnormality and / or disease condition may exist in(multiple) eyes. The output may also include a diagnosis based on the difference that triggered the abnormality report. As described with reference to Figure 4, specific diseases may exhibit characteristic differences in the appearance of pupil images, and some diseases, such as retinoblastoma, cataracts, corneal lacerations, etc., can be determined based on differences detected during(multiple) vision screening tests. Output generation component 138 can store the output as part of patient data 140 in a database such as screening database 144. Output generation component 138 can also display the output to an operator (e.g., operator 102) on a display screen (e.g., multiple displays 122, 220, 334).

[0105] In the absence of any discrepancy that meets or exceeds the corresponding threshold, during operation 718, the output generation unit 138 can generate an output reporting a normal screening result, for example, determining that the patient's eyes are normal based on (multiple) vision screening tests. The output generation unit 138 can store the output as part of patient data 140 in a database such as screening database 144, and / or can display the output on (multiple) displays 122, 220, 334 to the operator of the vision screening device (e.g., operator 102).

[0106] Figure 8 An example method 800 for vision screening, according to some implementations of this disclosure, is illustrated. The vision screening includes one or more individual vision screening tests, such as individual screenings for different conditions of (multiple) eyes. Various operations of method 800 can be referenced. Figure 6 and Figure 7 The methods described are substantially similar or identical. As discussed, the operation of method 800 will be described as being performed by the processor(s) 128 of the vision screening device 104, but the operation may alternatively or additionally be performed by the processor(s) 146 of the remote vision screening system 110.

[0107] In operation 802, the patient screening component 132 can select to perform vision screening tests on patients participating in the vision screening period. (See reference...) Figure 1As discussed, the patient screening component 132 can determine a list of vision screening tests(s) to be performed on the patient, at least in part, based on the patient's test categories (e.g., the patient's age group or medical history) stored in the patient data 140. The vision screening test selected in operation 802 can be the next incomplete vision screening test on the list of vision screening tests to be performed on the patient. In some examples, at 802, the patient screening component 132 can select a vision screening test based on input received from the clinician or operator performing the screening test.

[0108] In operation 804, the data analysis and visualization component 136 can receive images of (multiple) eyes of a patient undergoing vision screening from the image capture control component 134. For example, the images may include one or more of the following: a grayscale image captured under NIR radiation, a color image captured under white light, a color or grayscale video, and / or a composite image. See reference... Figure 6 The illustrated example method 600 captures images as described. In some examples, images may be captured in advance, for example, while performing a previously selected vision screening test, and new images may not be captured.

[0109] In operation 806, the data analysis and visualization component 136 can, for example, use the reference. Figure 7 The described method 700 performs a selected vision screening test. Each vision screening test may have a predetermined set of image requirements associated with it, indicating the most suitable images for detecting the condition being screened. For example, image requirements may include image type (e.g., color, grayscale, composite, video, etc.), illumination angle(s) or gaze direction(s), illumination type / level, etc. The data analysis and visualization component 136 may, for example, identify the images(s) to be analyzed based on the image requirements by matching them with metadata associated with the captured images(s), and perform the selected vision screening test based on the analysis of the identified images. For example, a screening test for cataracts in (multiple) eyes may require composite images as input, while a screening test for blastoma may require color images as input. In some examples, for example, by identifying frames associated with the desired illumination angle or gaze direction, images(s) captured at different illumination angles or gaze directions can be extracted from a video.

[0110] In operation 808, the patient screening component 132 can determine whether the vision screening period has been completed, for example, whether the vision screening test performed in operation 806 is the last in the list of vision screening tests(s) determined by the patient screening component 130. If in operation 808, the patient screening component 132 determines that the vision screening period has not been completed (operation 808 - No), the patient screening component 130 can proceed to operation 802 to select the next vision screening test to be performed. On the other hand, if in operation 808, the patient screening component 132 determines that the vision screening period has been completed (operation 808 - Yes), the output generation component 138 can generate a report in operation 810, which includes the results of the vision screening tests(s) ...(s))(s)(s)(s)(s)(s)(s)(s)(s)(s))(s)(s)(s)(s)(s)(s)(s)(s))(s)(s)(s)(s)(s)(s)(s)(s)(s))(s)(s)(s)(s)(s)(s)(s)(s)(s)(s))(s

[0111] Based at least on the description herein, it can be understood that the vision screening apparatus and related systems and methods disclosed herein can be used to assist in performing one or more vision screening tests, including multiple tests for screening for eye diseases and / or abnormalities in patients. Components of the vision screening apparatus described herein can be configured to: generate radiation of different wavelengths in addition to white light to illuminate the eyes of a patient undergoing vision screening; capture images of (multiple) eyes under different illumination conditions; generate visualizations that aid in the diagnosis of disease conditions; determine differences between pupil images; and determine outputs indicating diagnoses, recommendations, or results of the screening tests. Exemplary vision screening apparatuses may include (multiple) radiation sources for generating radiation of different wavelengths, (multiple) sensors for capturing reflected radiation from a patient's eyes, a white light source, and a camera configured to capture (multiple) color images of (multiple) eyes of a patient under white light illumination, and (multiple) displays for displaying the output to an operator of the vision screening apparatus. The apparatus described herein can be used to screen for eye diseases and abnormalities in patients without requiring input or feedback from the patient or eye dilation, thereby allowing the apparatus to be used to screen very young, very old, incapacitated, or uncooperative patients.

[0112] The foregoing only illustrates the principles of this disclosure, and those skilled in the art can make various modifications without departing from the scope of this disclosure. The examples above are presented for illustrative purposes and not for limitation. This disclosure may also take many forms other than those expressly described herein. Therefore, it should be emphasized that this disclosure is not limited to the methods, systems, and apparatus expressly disclosed, but is intended to include variations and modifications of the methods, systems, and apparatus within the spirit of the appended claims.

[0113] As a further example, variations can be made to the apparatus or process constraints (e.g., size, configuration, components, sequence of process steps, etc.) to further optimize the provided structures, apparatuses, and methods, as shown and described herein. In any case, the structures and apparatuses described herein, and the related methods, have numerous applications. Therefore, the disclosed subject matter should not be limited to any single example described herein, but should be interpreted in breadth and scope according to the appended claims.

Claims

1. A vision screening device, characterized in that, include: A radiation source configured to emit radiation of a first wavelength; A sensor configured to capture radiation reflected by the patient's eye; White light source; A camera configured to capture a color image of the patient's eye; A processor, operatively connected to the radiation source, the sensor, the white light source, and the camera; and The memory stores instructions that, when executed by the processor, cause the processor to: The radiation source emits radiation of the first wavelength during the first time period; The sensor captures a portion of the radiation reflected by the patient's eye during the first time period; The white light source is used to illuminate the patient's eyes during a second time period following the first time period; The camera captures a color image of the patient's eye during the second time period; A first region of a grayscale image indicating the captured portion of the radiation is determined, the first region corresponding to the pupil of the eye; Determine a second region of the color image corresponding to the pupil of the eye; A composite image of the eye is generated, at least in part based on the grayscale image and the color image indicating the captured portion of the radiation, the composite image being configured such that the first region and the second region are located in the same position in the composite image, wherein the composite image includes a first plurality of pixels representing the grayscale image and a second plurality of pixels representing the color image; Based on the synthesized image, a difference between a value related to the eye and a desired value is determined, wherein the difference is at least partially based on the first region or the second region; as well as The output indicating the condition related to the eye is generated at least in part based on the difference.

2. The vision screening device according to claim 1, wherein, The first wavelength is the near-infrared (NIR) band of the electromagnetic spectrum.

3. The vision screening device according to claim 2, wherein, The radiation source includes an array of NIR light-emitting diodes (LEDs) arranged such that multiple NIR LEDs share a common axis.

4. The vision screening device according to claim 1, further comprising a display unit disposed on a first side of the vision screening device and configured to display the output to an operator of the vision screening device. in, The radiation source, the sensor, the white light source, and the camera are disposed on the second side of the vision screening device opposite to the first side.

5. The vision screening device according to claim 4, wherein, The white light source includes an array of light-emitting diodes (LEDs) configured to illuminate the patient's eye from a first angle and a second angle different from the first angle relative to the visual axis associated with the patient's eye.

6. The vision screening device according to claim 5, wherein, The instruction also causes the processor to: The white light source illuminates the patient's eye from the first angle during the first portion of the second time period; The camera captures a first color image of the patient's eye during the first portion of the second time period; The white light source illuminates the patient's eye from the second angle during the second portion of the second time period; The camera captures a second color image of the patient's eye during the second portion of the second time period; An animated image sequence of the eyes is generated based on the first color image and the second color image; and The animated image sequence is displayed on the display unit.

7. The vision screening device according to claim 5, wherein, The instruction also causes the processor to: The camera captures a first color image of the patient's eye corresponding to the patient's first gaze direction; The camera captures a second color image of the patient's eye corresponding to the patient's second gaze direction; as well as An image of the retina of the eye is generated based on the first color image and the second color image.

8. A vision screening method, characterized in that, include: Irradiate the patient's eyes during the first time period; A grayscale image of the eye is captured during the first time period; The eyes were irradiated during a second time period separated from the first time period; Capture a color image of the eye during the second time period; A first region of the grayscale image of the captured portion indicating radiation is determined, the first region corresponding to the pupil of the eye; Determine a second region of the color image corresponding to the pupil of the eye; A composite image of the eye is generated based at least in part on the grayscale image and the color image, the composite image being configured such that the first region and the second region are located at the same position in the composite image, wherein the composite image derives a first plurality of pixel values ​​from the grayscale image and derives a second plurality of pixel values ​​from the color image; Based on at least one of the color image or the synthetic image, determine the difference between a color value associated with the eye and a desired color value, wherein the difference is at least partially based on the first region or the second region; and Based at least in part on the difference, an output related to the patient is generated, indicating normal eye screening.

9. The method according to claim 8, wherein, The irradiation during the first time period includes irradiating the eye with radiation in the near-infrared (NIR) band of the electromagnetic spectrum.

10. The method of claim 8, wherein, The color value is a first color value, and the method further includes: Obtain patient data related to the patient, wherein the patient data includes a second color value related to the eye; and The second color value is determined as the desired color value.

11. The method according to claim 8, wherein, The eye is the patient's first eye, and the desired color value is either a second color value associated with the patient's second eye or a third color value obtained from standard data related to the human eye.

12. The method of claim 8, further comprising: Feed grayscale or color images as input to a trained machine learning model; as well as The output is received from the trained machine learning model.

13. The method of claim 8, further comprising: Determine that the difference is equal to or greater than the threshold. The output is at least partially based on determining that the difference is equal to or greater than the threshold.

14. The method of claim 13, further comprising: Identify a portion of the synthesized image where the difference is equal to or greater than the threshold; as well as Generate a graphic indicating the portion of the composite image, wherein the output includes the composite image having the graphic.

15. The method of claim 14, further comprising: The composite image having the graphics is provided via a graphical user interface (GUI). The GUI allows users to turn the graphics on or off.

16. A vision screening system, characterized in that, include: Memory; processor; as well as Computer-executable instructions, stored in the memory and executable by the processor, to perform operations including: The radiation source emits near-infrared (NIR) radiation during the first time period; The sensor captures a portion of the NIR radiation reflected by the patient's eye during the first time period; The eye is illuminated by a white light source during a second time period separated from the first time period; The camera captures a color image of the eye during the second time period; Generate a grayscale image indicating the captured portion of the NIR radiation; Determine a first region of the grayscale image corresponding to the pupil of the eye; Determine a second region of the color image corresponding to the pupil of the eye; Based on the color image and the portion of the NIR radiation, a difference between a value related to the eye and a desired value is determined, wherein the difference is at least partially based on the first region or the second region; Determine that the difference is equal to or greater than the threshold; and An output indicating the condition of the eye is generated, at least in part based on determining that the difference is equal to or greater than the threshold.

17. The system according to claim 16, wherein, The operation also includes: A composite image is generated based on the grayscale image and the color image, wherein the composite image includes a first plurality of pixel values ​​from the grayscale image and a second plurality of pixel values ​​from the color image.

18. The system according to claim 16, wherein, The operation also includes: The white light source illuminates the patient's eyes from multiple angles during the second time period; The camera captures multiple color images of the patient's eyes, wherein each of the multiple color images corresponds to an angle among the multiple angles; A video is generated based on the plurality of color images, wherein each frame of the video corresponds to a color image among the plurality of color images; and The video is displayed on the screen.

Images