Vision screening device with oversampling sensor

Abstract

The present disclosure describes a vision screening device for performing a vision screening test on a patient, the vision screening device comprising an oversampling sensor, allowing the device to operate while maintaining different distances from the patient. The vision screening device may include associated methods and systems that may be configured to capture a series of oversampled images, detect a patient's face and eye in the images, track the eye, and output a stabilized eye image for a vision screening test. The apparatus may also be configured to determine a measurement relating to the eye position in the eye image, and to determine, based on the measurement, whether the eye image meets a criterion used in an ongoing vision screening test. The device-related method may be configured to provide guidance to a device operator to adjust the position of the device or patient so that a standard-compliant eye image may be generated.

Description

[0001] Related Applications

[0002] This application claims priority to U.S. patent application US18 / 211,027, filed on June 16, 2023, which claims priority to U.S. provisional application US63 / 352,695, filed on June 16, 2022, the entire disclosure of which is incorporated herein by reference. Technical Field

[0003] The present application relates to medical devices, and more particularly to a vision screening device and a related system and method for implementing vision screening tests while adapting to changes in patient distance and patient movement. Background Art

[0004] Vision screening typically includes screening tests for various eye diseases and defects. Such vision screening tests may include, for example, refractive error tests, autofocus tests, visual acuity tests, color vision screening, screening for diseases of the retina and intraocular media, etc. Vision screening tests may be performed using a vision screening device that may be compact and / or portable (e.g., a handheld device, a mobile device like a tablet or smartphone, etc.) so that the test can be performed anywhere, from conventional screening environments such as schools and medical clinics to doctor's offices, hospitals, eye care facilities, and / or other remote and / or mobile locations.

[0005] Screening tests may have distance requirements or other parameters that must be met to ensure accurate test results. For example, in some cases, the ideal distance between the patient and the vision screening device during a vision screening test may vary from about three feet to about 22 feet. Because of the narrow field of view of a typical vision screening device, it may be difficult for the operator of the vision screening device to center the patient's eye in the device's viewfinder or screen, especially at distances greater than three feet. In addition, at distances greater than three feet, the effects of jitter caused by slight movements of the operator holding the vision screening device or slight movements of the patient while performing the vision screening test may be magnified, and the imaging component of the vision screening device may not be able to obtain a clear image of the eye and eye parts as required by the screening test, nor maintain the eye centered in the image.

[0006] In view of this, the vision screening device should be advantageously configured to operate at different distances from the patient to achieve proper eye centering and stable imaging to accurately perform the vision screening test.

[0007] Various examples of the present disclosure are directed to overcoming one or more of the above-mentioned disadvantages. Summary of the invention

[0008] In an example of the present disclosure, a vision screening device includes a sensor configured to capture an image (e.g., an oversampling sensor), a processor operably connected to the sensor, and a memory storing instructions executable by the processor. When the instructions are executed, the processor: causes the sensor to capture a first image of an environment including a patient; identifies a first portion of the first image corresponding to the patient's face; based on the first portion, determines a second portion of the first image at least partially located within the first portion, the second portion illustrating the patient's first eye and the patient's second eye; determines a first position of the first eye within the second portion and a second position of the second eye within the second portion; determines whether the first position and the second position meet corresponding eye position standards. When the instructions are executed, the processor also: extracts image data from the first image based on determining that the first position and the second position meet corresponding eye position standards, the image data including data corresponding to the second portion, but not including data corresponding to the remaining portions of the first image other than the second portion; generates a second image illustrating the first eye and the second eye based on the image data.

[0009] In another example of the present disclosure, a method includes: capturing a first image, the first image including a patient; identifying a first portion of the first image, the first portion illustrating at least a portion of the patient's face; identifying a second portion of the first image at least partially located within the first portion, the second portion illustrating a first eye of the patient and a second eye of the patient; based on the second portion, determining a first position of a pupil of the first eye and a second position of a pupil of the second eye relative to a center point of the first portion. The method also includes: determining whether the first position and the second position meet corresponding pupil position criteria; based on determining that the first position and the second position meet corresponding pupil position criteria, extracting image data from the first image, the image data including data corresponding to the second portion, and not including data corresponding to the remaining portion of the first image other than the second portion. The method also includes: generating a second image illustrating the first eye and the second eye based on the image data; determining a condition of at least one of the first eye or the second eye based on the second image; and outputting an indication of the condition via an electronic device.

[0010] In another example of the present disclosure, a system includes a memory, a processor, and computer executable instructions stored in the memory and executable by the processor. When the instructions are executed, the processor performs the following operations: causing a sensor to capture a first image of a patient; determining a first portion of the first image corresponding to the patient's face; determining a second portion of the first image, the boundary of the second portion being at least partially located within the first portion, the second portion illustrating the patient's first eye and the patient's second eye. When the instructions are executed, the processor is also caused to: determine a first position of the first eye relative to the boundary of the second portion and a second position of the second eye relative to the boundary of the second portion; determine whether the first position and the second position meet corresponding eye position standards; based on determining that the first position and the second position meet corresponding eye position standards, extract image data from the first image, the image data does not contain data from outside the boundary in the first image. When the instructions are executed, the processor is also caused to: generate a second image illustrating the first eye and the second eye based on the image data; and determine the condition of at least one of the first eye or the second eye based on the second image. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] The features, properties and various advantages of the present disclosure will be more readily understood from the following detailed description considered in conjunction with the accompanying drawings.

[0012] Figure 1 The present disclosure shows an example vision screening device and a vision screening system. In certain embodiments, Figure 1 The example system components shown in can be used to perform one or more screening tests related to vision screening and / or detection of eye diseases or defects.

[0013] Figure 2A An example operating environment of the vision screening device of the present disclosure is shown.

[0014] Figure 2B A schematic diagram of an oversampling sensor of an example vision screening device of the present disclosure is provided in a top view of an operating environment of the vision screening device of the present disclosure.

[0015] Figure 2C Provided Figure 2B Schematic diagram of an image captured by an oversampled sensor.

[0016] Figure 3A A graphical flow chart of an example workflow for performing a vision screening test using the example systems and devices of the present disclosure is provided.

[0017] Figure 3B Provides vision screening during the test Figure 3A A graphical flow chart of example verification steps of the eye image captured in step 324 of FIG.

[0018] Figure 4A flowchart illustrating an example method of the present disclosure is provided.

[0019] In each figure, the leftmost digit or digits of a reference number identify the figure in which the reference number first appears. The use of the same reference number in different figures indicates similar or identical items or features. The figures are not drawn to scale. DETAILED DESCRIPTION

[0020] The present disclosure relates in part to vision screening devices and corresponding methods. Such an example vision screening device may be configured to perform one or more vision screening tests on a patient and output the results of the vision screening tests to an operator of the device, such as a physician or physician assistant. Specifically, the present disclosure relates to vision screening devices and methods, wherein the patient is positioned at different distances from the vision screening device based on the specific requirements of the vision screening test. The example vision screening device of the present disclosure may be a portable and / or handheld device and include an oversampling sensor to capture one or more images and / or videos of the patient's environment (such as an examination room). The captured range of the environment (e.g., the field of view of the oversampling sensor) may be based on the maximum distance between the patient and the vision screening device and the range of movement that may be expected during the vision screening. The method implemented by the vision screening device may use computer vision and machine learning techniques to determine the position of the patient's face and / or eyes in the captured image, and provide guidance to the operator to perform non-critical alignment of the vision screening device so that the eye is centered in the field of view. The vision screening device may also implement an automatic mode in which the operator does not need to center the eye within the field of view, but instead the device generates a centered image of the target area, such as the patient's eye, by automatically identifying the relevant portion of the oversampled sensor and extracting corresponding image data from the relevant portion. The field of view and resolution of the oversampled sensor may be selected so that the resolution of the extracted image data of the target area, such as the eye, is sufficient for subsequent analysis related to the vision screening test.

[0021] In the example vision screening device disclosed herein, the image processing and computer vision technology implemented by the vision screening device can track the patient's eyes continuously or intermittently at preset intervals in real time or near real time. The vision screening device can use the tracking output to update the eye position, and use the updated position to generate a stable eye image, and the position of the eye in the eye image remains substantially fixed. The tracking positioning of the eye can compensate for the effects of small movements (including movement of the device or the patient's head), enabling the vision screening device to provide real-time image stabilization during the duration of the vision screening.

[0022] In certain example environments, the patient may be exposed to ambient light during eye positioning. After the eye reaches the desired position in the field of view, different electromagnetic spectrum bands (e.g., infrared, near-infrared, visible light) can be used to illuminate the eye and / or internal structures of the eye, and images of the eye can be generated under different illumination conditions, depending on the requirements of the vision screening test being implemented. The vision screening device can determine one or more eye diseases and / or defects related to one or both eyes of the patient based on the analysis of the captured images, such as myopia, hyperopia, cataracts, tumors, refractive errors, retinal detachment or lesions, color vision disorders, etc.

[0023] At least refer to Figure 1 As described, an example vision screening device related to screening for eye diseases and defects may include a component for capturing an image of a patient's eye under visible light and near-infrared and other electromagnetic spectrum band radiation. The device may include a component for capturing an image under ambient light using an oversampled sensor of a camera and analyzing the captured oversampled image to extract a portion of image data corresponding to the patient's eye image (e.g., an eye image). The device may also include a component for analyzing the resulting eye image to determine whether the eye image meets certain standards. In each example, these standards may include determining whether the eye image has an acceptable resolution and clarity and whether the size and position of the eye in the eye image are within an acceptable range. The device may also include a component for tracking the eye and keeping a bounding box around the eye (e.g., an eye bounding box) within the boundaries of the oversampled image to indicate the corresponding area of ​​the eye image in the oversampled image. Image data within the eye bounding box can be extracted from the oversampled image to generate an eye image. In addition, the vision screening device may include a component for performing a vision screening test, analyzing an eye image to determine a patient's eye pathology and / or defect, and a component for determining and reporting an output indicating that a pathology or defect was detected during the vision screening.

[0024] Refer to the following Figure 1 , Figures 2A-2C , Figure 3A-3B and Figure 4 It should be understood that although these figures describe devices and systems that can utilize the disclosed methods, the methods, processes, functions, operations and / or techniques described herein are also applicable to other devices, systems, etc.

[0025] Figure 1 An example environment 100 is shown for performing vision screening tests, particularly screening tests for detecting eye diseases and / or defects, that require the patient to maintain different distances from the vision screening device, according to certain embodiments. Figure 1As shown, in some examples, operator 102 can perform a vision screening test on patient 106 via vision screening device 104 to determine the eye health of patient 106. As described herein, vision screening device 104 can perform one or more vision screening tests, including screening for eye diseases and / or defects. For example, vision screening device 104 can be configured to perform vision screening tests, such as visual acuity tests, refractive error tests, focus tests, dynamic eye tracking tests, color vision screening tests, and / or any other vision screening tests, the results of which can be used to assess the vision health of patient 106 or diagnose an eye condition of patient 106.

[0026] In various examples, the vision screening device 104 may include a portable device configured to perform one or more vision screening tests. In some examples, the vision screening device 104 may be a handheld device, such as a mobile device similar to a tablet or a smart phone. Due to its portability, the vision screening device 104 can perform vision screening tests in any location, from traditional screening environments (such as schools and medical clinics) to doctors' offices, hospitals, eye care facilities, various group settings outside of medical offices, and / or other remote and / or mobile locations. It is also contemplated that the vision screening device 104 can be used to perform vision screening tests on all age groups (including newborns, young children, and elderly patients).

[0027] As described herein, the vision screening device 104 may be configured to perform one or more vision screening tests on the patient 106. The vision screening test performed using the vision screening device 104 may have a corresponding recommended distance requirement (indicating the distance or distance range to be maintained between the vision screening device 104 and the patient 106 during the test), which must be met to ensure the accuracy of the test results. For example, during the refractive error test using the vision screening device 104, the patient 106 may be recommended to be separated from the vision screening device 106 by a distance of about three feet or about one meter. In other examples, the recommended distance between the vision screening device 104 and the patient 106 for performing a specific vision screening test may be six feet, twelve feet, or as far as twenty-two feet. In the disclosed example, the vision screening device 104 may be configured to operate at various distances from the patient 106 and generate a stable image of the patient 106's eye, thereby compensating for small relative movements between the vision screening device 108 and the patient 106. The vision screening device 104 can automatically acquire an image of the eye and keep the eye at a specific position within the eye image when operating at different distances from the patient 106. Alternatively or additionally, the vision screening device 104 may provide instructions to the operator 102 and / or the patient 106 to adjust the positioning of the patient 106 and / or the vision screening device 106 to achieve proper eye positioning as required by the vision screening test being performed.

[0028] To achieve the above features, the vision screening device 104 may include a camera 108 equipped with an oversampling sensor 110, such as Figure 1 . The oversampled sensor 110 of the camera 108 may be configured to capture reflected light from the environment in which the patient 106 is located under ambient lighting conditions. The oversampled sensor 110 of the camera 108 may have an extended field of view that includes the area surrounding the patient 106, so that the patient 106 can be detected within the field of view of the oversampled sensor 100 even if the patient 106 is centrally positioned in the field of view of the camera 108 of the vision screening device 104. The resolution of the oversampled sensor 110 may be selected so that even when the patient 106 is at the maximum operating distance supported by the vision screening device 104, the image data of the target area of ​​the patient 106 (such as the eye) has sufficient resolution for subsequent analysis related to the vision screening test. Refer to Figures 2A-2C An oversampling sensor 110 and methods for extracting target regions (eg, corresponding to eyes) of oversampled images and / or videos are described in detail.

[0029] The camera 108 may also include an optical assembly (not shown) including one or more lenses, windows, prisms, filters, mirrors, and / or any other device configured to collect and direct a reflected light beam from the patient 106 to the oversampled sensor 110. In some examples, the camera 108 may be equipped with a lens having wide-angle image capture capabilities. For example, the camera 108 operating in a wide-angle capture mode may enable the vision screening device 104 to locate the patient 106 in the examination room. The camera 108 may also capture a video of the patient while performing a vision screening test. The camera 108 may generate an oversampled image and / or video of the environment, which may be processed to generate image data corresponding to a target area (such as the eye of the patient 106). The oversampled image and / or video may be stored in various formats, such as JPEG, BITMAP, TIFF, etc. (image formats) and MP4, MOV, WMV, AVI, etc. (video formats).

[0030] While achieving and / or maintaining the spacing and alignment of the target areas using the camera 108 and the oversampling sensor 110 related methods, additional steps related to implementing one or more vision screening tests can be performed. In each example, one or more vision screening tests may include irradiating the eye of the patient 106 with radiation in one or more bands of the electromagnetic spectrum and capturing an image of the reflected radiation from the eye or part of the eye of the patient 106. For example, a refractive error screening test is based on a pupil image captured under different illumination modes generated by a near infrared (NIR) radiation source. In other examples, a vision screening test (such as a red light reflex test) may include irradiating the eye of the patient 106 with visible light and capturing an image of the eye under visible light illumination. The vision screening device 104 can acquire data, including images and / or video data of the eye under various illuminations, and detect the pupil, retina and / or lens of the patient 106 eye. These data can be used to help the operator 102 or clinician diagnose eye diseases and defects of the patient 106. The vision screening device 104 may also be configured to process image and / or video data to screen for eye diseases and defects, such as refractive errors, strabismus, and retinopathy. As such, other components of the vision screening device 104 described herein may also provide automatic diagnosis and / or recommendations based on analyzing images captured by the vision screening device 104.

[0031] like Figure 1 As shown in the schematic diagram of , the vision screening device 104 may include one or more radiation sources 112, which are configured to perform functions related to implementing color vision screening tests. The radiation source 112 may include individual radiation emitters, such as light emitting diodes (LEDs), which may be arranged in a pattern into an LED array. In each case, the radiation source 112 may include an infrared or near-infrared (NIR) radiation emitter, such as a NIR LED, which may provide coherent collimated illumination for measuring the refractive error of the patient 106 eye using a photorefraction method. The NIR radiation emitter of the radiation source 112 can also be used to measure the gaze angle or gaze direction of the patient 106 eye. In addition, the radiation source 112 may also include a color LED of visible light for generating colored light and white light stimuli for display to the patient 106 during various vision screening tests.

[0032] The vision screening device 104 may also include one or more radiation sensors 114, such as an infrared camera, configured to capture reflected radiation from the patient's eyes during the vision screening test. For example, the vision screening device 104 may emit one or more beams of radiation via the radiation source 112 and may be configured to direct the beams to the patient's 106 eyes. The vision screening device 104 may then capture the corresponding radiation reflected back (e.g., from the pupil of the eye) via the radiation sensor 114. In each example, the radiation sensor 114 may include a NIR radiation sensor to capture reflected NIR radiation when the patient's 106 eyes are illuminated by the NIR radiation source 112. The data captured by the NIR radiation sensor 114 may be used to measure the refractive error and / or gaze angle of the patient's 106 eyes.

[0033] The vision screening device 104 can send the data captured by the radiation sensor 114, the oversampled image data captured by the oversampled sensor 110, and / or the image data corresponding to the target area (e.g., an eye image) to the vision screening system 118 via the network 116 for analysis to determine the output 120 related to the patient 106. The data sent may include images and / or videos illustrating the pupil, retina, and / or lens of the patient's 106 eye. The data may be sent intermittently during a specific period of the vision screening test or during the entire test. In some examples, the vision screening device 104 may extract the image data corresponding to the target area locally and send only the extracted image data to the vision screening system 118 for further analysis. Alternatively or additionally, the vision screening device 104 may perform part or all of the analysis locally to determine the output 120. In fact, in any example described herein, some or all of the methods of the present disclosure may be performed in whole or in part by the vision screening device 104 independently (e.g., without the vision screening system 118 or its components), or by the vision screening system 118 independently (e.g., without the vision screening device 104 or its components). For example, in some examples, the vision screening device 104 may be configured to perform any of the vision screening tests and / or other methods described herein without being connected to or otherwise communicating with the vision screening system 118 via the network 116. In another example, the vision screening system 118 may include one or more components that are similar and / or the same as the components included in the vision screening device 104, and thus, the vision screening system 118 may be configured to perform any of the vision screening tests and / or other methods described herein without being connected to or otherwise communicating with the vision screening device 94.

[0034] The vision screening device 104 may also include one or more display screens, such as display screen 122 and display screen 124, which may be a color LCD (liquid crystal display) or OLED (organic light emitting diode) display screen. Display screen 122 may be an operator display screen facing the operator 102, configured to provide the operator 102 with information related to the vision screening test. Such information may include guidance on the positioning alignment of the patient 106 relative to the vision screening device 104. The vision screening device 104 may also display an eye image generated by the device 104 on the display screen 122. During the vision screening process, the eye image may be displayed in real time or near real time. In any example described herein, the display screen 122 facing the operator 102 may be configured to display and / or otherwise provide output 120 generated by the vision screening device 104 and / or generated by the vision screening system 118. The output 120 may include test parameters, the current status and progress of the screening test, measurements determined during the test, images captured or generated during the screening test, diagnoses determined based on one or more tests, and / or diagnosis-related recommendations. The display screen 122 facing the operator 102 may also display patient-related or patient-specific information and the patient's medical history.

[0035] In some examples, the vision screening device 104 may also include a display screen 124 facing the direction of the patient 106 and configured to display content to the patient 106. The content may include attention-grabbing images and / or videos to attract the patient's attention and lock the patient's line of sight to the vision screening device 104. Content corresponding to various vision screening tests may also be presented to the patient 106 on the display screen 124. For example, the display screen 124 may display color stimuli to the patient 106 during a color vision screening test, or display a Snellen Eye Chart during a visual acuity screening test. The display screens 122 and 124 may be integrated with the vision screening device 104, or may be a device peripheral controlled by a computer program of the device 104. The vision screening device 104 may refer to the example vision screening device configuration described in the U.S. patent application filed on August 12, 2022, application number US17 / 886,920, the entire disclosure of which is incorporated herein by reference.

[0036] The vision screening device 104 can use the network interface 126 of the vision screening device 104 to send the data captured by the camera 108 and / or the radiation sensor 114 via the network 116. In addition, the vision screening device 104 can also similarly send other test data related to the vision screening test being implemented (e.g., test type, test duration, patient identity, etc.). The network interface 126 of the vision screening device 104 can be operably connected to one or more processors 128 of the vision screening device 104, and can enable wired and / or wireless communication between the vision screening device 104 and one or more components of the vision screening system 118 and wired communication with one or more other remote systems and / or other networked devices. For example, the network interface 126 may include a personal area network component to enable one or more short-range wireless communication channels for communication and / or include a wide area network component to enable wide area network communication. In any example described herein, the network interface 126 can enable, for example, communication between the processor 128 of the vision screening device 104 and the vision screening system 118 via the network 116. Figure 1 The illustrated network 116 may be any type of wireless network or other communication network known in the art. Examples of network 116 include the Internet, an intranet, a wide area network (WAN), a local area network (LAN), and a virtual private network (VPN), a cellular network connection, and a connection established using protocols such as 802.11a, 802.11b, 802.11g, 802.11n, and / or 802.11ac.

[0037] As described herein, a processor (such as processor 128) can be a single processing unit or multiple processing units, and can include a single or multiple computing units or multiple processing cores. Processor 128 can be implemented as one or more microprocessors, microcomputers, microcontrollers, digital signal processors, central processing units, state machines, logic circuits, and / or any device that manipulates signals based on operating instructions. For example, processor 128 can be one or more hardware processors and / or any suitable type of logic circuit, specifically programmed or configured to execute the algorithms and processes described herein. Figure 1 As shown in the schematic diagram of , the vision screening device 104 may also include a computer-readable medium 130 operably connected to the processor 128. The processor 128 may be configured to obtain and execute computer-readable instructions stored in the computer-readable medium 130, and the computer-readable instructions may program the processor 128 to perform the functions described herein.

[0038] Computer readable medium 130 may include volatile and non-volatile memory and / or removable and non-removable media implemented by any type of information storage technology for storing information, such as computer readable instructions, data structures, program modules, or other data. Such computer readable medium 130 may include, but is not limited to, RAM, ROM, EEPROM, flash memory or other storage technology, optical storage, solid-state storage, magnetic tape, disk storage, RAID storage systems, storage arrays, network attached storage, storage area networks, cloud storage, or any other medium that can be used to store the desired information and can be accessed by a computing device. Computer readable medium 130 may be a computer readable storage medium and / or a tangible non-transitory medium, but reference to non-transitory computer readable media does not include media such as energy, carrier signals, electromagnetic waves, and signals themselves.

[0039] The computer readable medium 130 may be used to store any number of functional components executable by the processor 128. In various examples, the functional components include instructions or programs executable by the processor 128 that, when executed, specifically configure one or more processors 128 to perform actions related to one or more vision screening tests for detecting and diagnosing eye diseases and defects. For example, the computer readable medium 130 may store one or more functional components for performing a vision screening test and for aligning the eye during the vision screening test. For example, Figure 1 As shown, these components may include an image capture analysis component 132, an eye image verification component 134, an eye tracking and output generation component 136, and / or a patient screening component 138. At least some of the functional components of the vision screening device 104 will be described in detail below.

[0040] In various examples, the image capture control component 132 may be configured to operate the camera 108 of the vision screening device 104 to capture an oversampled image of the patient 106 and the surrounding environment, and identify a portion of the oversampled image corresponding to the patient's 106 eyes. In some examples, the image capture control component 132 may analyze the image and / or video data (e.g., oversampled images) captured by the oversampled sensor 110 to determine an area in the image data corresponding to the face of the patient 106 (e.g., a facial image). Algorithms for detecting human faces in digital images / videos are well known in the art of digital image processing, and typically generate outputs indicating a bounding box around each detected face and / or the location of the eyes in the face. The image capture control component 132 may execute a facial detection algorithm to detect a bounding box corresponding to the face of the patient 106, and determine an oversampled image area corresponding to the face, which includes an area within the bounding box. The image capture control component 132 may further process the area corresponding to the face to determine the location of the patient's 106 eyes. Algorithms for determining the positions of facial features (such as eyes, mouth, etc.) within facial regions in digital images are well known in the art, such as active appearance models and active shape models of the human face and other geometric models or appearance-based models of the eyes and / or eye pupils.

[0041] Alternatively or additionally, the image capture control component 132 can determine the position of the patient's 106 eyes by executing an eye detection algorithm. Machine learning algorithms known in the art can be based on a convolutional neural network (CNN) or a traditional object classifier, which can be trained to detect eyes in an image. Some facial detection algorithms can also generate eye positions as part of the output. The eye detection algorithm can output a bounding box around the eye, determine the center position of the eye, the inner and outer corners of the eye, and / or the position of eye features (such as the pupil position of the eye). Eye detection algorithms known in the art can also be used in a similar manner to process infrared and near infrared (NIR) images. In some examples, the image capture control component 132 can apply an eye detection algorithm to the NIR image captured by the radiation sensor 114 to determine the eye position.

[0042] Alternatively or additionally, when the eye of the patient 106 is illuminated by infrared radiation emitted by the emitter of the radiation source 112, the image capture control component 132 can determine the eye position from the infrared image captured by the radiation sensor 114. The reflection of infrared radiation from the iris of the patient's 106 eye causes the pupil of the patient's 106 eye to appear as a nearly circular spot of high brightness or high intensity in the captured infrared image. The image capture control component 132 can perform image processing techniques to determine a high-intensity area in the infrared image, which can be used as a candidate area for the pupil of the patient's 106 eye. The image capture control component 132 can determine the candidate area based on criteria (including a high-intensity average, a near-circular shape, and / or a diameter of an area that matches the expected diameter of the patient's 106 eye pupil based on the distance of the patient 106 from the vision screening device 104). The image capture control component 132 can determine the score of each candidate area based on the above criteria. Based on the distance between the patient 106 and the vision screening device 104, additional weights may be assigned to the candidate regions if the associated distances of the first candidate region and the second candidate region match the expected distance between the pupils of the patient's 106 eyes. In some examples, the image capture control component 132 may determine the highest scoring candidate region or a pair of candidate regions as the location of the patient's 106 eye pupils.

[0043] In other examples, the operator 102 may provide an initial position of the eye of the patient 106. For example, the oversampled image may be displayed to the operator 102 on the display screen 122, and the user interface of the vision screening device 104 may enable the operator 102 to click or otherwise indicate the center position of the eye in the displayed image.

[0044] Based on determining the eye position using one or more of the above methods, the image capture control component 132 can determine an eye bounding box indicating the area in the oversampled image corresponding to the eye of the patient 106. In addition, in some examples, the image capture control component 132 can extract image data corresponding to the area within the eye bounding box to generate an eye image of the patient 106. In other examples, the image capture control component 132 can specify an eye bounding box and provide the associated oversampled image without extracting the eye image.

[0045] In some examples, the camera 108 may be equipped with an auto-focus zoom lens that provides telephoto and wide-angle image capture capabilities. In these examples, the image capture control component 132 may operate the camera 108 in a telephoto zoom mode to capture a magnified image of the patient's eye based on the determined eye bounding box position. In other examples, the image capture control component 132 may apply a facial detection algorithm to determine a facial region, capture a magnified image of the facial region, and apply an eye detection algorithm to the magnified image to determine an eye bounding box.

[0046] The image capture control component 132 can specify an eye bounding box using the coordinates (x, y) of the eye bounding box corners relative to the oversampled image origin (0,0) to identify the portion of the oversampled image corresponding to one or both eyes of the patient 106. In some examples, the image capture control component 132 can process oversampled images captured and processed at frequent periodic intervals (e.g., at a rate of 2 images per second, 10 images per second, or more), or process oversampled images corresponding to frames of video 0 captured by the camera 108, thereby generating an eye bounding box in real time or near real time. In addition, in some examples, the image capture control component 132 can extract image data corresponding to the data within the eye bounding box from the oversampled image, thereby updating the eye image in real time or near real time. The output 120 can include one or more eye images. Refer to the following Figure 2B , Figure 2C and Figure 3A The functions and features of the image capture analysis component 132 are described in detail.

[0047] In each example, the computer-readable medium 130 may additionally store an eye image verification component 134 configured to determine whether an eye image as generated by the image capture analysis component 132 or an eye image indicated by an eye bounding box as determined by the image capture analysis component 132 meets the requirements specified in a predetermined validity standard. The validity standard may be based on the requirements of the vision screening test being implemented. For example, the eye image may be required to have a minimum pixel resolution and clarity level at which eye features that are detectable for determining eye diseases and / or defects may be present. In addition, the position of the eye pupil may be required to be centered and / or horizontally aligned in the eye image. For example, the distance between the pupil position of the first eye and the pupil position of the second eye may be reduced due to an out-of-plane rotation of the patient's head relative to the vision screening device 104, wherein the reduction in distance is proportional to the degree of out-of-plane rotation. In another example, the pupils may not be horizontally aligned due to an in-plane rotation of the patient's head. Without affecting the requirements of the vision screening test, the validity standard may be related to the maximum allowable amount of in-plane rotation and out-of-plane rotation that may be present in the eye image. Refer to the following Figure 3B The method related to the eye image verification component 134 is described in detail.

[0048] In addition, the eye image verification component 134 can determine the distance of the patient 106 relative to the vision screening device, for example, based on the output of a distance sensor associated with the vision screening device 104 and / or based on the eye size or the distance between the centers of the eyes. For example, the vision screening device 104 can be equipped with a distance meter, such as an ultrasonic distance meter, an infrared distance meter, and / or any other proximity sensor capable of determining the distance between the patient 106 and the screening device. In each case, each vision screening test can have a recommended distance between the vision screening device 104 and the patient 106 and / or a recommended position of the patient 106 relative to the vision screening device 106.

[0049] If the patient distance and / or eye position do not meet the recommendations for the vision screening test being performed, the eye image verification component 134 can issue instructions to the operator 102 via the display screen 122 indicating the adjustments needed to move the patient 106 and / or the vision screening device 104 and bring the patient to the correct position for performing the vision screening test. The eye image verification component 134 can determine whether the eye image meets the validity criteria.

[0050] In various examples, after the operator 102 repositions the patient 106 and / or the vision screening device 104 based on the suggested adjustments, the vision screening device 124 may rerun the steps associated with the image capture analysis component 132 .

[0051] In some examples, the vision screening device 104 may execute an eye tracking and output generation component 136 that is configured to generate a focused, centered eye image using an eye tracking algorithm known in the art for a patient video feed or images captured at preset intervals (e.g., videos and images captured using camera 108). The eye tracking and output generation component 136 may initialize an eye tracking algorithm (e.g., a deep learning-based or Kalman filter-based eye tracker) using the eye position generated by the image capture analysis component 132, and then determine the current eye position in real time or near real time in response to slight movements of the patient 106 or the vision screening device 104. The eye tracking algorithm may output the position of the pupil and / or other features of the eye and / or the center of the bridge of the nose. The eye tracking and output generation component 136 may continuously or intermittently (e.g., at preset intervals) update the initial position of the eye bounding box generated by the image capture analysis component 132 based on the eye position output by the eye tracking algorithm. In some examples, the size and dimensions of the eye bounding box may not change, and the eye tracking and output generation component 136 may only update the position of a particular corner (e.g., the upper left corner) of the eye bounding box to specify the new position. In addition, the eye tracking and output generation component 136 may extract image data corresponding to the eye bounding box from the oversampled image to generate a stabilized eye image, which may be presented to the operator 102 on the display screen 122 and / or further processed to determine an output 120 associated with the patient 106. The output 120 may include one or more stabilized eye images. Figure 2C The functions and features of the eye tracking and output generation component 136 are described in detail.

[0052] In each example, the image capture control component 132 can send the oversampled image to the vision screening system 118, and then the eye tracking and output generation component 136 can send a sequence of (x, y) coordinates of a specific corner (e.g., the upper left corner) of the eye bounding box over time, thereby continuously updating the current position of the eye bounding box. The vision screening system 118 can extract and process the current eye image from the oversampled image based on the current position of the eye bounding box. In other examples, other components of the vision screening device 104 can receive the oversampled image and the sequence of eye bounding box positions for further processing.

[0053] In each example, the computer-readable medium 130 may additionally store a patient screening component 138, which may be configured to store and / or access patient data 140 associated with the patient 106. For example, the patient data 140 may include demographic information such as name, age, race, etc. When the vision screening device 104 and / or the vision screening system 118 initiates a vision screening test, the patient 106 may provide or the operator 102 may request patient data 140 about patient demographic information, medical information, preferences, etc. from the patient 106 or the patient's 106 guardian. In these examples, the operator 102 may request data while the screening is ongoing or before the screening begins. In some examples, the operator 102 may be provided with a predetermined category associated with the patient 106, such as a predetermined age range (e.g., newborn to six months, six to twelve months, one to five years old, etc.), and the operator 102 may request the patient data 140 in order to select an appropriate category associated with the patient 106. In other examples, the operator 102 may provide free-form input associated with the patient data 140. In still other examples, the input elements may be provided directly to the patient 106 .

[0054] Alternatively or additionally, the patient screening component 132 may be configured to receive, access and / or store patient data 140 associated with the patient 106 and / or other patients. For example, the patient screening component 132 may store previous patient information associated with the patient 106 and / or other patients. For example, the patient screening component 132 may store previous screening histories of the patient 106, including data from previous screenings, such as color images, NIR images and / or videos of the patient's 106 eyes. The patient screening component 132 may receive the patient data 140 and / or may access such information via the network 116. For example, the patient screening component 132 may access an external database, such as a screening database 144, storing data associated with the patient 106 and / or other patients. The screening database 144 may be configured to store patient data 140 associated with a patient ID. When the operator 102 and / or the patient 106 enters a patient ID, the patient screening component 132 may access or receive the patient data 140 stored in association with the patient ID of the patient 106.

[0055] In each example, the patient screening component 138 may be configured to determine a vision screening test to be performed on the patient 106 based at least in part on the patient data 140. For example, the patient screening component 138 may use the patient data 140 to determine a test category to which the patient 106 belongs (e.g., a test category based on age, medical history, etc.). The patient screening component 138 may determine the vision screening test to be performed based on the test category. For example, if the patient data 140 indicates that the patient is a newborn, the selected vision screening test may include screening for congenital eye symptoms, such as congenital cataracts, retinoblastoma, corneal opacities, strabismus, etc. In addition, eye abnormalities may be associated with systemic genetic diseases, such as Marfan syndrome and Tay-Sachs disease. For example, a screening test for a characteristic erythema of the eye may indicate Tay-Sachs disease. As another example, if the patient data 140 indicates that the patient is over 50 years old, the patient screening component 138 may determine that the vision screening test includes screening for the onset of cataracts, macular degeneration, and other age-related eye diseases.

[0056] The patient screening component 138 can also determine a vision screening test based on the patient's medical history. For example, the screening database 144 can store the patient's 106 medical history related to previous vision screening tests in the patient data 140, including test results, eye images, measurements, recommendations, etc. The patient screening component 138 can access the patient data 140 including the medical history from the screening database 144 and determine the vision screening test to be implemented to monitor the status and changes of previously detected vision health problems. For example, if a progressive eye disease, such as cataracts or macular degeneration, is detected in a previous screening, further screening can be implemented to track the progression of the disease. As another example, if the patient 106 has undergone surgery to remove an eye tumor, the vision screening test may include further screening for eye tumors or scars.

[0057] Additionally, the patient screening component 138 may be configured to receive, access, and / or analyze data from other components of the vision screening device 104 and may store the output 120 (which may include recommendations, diagnoses, measurements, captured images / videos, and / or generated ocular images) in a database, such as a screening database 144, for evaluation by a clinician or for access during subsequent vision screenings of the patient 106. The screening database 144 may provide access to authorized medical professionals to enable printing of reports or further evaluation of data related to the patient 106 screening.

[0058] Figure 1Although an example processor 128 and a computer-readable medium 130 storing an image capture analysis component 132, an eye image verification component 134, an eye tracking and output generation component 136, a patient screening component 138, and / or other components of the vision screening device 104 are shown, the vision screening system 118 may include similar components and / or the same components in any of the examples described herein. In these examples, the vision screening system 118 may include a processor 146 and a computer-readable memory 148 that are configured to perform the functions of some or all of the components in the computer-readable memory 130 of the vision screening device 104. For example, one or more components of the computer-readable memory 130 may be included in an analysis component 150 of the computer-readable memory 148 and may be executed by the processor 146. In these examples, the vision screening system 118 may communicate with the vision screening device 104 via the network 116 using a network interface 152 to receive data from the vision screening device 104 and send results (e.g., output 120) back to the vision screening device 104. The vision screening system 118 may be implemented on a computer proximate to the vision screening device 104, or may be at a remote location. For example, the vision screening system 118 may be implemented as a cloud service on a remote cloud server.

[0059] The vision screening system 118 may be configured to receive data collected during the performance of a vision screening test (such as a color vision screening test) from the vision screening device 104 via the network 116. In some examples, the image capture analysis component 132 or the eye tracking and output generation component 136 may send the oversampled image to the vision screening system 118 via the network 116, and subsequently send information specifying an eye bounding box and an eye bounding box update to be applied to the oversampled image to extract an eye image. In some examples, based at least in part on processing data corresponding to the eye image, the vision screening system 118 may determine an output 120 related to the patient 106. For example, the output 120 may include a recommendation and / or diagnosis related to the eye health of the patient 106 based on analyzing the eye image and / or NIR image data indicating an eye-related disease and / or defect of the patient 106. The vision screening system 118 may communicate the output 120 to the processor 128 of the vision screening device 104 via the network 116. As described above, in any of the examples herein, the vision screening device 104 may alternatively or additionally generate one or more such recommendations, diagnoses, or other outputs.

[0060] The network interface 152 can enable wired and / or wireless communications between components and / or devices displayed in the system 100 and / or with one or more other remote systems and other networked devices. For example, at least some of the network interfaces 152 may include personal area network components to enable one or more short-range wireless communication channels for communication. In addition, at least some of the network interfaces 152 may include wide area network components to enable wide area network communication. Such network interfaces 152 can enable, for example, communication between the vision screening system 118 and the vision screening device 104 and / or other components of the system 100 via the network 116. For example, the network interface 152 can be configured to connect to an external database (e.g., a screening database 144) to receive, access, and / or send screening data using a wireless connection. The wireless connection can include a cellular network connection and a connection established using protocols such as 802.11a, 802.11b, 802.11g, and / or 802.11ac. In other examples, the vision screening device 104 can be directly connected to the external system using one or more wireless protocols such as Bluetooth, Wi-Fi Direct, radio frequency identification (RFID), infrared signals, and / or Zigbee, and other configurations may also be used. Transmitting data to an external database can print reports or further evaluate the patient's vision test data. For example, the collected data and corresponding test results can be wirelessly transmitted and stored in a remote database accessible to authorized medical professionals.

[0061] It should be understood that Figure 1 Although system 100 is depicted as including a single visual screening system 118, in other examples, system 100 may include any number of local or remote visual screening systems substantially similar to visual screening system 118, which are configured to operate independently and / or in combination, and are further configured to communicate via network 116.

[0062] As stated in this article, Figure 1 An exemplary vision screening device 104 is depicted, which includes components for performing a vision screening test on a patient, wherein the patient can be positioned at different distances from the vision screening device 104 based on the requirements of the vision screening test. The vision screening device can include a camera whose oversampled sensor is configured to capture an environmental area that is larger than a target area (such as an eye). The vision screening device 104 can include one or more components for extracting a target area image from image data captured by the oversampled sensor and determining that the target area image meets the requirements of the vision screening test. In some examples, the one or more components can be implemented on a remote vision screening system 118 that communicates with the vision screening device 104 via a network 116. The vision screening device 104 and its components are described in detail with reference to the remaining figures.

[0063] It should be understood that although the components of the vision screening device 104, such as the image capture analysis component 132 and the eye tracking and output generation component 136, are described in the context of vision screening, other applications of these components are also conceivable. For example, in the vision screening application, the target area determined and tracked corresponds to the patient's eye. In other applications, the target area may correspond to other parts of the patient that also need to be located and extracted from the oversampled image. For example, in a patient skin examination, a mole or lesion on the skin can be a target area for determination and tracking. The mole or lesion can be determined based on the mole or lesion area showing different colors or different spectral responses under the radiation source of the vision screening device. Then, the above-mentioned technology can be used to track the determined area to keep the area in the center of the field of view of the screening device. As another example, when examining the retina of the eye, the optic disc or fovea can be used as the target area. The optic disc or fovea can be determined in the retinal image based on the apparent difference between the optic disc or fovea and the background area of ​​the retina. The determined area can then be tracked to keep the optic disc or fovea in the same position in the retinal image during the retinal examination. In another example, in a patient monitoring application, the above components can be used to locate and track the position of the patient on the bed and / or track the patient's body parts to monitor biological signs, such as chest movement for monitoring breathing. In the above example, the method and system related to the device for examining a target area other than the eye can also adjust the extracted area corresponding to the target area in the oversampled image to compensate for the movement of the device (which can be a handheld device) or the patient (which can be a free patient), as described in the vision screening context.

[0064] Figure 2A (not drawn to scale) illustrates an example system 200 according to an example of the present disclosure, including a Figure 1 Schematic diagram of components such as a camera 108 equipped with an oversampling sensor 110 of the vision screening device 104 shown. In each case, the camera 108 can also be equipped with a lens assembly 202 having an optical center 204. The lens assembly 202 directs reflected light from the environment 206 in front of the camera 108 to the oversampling sensor 110. The environment can be an examination room where the patient 106 is located. During the implementation of the first vision screening test, the patient 106 can be required to be located at a distance 208 from the optical center 204, and the distance 208 can be an approximate distance or a range of distances. The patient's eye can be approximately located in an area 210 on an environmental plane 212. In this case, the reflected light from the area 210 corresponding to the eye can pass through the lens assembly 202 and focus on the oversampling sensor 110 to form an image 214.

[0065] Another example Figure 2AAs shown, during the implementation of the second vision screening test, the patient 106 may be required to maintain a distance 216 from the optical center 204, and the distance 216 may be an approximate distance or a range of distances. In this case, the patient's eye may be approximately located in an area 218 on an environmental plane 220. Reflected light from the area 218 corresponding to the eye may be focused onto the oversampling sensor 110 through the lens assembly 202, forming an image 222 on the oversampling sensor 110. In each example, the distance 208 may be the closest operating distance of the vision screening device 104, and the distance 216 may be the farthest operating distance of the vision screening device 104. For example, the distance 208 may be approximately three feet, and the distance 216 may be approximately twenty-two feet.

[0066] like Figure 2A As shown, the width 224 and height 226 of the oversampled sensor 110 of the vision screening device 104 may be sufficient to support a horizontal field of view 228 and a vertical field of view 230 at a distance 216 (which may be the maximum operating distance of the vision screening device). In addition, the resolution of the oversampled sensor 110 (e.g., the total number of pixels obtained by multiplying the number of pixels along the width 224 by the number of pixels along the height 226) may enable the patient's eye image 222 at the maximum operating distance to have a resolution (e.g., the number of pixels in the image 222) sufficient to meet the requirements of the second vision screening test. For example, if the vision screening test requires the eye image to be at least 8 million pixels, and at the farthest distance 216, the eye image occupies 10% of the area of ​​the oversampled sensor 110, then the resolution of the oversampled sensor needs to be at least 80 million pixels, for example, the size of the oversampled sensor along the width 224 is 10,966 pixels and the size along the height 226 is 7,296 pixels. It should be appreciated that if image 222 has adequate resolution at the farthest operating distance, other images of the patient's eye located closer to optical center 204 will be larger and therefore also have adequate resolution.

[0067] As stated in this article, Figure 2A An operating environment of an example vision screening device 104 is depicted, the vision screening device 104 including a camera 10 equipped with an oversampling sensor 110 to implement vision screening tests that require a patient to be positioned at different distances from the vision screening device 104. The vision screening device 104 is intended to perform a full range of vision screenings, which may include a variety of different vision screening tests, including screening for a variety of patient eye diseases, defects, and symptoms, which may require the patient to be positioned at a range of distances from the device. As described above, the vision screening device 104 is configured to capture eye images at a sufficient resolution at distances between the closest operating distance and the farthest operating distance.

[0068] Figure 2B A top view of an environment in which a vision screening test may be performed on a patient 106 is shown, including Figure 1 A schematic diagram of components such as a camera 108 equipped with an oversampling sensor 110 in a vision screening device 104 is shown. The patient 106 is positioned in front of the camera 108 at a distance required for the vision screening test. In the ideal scenario shown, the patient 106 is vertically aligned with the optical axis 232 of the lens assembly 202 passing through the optical center 204. As shown, the oversampling sensor 110, which can be mounted on the sensor assembly 234, captures an oversampled image 242 covering a horizontal field of view 236, which includes the patient 106 and a portion of the patient 106's surroundings. The image 242 includes an area 240 corresponding to a target area 238 (such as the patient's 106 eye), for example, the area 240 can correspond to an eye bounding box.

[0069] As above Figure 1 As described, during the vision screening, components of the vision screening device 104 (such as the image capture analysis component 132 and / or the eye tracking and output generation component 136) may be configured to extract image data from pixels 244 of an image 242 in a target area 240 to generate an eye image of the patient 106. As described above, the vision screening device 104 may be a handheld portable device. Due to movement of the operator of the handheld vision screening device and / or movement of the head of the patient 106, it is expected that the target area 238 will move slightly relative to the camera 108. As a result of this movement, the position of the area 240 corresponding to the eye bounding box may change relative to the boundaries of the oversampled image 242. In addition to translational motion (e.g., the patient's position), the patient's 106 head may also rotate, resulting in a change in the position and / or orientation of the area 240. For example, rotation of the patient's 106 head in the XY plane of the coordinate system 246 corresponds to an in-plane rotation relative to the camera 108 of the vision screening device, while rotation of the patient's 106 head in the YZ plane of the coordinate system 246 (for example, due to up and down movement of the head) or in the XZ plane (for example, due to left and right movement of the head) corresponds to an out-of-plane rotation relative to the camera 108 of the vision screening device.

[0070] Figure 2CTracking and extracting an eye bounding box within an oversampled image 242 is shown. As shown, the eye bounding box 240(1) can be at a first position at a first time, where a corner (e.g., the upper left corner) 248(1) of the eye bounding box 240(1) is located at coordinates (x1, y1) with respect to an origin (0,0) 250 of the oversampled image 242, where the x-coordinate x1 corresponds to a distance 252(1) and the y-coordinate y1 corresponds to a distance 254(1). The eye bounding box 240(1) is further characterized by a width 256(1) and a height 258(1). As is known in the art, any bounding box can be uniquely characterized by a specified corner (e.g., the upper left corner), a width, and a height. The image capture analysis component 132 or the eye tracking and output generation component 136 can present the eye bounding box by specifying the (x, y) coordinates of the upper left corner and the width and height of the eye bounding box. In other examples, the eye bounding box can be alternatively specified using two opposite corners of the bounding box (e.g., the upper left corner and the lower right corner). The image capture analysis component 132 or the eye tracking and output generation component 136 can extract image data (e.g., pixel values) from the area indicated by the eye bounding box to generate an eye image of the patient 106.

[0071] As reference Figure 1 As described, the eye tracking and output generation component 136 can track the patient's eyes and update the position of the eye bounding box continuously in real time or intermittently at predefined intervals. For example, the eye tracking and output generation component 136 can determine an eye bounding box 240(2) at a second time, where the eye bounding box 240(2) is located at a second position different from the first position. The eye bounding box 240(2) can be uniquely specified by the coordinates (x2, y2) of its upper left corner 248(2) and the width 256(2) and height 258(2). As described with respect to the eye bounding box 240(1), the x-coordinate x2 corresponds to the distance 252(2) and the y-coordinate y2 corresponds to the distance 254(2). In each case, the width 256(1) can be the same as the width 256(2), and the height 258(1) can be the same as the height 258(2). In such examples, the eye bounding box can be specified by updating the upper left corner at the second time after the first time. In other examples, the height and width of the eye bounding box may not remain constant over time and need to be specified to uniquely determine the position and extent of the eye edge box.

[0072] In various examples, such as this article reference Figures 2A-2CAs described above, the vision screening device 104 can be operated at different distances from the patient 106, depending on the requirements of the vision screening test being performed. The vision screening device 104 can capture an oversampled image of the patient 106 and the surrounding environment, locate the patient's face in the oversampled image, locate the patient's eyes by specifying an eye bounding box, and generate an eye image of the patient's eyes by extracting relevant image data from the oversampled image. In addition, the vision screening device 104 can track the patient's eyes by updating the eye bounding box and generating a stable image of the patient's 106 eyes, and compensate for small movements of the eyes relative to the vision screening device 106.

[0073] Figure 3A FIG. 3 is a flow chart showing an example workflow 300 for implementing one or more of the vision screening tests described above. For example, the vision screening test may be implemented using the vision screening device 104. Figure 2B As described above, the vision screening device 104 is configured so that the camera 108 of the vision screening device faces the patient who is located in an environment such as in front of the vision screening device in an examination room. Figure 3A , the workflow 300 is shown as a set of steps or boxes of a logical flow chart, and each step of the workflow 300 is associated with the step. The steps of the workflow 300 represent a sequence of operations that can be implemented in hardware, software, or a combination thereof. In terms of software, the steps represent computer executable instructions stored on one or more computer readable storage media, which when executed by a processor (e.g., the processor 128 or the processor 146 of the vision screening system 118) can perform the steps. The steps correspond to the reference Figure 1 The functionality of one or more components described herein, such as the image capture analysis component 132 , the eye image verification component 134 , and the eye tracking and output generation component 136 .

[0074] In some examples, the workflow 300 may be implemented as an app running on the processor 130, which may provide step-by-step instructions to an operator performing a vision screening test, for example, on the display screen 122 of the vision screening device 104. The instructions may include information about the correct positioning of the patient relative to the vision screening device 104 and / or an indication of whether the correct positioning is maintained during the performance of the vision screening test.

[0075] In step 302, the processor captures an oversampled image at the beginning of a vision screening period for the patient. The vision screening device 104 may, for example, use a camera 108 equipped with an oversampled sensor 110 to capture an oversampled image 304 of the patient and the surrounding environment, which may be captured as part of a series of oversampled images. In some examples, step 302 may be performed in response to input from an operator of the vision screening device 104, where the operator selects a vision screening test to be performed, for example, via a user interface displayed on the display screen 122. For example, the vision screening tests supported by the vision screening device 104 may include a photorefractive screening test, a visual acuity screening test, an eye tracking test, a focus test, a color vision screening test, and the like.

[0076] At step 306, the processor may identify a facial region in the oversampled image 304. For example, the processor may perform a facial detection algorithm on the oversampled image 304 to determine a bounding box 308 corresponding to the patient's face, as described with respect to the image capture analysis component 132 of the vision screening device 104. In some examples of the facial detection algorithm, a bounding box is output around the detected facial region. In such examples, the bounding box 308 may be a bounding box output by the facial detection algorithm, or a bounding box generated by the processor based on the output bounding box (e.g., by expanding or contracting the output bounding box). In other examples, the facial detection algorithm may output a partial or complete facial outline, eye position, and other features of the face, etc. In such examples, the processor may generate a bounding box 308 covering the facial region including the facial outline and features.

[0077] At step 310, the processor may detect eye locations corresponding to the first eye 312A and the second eye 312B in the identified facial region 308. For example, the processor may execute an eye detection algorithm (as described with respect to the image capture analysis component 132) to determine the eye locations. The processor may generate an eye bounding box 314 that covers a rectangular area around the first eye 312A and the second eye 312B. The eye bounding box 314 may be specified by its width and height and a corner (e.g., the upper left corner) 315(1) of the rectangular bounding box, as described with reference to FIG. Figure 2C described.

[0078] At step 316, the processor may determine position measurements of the first eye 312A and the second eye 312B within the eye bounding box 314. The position measurements may be based on eye features identified by the eye detection algorithm, such as the pupil, the center of the bridge of the nose, the corner of the eye, etc. For example, the position measurements may include a first distance 320A (e.g., number of pixels) between the pupil 318A and the center 320 of the first eye 312A, a second distance 320B (e.g., number of pixels) between the pupil 318B and the center 320 of the second eye 312B, a third distance 322A (e.g., number of pixels) between the pupil 318A and the nearest vertical edge of the eye bounding box 314, a fourth distance 322B between the pupil 318B and the nearest vertical edge of the eye bounding box 314, and / or an interpupillary distance obtained as the sum of the distance 320A and the distance 320B. Other measurements between eye features and the relative positions of features within the eye bounding box are also contemplated.

[0079] At step 324, the processor may determine whether the eye image corresponding to the eye bounding box 314 meets the criteria of the vision screening test being performed. Acceptable eye image criteria may be based on the requirements of other steps of the vision screening test. For example, the eye image may need to meet size and resolution requirements, positioning requirements of the eye in the eye image, etc., before eye features used to detect eye diseases and / or defects can be detected on the eye image. Figure 3B The criteria for step 324 and further details of determining whether the eye image meets the criteria are described in detail. Step 324 may be performed on an image corresponding to the area within the eye bounding box 314 extracted from the oversampled image 304, or may be performed directly on the oversampled image 314 by accessing pixel values ​​in the area corresponding to the eye bounding box 314.

[0080] At step 326 (step 324—no), the processor may notify the operator of the vision screening device 104, for example, via a user interface displayed on the display screen 122, that the eye image does not meet one or more criteria. In some examples, the notification may indicate that the eye image could not be found due to a failure of the facial detection and / or eye tracking algorithm. The notification may include information about the criteria that were not met and / or instructions for corrective actions by the operator, such as directing the device or patient position to adjust direction. Figure 3B More details of step 326 are described. In various examples, after the operator 102 repositions the patient 106 and / or the vision screening device 104 according to the instructions, the processor can continue to re-execute step 302 to capture a new oversampled image after the adjustments are completed.

[0081] In step 328 (step 324—yes), the processor may execute the eye tracking and output generation component 136 to track the eyes using an eye tracking algorithm known in the art and to keep the eye image substantially unchanged from the eye image that meets the vision screening test criteria in step 324. For example, during the vision screening, the eye position within the oversampled image 304 may change due to slight movements of the operator of the handheld vision screening device 104 and / or slight head movements of the patient 106. The eye tracking and output generation component 136 may determine the eye bounding box 314 at the new position by specifying a new position of a corner (e.g., the upper left corner) 315(2), as shown in reference to FIG. Figure 2C In the event that the eye tracking algorithm is unable to locate and / or track the eye, the workflow 300 can restart at step 302 by recapturing the oversampled image and continuing the sequence of operations as shown. In some examples, if the inability to track the eye continues for a threshold amount of time, an operator of the device can be notified, such as by executing the operations of step 326.

[0082] In step 330, the processor may execute the eye tracking and output generation component 136 to output a stabilized eye image that compensates for the above-mentioned small movements and maintains the eye position in a substantially fixed position within the stabilized eye image. The stabilized eye image is generated by extracting image data corresponding to the eye bounding box 314 area of ​​the oversampled image 304. It is worth noting that the size of the eye bounding box 314 image may also vary based on the distance between the vision screening device 104 and the patient 106. For example, when the patient is closer to the vision screening device 104, the eye bounding box may be larger. In each case, the stabilized eye image 314 can be scaled to a fixed size (e.g., a fixed width and height) to provide a stable, flicker-free, consistent eye image display to the operator.

[0083] As described above, step 324 may determine whether the eye image indicated by eye bounding box 314 satisfies one or more criteria. Figure 3B , column 332 shows an example of an oversampled image and a corresponding eye bounding box, and column 334 shows position measurements associated with a particular test criterion in step 324. As described with respect to the eye image verification component 134, the eye image may be required to have a minimum pixel resolution at which eye features useful for determining eye disease and / or defects may be detected. In step 336, the eye image verification component 134 may check whether the size of the eye image (e.g., the total number of pixels) is acceptable, e.g., is greater than a predefined minimum number of pixels. For example, if the distance of the patient's location exceeds the operating distance of the vision screening device 104, the size of the eye image may be too small, e.g., the number of pixels may be less than the minimum number of pixels.

[0084] The eye image verification component 134 can also check whether the position of the corresponding eye bounding box 314(1) relative to the oversampled image 304(1) is acceptable. If the upper left corner of the eye bounding box 314(1) is located at the coordinates (x1,y1) relative to the origin (0,0) of the upper left corner of the oversampled image 304(1), then the distance 338 is x1 and the distance 340 is y1. The distance 338 and the distance 340 (e.g., as a number of pixels) represent the distance of the eye bounding box 314(1) from the edge of the oversampled image 304(1). If both the distance 338 and the distance 340 are greater than a threshold distance (e.g., a threshold number of pixels), the eye image verification component 134 can determine that the position of the eye image is acceptable. If the distance 338 and / or the distance 340 are less than the threshold distance, it indicates that the eye bounding box 314(1) is too close to the edge of the oversampled image and slight movement during the vision screening test may result in part of the eye not being captured in the oversampled image. In some examples, distance 338 or distance 340 being less than the threshold distance may indicate that a portion of the eye is outside of the captured oversampled image. The patient and / or the vision screening device may need to be moved in a direction to move the patient's eye toward the center of the oversampled sensor 110 of the camera 108.

[0085] At step 342 (which may be included in step 324), the eye image verification component 134 may determine whether the degree of out-of-plane rotation of the patient's head relative to the vision screening device (if any) is below an acceptable threshold. For example, an eye image 314(2) may be generated from the oversampled image 304(2) and the eye image 314(2) may depict that the degree of out-of-plane rotation of the patient's head is greater than an acceptable threshold. Figure 3A As described, this situation can be determined based on the measurements 320A and 320B between the pupil and the center of the eye. In the absence of minimum out-of-plane rotation or only minimum out-of-plane rotation, the ratio of the measurement 320A to the measurement 320B is approximately equal to 1.0. The eye image verification component 134 can calculate the ratio of the measurement 320A to the measurement 320B, and if the absolute difference between the ratio and 1.0 is greater than a threshold score, the eye image verification module 134 can determine that the out-of-plane rotation is unacceptable. For example, if the threshold score is 0.1, the acceptable range of the ratio will be [0.9, 1.1].

[0086] In step 344 (which may be included in step 324), the eye image verification component 134 may determine whether the degree of in-plane rotation of the patient's head relative to the vision screening device (if any) is below an acceptable threshold. For example, an eye image 314(3) can be generated from the oversampled image 304(3), and the eye image 314(2) depicts that the degree of in-plane rotation of the patient's head is greater than an acceptable threshold. This situation can be determined by measuring the vertical distance 346 between the pupil of the first eye and the pupil of the second eye. In the case of no in-plane rotation or only minimal in-plane rotation, the distance 346 can be substantially zero. The eye image verification component 134 can calculate the difference between the y-coordinate value of the first eye and the y-coordinate value of the second eye, and if the absolute difference is greater than the threshold distance, the eye image verification component 134 can determine that the in-plane rotation is unacceptable, otherwise it is acceptable. For example, if the center of the first eye is located at (x f ,y f ), the center of the second eye is located at (x s ,y s ), then the absolute difference |y f -y s | is compared to a threshold distance (which may be expressed as a number of pixels). In other examples, the ratio of the absolute difference of the y-coordinate to the interpupillary distance (distance 320A + distance 320B) may be compared to a threshold ratio to determine acceptability. In certain examples, the eye image verification component 134 may determine whether in-plane rotation is acceptable after applying a rotation compensation technique to the eye image 314(3), for example, the threshold distance or threshold ratio may indicate a maximum amount of in-plane rotation remaining after applying the rotation compensation technique. In each example, the rotation compensation technique may rotate a bounding box corresponding to the eye image 314(3) within the oversampled image 304(3) until the distance 346 is reduced or substantially zero, in the example shown, for example by applying a counterclockwise rotation.

[0087] If each of steps 336, 342, and 344 conclude that the eye image is acceptable, then in step 324 it can be determined that the eye image meets the acceptance criteria, and the workflow 300 can proceed to step 328. Fewer or more validity criteria can be checked in step 324. For example, step 324 can include an additional step of determining whether the clarity level of the eye image is acceptable. As is known in the field of image processing technology, the clarity of the image can be determined based on edge features detected in the image. Similar techniques can be used to determine the clarity of the eye image and compare it to a minimum clarity level.

[0088] If one or more of steps 336, 342, and 344 conclude that the eye image is unacceptable, then in step 324 it can be determined that the eye image does not meet the acceptance criteria, and the workflow 300 can proceed to step 326. In step 326, an operator of the vision screening device 104 can be notified, for example, via a user interface displayed on the display screen 122, that the eye image does not meet one or more criteria, and an indication of which criteria are not met. As described above, the notification can include instructions for corrective actions for the operator. For example, the processor can use the display screen 122 to display an oversampled image and a visualization of the eye bounding box to the operator. The visualization can include a portion of the oversampled image, including the eye bounding box, wherein the color or other characteristics of the eye bounding box indicate an acceptable eye image, or a specific standard that the eye image does not meet. For example, in step 336, the eye image bounding box 314(4) can be determined to be unacceptable. In this example, in step 336, the outline color and / or line style of the eye image bounding box 314(4) can indicate such a determination. For example, if the eye image meets all the criteria, the outline color of the eye image bounding box can be green, but if the eye image does not meet one or more criteria, it can become yellow, orange, or red. In some examples, the color can indicate that the standard is not met. In some examples, when the eye bounding box approaches one or more acceptability thresholds, the outline color can transition from green to yellow before actually crossing the threshold area. In addition, the processor can display an indication of corrective action on the display screen 122. In the example shown, arrow 346 indicates that the patient and / or vision screening device 104 needs to be moved to move the patient's eyes toward the center of the display. In other examples, the arrow can indicate the direction of the corrective action required to eliminate unacceptable in-plane rotation and / or out-of-plane rotation (e.g., turning the patient's head left to right, clockwise, or counterclockwise, etc.).

[0089] As mentioned above Figure 3A and Figure 3B As described, an eye image that meets the acceptability criteria is extracted from the oversampled image captured by the camera 108 of the vision screening device 104, and the patient's eye is tracked during the vision screening to output a stable eye image by compensating for small movements. A visualization of the tracked eye bounding box can be presented to the operator of the vision screening device 104, indicating whether the eye image meets the acceptability criteria of the vision screening test being implemented.

[0090] Figure 4 A flow chart is provided illustrating an example method of performing vision screening using a vision screening device as described herein. Figure 4The method in the embodiment is illustrated as a set of blocks in a logical flow chart, which represent a series of operations that can be implemented in hardware, software, or a combination thereof. In terms of software, the blocks represent computer-executable instructions stored on one or more computer-readable storage media, which when executed by a processor can perform the described operations. Generally, computer-executable instructions include routines, programs, objects, components, data structures, etc. that perform specific functions or implement specific abstract data types. The order in which the operations are described is not intended to be construed as limiting, and any number of the described blocks may be combined in any order and / or in parallel to implement the described operations. Figure 4 In some embodiments, the method may be omitted entirely. Figure 4 One or more blocks of the illustrated method.

[0091] Below about Figure 4 The operations described in the illustrated method may be performed by the apparatus 104 or system 118 and / or its various components as described herein. Figure 1 The system 100 and vision screening device 104 are shown to describe Figure 4 Unless otherwise specified, the method shown. Figure 4 Any of the operations described in the illustrated method may be performed individually or in combination by an image capture analysis component 132 executed by the processor 128 of the vision screening device 104, an eye image verification component 134, an eye tracking and output generation component 136 and / or a patient screening component 138 and / or an analysis component 150 executed by the processor 146 of the vision screening system 118.

[0092] Reference Figure 4 In the illustrated example method 400, at operation 402, the image capture analysis component 132 and / or one or more processors associated therewith may cause a camera to capture an oversampled digital image of a patient and his / her surroundings. For example, the camera may include a camera 108 in a vision screening device 104 equipped with an oversampled sensor 110. The oversampled sensor 110 is configured to capture an oversampled image of the patient and his / her surroundings such that a target area (such as the patient's eye) including a portion of the oversampled area includes a sufficient number of pixels (e.g., a sufficiently high resolution) to perform one or more vision screening tests on the patient. Vision screening tests may have different requirements for the distance between the patient and the vision screening device. The camera 108 may be equipped with a lens having a field of view suitable for capturing oversampled images of patients at different operating distances from the vision screening device 104, and the resolution (e.g., number of pixels) of the oversampled sensor 110 may be sufficient to operate at the farthest distance from the patient required for the vision screening test implemented by the vision screening device 104. In some examples, the oversampled image may be captured under ambient lighting in a vision screening setting (e.g., an examination room).

[0093] At operation 404, the image capture analysis component 132 may identify the patient's face in the oversampled image captured in operation 402. The image capture analysis component 132 may apply a facial detection algorithm known in the art to the oversampled image to determine the location of the face in the oversampled image. The facial location may be determined as a bounding box around the detected face and may include facial feature locations (such as eye and mouth locations) or facial contours.

[0094] At operation 406, the image capture analysis component 132 may identify a first eye and a second eye of the patient. The image capture analysis component 132 may apply an eye detection algorithm known in the art to an area within a bounding box around the detected face or to a larger area including the area within the bounding box of the face. The eye detection algorithm may output an eye center location (e.g., x-coordinate, y-coordinate), and may also include locations of eye features (such as inner and outer corners of the eye) and a center location between the first eye and the second eye.

[0095] At operation 408, the image capture analysis component 132 can determine the position of the first eye and the second eye. In some examples, the eye detection algorithm applied in operation 408 can generate the position of the pupil of the eye as part of the detection output, which can be used as the eye position. The eye position can also be approximately determined by the coordinates of the center of the eye as output by the eye detection algorithm. Alternatively or additionally, the image capture analysis component 132 can apply the pupil detection algorithm to the area detected as an eye by the eye detection algorithm. In some examples, the pupil detection algorithm can be based on images captured under different wavelengths (such as infrared or near infrared radiation).

[0096] At operation 410, the eye image verification component 134 can determine whether the positions of the first eye and the second eye meet eye image standards. Eye image standards may vary based on vision screening test requirements. The eye image may include both the first eye and the second eye, and the standard may specify an eye pupil position requirement within the eye image or a center position requirement between the first eye and the second eye within the eye image. Figure 3B As described, these criteria may include pupil horizontal alignment requirements (e.g., in-plane rotation is less than a threshold), equal spacing requirements from the center of the eye image (e.g., out-of-plane rotation is less than a threshold), minimum inter-pupillary pixel number requirements (e.g., related to the eye image size), etc.

[0097] If the eye position does not meet the eye image criteria (operation 410-No), the eye image verification component 132 may notify the operator of the vision screening device 104. Figure 3BAs described, the notification may include information about the criteria not being met and / or operator corrective action guidance. For example, the notification may include a visualization using color coding of a bounding box around the eye to indicate that the criteria are met, and include an arrow indicating a direction of correction. The notification may be displayed to the operator on a display screen (e.g., display screen 122) of the vision screening device. After the corrective action is completed, the operation may be reset to operation 402.

[0098] If the eye position meets the eye image criteria (operation 410-yes), the eye tracking and output generation component 136 can generate an eye image. In each example, the eye tracking and output generation component 136 can generate an eye image by extracting image data from a region including a first eye and a second eye in the oversampled image captured in operation 402. In addition, the eye tracking and output generation component 136 can track the eye position and update the eye image so that the relative position of the eye within the eye image remains substantially unchanged in response to slight movements of the vision screening device or the patient. The eye tracking and output generation component 136 can generate an eye image that is stabilized by compensating for slight movements. For example, as shown in reference Figure 2C As described, by maintaining the current position of one corner (e.g., the upper left corner) of the bounding box around the eye, the bounding box around the eye can be tracked and updated in real time, near real time, or intermittently at preset intervals, and the bounding box indicates the area in the oversampled image corresponding to the eye image.

[0099] At operation 416, the patient screening component 138 and additional components of the vision screening device 104 may utilize the ocular images generated in operation 414 and other data including ocular images and / or video data captured under various illuminations to determine a condition associated with the first eye and / or the second eye of the patient. The condition may include diseases and defects detected in the pupil, retina, and / or lens of the patient's 106 eye, such as refractive errors, strabismus, and retinal conditions. In some examples, the captured and / or generated ocular images may be used to assist the operator 102 or clinician in diagnosing ocular diseases and defects of the patient 106. In this manner, additional components of the vision screening device 104 may also automatically determine an ocular condition based on analyzing images captured by the vision screening device 104 during a vision screening.

[0100] At operation 418, the patient screening component 138 can output an indication of a condition of the first eye and / or the second eye. The first indication can correspond to a situation where the eye condition indicates a diseased or abnormal state, and can include a recommendation for additional screening and / or a diagnosis of a detected disease or abnormality. The first indication can also include a link to stored images (including images of the eye, images captured under different wavelengths of illumination) and / or generated visualizations. The second indication can correspond to a situation where the eye condition is determined to be normal, and the second indication can indicate that the patient has passed the vision screening or that the patient's eyes appear normal and healthy.

[0101] As described above, the example method 400 can be performed by components of the vision screening device 104 executed by the processor 128 of the device 104. The example method 400 illustrates operations performed during at least a portion of a vision screening test performed on a patient (e.g., patient 106) to determine the patient's eye-related diseases and / or defects based on eye images captured under different wavelengths of illumination. In an alternative example, some or all of the operations of the method 400 can be performed by the processor 146 of the vision screening system 118 connected to the vision screening device 104 via the network 116.

[0102] At least based on this specification, it should be understood that the vision screening device and related systems and methods disclosed herein can be used to assist in performing one or more vision screening tests, including patient eye disease and / or defect screening tests, wherein the patient maintains different distances from the vision screening device based on the vision screening test requirements. The components of the vision screening device can be configured to capture oversampled images of the patient and the surrounding environment, locate the patient's eyes, and generate stable patient eye images at different operating distances. These components can perform image processing techniques to detect faces, detect eyes and eye features, track eye positions, and extract image data corresponding to the patient's eye images from the oversampled images. In addition, the vision screening device can be configured to generate radiation of different bands to illuminate the eyes of patients undergoing vision screening, capture eye images under different illumination conditions, determine eye conditions, and generate output indicating a diagnosis, recommendation, or result of a vision screening test. The device can be a portable and / or handheld device, thereby allowing the device to be used to screen patients in various environments without the need to use a tripod to mount the device, and provide a comfortable experience for the patient by compensating for minor patient movements. The device can generate stable ocular images with sufficient resolution for diagnosing ocular conditions and supporting vision screening tests that require the patient to maintain different distances from the vision screening device.

[0103] The above merely illustrates the principles of the present disclosure, and various modifications may be made by those skilled in the art without departing from the scope of protection of the present disclosure. The above examples are for illustrative purposes only and are not intended to be limiting. The present disclosure may also take many other forms besides those disclosed herein. Accordingly, it should be emphasized that the present disclosure is not limited to the methods, systems, and devices expressly disclosed, but is intended to cover changes and modifications made thereto within the spirit of the appended claims.

[0104] As another example, changes in equipment or process constraints (e.g., size, configuration, components, order of process steps, etc.) may be implemented to further optimize the structures, devices, and methods provided by the present disclosure as illustrated and described herein. Regardless, the structures and devices described herein and the related methods are widely applicable. Therefore, the subject matter of the present disclosure is not limited to any single example described herein, but should be interpreted in terms of the breadth and scope of the present disclosure in accordance with the appended claims.

[0105] Sample Clauses

[0106] This disclosure example is embodied in one or more of the following clauses:

[0107] Item 1. A vision screening device, comprising: a sensor configured to capture an image; a processor operably connected to the sensor; a memory storing instructions that, when executed by the processor, cause the processor to: cause the sensor to capture a first image of an environment including a patient; identify a first portion of the first image corresponding to the face of the patient; based on the first portion, determine a second portion of the first image that is at least partially located within the first portion, wherein the second portion illustrates a first eye of the patient and a second eye of the patient; determine a first position of the first eye within the second portion and a second position of the second eye within the second portion; determine whether the first position and the second position meet corresponding eye position standards; based on determining that the first position and the second position meet corresponding eye position standards, extract image data from the first image, wherein the image data: includes data corresponding to the second portion and does not include data corresponding to the remaining portion of the first image other than the second portion; based on the image data, generate a second image illustrating the first eye and the second eye.

[0108] Clause 2. The vision screening device according to Clause 1, further comprising a display screen, wherein the instructions further cause the processor to display the second image on the display screen.

[0109] Clause 3. A vision screening device according to any one of the above clauses, alone or in combination, wherein the instruction further causes the processor to: based on determining that the first position and the second position do not meet the corresponding eye position standards, display an instruction on the display screen indicating at least one of the following: changing the position of the vision screening device, changing the position of the patient, or adjusting the patient's head.

[0110] Clause 4. A vision screening device according to any one of the above clauses, alone or in combination, wherein the instructions further cause the processor to: determine a condition of at least one of the first eye or the second eye based on the second image; and display an output indicating the condition on the display screen.

[0111] Clause 5. A vision screening device as described in any of the above clauses, alone or in combination, wherein the condition is associated with at least one of visual acuity, abnormality, disease or normal vision.

[0112] Clause 6. A vision screening device according to any of the above clauses, alone or in combination, wherein the resolution of the sensor is based at least in part on the maximum distance between the sensor and the patient during operation of the vision screening device.

[0113] Clause 7. A vision screening device according to any one of the above clauses, alone or in combination, wherein the first image is captured at a first time, and the instructions also cause the processor to: cause the sensor to capture a third image at a second time, the second time being later than the first time; identify a third part of the second image, the third part including the patient's face; identify a fourth part of the third part, the fourth part including the patient's eyes; and cause the display to display the fourth part instead of the third part.

[0114] Clause 8. A vision screening device according to any one of the above clauses, alone or in combination, wherein the criteria include one or more of the following: the size of the first eye, the position of the first eye relative to the boundary of the first image, the position of the first eye relative to the boundary of the second part, or the distance between the first eye and the second eye.

[0115] Item 9. A method, comprising: capturing a first image, the first image including a patient; identifying a first portion of the first image, the first portion illustrating at least a portion of the patient's face; identifying a second portion of the first image that is at least partially located within the first portion, wherein the second portion illustrates a first eye of the patient and a second eye of the patient; determining a first position of a pupil of the first eye and a second position of a pupil of the second eye relative to a center point of the first portion based on the second portion; determining whether the first position and the second position meet corresponding pupil position criteria; based on determining that the first position and the second position meet corresponding pupil position criteria, extracting image data from the first image, wherein the image data: includes data corresponding to the second portion and does not include data corresponding to the remainder of the first image other than the second portion; based on the image data, generating a second image illustrating the first eye and the second eye; and outputting an indication of the condition via an electronic device.

[0116] Clause 10. A method according to Clause 9, wherein the first image is captured at a first time, and the method further includes: capturing a third image at a second time, the second time being later than the first time; identifying a third part of the second image, the third part including the face of the patient; identifying a fourth part of the third part, the fourth part including the eyes of the patient; and causing the electronic device to display the fourth part instead of the third part.

[0117] Clause 11. A method according to any one of the above clauses, alone or in combination, wherein the pupil position standard includes one or more of the following: the position of the pupil of the first eye relative to the boundary of the first image, the position of the pupil of the first eye relative to the boundary of the second part, the horizontal distance between the pupil of the first eye and the pupil of the second eye, or the vertical distance between the pupil of the first eye and the pupil of the second eye.

[0118] Clause 12. The method according to any one of the above clauses, alone or in combination, further includes: outputting an indication of satisfying the corresponding pupil position standard via the electronic device.

[0119] Clause 13. The method according to any one of the above clauses, alone or in combination, further includes: generating an instruction to adjust the patient's position based on at least one of the corresponding pupil position standards not being met; and outputting the instruction via the electronic device.

[0120] Clause 14. The method according to any one of the preceding clauses, alone or in combination, wherein the condition is associated with at least one of visual acuity, optical abnormality, optical disease or normal vision.

[0121] Item 15. A system comprising: a memory; a processor; computer executable instructions stored in the memory and executable by the processor to perform the following operations: cause a sensor to capture a first image of a patient; determine a first portion of the first image corresponding to the face of the patient; determine a second portion of the first image, the boundary of the second portion being at least partially located within the first portion, wherein the second portion illustrates a first eye of the patient and a second eye of the patient; determine a first position of the first eye relative to the boundary of the second portion and a second position of the second eye relative to the boundary of the second portion; determine whether the first position and the second position meet corresponding eye position standards; based on determining that the first position and the second position meet corresponding eye position standards, extract image data from the first image, wherein the image data does not contain data from outside the boundary in the first image; based on the image data, generate a second image illustrating the first eye and the second eye; based on the second image, determine a condition of at least one of the first eye or the second eye.

[0122] Clause 16. The system of clause 15, wherein the condition is associated with at least one of visual acuity, optical abnormality, optical disease, or normal vision.

[0123] Item 17. In the system described in any one of the above items alone or in combination, the operation also includes: determining to correct the second part based on that the first position and the second position do not meet the corresponding eye position standards, wherein after applying the correction, the first position and the second position meet the corresponding eye position standards within the second part; and generating a third image illustrating the first eye and the second eye based on image data extracted from the first image within the second part after applying the correction.

[0124] Clause 18. A system according to any one of the above clauses, alone or in combination, wherein the first image is captured at a first time, and the operation further includes: causing the sensor to capture a third image at a second time, the second time being later than the first time; identifying a third part of the second image, the third part including the face of the patient; identifying a fourth part of the third part, the fourth part including the eyes of the patient; causing the fourth part instead of the third part to be displayed on the display screen.

[0125] Clause 19. The system according to any one of the above clauses, alone or in combination, wherein the operation further comprises: generating an instruction to adjust the position of the patient or the position of the sensor based on at least one of the corresponding eye position criteria not being met.

[0126] Clause 20. A system according to any one of the above clauses, alone or in combination, wherein the eye position criteria include one or more of the following: the position of the first eye relative to the boundary of the first image, the position of the first eye relative to the boundary of the second part, or the distance between the first eye and the second eye.

Claims

1. A vision screening device, comprising: a sensor configured to capture an image; a processor operatively connected to the sensor; and A memory storing instructions, which, when executed by a processor, cause the processor to: causing the sensor to capture a first image of an environment including the patient; identifying a first portion of the first image corresponding to a face of the patient; determining, based on the first portion, a second portion of the first image that is at least partially within the first portion, wherein the second portion illustrates a first eye of the patient and a second eye of the patient; determining a first position of the first eye within the second portion and a second position of the second eye within the second portion; determining whether the first position and the second position satisfy corresponding eye position criteria; Based on determining that the first position and the second position satisfy corresponding eye position criteria, image data is extracted from the first image, wherein the image data: contains data corresponding to the second part, and does not contain data corresponding to a portion of the first image other than the second portion; and Based on the image data, a second image illustrating the first eye and the second eye is generated.

2. The vision screening device according to claim 1, further comprising a display screen, wherein: The instructions also cause the processor to display the second image on the display screen.

3. The vision screening device according to claim 2, wherein: The instructions further cause the processor to: Based on determining that the first position and the second position do not satisfy corresponding eye position criteria, displaying instructions on the display screen indicating at least one of the following: changing the location of the vision screening device, Change the patient's location, or Adjust the patient's head.

4. The vision screening device according to claim 2, wherein: The instructions further cause the processor to: determining a condition of at least one of the first eye or the second eye based on the second image; and An output indicative of the condition is displayed on the display screen.

5. The vision screening device according to claim 4, wherein: The condition is associated with at least one of visual acuity, abnormality, disease, or normal vision.

6. The vision screening device according to claim 1, wherein: The resolution of the sensor is based at least in part on a maximum distance between the sensor and the patient during operation of the vision screening device.

7. The vision screening device according to claim 2, wherein: capturing the first image at a first time, and the instructions further cause the processor to: causing the sensor to capture a third image at a second time, the second time being later than the first time; identifying a third portion of the second image, the third portion including a face of the patient; identifying a fourth portion of the third portion, the fourth portion including an eye of the patient; and The display is caused to display the fourth portion without displaying the third portion.

8. The vision screening device according to claim 1, wherein: The criteria include one or more of the following: the size of the first eye, the position of the first eye relative to a boundary of the first image, the position of the first eye relative to a boundary of the second portion, or The distance between the first eye and the second eye.

9. A method comprising: capturing a first image, the first image including a patient; identifying a first portion of the first image, the first portion illustrating at least a portion of a face of the patient; identifying a second portion of the first image that is at least partially within the first portion, wherein the second portion illustrates a first eye of the patient and a second eye of the patient; determining, based on the second portion, a first position of a pupil of the first eye and a second position of a pupil of the second eye relative to a center point of the first portion; determining whether the first position and the second position satisfy corresponding pupil position criteria; Based on determining that the first position and the second position satisfy corresponding pupil position criteria, image data is extracted from the first image, wherein the image data: contains data corresponding to the second part, and does not contain data corresponding to the remaining portion of the first image other than the second portion; generating, based on the image data, a second image illustrating the first eye and the second eye; determining a condition of at least one of the first eye or the second eye based on the second image; and An indication of the condition is output via an electronic device.

10. The method according to claim 9, wherein: At a first time capturing the first image, the method further comprises: capturing a third image at a second time, the second time being later than the first time; identifying a third portion of the second image, the third portion including a face of the patient; identifying a fourth portion of the third portion, the fourth portion including an eye of the patient; and The electronic device is caused to display the fourth portion without displaying the third portion.

11. The method according to claim 9, wherein: The pupil position criteria include one or more of the following: the position of the pupil of the first eye relative to a boundary of the first image, the position of the pupil of the first eye relative to the boundary of the second portion, the horizontal distance between the pupil of the first eye and the pupil of the second eye, or The vertical distance between the pupil of the first eye and the pupil of the second eye.

12. The method according to claim 9, further comprising: An indication is outputted via the electronic device that the corresponding pupil position criteria are met.

13. The method according to claim 9, further comprising: generating instructions to adjust the position of the patient based on at least one of the corresponding pupil position criteria not being met; and The instruction is outputted via the electronic device.

14. The method according to claim 9, wherein: The condition is associated with at least one of visual acuity, optical abnormality, optical disease, or normal vision.

15. A system comprising: Memory; processor; and Computer executable instructions, stored in the memory and executable by the processor, to perform the following operations: causing the sensor to capture a first image of the patient; determining a first portion of the first image corresponding to a face of the patient; determining a second portion of the first image, a boundary of the second portion being at least partially within the first portion, wherein the second portion illustrates a first eye of the patient and a second eye of the patient; determining a first position of the first eye relative to a boundary of the second portion and a second position of the second eye relative to a boundary of the second portion; determining whether the first position and the second position satisfy corresponding eye position criteria; extracting image data from the first image based on determining that the first position and the second position satisfy corresponding eye position criteria, wherein the image data does not include data in the first image from outside the boundary; generating, based on the image data, a second image illustrating the first eye and the second eye; and Based on the second image, a condition of at least one of the first eye or the second eye is determined.

16. The system of claim 15, wherein: The condition is associated with at least one of visual acuity, optical abnormality, optical disease, or normal vision.

17. The system of claim 15, the operations further comprising: determining to modify the second portion based on that the first position and the second position do not meet corresponding eye position standards, wherein after applying the correction, the first position and the second position satisfy corresponding eye position criteria within the second portion; and based on image data extracted from the first image in the second portion after applying the correction, A third image illustrating the first eye and the second eye is generated.

18. The system of claim 15, wherein: At a first time, capturing the first image, the operations further include: causing the sensor to capture a third image at a second time, the second time being later than the first time; identifying a third portion of the second image, the third portion including a face of the patient; identifying a fourth portion of the third portion, the fourth portion including an eye of the patient; and The fourth portion is caused to be displayed on the display screen instead of the third portion.

19. The system of claim 15, the operations further comprising: Based on at least one of the corresponding eye position criteria not being met, instructions are generated to adjust the position of the patient or the position of the sensor.

20. The system of claim 15, wherein: The eye position criteria include one or more of the following: the position of the first eye relative to a boundary of the first image, the position of the first eye relative to a boundary of the second portion, or The distance between the first eye and the second eye.

Images