Image retention and stitching for small flash eye disease diagnosis

Abstract

To provide image retention and stitching for minimal flash eye disease diagnosis.SOLUTION: Systems and methods for minimizing retinal exposure to flash during image acquisition for diagnosis are provided herein. In certain embodiments, the system captures multiple retinal images of different retinal regions. The system determines that a first portion of the first image does not satisfy a criterion while a second portion of the first image satisfies the criterion, identifies a portion of the retina depicted in the first portion that does not satisfy the criterion, and determines whether the portion of the retina is depicted in a third portion of the second image and whether the third portion satisfies the criterion. In response to determining that the third portion satisfies the criteria, the system performs a diagnosis. In response to determining that the portion of the retina is not depicted in the second image, the system captures an additional image of the retinal region.SELECTED DRAWING: Figure 6

Description

[Technical Field]

[0001] (background) The present invention relates generally to autonomous diagnosis of retinal abnormalities, and more particularly to image retention to reduce the need to fire a flash at a patient's retina. [Background technology]

[0002] An autonomous system for diagnosing retinal abnormalities captures images of a patient's retina (used interchangeably herein with the word "fundus") and analyzes those images for abnormalities. Typically, images of different portions of the retina (e.g., an image centered on the fovea and an image centered on the optic disc) are captured. When the captured images are insufficient to perform a sufficient diagnosis, additional images are captured until sufficient images of each portion of the retina have been obtained. Repeated exposure to flashes by the imaging device can further cause pupil restriction, making successful imaging less likely after each flash. Therefore, reducing the need for additional images to be captured until sufficient images are captured reduces the number of flashes and reduces the need to take numerous photographs or have the patient wait until their pupils have returned to normal dilation to be diagnosed. Additionally, reducing the need for additional images may allow for more rapid diagnosis in that a number of flashes that would significantly increase pupil constraints are unlikely to be achieved in a seated position, thus preventing the need to fire a flash at the patient's eye, delaying diagnosis of the patient until such time as it is likely to again result in a viable image. Summary of the Invention [Means for solving the problem]

[0003] (overview) Provided herein are systems and methods for reducing the need to re-flash a patient's eye to capture an image of a retinal region when a previously captured image is insufficient. As an example, two images may be captured to perform a diagnosis of a patient's right eye: one centered on the fovea and one centered on the optic disc. There is some overlap between the portions of the retina depicted in these two images. If the region of the retina depicted in one of the images is unusable for some reason (e.g., overexposure or underexposure), rather than recapturing the image, the system may determine whether the same region is depicted without the same deficiencies in another image. If the region is usable from the other image, the system may avoid the need to re-flash a patient's right eye to capture another image of the region and instead stitch portions of the two images together to perform a diagnosis.

[0004] To minimize retinal exposure to flash during diagnostic image acquisition, in one embodiment, a retinal image preprocessing tool captures multiple retinal images (e.g., by instructing an imaging device to take images and send them to the retinal image preprocessing tool). Each retinal image may correspond to a different retinal region of multiple retinal regions (e.g., a region centered on the fovea and a region centered on the optic disc). The retinal image preprocessing tool may determine that a first portion of a first image (e.g., the portion centered on the fovea) does not meet a criterion (e.g., the image is overexposed or underexposed, the image has shadows or other artifacts, etc.), while a second portion of the first image (e.g., the portion centered on the optic disc) meets the criterion (e.g., the second image is properly exposed).

[0005] The retinal image preprocessing tool may identify a portion of the retina depicted in the first portion that does not meet the criteria and may determine that the same portion of the retina is depicted in a third portion of the second image. The retinal image preprocessing tool may determine whether the third portion meets the criteria. In response to determining that the third portion meets the criteria, a diagnosis may be performed using multiple retinal images. In response to determining that the portion of the retina is not depicted in the second image, the retinal image preprocessing tool may capture additional images of the retinal area depicted in the first image. The present invention provides, for example, the following items. (Item 1) 1. A method for minimizing retinal exposure to flash light during diagnostic image acquisition, comprising: capturing a plurality of retinal images, each retinal image corresponding to a different retinal region of a plurality of retinal regions, the plurality of retinal images including a first image and a second image; determining that a first portion of the first image does not meet a criterion, while a second portion of the first image meets the criterion; identifying a portion of the retina depicted within the first portion that does not meet the criteria; and determining whether the portion of the retina is depicted within a third portion of the second image; and determining whether the third portion meets the criteria; and performing the diagnosis using the plurality of retinal images in response to determining that the third portion meets the criteria; and capturing an additional image of the retinal area depicted in the first image in response to determining that the portion of the retina is not depicted in the second image; and A method comprising: (Item 2) 2. The method of claim 1, wherein performing the diagnosis includes passing the images through a fully autonomous machine learning model that outputs a likelihood of disease based on the plurality of retinal images. (Item 3) Item 10. The method of item 1, wherein each retinal image of the plurality of retinal images comprises a multi-frame image, each frame of the multi-frame image capturing an image at a different level of flash exposure. (Item 4) 2. The method of claim 1, wherein determining that the first portion of the first image does not meet the criteria comprises determining that the first portion of the first image is either overexposed or underexposed. (Item 5) Item 10. The method of item 1, wherein the second image is an image of the retina of the same eye as depicted by the first image. (Item 6) performing the diagnosis using the plurality of retinal images, generating a composite image including the first portion of the image with the third portion of the second image stitched into the first image; performing said diagnosis using said composite image; and The method according to item 1, comprising: (Item 7) performing the diagnosis using the plurality of retinal images, analyzing the first image without taking into account the first portion to generate a first analysis; analyzing the third portion of the second image to generate a second analysis; and performing said diagnosis using said first analysis and said second analysis; The method according to item 1, comprising: (Item 8) Capturing the additional image comprises: transmitting, using an application protocol interface (API), instructions to an imaging device to capture an additional image using the viewpoint used to capture the first image; receiving the additional image from the imaging device; The method according to item 1, comprising: (Item 9) Item 11. The method of item 1, wherein performing the diagnosis using the plurality of retinal images includes using the third portion in the diagnosis. (Item 10) 1. A computer program product for minimizing retinal exposure to flash during diagnostic image acquisition, comprising: capturing a plurality of retinal images, each retinal image corresponding to a different retinal region of a plurality of retinal regions, the plurality of retinal images including a first image and a second image; determining that a first portion of the first image does not meet a criterion, while a second portion of the first image meets the criterion; identifying a portion of the retina depicted within the first portion that does not meet the criteria; and determining whether the portion of the retina is depicted within a third portion of the second image; and determining whether the third portion meets the criteria; and performing the diagnosis using the plurality of retinal images in response to determining that the third portion meets the criteria; and capturing an additional image of the retinal area depicted in the first image in response to determining that the portion of the retina is not depicted in the second image; and 1. A computer program product comprising a non-transitory computer-readable storage medium containing computer program code for performing the steps of: (Item 11) Item 11. The computer program product of item 10, wherein the computer program code for performing the diagnosis includes computer program code for passing the images through a fully autonomous machine learning model that outputs a likelihood of disease based on the plurality of retinal images. (Item 12) Item 11. The computer program product of item 10, wherein each retinal image of the plurality of retinal images comprises a multi-frame video, each frame of the multi-frame video capturing an image at a different level of flash exposure. (Item 13) Item 11. The computer program product of item 10, wherein the computer program code for determining that the first portion of the first image does not satisfy the criterion comprises computer program code for determining that the first portion of the first image is either overexposed or underexposed. (Item 14) Item 11. The computer program product of item 10, wherein the second image is an image of the retina of the same eye as that depicted by the first image. (Item 15) The computer program code for performing the diagnosis using the plurality of retinal images comprises: generating a composite image including the first portion of the image with the third portion of the second image stitched into the first image; performing said diagnosis using said composite image; and Item 11. A computer program product according to item 10, comprising computer program code for performing the steps of: (Item 16) The computer program code for performing the diagnosis using the plurality of retinal images comprises: analyzing the first image without taking into account the first portion to generate a first analysis; analyzing the third portion of the second image to generate a second analysis; and performing said diagnosis using said first analysis and said second analysis; Item 11. A computer program product according to item 10, comprising computer program code for performing the steps of: (Item 17) The computer program code for capturing the additional images comprises: transmitting, using an application protocol interface (API), instructions to an imaging device to capture an additional image using the viewpoint used to capture the first image; receiving the additional image from the imaging device; Item 11. A computer program product according to item 10, comprising computer program code for performing the steps of: (Item 18) Item 11. The computer program product of item 10, wherein performing the diagnosis using the plurality of retinal images includes using the third portion in the diagnosis. (Item 19) 1. A computer program product for minimizing retinal exposure to flash during diagnostic image acquisition, comprising: a first module for capturing a plurality of retinal images, each retinal image corresponding to a different retinal region of a plurality of retinal regions, the plurality of retinal images including a first image and a second image; a second module, determining that a first portion of the first image does not meet a criterion, while a second portion of the first image meets the criterion; identifying a portion of the retina depicted within the first portion that does not meet the criteria; and determining whether the portion of the retina is depicted within a third portion of the second image; and determining whether said third portion satisfies said criteria; a second module for performing a third module for performing the diagnosis using the plurality of retinal images in response to determining that the third portion meets the criteria; and a fourth module for capturing an additional image of the retinal area depicted in the first image in response to determining that the portion of the retina is not depicted in the second image; and 10. A computer program product comprising a computer readable storage medium containing computer program code comprising: (Item 20) 20. The computer program product of claim 19, wherein performing the diagnosis includes passing the images through a fully autonomous machine learning model that outputs a likelihood of disease based on the plurality of retinal images. [Brief explanation of the drawings]

[0006] BRIEF DESCRIPTION OF THE DRAWINGS [Figure 1] FIG. 1 is an exemplary block diagram of system components in an environment for utilizing a retinal image preprocessing tool, according to one embodiment.

[0007] [Figure 2] FIG. 2 is an exemplary block diagram of modules and components of an imaging device, according to one embodiment.

[0008] [Figure 3] FIG. 3 is an exemplary block diagram of modules and components of a retinal image preprocessing tool, according to one embodiment.

[0009] [Figure 4] FIG. 4 is a block diagram illustrating components of an exemplary machine capable of reading instructions from a machine-readable medium and executing them within a processor (or controller).

[0010] [Figure 5] FIG. 5 depicts an exemplary image of a patient's eye according to one embodiment.

[0011] [Figure 6] FIG. 6 is an exemplary flow chart for minimizing retinal exposure to flash during diagnostic image acquisition, according to one embodiment.

[0012] The figures depict various embodiments of the present invention for illustrative purposes only. Those skilled in the art will readily recognize from the following discussion that alternative embodiments of the structures and methods illustrated herein may be employed without departing from the inventive principles described herein. DETAILED DESCRIPTION OF THE INVENTION

[0013] (Detailed explanation) (a) Environmental Overview FIG. 1 is an exemplary block diagram of system components in an environment for utilizing a retinal image preprocessing tool, according to one embodiment. The environment 100 includes an imaging device 110, a network 120, a retinal image preprocessing tool 130, and a retinal disease diagnosis tool 140. The imaging device 110 is a device configured to capture one or more images of a retina of a patient's eye. The imaging device 110 may be caused to capture such images autonomously when commanded through manual operation, by computer program instructions, or an external signal (e.g., received from a retinal pigmentation determination tool 130), or a combination thereof. Examples of what these images may look like and how they are derived are described in commonly owned U.S. patent application Ser. No. 15 / 466,636, filed March 22, 2017, the disclosure of which is hereby incorporated by reference in its entirety. Another example is described with respect to FIG. 5 of this disclosure.

[0014] After capturing the image, the imaging device 110 transmits the image to the retinal disease diagnostic tool 140. The retinal disease diagnostic tool 140 may take one or more images as input and may use a machine learning model to fully autonomously output a diagnosis based on the input. The retinal disease diagnostic tool 140 autonomously analyzes the retinal images and determines a diagnosis using machine learning analysis of biomarkers therein. The diagnosis may specifically be a determination that the user has a particular disease, such as diabetic retinopathy, or may be a determination that the user likely has a disease and should therefore see a physician for confirmation and treatment. The manner in which the retinal disease diagnostic tool 140 performs the analysis and determines a diagnosis is further discussed in commonly owned U.S. Patent No. 10,115,194, issued October 30, 2019, the disclosure of which is hereby incorporated by reference in its entirety.

[0015] Prior to performing a diagnosis, retinal disease diagnosis tool 140 may have a retinal image preprocessing tool 130 that determines whether the image(s) captured by imaging device 110 are sufficient for performing a diagnosis. The manner in which retinal image preprocessing tool 130 performs this analysis is described in further detail below with respect to FIG. 3. While depicted as a component of retinal disease diagnosis tool 140, retinal image preprocessing tool 130 may be a standalone entity that receives images, reviews the images for sufficiency, and then transmits the images to retinal disease diagnosis tool 140 if the images are sufficient for diagnosis. Retinal image preprocessing tool 130 and / or retinal disease diagnosis tool 140 may be instantiated on one or more servers. Furthermore, retinal image preprocessing tool 130 and / or retinal disease diagnosis tool 140 may be instantiated in whole or in part on imaging device 110, thus eliminating the need for some or all of the communication traffic over network 120.

[0016] (b) Exemplary Imaging Device Components 2 is an exemplary block diagram of modules and components of an imaging device, according to one embodiment. The imaging device 110 includes an image capture component 211, a flash component 212, a retinal disease diagnosis tool application protocol interface (API) 214, and a user interface 215. While not depicted, the imaging device 110 may include other components, such as built-in instances of either or both of the retinal image preprocessing tool 130 and the retinal disease diagnosis tool 140, as well as any components thereof. The imaging device 110 may also include any databases or memory for performing any functionality described herein. The imaging device 110 may also exclude some depicted components.

[0017] Image capture component 211 may be any sensor configured to capture an image of a patient's retina. For example, a specialized lens may be used to capture an image of a patient's retina. Flash component 212 may be any component capable of illuminating a patient's retina during image capture by image capture component 211 and may be configured to emit light in coordination with the image capture operation of image capture component 211.

[0018] The retinal disease diagnosis tool API 214 interfaces with the retinal disease diagnosis tool 130 to translate commands from the retinal image pre-processing tool 130 to the imaging device 110. Exemplary commands may include commands to capture or recapture an image, commands to adjust the intensity of light emitted by the flash component 212, and the like. These commands and how they are generated are discussed in further detail below with reference to FIG. 3.

[0019] User interface 215 is an interface through which an operator of imaging device 110 may instruct imaging device 110 to perform any function, such as capturing an image, adjusting flash intensity, and the like. User interface 215 may be any hardware or software interface and may include physical components (e.g., buttons) and / or graphical components (e.g., on a display such as a touchscreen display). User interface 215 may be located on imaging device 110, may be a peripheral device of imaging device 110, or may be located on a device separated from imaging device 110 by network 120, thereby enabling remote operation of imaging device 110. An exemplary user interface is shown and discussed in further detail with respect to FIG. 5 .

[0020] (c) Exemplary retinal image preprocessing tool components 3 is an exemplary block diagram of modules and components of a retinal image preprocessing tool, according to one embodiment. The retinal image preprocessing tool 130 includes an initial image capture module 331, a preprocessing module 232, a preliminary image evaluation module 233, an image recapture module 234, and an image stitching module 235. Although not depicted, the retinal image preprocessing tool 110 may include other components, such as additional modules, and any databases or memories for performing any of the functions described herein. Additionally, the retinal image preprocessing tool 130 may exclude some of the depicted components.

[0021] The initial image capture module 331 captures an image of one or both retinas of the patient's eye. The term "capture," as used herein, may refer to causing an image to be taken, i.e., "capturing" may include instructing the imaging device 110 from a remote device or server to take an image and transmit the image over the network 120 to the initial image capture module 331. Alternatively, if the initial image capture module 331 resides on the imaging device 110, the initial image capture module 331 may capture the image by instructing the imaging device to take an image and route it internally to the pre-processing module 332.

[0022] The initial images captured by the initial image capture module 331 may include images of different portions of the patient's retina. The images of the different portions, when used together as inputs to one or more machine learning models of the retinal disease diagnostic tool 140, result in an output of a diagnosis of retinal disease. The retinal regions may be predefined. For example, an administrator of the retinal disease diagnostic tool 140 may indicate that certain retinal regions should be captured for diagnosis. The regions may be defined as images centered on the fovea and as images centered on the optic disc. Thus, the initial image capture module 331 may capture images of one or both of the patient's eyes centered on those defined retinal regions.

[0023] In some embodiments, the captured images may be multi-frame footage spanning a period of time. For example, similar to "Live Photos," each frame may be captured from the moment the flash first fires until the moment the flash completely dissipates, and the frames, when displayed consecutively, form a video. Capturing multiple frames improves the likelihood that one of the frames will contain a portion of the retina depicted by the image that satisfies preprocessing criteria (described below) (even if that portion in another frame does not meet the criteria). For example, continuing from the flash example, if underexposure, overexposure, or shadowing caused by one flash intensity is corrected when the flash is adjusted to another intensity (e.g., the flash dims after 0.05 seconds as it is turned off), the portion of the retina depicted in the frame at the other intensity may be used to satisfy the criteria.

[0024] The preprocessing module 332 is used to determine whether a captured image is sufficient for input into a machine learning model. Sufficiency may be defined by an administrator and may be based on parameters such as underexposure (e.g., the image was captured using a very low flash intensity), overexposure (e.g., the image was captured using a very high flash intensity), blurring (e.g., the patient moved when the image was captured, obscuring the image), shadows, and / or any other defined parameters. Criteria may be established based on these parameters to determine whether an image is sufficient, such as "the image must be within a certain distance of exposure" and / or "landmarks (e.g., the optic disc boundary) must be sufficiently narrow" (where a wide boundary indicates blurring). The preprocessing module 332 compares the image parameters to the criteria to determine whether each image is sufficient for input into a machine learning model.

[0025] In one embodiment, if the preprocessing module 332 determines that an image is insufficient for input into the machine learning model, the retinal image preprocessing tool 130 may instruct the insufficient image to be recaptured. However, this embodiment poses a disadvantage in that an extra flush of the patient's eye is required, which may affect the patient's health. There is also a technical disadvantage because extra bandwidth and processing power is required to capture and retransmit the image to the retinal image preprocessing tool 130. In one embodiment, a preliminary image evaluation module 333 is used to determine whether a diagnosis can be made without recapturing the image, despite the insufficiency of a given image. The preliminary image assessment model 333 identifies one or more depicted portions of the retina in the image that do not meet one or more criteria. For example, an image may be partially overexposed or underexposed because retinal pigmentation in a patient's retina is inconsistent, leading to a consistent level of flash causing proper exposure in some portions of the patient's retina while causing improper exposure in other portions of the patient's retina. As another example, an image may have shading that may obscure one or more biomarkers depicted in portions of the retina while leaving the remainder of the image with sufficient quality. Thus, portion(s) that meet and / or do not meet one or more criteria may be isolated by the preliminary image assessment model 333. The preliminary image assessment model 333 may identify portions of the image that do and do not meet the criteria by examining these portions on any basis (e.g., pixel-by-pixel, quadrant-by-quadrant, area-by-area, etc.).

[0026] If the portion of the retina depicted by an image is determined to be insufficient, the preliminary image assessment model 333 examines one or more other images captured by the initial image capture module 331 to determine whether the portion is recoverable without recapturing the image and eliminates the insufficiency. For example, if a portion of the retina is insufficiently depicted in an image centered on the fovea of ​​the patient's retina, the preliminary image assessment model 333 may determine whether the same portion of the retina is depicted in an image centered on the optic disc of the patient's retina that was also captured during the initial image capture. Another example may include evaluating whether a frame of multi-frame video of which the insufficient image is a part can be used. Because the preliminary image assessment model 333 evaluates whether each captured image is sufficient for diagnosis, if the same portion of the retina is depicted in another image, the preliminary image assessment model 333 can immediately determine whether the same portion is of sufficient quality. If the same portion is of sufficient quality, the preliminary image assessment module 333 determines that recapturing the image is not necessary when a preliminary image is available. If the preliminary image evaluation module 333 determines that the same image of the retina is not depicted with sufficient quality in another image, the preliminary image evaluation module 333 determines that the image should be recaptured. In one embodiment, two or more preliminary images may be used together as preliminary to a single image, where different deficient portions of the image are repaired by portions from two or more preliminary images.

[0027] The image recapture module 334 recaptures an image if the image is insufficient, and in some embodiments, if there is no suitable preliminary image to remove the insufficiency. The image recapture module 334 functions in the same manner as the image capture module 331, except that it is used to capture images of already captured regions of the retina.

[0028] The image stitching module 335 stitches sufficient portions of an image with replacement content for insufficient portions of the image, the replacement content being from a preliminary image. The term "stitching," as used herein, refers to either the actual or logical assembly of portions from two images. Actual stitching refers to generating a composite image including portions from at least two images. Logical stitching refers to obtaining different portions from different images using those separate portions for diagnosis. For example, each portion is input separately into a machine learning model without generating a composite image, from which a diagnosis is output. As another example, a sufficient portion of an image may be used to generate a first analysis without taking into account the insufficient portion of the image. A preliminary image may have portions corresponding to the insufficient portion(s) of the first image that are used to generate a second analysis. Two analyses may be used to perform a diagnosis (e.g., by inputting the analysis into a machine learning model with or without the image itself). In one embodiment, the image stitching module 335 weights areas of poor image quality based on the distribution rate of biomarkers in those areas with respect to their distance from anatomical markers, such as the fovea and the optic nerve head, allowing for greater confidence in processing studies even when areas with poor image quality exist at the fovea of ​​the stitched images. Such weighting can be based on training a random forest, ANN, or the like on a representative dataset to learn and weight the importance of various areas of sufficient image quality at the fovea. While the image stitching module 335 is depicted as a module of the retinal image preprocessing tool 130, the image stitching module 335 may instead be, in whole or in part, a module of the retinal disease diagnosis tool 140.

[0029] (d) Exemplary Computer Architecture 4 is a block diagram illustrating components of an exemplary machine capable of reading instructions from a machine-readable medium and executing them within a processor (or controller). Specifically, FIG. 4 shows a diagrammatic representation of a machine in the exemplary form of a system 400, in which program code (e.g., software) may be implemented to cause the machine to perform any one or more of the methods discussed herein. The program code may consist of instructions 424 executable by one or more processors 402. In alternative embodiments, the machine may operate as a standalone device or may be connected (e.g., networked) to other machines. In a networked deployment, the machine may operate in the capacity of a server machine or a client machine in a server-client network environment, or as a peer machine in a peer-to-peer (or distributed) network environment.

[0030] The machine may be a server computer, a client computer, a personal computer (PC), a tablet PC, a set-top box (STB), a personal digital assistant (PDA), a mobile phone, a smartphone, a web appliance, a network router, a switch, or a bridge, or any machine capable of executing (sequentially or otherwise) instructions 424 that prescribe actions to be taken by the machine. Additionally, although only a single machine is illustrated, the term "machine" should also be taken to include any collection of machines that individually or together execute instructions 124 to perform any one or more of the methodologies discussed herein.

[0031] The exemplary computer system 400 includes a processor 402 (e.g., a central processing unit (CPU), a graphics processing unit (GPU), a digital signal processor (DSP), one or more application specific integrated circuits (ASICs), one or more radio frequency integrated circuits (RFICs), or any combination thereof), a main memory 404, and a static memory 406, which are configured to communicate with each other via a bus 408. The computer system 400 may further include a visual display interface 410. The visual interface may include software drivers that enable a user interface to be displayed on a screen (or display). The visual interface may display the user interface directly (e.g., on a screen) or indirectly (e.g., via a visual projection unit) on a surface, window, or the like. For ease of discussion, the visual interface may be described as a screen. The visual interface 410 may include or interface with a touch-enabled screen. The computer system 400 may also include an alphanumeric input device 412 (e.g., a keyboard or touchscreen keyboard), a cursor control device 414 (e.g., a mouse, trackball, joystick, motion sensor, or other pointing instrument), a storage unit 416, a signal generating device 418 (e.g., a speaker), and a network interface device 420, which are also configured to communicate via the bus 408.

[0032] The storage unit 416 may include a machine-readable medium 422 on which are stored instructions 424 (e.g., software) that embody any one or more of the methods or functions described herein. The instructions 424 (e.g., software) may reside, completely or partially, within the main memory 404 or within the processor 402 (e.g., within a processor's cache memory) during execution by the computer system 400, and the main memory 404 and the processor 402 may constitute machine-readable media. The instructions 424 (e.g., software) may be transmitted or received over a network 426 via the network interface device 420.

[0033] While in an exemplary embodiment, machine-readable medium 422 is shown to be a single medium, the term "machine-readable medium" should be taken to include a single medium or multiple media (e.g., a centralized or distributed database, or associated caches and servers) capable of storing instructions (e.g., instructions 424). The term "machine-readable medium" should also be taken to include any medium capable of storing instructions (e.g., instructions 424) for execution by a machine, causing the machine to perform any one or more of the methodologies disclosed herein. The term "machine-readable medium" includes, but is not limited to, data repositories in the form of solid-state memory, optical media, and magnetic media.

[0034] (e) Example Images and User Interfaces FIG. 5 depicts exemplary images of a patient's eye according to one embodiment. User interface 500 may be displayed to the operator of imaging device 110 as part of user interface 215. User interface 500 includes images 510, 520, 530, and 540. As depicted, the patient has two captured images of each eye: one centered on the fovea and one centered on the optic disc. As depicted, these images are merely exemplary, and any parameters may be used to indicate where the images should be centered, as input by the operator of imaging device 110 and / or an administrator who determines what images are needed to perform a diagnosis. Furthermore, while the depicted example uses two images, any number of images may be used. Although the images are centered around the optic disc and fovea, portions of the images overlap; therefore, if any of the overlapping portions of one image do not meet the specified criteria, each image may be used as a backup for the other.

[0035] (f) Exemplary data flow for minimizing retinal exposure to flash during diagnostic image acquisition. 6 is an exemplary flowchart for minimizing retinal exposure to flash during diagnostic image acquisition, according to one embodiment. Process 600 begins with one or more processors (e.g., processor 402) of a device used to launch retinal image preprocessing tool 130 capturing (602) multiple retinal images (e.g., using initial image capture module 331), each corresponding to a different retinal region of multiple retinal regions, the multiple retinal images including a first image and a second image. The multiple retinal images may include, for example, a fovea-centered image and an optic disc-centered image for each eye. The first image may be fovea-centered image 510 of the patient's left eye, and the second image may be optic disc-centered image 530 of the patient's left eye, with the different retinal regions corresponding to the retinal regions within which each image is centered.

[0036] The retinal image preprocessing tool 130 determines that a first portion of the first image does not meet the criteria, while a second portion of the first image meets the criteria (604). For example, a portion of the fovea-centered left eye image 510 is determined to be insufficient due to retinal striations, overexposure, underexposure, shadowing, or other issues, but the remaining portion of the fovea-centered left eye image 510 is determined to be sufficient. The determination of sufficiency may be performed by the preprocessing module 332 using any manner described herein.

[0037] The retinal image preprocessing tool 130 identifies a portion of the retina depicted in the first portion that does not meet the criteria (606). That is, a portion of the retina itself may be depicted at different coordinates in the preliminary image, and therefore is the identified portion of the retina (not a portion of the image). The preliminary image evaluation module 333 may perform this determination in any manner described herein. The retinal image preprocessing tool 130 then determines whether the portion of the retina is depicted in the third portion of the second image (608). That is, the preliminary image evaluation module 333 determines whether another captured image includes a depiction of the same portion of the retina in some portion of the other captured image. For example, the retinal image preprocessing tool 130 determines whether the left eye image 530, which is centered on the optic disc, depicts the same portion of the retina that was insufficient in the left eye image 510, which is centered on the fovea.

[0038] The retinal image preprocessing tool 130 determines whether a third portion of the second image (e.g., a portion of the optic disc-centered left eye image 530 that captures the same portion of the retina that is insufficient in the fovea-centered left eye image 510) meets the criteria (610). In response to the third portion meeting the criteria, the retinal image preprocessing tool 130 performs a diagnosis using the multiple retinal images (e.g., using the image stitching module 335) (612). In response to the third portion not meeting the criteria, the retinal image preprocessing tool 130 captures additional images of the retinal region depicted in the first image (e.g., using the image recapture module 334) (614).

[0039] (g) Overview The foregoing description of embodiments of the invention has been presented for purposes of illustration and is not intended to be exhaustive or to limit the invention to the precise form disclosed. Those skilled in the relevant art will recognize that many modifications and variations are possible in light of the above disclosure.

[0040] Although this disclosure focuses specifically on retinal diseases, the disclosure applies generally to disease diagnosis in general, as well as to the capture (and possible recapture) and pre-processing of images of other parts of a patient's body. For example, the disclosed pre-processing may be applied to the diagnosis of other organs, such as the kidney, liver, brain, or heart, or portions thereof, and the images discussed herein may be of the relevant organs.

[0041] Some portions of this description will describe embodiments of the invention in terms of algorithms and symbolic representations of operations on information. These algorithmic descriptions and representations are commonly used by those skilled in the data processing arts to convey the substance of their work to others skilled in the art. While these operations are described functionally, computationally, or logically, it will be understood that they may be implemented by computer programs or equivalent electrical circuits, microcode, or the like. Further, it has proven convenient at times to refer to these arrangements of operations as modules, without loss of generality. The described operations and their associated modules may be embodied in software, firmware, hardware, or any combination thereof.

[0042] Any of the steps, operations, or processes described herein may be performed or implemented using one or more hardware or software modules, alone or in combination with other devices. In one embodiment, a software module is implemented using a computer program product that includes a computer-readable medium containing computer program code, which can be executed by a computer processor to perform any or all of the steps, operations, or processes described.

[0043] Embodiments of the present invention may relate to apparatus for performing the operations herein. The apparatus may be specially constructed for the required purposes and / or may comprise a general-purpose computing device selectively activated or reconfigured by a computer program stored in the computer. Such computer programs may be stored in a non-transitory tangible computer-readable storage medium, or any type of medium suitable for storing electronic instructions, which may be coupled to a computer system bus. Furthermore, any computing system referred to herein may include a single processor or may be an architecture employing a multiple processor design to increase computing power.

[0044] Embodiments of the present invention may relate to products produced by the computational processes described herein. Such products may comprise information resulting from the computational processes, and when the information is stored on a non-transitory tangible computer-readable storage medium, may include any embodiment of a computer program product or other data combination described herein.

[0045] Finally, the terminology used herein has been selected primarily for readability and guidance purposes, and not to precisely describe or delineate the subject matter of the invention. Accordingly, the scope of the invention is intended to be limited not by this detailed description, but by any claims that may be issued upon filing based on this specification. Thus, the disclosure of embodiments of the invention is intended to be illustrative but not limiting of the scope of the invention, which is set forth in the following claims.

Claims

[Claim 1] Devices, systems, methods, etc.

Images