Image retention and stitching for minimal-flash eye disease diagnosis

Abstract

To provide image retention and stitching for minimal-flash eye disease diagnosis.SOLUTION: There are provided systems and methods for minimizing retinal exposure to flash during image gathering for diagnosis. In an embodiment, a system captures a plurality of retinal images of different retinal regions. The system determines that a first portion of a first image does not meet a criterion while a second portion of the first image does meet the criterion, identifies a portion of the retina depicted in the first portion that does not meet the criterion, and determines whether the portion of the retina is depicted in a third portion of a second image and whether the third portion meets the criterion. Responsive to determining that the third portion meets the criterion, the system performs the diagnosis. Responsive to determining that the portion of the retina is not depicted in the second image, the system captures an additional image of the retinal region.SELECTED DRAWING: Figure 6

Description

[Technical field]

[0001] (background) The present invention relates generally to autonomous diagnosis of retinal abnormalities, and more particularly to image retention to reduce the need to fire a flash at a patient's retina. [Background technology]

[0002] An autonomous system for diagnosing retinal abnormalities captures images of a patient's retina (used interchangeably herein with the word "fundus") and analyzes those images for abnormalities. Typically, images of different parts of the retina are captured (e.g., an image centered on the fovea and an image centered on the optic disc). When the captured images are insufficient to perform a sufficient diagnosis, additional images are captured until sufficient images of each part of the retina are obtained. Repeated exposure to flashes by the imaging device may further cause pupil constraints that make successful imaging less likely after each flash. Thus, reducing the need for additional images to be captured until sufficient images are captured reduces the number of flashes and reduces the need to take countless pictures or have the patient wait to be diagnosed until the pupils have recovered to normal dilation. Additionally, reducing the need for further images may allow for more rapid diagnosis in that a number of flashes that would significantly increase pupil constraint is unlikely to be achieved in a seated position, thus preventing the need to delay diagnosis of the patient until such time as firing a flash at the patient's eye is likely to again result in a viable image. Summary of the Invention [Means for solving the problem]

[0003] (overview) A system and method are provided herein for reducing the need to re-flash a patient's eye to capture an image of a retinal region when a previously captured image is insufficient. As an example, two images may be captured to perform a diagnosis of a patient's right eye, one with the fovea at the center and one with the optic disc at the center. There is some overlap between the portions of the retina depicted in these two images. If the area of ​​the retina depicted in one of the images is unusable for some reason (e.g., overexposure or underexposure), rather than recapturing the image, the system may determine whether the same area is depicted in the other image without the same deficiencies. If the area is usable from the other image, the system may avoid the need to re-flash a patient's right eye to capture another image of the area, and instead stitch portions of the two images to perform a diagnosis.

[0004] During diagnostic image collection, in order to minimize retinal exposure to flash, in an embodiment, a retinal image preprocessing tool captures multiple retinal images (e.g., by instructing an imaging device to take images and send them to the retinal image preprocessing tool). Each retinal image may correspond to a different retinal region of the multiple retinal regions (e.g., a region with the fovea at the center and a region with the optic disc at the center). The retinal image preprocessing tool may determine that a first portion of the first image (e.g., the portion with the fovea at the center) does not meet a criterion (e.g., the image is overexposed or underexposed, the image has a shadow or other artifact, etc.), while a second portion of the first image (e.g., the portion with the optic disc at the center) meets a criterion (e.g., the second image is properly exposed).

[0005] The retinal image preprocessing tool may identify a portion of the retina depicted in the first portion that does not meet the criteria and may determine that the same portion of the retina is depicted in a third portion of the second image. The retinal image preprocessing tool may determine whether the third portion meets the criteria. In response to determining that the third portion meets the criteria, a diagnosis may be performed using the multiple retinal images. In response to determining that the portion of the retina is not depicted in the second image, the retinal image preprocessing tool may capture additional images of the retinal area depicted in the first image. The present invention provides, for example, the following items. (Item 1) 1. A method for minimizing retinal exposure to flash during diagnostic image acquisition, comprising: capturing a plurality of retinal images, each retinal image corresponding to a different retinal region of a plurality of retinal regions, the plurality of retinal images including a first image and a second image; determining that a first portion of the first image does not meet a criterion while a second portion of the first image meets the criterion; identifying a portion of the retina depicted within the first portion that does not meet the criterion; and determining whether the portion of the retina is depicted within a third portion of the second image; and determining whether the third portion meets the criteria; and performing the diagnosis using the plurality of retinal images in response to determining that the third portion meets the criterion; and capturing an additional image of the retinal area depicted in the first image in response to determining that the portion of the retina is not depicted in the second image; and The method includes: (Item 2) 2. The method of claim 1, wherein performing the diagnosis includes passing the images through a fully autonomous machine learning model that outputs a likelihood of disease based on the multiple retinal images. (Item 3) 2. The method of claim 1, wherein each retinal image of the plurality of retinal images comprises a multi-frame image, each frame of the multi-frame image capturing an image at a different level of flash exposure. (Item 4) 2. The method of claim 1, wherein determining that the first portion of the first image does not meet the criterion comprises determining that the first portion of the first image is either overexposed or underexposed. (Item 5) 2. The method of claim 1, wherein the second image is an image of the retina of the same eye depicted by the first image. (Item 6) performing the diagnosis using the plurality of retinal images, generating a composite image including the first portion of the image with the third portion of the second image stitched into the first image; performing said diagnosis using said composite image; and The method according to item 1, comprising: (Item 7) performing the diagnosis using the plurality of retinal images, analyzing the first image without taking into account the first portion to generate a first analysis; analyzing the third portion of the second image to generate a second analysis; and performing said diagnosis using said first analysis and said second analysis; and The method according to item 1, comprising: (Item 8) Capturing the additional image includes: transmitting, using an application protocol interface (API), instructions to an imaging device to capture an additional image using the viewpoint used to capture the first image; receiving the additional image from the imaging device; The method according to item 1, comprising: (Item 9) 2. The method of claim 1, wherein performing the diagnosis using the plurality of retinal images includes using the third portion in the diagnosis. (Item 10) 1. A computer program product for minimizing retinal exposure to flash during diagnostic image acquisition, comprising: capturing a plurality of retinal images, each retinal image corresponding to a different retinal region of a plurality of retinal regions, the plurality of retinal images including a first image and a second image; determining that a first portion of the first image does not meet a criterion while a second portion of the first image meets the criterion; identifying a portion of the retina depicted within the first portion that does not meet the criterion; and determining whether the portion of the retina is depicted within a third portion of the second image; and determining whether the third portion meets the criteria; and performing the diagnosis using the plurality of retinal images in response to determining that the third portion meets the criterion; and capturing an additional image of the retinal area depicted in the first image in response to determining that the portion of the retina is not depicted in the second image; and A computer program product comprising a non-transitory computer readable storage medium containing computer program code for performing the steps of: (Item 11) 11. The computer program product of claim 10, wherein the computer program code for performing the diagnosis comprises computer program code for passing the images through a fully autonomous machine learning model that outputs a likelihood of disease based on the plurality of retinal images. (Item 12) Item 11. The computer program product of item 10, wherein each retinal image of the plurality of retinal images comprises a multi-frame image, each frame of the multi-frame image capturing an image at a different level of flash exposure. (Item 13) Item 11. The computer program product of item 10, wherein the computer program code for determining that the first portion of the first image does not satisfy the criterion comprises computer program code for determining that the first portion of the first image is either overexposed or underexposed. (Item 14) Item 11. The computer program product of item 10, wherein the second image is an image of a retina of the same eye depicted by the first image. (Item 15) The computer program code for performing the diagnosis using the plurality of retinal images comprises: generating a composite image including the first portion of the image with the third portion of the second image stitched into the first image; performing said diagnosis using said composite image; and Item 11. A computer program product according to item 10, comprising computer program code for carrying out the steps of: (Item 16) The computer program code for performing the diagnosis using the plurality of retinal images comprises: analyzing the first image without taking into account the first portion to generate a first analysis; analyzing the third portion of the second image to generate a second analysis; and performing said diagnosis using said first analysis and said second analysis; and Item 11. A computer program product according to item 10, comprising computer program code for carrying out the steps of: (Item 17) The computer program code for capturing the additional images further comprises: transmitting, using an application protocol interface (API), instructions to an imaging device to capture an additional image using the viewpoint used to capture the first image; receiving the additional image from the imaging device; Item 11. A computer program product according to item 10, comprising computer program code for carrying out the steps of: (Item 18) 11. The computer program product of claim 10, wherein performing the diagnosis using the plurality of retinal images includes using the third portion in the diagnosis. (Item 19) 1. A computer program product for minimizing retinal exposure to flash during diagnostic image acquisition, comprising: a first module for capturing a plurality of retinal images, each retinal image corresponding to a different retinal region of a plurality of retinal regions, the plurality of retinal images including a first image and a second image; A second module, determining that a first portion of the first image does not meet a criterion while a second portion of the first image meets the criterion; identifying a portion of the retina depicted within the first portion that does not meet the criterion; and determining whether the portion of the retina is depicted within a third portion of the second image; and determining whether said third portion meets said criteria; and a second module for performing a third module for performing the diagnosis using the plurality of retinal images in response to determining that the third portion meets the criterion; and a fourth module for capturing an additional image of the retinal area depicted in the first image in response to determining that the portion of the retina is not depicted in the second image; and A computer program product comprising a computer readable storage medium containing computer program code comprising: (Item 20) 20. The computer program product of claim 19, wherein performing the diagnosis includes passing the images through a fully autonomous machine learning model that outputs a likelihood of disease based on the plurality of retinal images. [Brief description of the drawings]

[0006] BRIEF DESCRIPTION OF THE DRAWINGS [Figure 1] FIG. 1 is an exemplary block diagram of system components in an environment for utilizing a retinal image pre-processing tool, according to one embodiment.

[0007] [Diagram 2] FIG. 2 is an exemplary block diagram of modules and components of an imaging device, according to one embodiment.

[0008] [Diagram 3] FIG. 3 is an exemplary block diagram of modules and components of a retinal image pre-processing tool, according to one embodiment.

[0009] [Figure 4] FIG. 4 is a block diagram illustrating components of an exemplary machine capable of reading instructions from a machine-readable medium and executing them within a processor (or controller).

[0010] [Diagram 5] FIG. 5 depicts an exemplary image of a patient's eye according to one embodiment.

[0011] [Figure 6] FIG. 6 is an exemplary flow chart for minimizing retinal exposure to flash during diagnostic image acquisition, according to one embodiment.

[0012] The figures depict various embodiments of the invention for illustrative purposes only. Those skilled in the art will readily recognize from discussion below that alternative embodiments of the structures and methods illustrated herein may be employed without departing from the inventive principles described herein. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0013] (Detailed Description) (a) Environmental Overview FIG. 1 is an exemplary block diagram of system components in an environment for utilizing a retinal image preprocessing tool, according to one embodiment. The environment 100 includes an imaging device 110, a network 120, a retinal image preprocessing tool 130, and a retinal disease diagnosis tool 140. The imaging device 110 is a device configured to capture one or more images of a retina of a patient's eye. The imaging device 110 may be caused to capture such images autonomously, through manual operation, when commanded by computer program instructions or an external signal (e.g., received from a retinal pigmentation determination tool 130), or by a combination thereof. Examples of what these images may look like and how they are derived are described in self-filed U.S. patent application Ser. No. 15 / 466,636, filed Mar. 22, 2017, the disclosure of which is hereby incorporated by reference in its entirety. Other examples are described with respect to FIG. 5 of the present disclosure.

[0014] After capturing the image, the imaging device 110 transmits the image to the retinal disease diagnostic tool 140. The retinal disease diagnostic tool 140 may take one or more images as input and may use a machine learning model to output a diagnosis based on the input, fully autonomously. The retinal disease diagnostic tool 140 autonomously analyzes the retinal image and determines a diagnosis using machine learning analysis of biomarkers therein. The diagnosis may be specifically a determination that the user has a particular disease, such as diabetic retinopathy, or may be a determination that the user is likely to have a disease and therefore should see a physician for confirmation and treatment. The manner in which the retinal disease diagnostic tool 140 performs the analysis and determines a diagnosis is further discussed in commonly owned U.S. Patent No. 10,115,194, issued October 30, 2019, the disclosure of which is hereby incorporated by reference in its entirety.

[0015] Prior to performing a diagnosis, the retinal disease diagnosis tool 140 may have a retinal image preprocessing tool 130 that determines whether the image(s) captured by the imaging device 110 are sufficient to perform a diagnosis. The manner in which the retinal image preprocessing tool 130 performs this analysis is described in more detail below with respect to FIG. 3. While depicted as a component of the retinal disease diagnosis tool 140, the retinal image preprocessing tool 130 may be a standalone entity that receives an image, vets the image for sufficiency, and then transmits the image to the retinal disease diagnosis tool 140 if the image is sufficient for diagnosis. The retinal image preprocessing tool 130 and / or the retinal disease diagnosis tool 140 may be instantiated on one or more servers. Additionally, the retinal image preprocessing tool 130 and / or the retinal disease diagnosis tool 140 may be instantiated in whole or in part on the imaging device 110, thus eliminating the need for some or all of the communication traffic via the network 120.

[0016] (b) Exemplary Imaging Device Components 2 is an exemplary block diagram of modules and components of an imaging device, according to one embodiment. The imaging device 110 includes an image capture component 211, a flash component 212, a retinal disease diagnosis tool application protocol interface (API) 214, and a user interface 215. Although not depicted, the imaging device 110 may include other components, such as built-in instances of either or both of the retinal image pre-processing tool 130 and the retinal disease diagnosis tool 140, as well as any components thereof. The imaging device 110 may include any database or memory for performing any functionality described herein. The imaging device 110 may also exclude some depicted components.

[0017] The image capture component 211 may be any sensor configured to capture an image of the patient's retina. For example, a special lens may be used to capture an image of the patient's retina. The flash component 212 may be any component capable of illuminating the patient's retina during image capture by the image capture component 211 and may be configured to emit light in coordination with the image capture operation of the image capture component 211.

[0018] The retinal disease diagnosis tool API 214 interfaces with the retinal disease diagnosis tool 130 to translate instructions from the retinal image pre-processing tool 130 to the imaging device 110. Exemplary instructions may include instructions for capturing or recapturing an image, instructions for adjusting the intensity of light emitted by the flash component 212, and the like. These instructions and how they are generated are discussed in further detail below with reference to FIG.

[0019] User interface 215 is an interface by which an operator of imaging device 110 may instruct imaging device 110 to perform any function capable of performing such as capturing an image, adjusting flash intensity, and the like. User interface 215 may be any hardware or software interface and may include physical components (e.g., buttons) and / or graphical components (e.g., on a display such as a touch screen display). User interface 215 may be located on imaging device 110, may be a peripheral device of imaging device 110, or may be located on a device separated from imaging device 110 by network 120, thereby enabling remote operation of imaging device 110. An exemplary user interface is shown and discussed in further detail with respect to FIG. 5.

[0020] (c) Exemplary Retinal Image Pre-Processing Tool Components 3 is an exemplary block diagram of modules and components of a retinal image preprocessing tool, according to one embodiment. The retinal image preprocessing tool 130 includes an initial image capture module 331, a preprocessing module 232, a preliminary image evaluation module 233, an image recapture module 234, and an image stitching module 235. Although not depicted, the retinal image preprocessing tool 110 may include other components, such as additional modules, and any databases or memories for performing any functionality described herein. Additionally, the retinal image preprocessing tool 130 may exclude some depicted components.

[0021] The initial image capture module 331 captures a retinal image of one or both of the patient's eyes. The term "capture," as used herein, may refer to causing an image to be taken, i.e., "capturing" may include instructing the imaging device 110 from a remote device or server to take an image and transmit the image over the network 120 to the initial image capture module 331. Alternatively, if the initial image capture module 331 is present on the imaging device 110, the initial image capture module 331 may capture the image by instructing the imaging device to take an image and route it internally to the pre-processing module 332.

[0022] The initial images captured by the initial image capture module 331 may include images of different portions of the patient's retina. The images of the different portions, when used together as inputs to one or more machine learning models of the retinal disease diagnosis tool 140, result in an output of a diagnosis of a retinal disease. The retinal regions may be predefined. For example, an administrator of the retinal disease diagnosis tool 140 may indicate that certain retinal regions should be captured for diagnosis. The regions may be defined as images with the fovea at the center and as images with the optic disc at the center. Thus, the initial image capture module 331 may capture images for one or both of the patient's eyes with those defined retinal regions at the center.

[0023] In some embodiments, the captured image may be a multi-frame image over a period of time. For example, similar to a "live photo," each frame may be captured from the moment the flash is first emitted until the moment the flash completely dissipates, and each frame, when displayed consecutively, forms an image. Capturing multiple frames improves the likelihood that one of the frames will contain a portion of the retina depicted by the image that satisfies a pre-processing criterion (described below) (even if that portion of another frame does not meet that criterion). For example, continuing from the flash example, if underexposure, overexposure, or shadowing caused by one flash intensity is corrected when the flash is adjusted to another intensity (e.g., the flash dims after 0.05 seconds as it is shut off), the portion of the retina depicted in the frame at the other intensity may be used to satisfy the criterion.

[0024] The pre-processing module 332 is used to determine whether a captured image is sufficient for input into a machine learning model. Sufficiency may be defined by an administrator and may be based on parameters such as underexposure (e.g., the image was captured using a very low flash intensity), overexposure (e.g., the image was captured using a very high flash intensity), blurring (e.g., the patient moved when the image was captured, obscuring the image), shadows, and / or any other defined parameters. Criteria may be established based on these parameters to determine whether an image is sufficient, such as "the image must be within a certain distance of exposure" and / or "landmarks (e.g., the border of the optic disc) must be sufficiently narrow" (where a wider border indicates blurring), etc. The pre-processing module 332 compares the parameters of the image to the criteria and determines whether each image is sufficient for input into a machine learning model.

[0025] In an embodiment, if the pre-processing module 332 determines that an image is insufficient for input into the machine learning model, the retinal image pre-processing tool 130 may command the insufficient image to be recaptured. However, this embodiment causes a disadvantage that the patient's health may be affected in that an extra flush of the patient's eye is required. There is also a technical disadvantage, since extra bandwidth and processing power is required to capture and retransmit the image to the retinal image pre-processing tool 130. In an embodiment, the preliminary image evaluation module 333 is used to determine whether a diagnosis can be performed without recapturing the image, despite the insufficiency of a given image. The preliminary image assessment model 333 identifies one or more depicted portions of the retina in the image that do not meet one or more criteria. For example, the image may be partially overexposed or underexposed because the retinal pigmentation in the patient's retina is inconsistent, leading to a consistent level of flash causing proper exposure in some portions of the patient's retina while causing inappropriate exposure in other portions of the patient's retina. As another example, the image may have shadows that may obscure one or more biomarkers depicted in the portion of the retina while leaving the remaining image with sufficient quality. Thus, the portion(s) that meet and / or do not meet the one or more criteria may be isolated by the preliminary image assessment model 333. The preliminary image assessment model 333 may identify these portions by examining the portions of the image that do and do not meet the criteria on any basis (e.g., pixel by pixel, quadrant by quadrant, area by area, etc.).

[0026] If a portion of the retina depicted by an image is determined to be insufficient, the preliminary image assessment model 333 examines one or more other images captured by the initial image capture module 331 to determine whether the portion is recoverable without recapturing the image and removes the insufficiency. For example, if a portion of the retina depicted in an image centered on the fovea of ​​the patient's retina is insufficient, the preliminary image assessment model 333 may determine whether the same portion of the retina is depicted in an image centered on the optic disc of the patient's retina that was also captured during the initial image capture. Another example may include evaluating whether a frame of a multi-frame video of which the insufficient image is a part can be used. Because the preliminary image assessment model 333 evaluates whether each captured image is sufficient for diagnosis, if the same portion of the retina is depicted in another image, the preliminary image assessment model 333 can immediately determine whether the same portion is of sufficient quality. If the same portion is of sufficient quality, the preliminary image assessment module 333 determines that recapture of the image is not necessary when a preliminary image is available. If the preliminary image assessment module 333 determines that the same image of the retina is not depicted with sufficient quality in another image, the preliminary image assessment module 333 determines that the image should be recaptured. In an embodiment, two or more preliminary images may be used together as a preliminary to a single image, where different deficient portions of an image are repaired by portions from two or more preliminary images.

[0027] The image recapture module 334 recaptures an image when an image is insufficient, and in some embodiments when there is no adequate backup image to remove the insufficiency. The image recapture module 334 functions in the same manner as the image capture module 331, except that it is used to capture images of already captured regions of the retina.

[0028] The image stitching module 335 stitches a sufficient portion of the image with replacement content for the insufficient portion of the image, the replacement content being from the preliminary image. The term "stitching" as used herein refers to either an actual or logical assembly of portions from two images. Actual stitching refers to generating a composite image that includes portions from at least two images. Logical stitching refers to obtaining different portions from different images using their separate portions for diagnosis. For example, each portion is input separately into a machine learning model without generating a composite image from which a diagnosis is output. As another example, the sufficient portion of the image may be used to generate a first analysis without taking into account the insufficient portion of the image. The preliminary image may have portions corresponding to the insufficient portion(s) of the first image that are used to generate a second analysis. The two analyses may be used to perform a diagnosis (e.g., by inputting the analysis into a machine learning model with or without the image itself). In an embodiment, the image stitching module 335 weights areas of poor image quality based on the distribution rate of biomarkers in those areas with respect to distance from anatomical markers such as the fovea and the optic disc, allowing for a higher degree of confidence in processing those studies even if there are areas with poor image quality at the fovea of ​​the stitched images. Such weighting can be based on training random forests, ANNs, and the like on a representative data set to learn and weight the importance of various areas of sufficient image quality at the fovea. While the image stitching module 335 is depicted as a module of the retinal image pre-processing tool 130, the image stitching module 335 may instead be a module of the retinal disease diagnosis tool 140, in whole or in part.

[0029] (d) Exemplary Computer Architecture FIG. 4 is a block diagram illustrating components of an exemplary machine capable of reading instructions from a machine-readable medium and executing them in a processor (or controller). In particular, FIG. 4 shows a diagrammatic representation of a machine in the exemplary form of a system 400 in which program code (e.g., software) for causing the machine to perform any one or more of the methods discussed herein may be implemented. The program code may consist of instructions 424 executable by one or more processors 402. In alternative embodiments, the machine may operate as a standalone device or may be connected (e.g., networked) to other machines. In a networked deployment, the machine may operate in the capacity of a server machine or a client machine in a server-client network environment, or may operate as a peer machine in a peer-to-peer (or distributed) network environment.

[0030] The machine may be a server computer, a client computer, a personal computer (PC), a tablet PC, a set-top box (STB), a personal digital assistant (PDA), a mobile phone, a smart phone, a web appliance, a network router, a switch or bridge, or any machine capable of executing (sequentially or otherwise) instructions 424 that specify actions to be taken by the machine. Additionally, although only a single machine is illustrated, the term "machine" should also be taken to include any collection of machines that individually or together execute instructions 124 to perform any one or more of the methodologies discussed herein.

[0031] The exemplary computer system 400 includes a processor 402 (e.g., a central processing unit (CPU), a graphics processing unit (GPU), a digital signal processor (DSP), one or more application specific integrated circuits (ASICs), one or more radio frequency integrated circuits (RFICs), or any combination thereof), a main memory 404, and a static memory 406, which are configured to communicate with each other via a bus 408. The computer system 400 may further include a visual display interface 410. The visual interface may include software drivers that allow for displaying a user interface on a screen (or display). The visual interface may display the user interface directly (e.g., on a screen) or indirectly (e.g., via a visual projection unit) on a surface, window, or the like. For ease of discussion, the visual interface may be described as a screen. The visual interface 410 may include or interface with an interface with a touch-enabled screen. Computer system 400 may also include an alphanumeric input device 412 (e.g., a keyboard or touch screen keyboard), a cursor control device 414 (e.g., a mouse, trackball, joystick, motion sensor, or other pointing instrument), a memory unit 416, a signal generating device 418 (e.g., a speaker), and a network interface device 420, which are also configured to communicate over bus 408.

[0032] The storage unit 416 may include a machine-readable medium 422 on which are stored instructions 424 (e.g., software) embodying any one or more of the methods or functions described herein. The instructions 424 (e.g., software) may reside, completely or partially, within the main memory 404 or within the processor 402 (e.g., within a processor's cache memory) during its execution by the computer system 400, and the main memory 404 and the processor 402 may constitute machine-readable media. The instructions 424 (e.g., software) may be transmitted or received over a network 426 via the network interface device 420.

[0033] While in an exemplary embodiment, the machine-readable medium 422 is shown to be a single medium, the term "machine-readable medium" should be taken to include a single medium or multiple media (e.g., a centralized or distributed database, or associated caches and servers) capable of storing instructions (e.g., instructions 424). The term "machine-readable medium" should also be taken to include any medium capable of storing instructions (e.g., instructions 424) for execution by a machine and causing a machine to perform any one or more of the methods disclosed herein. The term "machine-readable medium" includes, but is not limited to, data repositories in the form of solid-state memory, optical media, and magnetic media.

[0034] (e) Example Images and User Interfaces FIG. 5 depicts exemplary images of a patient's eye according to one embodiment. User interface 500 may be displayed to an operator of imaging device 110 as part of user interface 215. User interface 500 includes images 510, 520, 530, and 540. As depicted, the patient has two images captured of each eye, one centered on the fovea and one centered on the optic disc. As depicted, these images are merely exemplary and any parameter may be used to indicate where the images should be centered as input by the operator of imaging device 110 and / or an administrator who determines what images are needed to perform a diagnosis. Furthermore, while in the depicted example, two images are used, any number of images may be used. Although the images are centered around the optic disc and the fovea, portions of the images overlap, and therefore each image may be used as a backup for the other image if any of the overlapping portions of one image do not meet the defined criteria.

[0035] (f) Exemplary data flow for minimizing retinal exposure to flash during diagnostic image acquisition. 6 is an exemplary flow chart for minimizing retinal exposure to flash during diagnostic image acquisition, according to one embodiment. The process 600 begins with one or more processors (e.g., processor 402) of a device used to launch the retinal image pre-processing tool 130 capturing (602) a plurality of retinal images (e.g., using the initial image capture module 331), each retinal image corresponding to a different retinal region of a plurality of retinal regions, the plurality of retinal images including a first image and a second image. The plurality of retinal images may include, for example, a fovea-centered image and an optic disc-centered image for each eye. The first image may be a fovea-centered image 510 of the patient's left eye, and the second image may be an optic disc-centered image 530 of the patient's left eye, and the different retinal regions may correspond to the retinal regions within which each image is centered.

[0036] The retinal image preprocessing tool 130 determines that a first portion of the first image does not meet the criteria while a second portion of the first image does meet the criteria (604). For example, a portion of the fovea-centered left eye image 510 is determined to be insufficient due to retinal striations, overexposure, underexposure, shadowing, or other issues, but a remaining portion of the fovea-centered left eye image 510 is determined to be sufficient. The determination of sufficiency may be performed by the preprocessing module 332 using any manner described herein.

[0037] The retinal image preprocessing tool 130 identifies (606) a portion of the retina depicted in the first portion that does not meet the criteria. That is, a portion of the retina itself may be depicted in different coordinates of the preliminary image, and thus it is a portion of the retina that is identified (not a portion of the image). The preliminary image evaluation module 333 may perform this determination in any manner described herein. The retinal image preprocessing tool 130 then determines (608) whether a portion of the retina is depicted in a third portion of the second image. That is, the preliminary image evaluation module 333 determines whether another captured image includes a depiction of the same portion of the retina in some portion of the other captured image. For example, the retinal image preprocessing tool 130 determines whether the left eye image 530 with the optic disc at the center depicts the same portion of the retina that was insufficient in the left eye image 510 with the fovea at the center.

[0038] The retinal image preprocessing tool 130 determines whether a third portion of the second image (e.g., a portion of the optic disc-centered left eye image 530 that captures the same portion of the retina that is deficient in the fovea-centered left eye image 510) meets the criteria (610). In response to the third portion meeting the criteria, the retinal image preprocessing tool 130 performs a diagnosis (612) using the multiple retinal images (e.g., using the image stitching module 335). In response to the third portion not meeting the criteria, the retinal image preprocessing tool 130 captures additional images of the retinal area depicted in the first image (e.g., using the image recapture module 334) (614).

[0039] (g) Overview The foregoing description of embodiments of the invention has been presented for purposes of illustration and is not intended to be exhaustive or to limit the invention to the precise form disclosed. Those skilled in the relevant art will recognize that many modifications and variations are possible in light of the above disclosure.

[0040] Although the present disclosure focuses specifically on retinal diseases, the present disclosure applies generally to disease diagnosis in general, as well as to the capture (and possible recapture) and pre-processing of images of other parts of a patient's body. For example, the disclosed pre-processing may be applied to the diagnosis of other organs, such as the kidney, liver, brain, or heart, or portions thereof, and the images discussed herein may be of the relevant organs.

[0041] Some portions of this description describe embodiments of the invention in terms of algorithms and symbolic representations of operations on information. These algorithmic descriptions and representations are commonly used by those skilled in the data processing arts to convey the substance of their work to others skilled in the art. While these operations are described functionally, computationally, or logically, it will be understood that they are implemented by computer programs or equivalent electrical circuits, microcode, or the like. Moreover, it has proven convenient at times to refer to these arrangements of operations as modules, without loss of generality. The described operations and their associated modules may be embodied in software, firmware, hardware, or any combination thereof.

[0042] Any of the steps, operations, or processes described herein may be performed or implemented using one or more hardware or software modules, alone or in combination with other devices. In one embodiment, a software module is implemented using a computer program product that includes a computer readable medium containing computer program code, which can be executed by a computer processor to perform any or all of the steps, operations, or processes described.

[0043] An embodiment of the present invention may relate to an apparatus for performing the operations herein. The apparatus may be specially constructed for the required purposes and / or may comprise a general-purpose computing device selectively activated or reconfigured by a computer program stored in the computer. Such a computer program may be stored in a non-transitory tangible computer-readable storage medium, or any type of medium suitable for storing electronic instructions, which may be coupled to a computer system bus. Furthermore, any computing system referred to herein may include a single processor or may be an architecture employing a multiple processor design to increase computing power.

[0044] Embodiments of the invention may relate to products produced by the computational processes described herein. Such products may comprise information resulting from the computational processes, and when the information is stored on a non-transitory tangible computer-readable storage medium, may include any embodiment of a computer program product or other data combination described herein.

[0045] Finally, the terms used herein are primarily selected for readability and instructional purposes, and not for precisely describing or delineating the subject matter of the invention. Thus, the scope of the invention is intended to be limited not by this detailed description, but by any claims that may be issued upon filing based on this specification. Thus, the disclosure of embodiments of the invention is intended to be illustrative, but not limiting, of the scope of the invention, which is set forth in the following claims.

Claims

[Claim 1] The invention described in this specification.

Images