Real-time detection and correction of shadows in hyperspectral retinal images

Abstract

A method for real-time detection and correction of shadows in a hyperspectral retinal image may include: capturing a hyperspectral image of a retina of a receiving patient using a processor; detecting shadows in the hyperspectral image by the processor; determining by the processor that the shadows in the hyperspectral image exceed a threshold; and in response to determining that the shadows in the hyperspectral image exceed the threshold, initiating the capture of an additional hyperspectral image of the retina of the patient using the processor. Various other methods, systems, and computer-readable media are also disclosed.

Description

Background Art

[0001] The present invention relates generally to computer-based image analysis and, more particularly, to a computer system configured and arranged for real-time detection and correction of shadows in hyperspectral retinal images.

[0002] Hyperspectral sensors can be used to capture image frames at specific wavelengths λ (lambda) (e.g., 900nm to 450nm in steps of 5nm). Each image can represent a spectral band (e.g., a narrow wavelength range of the electromagnetic spectrum), and these spectral bands can be combined to form a three-dimensional (x, y, λ) hyperspectral data shape (e.g., a cube) for processing and analysis. The variables x and y can represent the two spatial dimensions of the scene, and λ can represent the spectral dimension, which can include a wavelength range.

[0003] Known techniques for capturing retinal images using hyperspectral sensors suffer from various artifacts, such as shadows due to uneven illumination or insufficient pupil dilation. Shadows result in poor image quality, which may require shadow correction or reacquisition of the retinal image. Unfortunately, conventional methods for shadow detection in retinal images captured by hyperspectral sensors cannot provide results in real time. Reacquiring images in a clinical setting can be very time-consuming when correcting shadows in the image is difficult. Summary of the invention

[0004] According to one aspect, a computer-implemented method is provided, comprising: receiving, using a processor, a hyperspectral image of a patient's retina; detecting, by the processor, a shadow in the hyperspectral image; determining, by the processor, that the shadow in the hyperspectral image exceeds a threshold; and in response to determining that the shadow in the hyperspectral image exceeds the threshold, initiating, by the processor, the capture of an additional hyperspectral image of the patient's retina.

[0005] According to another aspect, a system is provided, comprising: a memory; and a processor coupled to the memory, the processor configured to execute a method, the method comprising: receiving a hyperspectral image of a patient's retina; detecting a shadow in the hyperspectral image; determining that the shadow of the hyperspectral image exceeds a threshold; and in response to determining that the shadow of the hyperspectral image exceeds the threshold, prompting an image sensor to capture an additional hyperspectral image of the patient's retina.

[0006] According to another aspect, a computer program product is provided, comprising a computer-readable storage medium having computer-executable instructions stored thereon, which, when executed by a processor, causes the processor to perform a method, comprising: receiving a hyperspectral image of a patient's retina; detecting a shadow in the hyperspectral image; determining that the shadow of the hyperspectral image exceeds a threshold; and in response to determining that the shadow of the hyperspectral image exceeds the threshold, prompting an image sensor to capture an additional hyperspectral image of the patient's retina.

[0007] According to another aspect, a computer-implemented method is provided, comprising: receiving, using a processor, a hyperspectral image of a patient's retina; detecting, by the processor, a shadow in the hyperspectral image by calculating a shift vector of the hyperspectral image; determining, by the processor, that the shift vector of the hyperspectral image exceeds a threshold; and in response to determining that the shift vector of the hyperspectral image exceeds the threshold, initiating, using the processor, the capture of an additional hyperspectral image of the patient's retina.

[0008] According to another aspect, a computer-implemented method is provided, comprising: receiving, using a processor, a hyperspectral image of a patient's retina; registering, by the processor, the hyperspectral images to correct for eye motion detected between the hyperspectral images; detecting, by the processor, shadows in the registered hyperspectral images; determining, by the processor, that the shadows in the hyperspectral images exceed a threshold; and in response to determining that the shadows in the hyperspectral images exceed the threshold, initiating, using the processor, the capture of an additional hyperspectral image of the patient's retina.

[0009] According to one or more embodiments of the present invention, a method for real-time detection and correction of shadows in a hyperspectral retinal image includes: receiving a hyperspectral image of a patient's retina using a processor, detecting shadows in the hyperspectral image by the processor, determining by the processor that the shadows in the hyperspectral image exceed a threshold, and in response to determining that the shadows in the hyperspectral image exceed the threshold, initiating the capture of an additional hyperspectral image of the patient's retina using the processor. Identifying shadows and other artifacts in the image enables the systems and methods described herein to correct and / or reacquire images in real time to avoid wasting time in a clinical setting and to improve the accuracy of the final image.

[0010] In one or more embodiments of the invention, the method includes generating an alert to a user, the alert including a message indicating that a new hyperspectral image of the retina is needed due to the shadow. Detecting the shadow in the hyperspectral image includes calculating a shift vector of the shadow in the hyperspectral image. The shift vector indicates an amount and an angle of the shadow. Determining that the shadow of the hyperspectral image exceeds a threshold includes comparing the shift vector to the threshold.

[0011] In one or more embodiments of the present invention, the method includes detecting, by a processor, a field of view (FOV) and an optic disc (OD) of a retina in a hyperspectral image, calculating, by the processor, a soft mask using the FOV and OD of the retina in the hyperspectral image, calculating, by the processor, a first centroid using the soft mask and the hyperspectral image, normalizing, by the processor, the hyperspectral image, applying, by the processor, a tensor to the normalized hyperspectral image, calculating, by the processor, a second centroid using the normalized hyperspectral image with the applied tensor, and generating, by the processor, a shift vector using the first centroid and the second centroid. In some examples, the method includes detecting, by the processor, a second shadow in an additional hyperspectral image, determining, by the processor, that the second shadow of the additional hyperspectral image does not exceed a threshold, and in response to determining that the second shadow of the additional hyperspectral image does not exceed the threshold, transmitting, by the processor, the hyperspectral image and the additional hyperspectral image, wherein the hyperspectral image includes an overlay of values ​​of the shadow and the shift vector.

[0012] In one or more embodiments of the present invention, the technical solution may also be implemented through a system, a computer program product, an apparatus, a machine, a device or other practical applications.

[0013] In addition, in one or more embodiments of the present invention, a method for real-time detection and correction of shadows in a hyperspectral retinal image includes receiving a hyperspectral image of a patient's retina using a processor, detecting, by the processor, a shadow in the hyperspectral image by calculating a shift vector of the hyperspectral image, determining, by the processor, that the shift vector of the hyperspectral image exceeds a threshold, and responsive to determining that the shift vector of the hyperspectral image exceeds the threshold, prompting an image sensor to capture an additional hyperspectral image of the patient's retina. The shift vector indicates an amount and angle of the shadow. Identifying shadows and other artifacts in an image reduces time-consuming image reacquisition in a clinical setting and increases the accuracy of a final image.

[0014] Additionally, in one or more embodiments of the present invention, a method for real-time detection and correction of shadows in a hyperspectral retinal image includes: receiving a hyperspectral image of a patient's retina using a processor, recording the hyperspectral image by the processor to correct for eye movement detected between hyperspectral images, detecting shadows in the recorded hyperspectral image by the processor, determining by the processor that the shadow of the hyperspectral image exceeds a threshold, and in response to determining that the shadow of the hyperspectral image exceeds the threshold, facilitating the image sensor to capture an additional hyperspectral image of the patient's retina. Detecting the shadow in the hyperspectral image includes calculating a shift vector for the shadow in the hyperspectral image. Real-time identification of poor image quality in the captured image enables the systems and methods described herein to quickly correct and / or request images in a clinical setting.

[0015] Other technical features and benefits are achieved through the technology of the present invention. Embodiments and aspects of the present disclosure are described in detail herein and are considered part of the subject matter claimed for protection. For a better understanding, please refer to the detailed description and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Preferred embodiments of the present invention will now be described, by way of example only, with reference to the following drawings:

[0017] Figure 1A shows an example retinal image captured by a hyperspectral sensor in accordance with one or more embodiments of the present disclosure;

[0018] Figure 1B shows an example retinal image captured by a hyperspectral sensor in accordance with one or more embodiments of the present disclosure;

[0019] Figure 2 is a schematic hybrid data flow / block diagram illustrating real-time shadow detection and correction in hyperspectral images according to one or more example embodiments of the present disclosure;

[0020] Figure 3 is a schematic hybrid data flow / block diagram illustrating shadow detection in hyperspectral retinal images according to one or more example embodiments of the present disclosure;

[0021] Figure 4A is an example retinal image showing a soft mask generated during shadow detection in a hyperspectral retinal image according to one or more embodiments of the present disclosure;

[0022] Figure 4B is an example retinal image showing the center of gravity of a soft mask generated during shadow detection in a hyperspectral retinal image in accordance with one or more embodiments of the present disclosure;

[0023] Figure 5A is an example retinal image weighted with a soft mask generated during shadow detection according to one or more embodiments of the present disclosure

[0024] Figure 5B is an example retinal image with overlay of data generated during shadow detection in accordance with one or more embodiments of the present disclosure;

[0025] Figure 6 is a process flow diagram of an illustrative method for detecting and correcting shadows in hyperspectral retinal images according to one or more example embodiments of the present disclosure; and

[0026] Figure 7 A computer system is depicted in accordance with one or more embodiments of the present disclosure.

[0027] The diagrams described herein are illustrative. Many variations may be made to the diagrams or the operations described therein without departing from the spirit of the present disclosure. For example, the actions may be performed in a different order, or actions may be added, deleted, or modified. In addition, the term "coupled" and its variants describe having a communication path between two elements and do not imply a direct connection between the elements without intermediate elements / connections between them. All of these variations are considered part of the specification. DETAILED DESCRIPTION

[0028] Exemplary embodiments of the present invention relate to devices, systems, methods, computer-readable media, techniques, and methods for real-time shadow detection and correction in hyperspectral images. A hyperspectral imaging device includes a hyperspectral image sensor and a processor for facilitating image storage, processing, and visualization. The image sensor captures image frames at different wavelengths, and the resulting data can be an ordered collection of frames.

[0029] Capturing retinal images using hyperspectral imaging devices produces images that contain various artifacts, such as shadows resulting from uneven illumination or insufficient dilation of the patient's pupil. The presence of shadows in retinal images results in poor image quality, which necessitates that the captured retinal images be rejected and the captured images need to be reacquired and / or corrected for shadows. Conventional methods for shadow detection are slow (e.g., non-real-time), which makes reacquiring images time-consuming in a clinical setting. Furthermore, conventional methods do not provide readily available tools for real-time correction of shadows detected in hyperspectral retinal images.

[0030] The systems and methods described herein are directed to real-time shadow detection for triggering real-time reacquisition of images that are deemed inadequate due to poor quality caused by the detection of shadows in the image. Traditional methods for image reacquisition can take several seconds, and shadow detection requires additional time, which increases the time burden in the clinical environment. Real-time shadow detection (and correction) and reacquisition of images reduces the time required to obtain accurate and reliable images, which translates into overall cost savings. In addition, since more and higher quality images can be captured in a shorter time, doctors and technicians can make more accurate diagnoses, thereby improving patient outcomes. In addition, the system can provide additional information generated during shadow detection (e.g., angle and shift for shadow correction) using newly acquired images, which provides context and is used to make a more accurate diagnosis using the image. The system can detect shadows and / or other image areas that have a different brightness from the retina due to image artifacts, such as the optic disc or other retinal organs.

[0031] In some exemplary instances, a hyperspectral camera or image sensor of a hyperspectral imaging device captures an image of the patient's retina. The image is registered (e.g., aligned) to correct for eye movement between scans. The system then processes the registered image and detects the field of view (FOV) of the retina and retinal features, such as the optic disc (OD) of the retina. Using the FOV and the detected retinal features, the system calculates and / or generates a soft mask. The soft mask is a weight tensor used to blur image features to ensure the accuracy of accurate detection of shadows in the image. The soft mask is used to obtain a shift vector for the hyperspectral image. The shift vector includes at least two values ​​indicating the amount and direction (e.g., angle) of the shadow detected in the hyperspectral image. Shadow correction of the hyperspectral image is performed by weighting the image pixels under the soft mask, as indicated by the shift vector. The system uses the shift vector to determine whether to reacquire the hyperspectral image. For example, the system compares the value of the shift vector with a predetermined threshold. If the value of the shift vector exceeds the predetermined threshold, a new image is automatically acquired. When acquiring a new image, the original image and the new image are presented to the doctor or technician. In some embodiments, before the image is presented to the physician or technician, the original image may be enhanced with an overlay map presenting the data generated during shadow detection (eg, shift vectors, retinal features, etc.).

[0032] Although described in the context of capturing retinal images, the systems and methods described herein are applicable to any device, scenario, and / or field that utilizes hyperspectral imaging and requires reacquisition of images due to artifacts or quality issues with the initially captured images.

[0033] Reference now Figure 1A , an example retinal image 100 captured by a hyperspectral sensor is depicted in accordance with one or more embodiments of the present invention. The image depicts a single frame in a retinal hyperspectral image. One or more artifacts, such as shadows 110, are present in the hyperspectral retinal image. The shadows may be the result of uneven illumination of the patient's pupil and / or insufficient pupil dilation. Figure 1B An exemplary retinal image 150 showing a field of view (FOV) 170 and an optic disc (OD) 160 of a patient's retina is depicted. In some embodiments, FOV 170 refers to the portion of the retina visible by a hyperspectral camera or sensor at a specific location and orientation. Objects outside of FOV 170 are not captured in the retinal imaging. OD 160 refers to a small blind spot in the eye, which is the point where the axons of the retinal ganglion cells are concentrated. OD 160 appears as a bright spot in the retinal image.

[0034] Figure 2To illustrate a hybrid data flow / block diagram, a data flow 200 for real-time shadow detection and correction of hyperspectral images in an imaging device according to one or more exemplary embodiments of the present invention is shown. In some examples, an image sensor 210 of an imaging device 205 can be used to capture an image 100 of a retina of a patient 230. The image sensor 210 can be any type of imaging device capable of capturing a retinal image 100. Examples of the image sensor 210 include a hyperspectral device or camera for retinal imaging. A hyperspectral camera captures image frames of a specific wavelength (e.g., 900nm to 450nm, with a step size of 5nm), and the resulting data is an ordered set of frames or a three-dimensional (x, y, λ) hyperspectral data shape for processing and analysis, where x and y represent two spatial dimensions of the scene, and λ represents a spectral dimension, which includes a wavelength range. An example of the imaging device 205 is a hyperspectral device for retinal imaging, such as a metabolic hyperspectral retinal camera (MHRC), which includes an image sensor 210 and a processor for image storage, processing, and visualization. Although image sensor 210 is depicted as an integrated component of imaging device 205 , in some embodiments, image sensor 210 is an external component that communicates with imaging device 205 .

[0035] The image captured by the image sensor 210 is transmitted to the shadow detection module 220 coupled to the physical processor 215 of the imaging device 205. The physical processor 215 can generally represent any type or form of hardware-implemented processing unit capable of interpreting and / or executing computer-readable instructions. In one example, the physical processor 215 accesses and / or modifies one or more modules (e.g., the shadow detection module 220 and / or the image management module 225) stored in the memory. Additionally or alternatively, the physical processor 215 executes one or more modules to perform real-time shadow detection and correction in the hyperspectral image. Examples of the physical processor 215 include, but are not limited to, a microprocessor, a microcontroller, a central processing unit (CPU), a graphics processing unit (GPU), a field programmable gate array (FPGA) implementing a soft-core processor, an application-specific integrated circuit (ASIC), a portion of one or more of these components, a variation or combination of one or more of these components, or any other suitable physical processor.

[0036] The shadow detection module 220 processes the image captured by the image sensor 210 to detect shadows in the image in real time. In some examples, the shadow detection module 220 detects the shadow 110 in the image and transmits data associated with the shadow 110 to the image management module 225 of the imaging device 205. The image management module 225, coupled to the processor 215, compares the data associated with the shadow 110 with a predetermined threshold. In some examples, the predetermined threshold is manually set by a user of the imaging device 205, or can be dynamically set and / or adjusted based on the previous actions of the user (such as a technician or doctor 240) when using the imaging device 205. If the image management module 225 determines that the data associated with the detected shadow 110 exceeds the predetermined threshold, the image management module 225 triggers an action in real time. Examples of actions include, but are not limited to, automatically facilitating the reacquisition of a hyperspectral image of the retina of the patient 230, and alerting the technician or doctor 240 that a new image needs to be obtained due to insufficient quality detected in the image caused by the shadow. In some examples, the action includes alerting the technician or doctor 240 before initiating the reacquisition of the image and requesting confirmation to proceed with the reacquisition.

[0037] In response to the image management module 225 triggering a real-time action due to the detection of a shadow in the hyperspectral image, the processor 215 activates the image sensor 210 to capture a new hyperspectral image of the retina of the patient 230. The image is transmitted to the shadow detection module 220 for processing. The image management module 225 receives the data generated by the shadow detection module 220 for any shadows detected in the new hyperspectral image of the retina of the patient 230, and determines that the data associated with the shadows detected in the new hyperspectral image does not exceed a predetermined threshold. The image management module 225 transmits the original hyperspectral image and the new hyperspectral image of the retina of the patient 230 to the doctor 240. In some examples, the original hyperspectral image is enhanced with information associated with the shadow (e.g., the location and / or amount of the shadow detected in the image).

[0038] The imaging device 205 may perform multiple repetitions of image reacquisition to obtain a satisfactory image (e.g., not exceeding a threshold) for transmission to the physician 240. In some examples, the imaging device 205 has a limit (e.g., manually set and / or automatically determined) that limits the number of iterations of image reacquisition. When the limit is reached and a satisfactory image is not captured, the image management module 225 may generate and send a message to the physician 240 that a satisfactory image could not be captured. The message may include an indication of the number of attempts to acquire an image, one or more sets of previously acquired images, and / or possible reasons for poor or insufficient image quality.

[0039] In some examples, images captured by the imaging device 205 are stored in a data storage device that is local to the imaging device 205 and / or external to the imaging device 205 (eg, cloud storage, remote storage, etc.).

[0040] Figure 3 2 is a schematic hybrid data flow / block diagram illustrating a process 300 for shadow detection in a hyperspectral retinal image. According to an embodiment of the present invention, the process 300 is performed by a computer-based system, including, for example, data flow 200 (e.g., Figure 2 ). In box 302, an image sensor such as image sensor 210 of a retinal imaging device such as imaging device 205 captures a hyperspectral image of the retina of patient 230. The captured images may be taken at different wavelengths and stored as a collection of ordered frames. In box 304, the captured images are registered. For example, the captured images are registered by shadow detection module 220. For example, the images are registered to correct for eye movement between scans of the retina. The images are registered using any known techniques and / or algorithms for registering hyperspectral images. In some examples, registration of the images may include aligning frames of the hyperspectral image using spatial information contained in the image and spectral information when the image is taken at different wavelengths. In box 306, shadow detection module 220 processes the registered images and detects different components of the image, such as the FOV 170 of the image, and retinal features in the image, such as OD 160.

[0041] Referring now to block 308, the shadow detection module 220 may compute a soft mask for the image. The soft mask is computed using the detected FOV 170 and OD 160 or other relevant retinal organs. The soft mask is a weight tensor that may be used to blur detected components of the image that may obstruct or otherwise negatively affect the image for diagnostic purposes. At block 310, the shadow detection module 220 uses the soft mask to compute a center of gravity of the soft mask. In some examples, the center of gravity of the soft mask may be computed by computing the center of gravity of an image or component (e.g., a soft mask) using any known technique or algorithm. For example, the center of gravity of the soft mask may be computed by performing a calculation on the computed soft mask, such as averaging the soft mask row by row and column by column.

[0042] Referring now to block 312, the shadow detection module 220 normalizes the registered image from block 304. In some embodiments, the shadow detection module 220 element-wise multiplies the registered image from block 304 by the calculated soft mask from block 308 and then normalizes the image frame-by-frame (e.g., the weighted pixels of the frame sum equal to 1). At block 314, one or more tensors are applied to the normalized image from block 312. For example, the weighted (with soft mask) and normalized image from block 312 are element-wise multiplied with the row and column coordinate tensors. At block 316, the image with the applied tensors is used to calculate the centroid within the weighted image. Any known technique or algorithm for calculating the centroid can be applied to the weighted image to derive the centroid. At block 318, a shift vector can be calculated. In some examples, the centroid from block 316 and the centroid from block 310 are used to calculate the shift vector. The shift vector is calculated by taking the element-wise difference between the centroid of the soft mask and the centroid of the weighted image.

[0043] The shift vector includes at least two values. In some examples, the shift value includes a length value indicating the amount of shadow in the image. The shift vector also includes a direction (e.g., angle) value indicating the location of the shadow in the image. In some examples, the shadow detection module 220 uses a shift vector, such as the shift vector calculated in box 318, to detect shadows in the image. Shadow detection is performed by comparing the length value of the shift vector to an established predetermined threshold. The shadow detection module 220 performs shadow correction by shifting an annular weight mask (e.g., the same diameter as the FOV 170) as indicated by the shift vector and multiplying the image frame by the weight mask.

[0044] Reference now Figure 4A , a soft mask 400 generated during shadow detection in a hyperspectral retinal image. The soft mask 400 is a weight tensor generated by the shadow detection module 220 using data obtained from the hyperspectral image (e.g., FOV and / or OD measurements) and is used for shadow correction. The soft mask 400 is used to blur artifacts and features of the retina that may hinder or negatively affect the use of the image for diagnostic purposes. For example, component 410 is a portion of the soft mask 400 that is generated to blur the OD 160 detected in the original hyperspectral image, and component 415 is a portion of the soft mask 400 that is generated to blur the shadow 110 detected in the original hyperspectral image. Figure 4B Depicted is an image 450 showing a centroid 460 of the soft mask 400 calculated by the shadow detection module 220. The centroid 460 of the soft mask is used to calculate a shift vector that the image management module 225 uses to determine whether to reacquire a new hyperspectral image of the patient's 230 retina.

[0045] Reference now Figure 5A , depicts an image (or frame) 500 of a hyperspectral image set captured by the image sensor 210. Figure 3 As shown, the image 500 is weighted or corrected using the soft mask 400 calculated by the shadow detection module 220 . Figure 5B 5 is an image or frame 550 of a hyperspectral image of the retina of the patient 230. The image 550 includes an overlay of information generated by the shadow detection module 220. For example, the image 550 includes the soft mask centroid 460, the weighted image centroid 560, and a shift vector 570 indicating the size and direction of the shadow in the original hyperspectral image. This information is provided to the doctor 240 or technician to help them use the image for diagnostic purposes.

[0046] An illustrative method according to an exemplary embodiment of the present invention and corresponding data structures (e.g., modules, units, and other such components) for performing the method are now described. It should be noted that Figure 6 Each operation of the method 600 described in is performed by one or more modules described herein, etc. These modules can be implemented in any combination of hardware, software and / or firmware described herein. In certain example embodiments, one or more of these modules can be implemented at least in part as software and / or firmware modules including computer executable instructions, which cause one or more operations to be performed when executed by a processing circuit. The system or device configured to implement the example embodiments described herein may include one or more processing circuits, each of which may include one or more processing units or nodes. Computer executable instructions may include computer executable program code, which, when executed by a processing unit, may cause input data contained in or referenced by the computer executable program code to be accessed and processed to generate output data.

[0047] Figure 6 is a flow chart of an example computer-implemented method 600 for providing real-time shadow detection and correction in hyperspectral imagery. Figure 6 The steps shown can be performed by any suitable computer executable code and / or computing system. In one example, Figure 6 Each step shown in may represent an algorithm whose structure includes and / or is represented by multiple sub-steps, examples of which are provided in more detail below.

[0048] like Figure 6As shown, at step 602, one or more systems described herein receive a hyperspectral image of a patient's retina. The systems described herein may perform step 602 in any suitable manner. For example, the image sensor 210 of the imaging device 205 captures a hyperspectral image of the patient's retina. As discussed herein, the image sensor 210 may be any type of hyperspectral sensor capable of taking images at different wavelengths, which may be combined to form a three-dimensional (x, y, λ) hyperspectral data shape for processing and analysis, where x and y represent two spatial dimensions of the scene, and λ represents a spectral dimension, which includes a wavelength range.

[0049] At step 604, one or more systems described herein detect shadows in the hyperspectral image. The systems described herein may perform step 604 in any suitable manner. For example, the shadow detection module 220 of the imaging device 205 detects the shadow 110 in the hyperspectral image captured by the image sensor 210.

[0050] In some examples, the frames of the hyperspectral image captured at step 602 are registered. The shadow detection module 220 performs or facilitates the registration (e.g., alignment) of the captured images. The images are registered to correct and / or compensate for variations in the frames, such as for correcting for eye movement of a patient between scans. Using the registered images, the shadow detection module 220 processes the images to detect and identify different components and retinal features, such as the FOV 170 of the image, the OD 160 of the retina, etc.

[0051] In some examples, the shadow detection module 220 uses components and features identified from the image to calculate a soft mask 400. The soft mask 400 is a feature weight tensor for blurry images that can prevent accurate diagnosis or result in poor image quality.

[0052] like Figure 3 As described, using the calculated soft mask 400, the shadow detection module 220 calculates the center of gravity 460 of the soft mask. The shadow detection module then multiplies the soft mask and the registered image element by element, and then normalizes the image frame by frame. The shadow detection module 220 applies one or more tensors to the normalized and weighted image, and uses the resulting image to calculate the center of gravity 560 of the weighted normalized image. The shadow detection module 220 then uses the center of gravity 560 of the weighted, normalized image and the center of gravity 460 of the soft mask 400 to calculate or generate a shift vector 570. The shift vector 570 includes at least two values, a length and an angle. The length indicates the amount of the shadow, and the angle of the shift vector indicates the position of the shadow 110 in the image.

[0053] At step 606, one or more systems described herein determine that the shadow of the hyperspectral image exceeds a threshold. The systems described herein may perform step 606 in any suitable manner. For example, the image management module 225 of the imaging device 205 determines that the shadow 110 of the hyperspectral image exceeds a threshold.

[0054] In some examples, the image management module 225 receives the shift vector 570 calculated by the shadow detection module 220. The image management module 225 compares at least one value (such as a length value) of the shift vector with an identified threshold. In some examples, the threshold may be manually entered by a technician or doctor using the imaging device 205. The threshold may be a value stored in a profile associated with the technician or doctor using the device, and may vary from person to person. The threshold may be a global value applied to all images captured by the imaging device 205. In some examples, the threshold may include multiple components. The threshold may be a conditional or multi-part threshold that may specify different actions for different values. For example, the threshold may indicate that if the shift vector value falls within a first range, then shadow correction may be applied, and if the shift vector value falls within a second range, then a new image needs to be acquired.

[0055] At step 608, one or more systems described herein may initiate the capture of an additional hyperspectral image of the patient's retina. The systems described herein may perform step 608 in any suitable manner. For example, in response to determining that the shadowing of the hyperspectral image exceeds a threshold, the image management module 225 of the imaging device 205 may facilitate the capture of an additional hyperspectral image of the patient's retina.

[0056] In some examples, the image management module 225 determines that the shift vector value does not exceed a threshold, in which case the image is transmitted to the physician without further processing or any additional action.

[0057] In some instances, the image management module 225 determines that the length value of the shift vector falls within a threshold range that requires shading correction, and the image management module 225 facilitates shading correction of the image. In some examples, shading correction is performed by shifting a circular weighted soft mask (e.g., having a diameter the same as the FOV) indicated by the shift vector and multiplying the image frame by the weighted mask. The shadow detection module 220 then processes the shading-corrected image to calculate a new shift vector. The image management module 225 uses the value from the new shift vector to determine whether the shading-corrected image is sufficient for transmission to the physician, or whether a new image needs to be acquired regardless of shading correction.

[0058] In some examples, if the value of the shift vector exceeds a threshold or falls within a threshold range, indicating that a new image needs to be acquired, the image management module 225 sends a message to the technician or physician indicating that a new image is needed and requesting an indication as to whether to acquire a new image. The imaging device 205 receives the indication from the technician or physician. If the indication is not to acquire a new image, the original image is transmitted to the technician or physician with a note indicating that a new image is recommended. In some examples, if the indication is to acquire a new image, the image management module 225 facilitates a new iteration of acquiring and processing a new image in response to receiving the indication from the user.

[0059] In some examples, the image management module 225 determines that the length value of the shift vector exceeds a predetermined threshold or falls within a threshold range indicating that a new image needs to be acquired. The image management module 225 automatically initiates reacquisition of the image (e.g., prompting the image sensor 210 to acquire a new image). The image management module 225 also generates and sends an alert to the physician or technician indicating that a new image is being acquired. The alert indicates the reason for acquiring the new image. The new image is then processed by the shadow detection module 220, which generates a new shift vector for the newly acquired image. The image management module 225 uses the new shift vector to determine whether the newly acquired image is of sufficient quality to be transmitted to the physician or technician, or whether the image needs to be acquired again.

[0060] If the new image is satisfactory (e.g., does not exceed a threshold), the new image is transmitted to the physician or technician. In some examples, the old image is also transmitted to the physician or technician. The old images are transmitted in their original state and / or can be enhanced with data generated by the shadow detection module 220, such as shift vectors, components of the identification (e.g., FOV, OD, etc.), location and size of the shadow, center of gravity of the soft mask, center of gravity of the weighted normalized image, etc. This additional data provides the physician or technician with additional context to the image for diagnostic purposes.

[0061] In some instances, the systems and methods described herein perform multiple iterations of image acquisition and processing to capture satisfactory images. The imaging device uses a limit to limit the number of iterations of the process to avoid excessive burden on the system and waste time. The limit can be manually set by the user of the imaging device 205, or can be dynamically determined based on image quality. For example, if the shift vector of each group of acquired images remains within the identified range for a certain number of image acquisitions, the system can determine that a satisfactory image cannot be captured at this time. The image management module 225 generates a message to the doctor or technician indicating that image re-acquisition is not recommended. The message indicates the number of attempts to re-acquire and data indicating poor image quality. In some examples, the message includes a previously captured image or a subset of a previously captured image, and / or includes a link to access such data, which can be stored in a data storage device, locally or remotely stored in the imaging device 205.

[0062] In some example embodiments, a method for real-time detection and correction of shadows in a hyperspectral retinal image includes: receiving a hyperspectral image of a patient's retina using a processor; detecting, by the processor, a shadow in the hyperspectral image by calculating a shift vector of the hyperspectral image; determining, by the processor, that the shift vector of the hyperspectral image exceeds a threshold; and in response to determining that the shift vector of the hyperspectral image exceeds the threshold, causing, by the processor, an image sensor to capture an additional hyperspectral image of the patient's retina. The shift vector includes a length of the shadow and an angle of the shadow.

[0063] In another exemplary embodiment, a method for real-time detection and correction of shadows in a hyperspectral retinal image includes: receiving a hyperspectral image of a patient's retina using a processor; registering the hyperspectral images by the processor to correct for eye motion detected between the hyperspectral images; detecting shadows in the registered hyperspectral images by the processor; determining by the processor that the shadows in the hyperspectral images exceed a threshold; and in response to determining that the shadows in the hyperspectral images exceed the threshold, facilitating, by the processor, an image sensor to capture an additional hyperspectral image of the patient's retina. Detecting the shadows in the hyperspectral images includes calculating a shift vector for the shadows in the hyperspectral images.

[0064] Now turn to Figure 7, according to an embodiment of the present invention, a computer system 700 is generally shown. The computer system 700 can be an electronic computer framework including and / or employing any number of computing devices and networks utilizing various communication technologies and combinations thereof, as described herein. The computer system 700 can be easily upgraded, expanded, and modularized, with the ability to change to different services or reconfigure some features independently of other features. The computer system 700 can be, for example, a server, a desktop computer, a laptop computer, a tablet computer, or a smart phone. In some examples, the computer system 700 can be a cloud computing node. The computer system 700 can be described in the general context of computer system executable instructions executed by the computer system, such as program modules. Generally, program modules can include routines, programs, objects, components, logic, data structures, etc. that perform specific tasks or implement specific abstract data types. The computer system 700 can be implemented in a distributed cloud computing environment, where tasks are performed by remote processing devices linked through a communication network. In a distributed cloud computing environment, program modules can be located in local and remote computer system storage media including memory storage devices.

[0065] like Figure 7 As shown, the computer system 700 has one or more central processing units (CPUs) 701a, 701b, 701c, etc. (collectively or collectively referred to as (one or more) processors 701). The processor 701 can be a single-core processor, a multi-core processor, a computing cluster, or any number of other configurations. The processor 701, also referred to as a processing circuit, is coupled to a system memory 703 and various other components via a system bus 702. The system memory 703 may include a read-only memory (ROM) 704 and a random access memory (RAM) 705. The ROM 704 is coupled to the system bus 702 and may include a basic input / output system (BIOS) that controls certain basic functions of the computer system 700. The RAM is a read-write memory coupled to the system bus 702 for use by the processor 701. The system memory 703 provides temporary storage space for the operation of the instructions during operation. The system memory 703 may include a random access memory (RAM), a read-only memory, a flash memory, or any other suitable storage system.

[0066] Computer system 700 includes an input / output (I / O) adapter 706 and a communication adapter 707 coupled to system bus 702. I / O adapter 706 may be a small computer system interface (SCSI) adapter that communicates with hard disk 708 and / or any other similar components. I / O adapter 706 and hard disk 708 are collectively referred to herein as mass storage device 710.

[0067] Software 711 for execution on computer system 700 may be stored in mass storage device 710. Mass storage device 710 is an example of a tangible storage medium readable by processor 701 in which software 711 is stored as instructions executed by processor 701 to cause computer system 700 to operate, as described below with reference to the various figures. Examples of computer program products and execution of such instructions are discussed in greater detail herein. Communications adapter 707 interconnects system bus 702 with network 712, which may be an external network, to enable computer system 700 to communicate with other such systems. In one embodiment, a portion of system memory 703 and mass storage device 710 together store an operating system, which may be any suitable operating system, such as z / OS or AIX operating systems from IBM Corporation, to coordinate Figure 7 The functions of the various components are shown.

[0068] Additional input / output devices are shown connected to the system bus 702 via a display adapter 715 and an interface adapter 716. In one embodiment, adapters 706, 707, 715, and 716 may be connected to one or more I / O buses that are connected to the system bus 702 via an intermediate bus bridge (not shown). A display 719 (e.g., a screen or display monitor) is connected to the system bus 702 via a display adapter 715, which may include a graphics controller to improve performance for graphics-intensive applications and a video controller. A keyboard 721, a mouse 722, speakers 723, and the like may be interconnected to the system bus 702 via an interface adapter 716, which may include, for example, a super I / O chip that integrates multiple device adapters into a single integrated circuit. Suitable I / O buses for connecting peripheral devices such as hard disk controllers, network adapters, and graphics adapters typically include common protocols, such as the Peripheral Component Interconnect (PCI). Thus, as Figure 7 As configured in , computer system 700 includes processing capabilities in the form of processor 701, storage capabilities including system memory 703 and mass storage device 710, input devices such as keyboard 721 and mouse 722, and output capabilities including speakers 723 and display 719.

[0069] In some embodiments, the communication adapter 707 can transmit data using any suitable interface or protocol, such as an Internet Small Computer System Interface, etc. The network 712 can be a cellular network, a radio network, a wide area network (WAN), a local area network (LAN), or the Internet, etc. An external computing device can be connected to the computer system 700 through the network 712. In some examples, the external computing device can be an external network server or a cloud computing node.

[0070] It should be understood that Figure 7 The block diagram does not represent that the computer system 700 includes Figure 7 Instead, the computer system 700 may include Figure 7 700 (e.g., additional memory components, embedded controllers, modules, additional network interfaces, etc.). In addition, the embodiments described herein with respect to computer system 700 may be implemented with any suitable logic, wherein in various embodiments, logic as referred to herein may include any suitable hardware (e.g., a processor, an embedded controller, or an application specific integrated circuit, etc.), software (e.g., an application, etc.), firmware, or any suitable combination of hardware, software, and firmware.

[0071] The present invention may be a system, method and / or computer program product at any possible level of technical detail integration. The computer program product may include a computer-readable storage medium (or medium) having computer-readable program instructions thereon for causing a processor to perform various aspects of the present invention.

[0072] A computer-readable storage medium may be a tangible device that can retain and store instructions for use by an instruction execution device. A computer-readable storage medium may be, for example, but not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. A non-exhaustive list of more specific examples of computer-readable storage media includes the following: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disk read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanical encoding device such as a punch card or a raised structure in a groove on which instructions are recorded, and any suitable combination of the foregoing. As used herein, a computer-readable storage medium should not be interpreted as being a transient signal in itself, such as a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagating through a waveguide or other transmission medium (e.g., a light pulse through a fiber optic cable), or an electrical signal transmitted through a wire.

[0073] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to a corresponding computing / processing device, or downloaded to an external computer or external storage device, via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network can include copper transmission cables, optical transmission fibers, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions to be stored in a computer-readable storage medium within the corresponding computing / processing device.

[0074] The computer-readable program instructions for performing the operation of the present invention can be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-related instructions, microcode, firmware instructions, state setting data, integrated circuit configuration data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages, such as Smalltalk, C++, etc., and procedural programming languages, such as "C" programming language or similar programming languages. The computer-readable program instructions can be executed completely on the user's computer, partially on the user's computer, as an independent software package, partially on the user's computer, partially on a remote computer, or completely on a remote computer or server. In the latter case, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (for example, by using the Internet of an Internet service provider). In some embodiments, the electronic circuit including, for example, a programmable logic circuit, a field programmable gate array (FPGA) or a programmable logic array (PLA) can execute the computer-readable program instructions by utilizing the state information of the computer-readable program instructions to personalize the electronic circuit, thereby performing aspects of the present invention.

[0075] This article describes various aspects of the present invention with reference to the flowchart and / or block diagram of the method, device (system) and computer program product according to an embodiment of the present invention. It will be understood that each block of the flowchart and / or block diagram and the combination of blocks in the flowchart and / or block diagram can be implemented by computer-readable program instructions.

[0076] These computer-readable program instructions may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device to produce a machine, whereby instructions executed by a processor of a computer or other programmable data processing device may produce a device for implementing the functions / actions specified in the flowchart and / or block diagram. These computer-readable program instructions may also be stored in a computer-readable storage medium, which may direct a computer, a programmable data processing device, and / or other device to operate in a particular manner, such that the computer-readable storage medium having the instructions stored therein comprises an article of manufacture, which comprises instructions for implementing various aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0077] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operating steps to be performed on the computer, other programmable apparatus, or other device, thereby producing a computer-implemented process, so that the instructions executed on the computer, other programmable apparatus, or other device implement the functions / actions specified in one or more boxes of the flowchart and / or block diagram.

[0078] The flow chart and block diagram in the figure illustrate the possible implementation architecture, function and operation of the system, method and computer program product according to various embodiments of the present invention.In this regard, each block in the flow chart or block diagram can represent a module, fragment or part of an instruction, which includes one or more executable instructions for realizing a specified logical function.In some alternative implementations, the function marked in the frame may not occur in the order marked in the figure.For example, the two frames shown in succession can actually be performed substantially simultaneously, or these frames can sometimes be performed in reverse order, depending on the function involved.It will also be noted that each frame in the block diagram and / or the flow chart diagram and the combination of the frames in the block diagram and / or the flow chart diagram can be implemented by a system based on special-purpose hardware, which performs a specified function or action or performs a combination of special-purpose hardware and computer instructions.

[0079] The description of various embodiments of the present invention is for illustrative purposes only and is not intended to be exhaustive or limit the disclosed embodiments. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The terms used herein are selected to best explain the principles of the embodiments, practical applications, or technical improvements to the technology found on the market, or to enable other persons of ordinary skill in the art to understand the embodiments described herein.

[0080] Various embodiments of the present invention are described herein with reference to the accompanying drawings. Without departing from the scope of the invention, alternative embodiments of the present invention may be designed. Various connections and positional relationships (e.g., above, below, adjacent, etc.) are described below and are described between the elements in the accompanying drawings. Unless otherwise indicated, these connections and / or positional relationships may be direct or indirect, and the present invention is not intended to be limited in this respect. Therefore, the coupling of entities may refer to direct or indirect coupling, and the positional relationship between entities may be a direct or indirect positional relationship. In addition, the various tasks and process steps described herein may be incorporated into a more comprehensive program or process with additional steps or functions not described in detail herein.

[0081] The following definitions and abbreviations are used to interpret the claims and specification. As used herein, the terms "comprises," "comprising," "containing," or "having," or any other variation thereof, are intended to cover a non-exclusive inclusion. For example, a composition, mixture, process, method, article, or apparatus that includes a list of elements is not necessarily limited to only those elements but may include other elements not expressly listed or inherent to such composition, mixture, process, method, article, or apparatus.

[0082] In addition, the term "exemplary" is used herein to mean "serving as an example, instance, or illustration." Any embodiment or design described herein as "exemplary" is not necessarily to be construed as preferred or advantageous over other embodiments or designs. The terms "at least one" and "one or more" may be understood to include any integer greater than or equal to one, i.e., one, two, three, four, etc. The term "plurality" may be understood to include any integer greater than or equal to 2, i.e., 2, 3, 4, 5, etc. The term "connected" may include both indirect "connected" and direct "connected."

[0083] The terms "about," "substantially," "approximately," and variations thereof are intended to include the degree of error associated with measurement of a particular quantity based on the equipment available at the time the application was filed. For example, "about" may include a range of 8% or 5% or 2% of a given value.

[0084] For the sake of brevity, conventional techniques related to making and using various aspects of the present invention may or may not be described in detail herein. In particular, various aspects of computing systems and specific computer programs that implement the various technical features described herein are well known. Therefore, for the sake of brevity, many conventional implementation details are only briefly mentioned or omitted entirely herein, without providing well-known system and / or process details.

Claims

1. A computer-implemented method comprising: receiving a hyperspectral image of a patient's retina using a processor; detecting, by the processor, shadows in the hyperspectral image, wherein detecting shadows in the hyperspectral image comprises calculating a shift vector of the shadows in the hyperspectral image; determining, by the processor, that a shadow of the hyperspectral image exceeds a threshold by comparing the shift vector with a threshold; as well as Responsive to determining that the shadowing of the hyperspectral image exceeds the threshold, capturing, by the processor, an additional hyperspectral image of the patient's retina is initiated.

2. The computer-implemented method of claim 1 , further comprising generating an alert to a user, the alert comprising a message indicating that a new hyperspectral image of the retina is required due to the shadow.

3. The computer-implemented method of claim 1 , wherein: The shift vector indicates the amount and angle of the shadow.

4. The computer-implemented method of claim 1 , further comprising: detecting, by the processor, the visual field and the optic disc of the retina in the hyperspectral image; calculating, by the processor, a soft mask using the field of view and the optic disc of the retina in the hyperspectral image; calculating, by the processor, a first centroid using the soft mask and the hyperspectral image; normalizing, by the processor, the hyperspectral image; applying, by the processor, the tensor to the normalized hyperspectral image; calculating, by the processor, a second centroid using the normalized hyperspectral image with the tensor; as well as A shift vector is generated by the processor using the first center of gravity and the second center of gravity.

5. The computer-implemented method of claim 1 , further comprising: detecting, by the processor, a second shadow in the additional hyperspectral image; determining, by the processor, that a second shadow of the additional hyperspectral image does not exceed the threshold; as well as In response to determining that the second shadow of the additional hyperspectral image does not exceed the threshold, the processor transmits the hyperspectral image and the additional hyperspectral image, wherein the hyperspectral image includes an overlap of the shadow and the value of the shift vector.

6. The computer-implemented method of claim 1 , comprising: The hyperspectral images are registered, by the processor, to correct for detected eye motion between the hyperspectral images, and wherein the detecting step includes detecting, by the processor, shadows in the registered hyperspectral images.

7. The computer-implemented method of claim 1 , wherein: The shift vector includes the length of the shadow and the angle of the shadow.

8. A system comprising: Image sensor; Memory; as well as A processor coupled to the image sensor and the memory, the processor being configured to perform a method comprising: receiving a hyperspectral image of the patient's retina; detecting shadows in the hyperspectral image, wherein detecting shadows in the hyperspectral image comprises calculating a shift vector of the shadows in the hyperspectral image; determining that the shadow of the hyperspectral image exceeds the threshold by comparing the shift vector with the threshold; and In response to determining that the shadowing of the hyperspectral image exceeds the threshold, capturing, by the image sensor, an additional hyperspectral image of the patient's retina is initiated.

9. The system according to claim 8, wherein: The method performed by the processor also includes generating an alert to a user, the alert including a message indicating that a new hyperspectral image of the retina is needed due to the shadow.

10. The system according to claim 8, wherein: The shift vector indicates the amount and angle of the shadow.

11. The system according to claim 8, wherein: The method executed by the processor further includes: detecting the visual field and optic disc of the retina in the hyperspectral image; calculating a soft mask using the visual field and the optic disc of the retina in the hyperspectral image; calculating a first centroid using the soft mask and the hyperspectral image; Normalizing the hyperspectral image; Apply the tensor to the normalized hyperspectral image; calculating a second centroid using the normalized hyperspectral image with the tensor; and A shift vector is generated using the first centroid and the second centroid.

12. The system according to claim 8, wherein: The method executed by the processor further includes: detecting a second shadow in the additional hyperspectral image; determining that a second shadow of the additional hyperspectral image does not exceed the threshold; and In response to determining that the second shadow of the additional hyperspectral image does not exceed the threshold, the hyperspectral image and the additional hyperspectral image are transmitted, wherein the hyperspectral image includes an overlap of the shadow and the value of the shift vector.

13. A computer program product comprising a computer readable storage medium having computer executable instructions stored thereon, which when executed by a processor causes the processor to perform a method comprising: receiving a hyperspectral image of the patient's retina; detecting shadows in the hyperspectral image, wherein detecting shadows in the hyperspectral image comprises calculating a shift vector of the shadows in the hyperspectral image; determining that the shadow of the hyperspectral image exceeds the threshold by comparing the shift vector with the threshold; as well as In response to determining that the shadowing of the hyperspectral image exceeds the threshold, an image sensor is facilitated to capture an additional hyperspectral image of the patient's retina.

14. The computer program product of claim 13, wherein: The method performed by the processor also includes generating an alert to a user, the alert including a message indicating that a new hyperspectral image of the retina is needed due to the shadow.

15. The computer program product of claim 13, wherein: The shift vector indicates the amount and angle of the shadow.

16. The computer program product of claim 13, wherein: The method executed by the processor further includes: detecting the visual field and optic disc of the retina in the hyperspectral image; calculating a soft mask using the visual field and the optic disc of the retina in the hyperspectral image; calculating a first centroid using the soft mask and the hyperspectral image; Normalizing the hyperspectral image; Apply the tensor to the normalized hyperspectral image; calculating a second centroid using the normalized hyperspectral image with the tensor; and A shift vector is generated using the first centroid and the second centroid.

17. The computer program product of claim 13, wherein: The method executed by the processor further includes: detecting a second shadow in the additional hyperspectral image; determining that a second shadow of the additional hyperspectral image does not exceed the threshold; and In response to determining that the second shadow of the additional hyperspectral image does not exceed the threshold, the hyperspectral image and the additional hyperspectral image are transmitted, wherein the hyperspectral image includes an overlap of the shadow and the value of the shift vector.

18. A computer-implemented method comprising: receiving a hyperspectral image of a patient's retina using a processor; detecting, by the processor, shadows in the hyperspectral image by calculating a shift vector of the hyperspectral image; determining, by the processor, that a shift vector of the hyperspectral image exceeds a threshold by comparing the shift vector with a threshold; as well as In response to determining that the shift vector of the hyperspectral image exceeds the threshold, initiating, using the processor, capture of an additional hyperspectral image of the patient's retina.

19. The computer-implemented method of claim 18, wherein: The shift vector includes the length of the shadow and the angle of the shadow.

20. A computer-implemented method comprising: receiving a hyperspectral image of a patient's retina using a processor; registering, by the processor, the hyperspectral images to correct for eye motion detected between the hyperspectral images; detecting, by the processor, shadows in the registered hyperspectral image, wherein detecting shadows in the hyperspectral image comprises calculating a shift vector of the shadows in the hyperspectral image; determining, by the processor, that a shadow of the hyperspectral image exceeds a threshold by comparing the shift vector with a threshold; as well as In response to determining that the shadowing of the hyperspectral image exceeds the threshold, initiating, using the processor, capture of an additional hyperspectral image of the patient's retina.

21. The computer-implemented method of claim 20, wherein: Detecting shadows in the hyperspectral image includes calculating a shift vector of the shadows in the hyperspectral image.

22. A computer program product comprising a computer-readable storage medium having computer-executable instructions stored thereon, which, when executed by a processor, cause the processor to perform the method according to any one of claims 1 to 7 or any one of claims 18 to 21.

Images