Detection of Lesions in Eye Images

Abstract

To efficiently determine an eye image without mistakes.SOLUTION: There is provided a method executed by a computer to search for a region showing a lesion in an image of an eye part acquired by an eye imaging system. The method includes: reception of image data defining the image; search for the region in the image by processing the received image data using a learning algorithm (S12); determination of the position of the region in the image when the region in the image showing the lesion is found (S14); generation of a command to generate measurement data by causing an eye measuring device to perform measurement of the eye part using a reference point based on the determined position for setting the position of the measurement for the eye part (S16); and reception of the measurement data from the eye measuring device (S18).SELECTED DRAWING: Figure 3

Description

[Technical field]

[0001] Exemplary aspects herein relate generally to the field of image processing, and more specifically to processing of ocular images to facilitate identifying the presence and / or location of a lesion in an imaged portion of the eye. [Background technology]

[0002] An ocular image of a portion of a subject's eye acquired by an ocular imaging system may have one or more areas that contain some sign of a pathology that is not present in a comparative image of a healthy eye, and the presence of such area(s) may indicate a disease or other unhealthy condition of the subject. As an example, a fluorescein angiogram of the subject's retina may include one or more areas where blood vessels appear to be leaking, which may be an indication of macular edema. The ocular images must be acquired and interpreted by an experienced ophthalmologist. However, existing techniques are time-consuming for the ophthalmologist and / or the patient, and the interpretations made are often incorrect. Summary of the Invention

[0003] The inventors of the present invention have invented, according to a first exemplary embodiment herein, a computer-implemented method for searching for a region indicative of a lesion in an image of a portion of an eye acquired by an ophthalmic imaging system, the method comprising: receiving image data defining the image; and searching for the region in the image by processing the received image data with a learning algorithm trained on image data defining a plurality of images of the portion of a healthy eye and image data defining a plurality of images of the portion of an unhealthy eye, each image having at least one region indicative of the lesion. If a region in the image indicative of the lesion is found in the searching, a location of the region in the image is determined, instructions are generated to cause an ophthalmic measurement device to perform the measurement on the portion of the eye using a reference point based on the determined location to set a position of the measurement on the portion of the eye and generate measurement data, the measurement data being received from the ophthalmic measurement device.

[0004] The inventors of the present invention have further invented a computer-implemented method for searching for the presence of a lesion in an image of a portion of an eye acquired by an ophthalmic imaging system according to a second exemplary embodiment herein, the method comprising: receiving image data defining the image; and searching for the presence of at least one of a plurality of different types of lesions in the image by processing the received image data with a learning algorithm trained on image data defining a plurality of images of a healthy eye and a plurality of images of an unhealthy eye, each having a corresponding one of a plurality of different types of lesions. If at least one of the plurality of different types of lesions is found to be present in the image, the method further comprises the process of selecting, for each of the at least one type of lesion found to be present in the image, a corresponding one of a plurality of different types of measurement modalities to be used to perform a measurement on the portion of the eye; generating, for each of the at least one type of lesion found to be present in the image, corresponding instructions to cause an ophthalmic measurement device of the corresponding selected measurement modality to perform the measurement on the portion of the eye; and receiving measurement data of the measurements performed by the ophthalmic measurement device of each selected measurement modality.

[0005] The inventors of the present invention have further invented a computer program according to a third exemplary aspect herein, which, when executed by a computer, causes the computer to perform a method according to the first exemplary aspect or the second exemplary aspect herein.

[0006] The inventors of the present invention have further invented, according to a fourth exemplary embodiment herein, a non-transitory computer-readable storage medium storing a computer program according to the third exemplary embodiment herein.

[0007] The inventors of the present invention have further invented, according to a fifth exemplary embodiment herein, a signal carrying a computer program according to the third exemplary embodiment herein.

[0008] The inventors of the present invention have further invented an apparatus for searching for an area indicative of a lesion in an image of an eye portion acquired by an ophthalmic imaging system according to a sixth exemplary embodiment herein. The apparatus includes a receiving module configured to receive image data defining the image. The apparatus further includes a searching module configured to search for the area in the image by processing the received image data with a learning algorithm trained on image data defining a plurality of images of the eye portion of a healthy eye and image data defining a plurality of images of the eye portion of an unhealthy eye, each image having at least one area indicative of the lesion. The apparatus further includes an instruction generating module configured to execute, in response to the area in the image indicative of the lesion found by the searching module, a process of determining a location of the area in the image, and a process of generating instructions for causing an ophthalmic measurement device to perform the measurement on the eye portion using a reference point based on the determined location for setting a location of the measurement on the eye portion and generating measurement data. The receiving module is further configured to receive the measurement data from the ophthalmic measurement device.

[0009] The inventors of the present invention have further invented an apparatus for searching for the presence of a lesion in an image of a portion of an eye acquired by an ophthalmic imaging system according to a seventh exemplary embodiment herein, the apparatus including a receiving module configured to receive image data defining the image, and a searching module configured to search for the presence of at least one of a plurality of different types of lesions in the image by processing the received image data with a learning algorithm trained based on image data defining a plurality of images of a healthy eye and a plurality of images of an unhealthy eye, each image having a corresponding one of a plurality of different types of lesions. The apparatus further includes an instruction generating module configured to perform, in response to at least one of the plurality of different types of lesions found to be present in the image by the searching module, a process of selecting, for each of the at least one type of lesion found to be present in the image, a corresponding one of a plurality of different types of measurement modalities to be used to perform a measurement on the portion of the eye, and a process of generating, for each of the at least one type of lesion found to be present in the image, a corresponding instruction to cause an ophthalmic measurement device of the corresponding selected measurement modality to perform the measurement on the portion of the eye. The receiving module is further configured to receive measurement data of the measurements performed by the ophthalmic measurement device for each selected measurement modality.

[0010] Embodiments of the invention will now be described in detail, by way of non-limiting example only, with reference to the accompanying drawings, in which like reference signs appearing in different figures of the drawing may indicate identical or functionally similar elements, unless otherwise stated, in which: [Brief description of the drawings]

[0011] [Figure 1] 1 is a schematic diagram illustrating an apparatus for searching for regions indicative of a pathology in an image of an eye, according to a first exemplary embodiment herein. [Diagram 2]2 is a block diagram illustrating an example signal processing hardware configuration of the device of FIG. 1 according to one example embodiment described herein. [Diagram 3] 2 is a flow diagram illustrating a process by which the device of FIG. 1 searches for regions indicative of a pathology within an image of the eye, according to a first exemplary embodiment herein. [Figure 4(a)] FIG. 2 is a schematic diagram illustrating an image of a portion of an eye acquired by an ocular imaging system. [Figure 4(b)] FIG. 4(b) is a diagram of the image of FIG. 4(a) with marked areas showing lesions found, according to an exemplary embodiment herein. [Diagram 5] FIG. 1 is a schematic diagram illustrating how a learning algorithm may be trained based on image data defining a plurality of images of a portion of a healthy eye and image data defining a plurality of images of said portion of an unhealthy eye, each of said portions having at least one region indicative of a pathology, according to a first exemplary embodiment herein. [Figure 6] FIG. 11 is a schematic diagram illustrating how a learning algorithm may be trained based on image data defining a plurality of images of a portion of a healthy eye and a plurality of images of said portion of an unhealthy eye, each having at least one region indicative of a corresponding one of a plurality of different types of lesions, according to a second exemplary embodiment herein. [Figure 7] FIG. 1 is a schematic diagram of a convolutional neural network that includes artificial neurons in the input layer, hidden layer, and output layer. [Figure 8] 2 is a flow diagram illustrating a process by which a search module of the device of FIG. 1 may search for regions indicative of a lesion within an image, according to an exemplary embodiment herein. [Figure 9] FIG. 1 is a schematic diagram of an apparatus for searching for the presence of a lesion in an image of an eye, according to an exemplary embodiment herein. [Figure 10] 10 is a flow diagram illustrating a process by which the device of FIG. 9 searches for the presence of a lesion in an image of the eye, according to an exemplary embodiment herein. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] In certain exemplary embodiments described herein, the image of the eye is processed using image processing techniques described below to automatically detect lesions in the image of the eye, and then further measurements of the eye are automatically ordered to obtain supplementary data that allows the presence of the lesion to be determined (and / or the lesion to be characterized) with greater confidence. For example, the eye may be imaged using a different imaging modality (e.g., optical coherence tomography (OCT)) or its functional response to a measured light stimulus. As a further example, in the case of a fluorescein angiogram, where the image processing detects signs of possible vascular leakage in an area of ​​the retina, an OCT scan of the area may be ordered, and thickness or volume measurements may be obtained from the data of the OCT scan to determine the presence and location of macular edema and, optionally, diagnose its severity. Thus, embodiments can facilitate rapid and reliable detection of lesions for early detection of disease, and the advantages of the image of the eye being a wide-field or ultra-wide-field image covering a large portion of the retina or other part of the eye are very clear.

[0013] Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

[0014] [Embodiment 1]

[0015] FIG. 1 is a schematic diagram of an apparatus 100 for searching for a region indicative of a lesion in an image of an eye portion acquired by an ophthalmic imaging system (indicated at 520 in the schematic diagram of FIG. 5) according to a first exemplary embodiment. The apparatus 100 includes a receiving module 110 configured to receive image data defining an image created by the ophthalmic imaging system 520, and a searching module 120 configured to automatically search for at least one region indicative of a lesion in the image by processing the received image data using a learning algorithm, which will be described in detail below. The apparatus 100 further includes an instruction generating module 130 configured to automatically execute, in response to a region in the image indicative of the lesion found by the searching module 120, a process for determining a position of the region in the image, and a process for generating instructions for causing an ophthalmic measurement device to perform the measurement on the eye portion using a reference point based on the determined position for setting the position of the measurement on the eye portion and to generate measurement data. If more than one such region is found by the searching module 120, these processes may be performed for each of the regions. The receiving module 110 is further configured to receive the measurement data from the eye measurement device. Optionally, the device 100 may include, as in the present illustrated embodiment, a display control signal generator 140 (this optional component is shown in dashed lines in FIG. 1).

[0016] A region indicative of a lesion may be a region of the image defined by the received image data that contains image features indicative of damage, structural impairment, or any other abnormality that is not present in an image similarly acquired of a healthy eye.

[0017] The device 100 may be configured to search for areas indicative of any possible pathology within the eye, which may include (but are not limited to) glaucoma, moderate diabetic retinopathy, severe diabetic retinopathy, tumors, fungal masses, edema, and atrophy.

[0018] In this exemplary embodiment, the receiving module 110 is configured to receive image data defining an image of a portion of the retina of the eye generated by the ocular imaging system 520. However, in other exemplary embodiments, the received image data may define an image of a portion of the eye other than the retina, such as, for example, a portion of the anterior segment of the eye, or a portion of the posterior segment of the eye. Furthermore, the received image data may define a two-dimensional image, as in this exemplary embodiment, or alternatively, may define a three-dimensional image of the imaged portion of the eye. The received image data may be provided in any suitable format (which may be compressed or uncompressed) known to those skilled in the art. The image data received by the receiving module 110 is in a first imaging modality (described in more detail below) and represents the result of imaging the retina of the eye using appropriately selected values ​​of imaging parameters, which may include imaging resolution, aperture size, and wavelength.

[0019] The ocular imaging system 520 may be any ocular imaging system suitable for imaging the retina (or other selected portion) of the eye. The ocular imaging system 520 may be, for example, a fundus camera or a type of scanning imaging system. By way of example, the ocular imaging system 520 of the exemplary embodiment is a scanning imaging system configured to acquire an image of the retina of the subject's eye, an exemplary type being a scanning laser ophthalmoscope (SLO). The SLO of the exemplary embodiment is configured to capture an autofluorescence (AF) image (which may also be configured to capture a red-green (RG) reflectance image or an image from another fluorescence mode), but may alternatively or additionally be configured to acquire one or more other types of images. The SLO may be, for example, an ultra-wide field SLO (UWF-SLO) capable of generating an ultra-wide field image of up to 80% of the surface of the retina. Alternatively, the ophthalmic imaging system 520 may be another imaging modality, such as an optical coherence tomography (OCT) scanner, in which case the image processing techniques described herein are applicable to tomographic images acquired by the OCT scanner. As a further alternative, the ophthalmic imaging system 520 may be a combined SLO and OCT scanner, in which case the image processing techniques described herein are applicable to both the SLO retinal scans and the OCT scans acquired by the combined SLO and OCT scanner.

[0020] The receiving module 110 is configured to receive image data defining an image acquired by the ophthalmic imaging system 520 or any suitable means known to one of skill in the art. For example, the receiving module 110 may receive the image data from the ophthalmic imaging system 520 via a direct communication link (which may be provided by any suitable wired or wireless connection, such as a Universal Serial Bus (USB) or Bluetooth connection), or via an indirect communication link (which may be provided by a network, including a Local Area Network (LAN), a Wide Area Network (WAN), and / or the Internet). Furthermore, the image data may be received by the receiving module 110 receiving such image data after it has been acquired by the ophthalmic imaging system (e.g., by being read from a storage medium, such as a CD or hard disk, or by being received over a network, such as the Internet).

[0021] Moreover, the image data may be received by the receiving module 110 while it is being generated by the ophthalmic imaging system (and subsequently processed to search for regions indicative of a pathology within the image of the portion of the eye, as described below). That is, the image data may be obtained "on the fly", without waiting for the ophthalmic imaging system 520 to finish generating all of the image data forming the image of the portion of the retina. However, in the present exemplary embodiment, and for purposes of this specification, the receiving module 110 is configured to receive all of the image data defining the image of the portion of the eye, and then the searching module 120 begins processing the image data.

[0022] In embodiments such as the present exemplary embodiment in which the apparatus 100 includes a display control signal generator 140, the display control signal generator 140 may be configured to generate display control signals for controlling a display device, such as an LCD screen or other type of visual display unit (such as shown at 215 in FIG. 2 ), to display both the location of the area indicative of the lesion within the image of the portion of the eye and a representation of the received measurement data.

[0023] Figure 2 is a schematic diagram of programmable signal processing hardware 200, which may be configured in this exemplary embodiment to function as device 100 of Figure 1. The programmable signal processing hardware 200 includes a communications interface (I / F) 210 for receiving image data as described above, generating instructions for causing an ophthalmic measurement device 300 to perform the measurements on the portion of the eye and generate the measurement data, receiving the measurement data from the ophthalmic measurement device 300, and, optionally, outputting display control signals for controlling a display device 215 to display both the image of the portion of the eye and a display of the measurement data. The signal processing device 200 further comprises a processor 220 (e.g., a central processing unit (CPU) or a graphics processing unit (GPU)), a working memory 230 (e.g., a random access memory) and an instruction storage device 240 for storing a computer program comprising computer readable instructions, which, when executed by the processor 220, causes the processor 220 to perform various functions, including those of the search module 120, the instruction generation module 130 and, optionally, the display control signal generator 140, as described above. The instruction storage device 240 may comprise a ROM (e.g., in the form of an Electrically Erasable Programmable Read Only Memory (EEPROM) or Flash memory) preloaded with computer readable instructions. Alternatively, the instruction storage device 240 may comprise a RAM or similar type of memory, and the computer readable instructions of the computer program may be entered into the RAM or similar type of memory from a computer program product, such as a non-transitory computer readable storage medium 250 in the form of a CD-ROM, DVD-ROM or the like, or from a computer readable signal 260 carrying computer readable instructions. In any case, when the computer program is executed by the processor, the processor performs at least one of the methods of searching for areas indicative of a pathology or the presence of a pathology in an image of a portion of an eye acquired by an ocular imaging system as described herein. It should be noted, however, that the device 100 may alternatively be implemented as non-programmable hardware, such as an application specific integrated circuit (ASIC).

[0024] In the present exemplary embodiment, a combination of hardware components 270 including a processor 220, a working memory 230, and an instruction storage device 240, as shown in Figure 2, are configured to perform the functions of the search module 120 and the instruction generation module 130, which are now described in more detail below. In an embodiment such as the present exemplary embodiment in which the device 100 includes a display control signal generator 140, the functionality of this desired component is also provided by the combination of hardware components 270 and the communications I / F 210.

[0025] As will become clearer from the following description of the operations performed by the device 100 of this exemplary embodiment, the device 100 automatically processes image data defining an image of the portion of the eye acquired by the ocular imaging system 520 to locate and record the location in the image of each area indicative of the lesion, and generates corresponding instructions for the ocular measurement device 300 to perform the measurement on the portion of the eye using reference points based on the recorded locations of the corresponding areas to set the corresponding locations of the measurement on the portion of the eye, and generate measurement data. Thus, the device 100 acquires both the location of each image area suspected of containing a lesion and additional measurement data regarding the corresponding area. These different but complementary types of information acquired automatically by the device 100 can be presented together for examination by a physician, which may enable the identification of any lesion-containing areas of the received ocular image more quickly and with greater confidence than by examining the ocular image alone. For example, in an exemplary embodiment in which both the recorded location of the region in the image and a representation of the received measurement data are displayed on display device 215 (preferably overlapping one another), a person viewing display device 215 may be able to easily discern whether and where such a region is found in the image, which may be difficult to discern by viewing only one type of information, either the recorded location of the region in the image or a representation of the received measurement data, with only one of these being displayed on display device 215. Similar advantages may be obtained in alternative exemplary embodiments in which the recorded location of the region in the image and the received measurement data are automatically processed to identify regions of interest.

[0026] FIG. 3 is a flow diagram illustrating a process by which the device 100 of FIG. 1 searches for regions indicative of a pathology within an image of a portion of an eye acquired by an ocular imaging system 520, according to a first exemplary embodiment herein.

[0027] In process S10 of Fig. 3, the receiving module 110 receives image data of a first imaging modality acquired by an ophthalmic imaging system 520. The imaging modality of the ophthalmic imaging system may be one of many different types known to those skilled in the art, such as, for example, OCT, color fundus photography, fluorescein fundus photography (FA), indocyanine green photography (ICG), and autofluorescence (AF). The received image data defines an image of a portion of the eye (in this exemplary embodiment, the retina) acquired by the ophthalmic imaging system 520. Fig. 4(a) is a schematic diagram of an image 400 of the portion of the retina defined by the received image data acquired by the ophthalmic imaging system 520.

[0028] In process S12 of Figure 3, search module 120 searches for a region indicative of a type of pathology (410 in Figure 4(a)) in image 400 by processing the received image data using a learning algorithm trained on image data defining a plurality of images of a retina of a healthy eye and a plurality of images of a retina of an unhealthy eye, each of which has at least one region indicative of a pathology. The location of each region 410 found in the search within image 400 is recorded by search module 120, for example in working memory 230, when device 100 is implemented in programmable signal processing hardware 200 of Figure 2.

[0029] FIG. 5 is a schematic diagram illustrating how a learning algorithm may be trained based on image data 502 defining multiple images of a healthy eye and image data 501 defining multiple images of an unhealthy eye. A learning algorithm 530 is configured to learn from the input data by building a model 540 from an exemplary training set 500 of input data including image data 502 defining multiple images of the retina of a healthy eye and image data 501 defining multiple images of the retina of an unhealthy eye and to make predictions based on said input data. By way of example, image data 501 defines multiple images of the portion of an unhealthy eye, each image having an area indicative of severe diabetic retinopathy. The multiple images defined by image data 501 in the exemplary training set 500 may be collected by acquiring images of the retinas of multiple subjects. More generally, each image relates to the same portion of the eye (or substantially the same portion of eye 510, or a portion of the eye including the same portion) as an image defined by the received image data. Moreover, each image defined by the image data in the exemplary training set 500 is acquired by an ophthalmic imaging system 520, or the same type of ophthalmic imaging system operating in the same imaging modality. By way of example, the image data 501 defines multiple images of the portion of an unhealthy eye, each image having an area indicative of severe diabetic retinopathy.

[0030] In an embodiment where the learning algorithm 530 is a supervised learning algorithm (e.g., a neural network, a support vector machine, or an evolutionary algorithm), each example image in the example training set 500 is a pair of input image data defining the image of the portion of the eye and a desired output value indicating whether the image relates to a "healthy" or "unhealthy" portion of the eye. The supervised learning algorithm 530 analyzes the image data in the example training set 500 to create a model 540 that can be used to classify new first-look image data defining the image of the portion of the eye as "healthy" or "unhealthy". Because the learning algorithm 530 is trained on image data 501 defining multiple images of an unhealthy eye having only one type of pathology, the model 540 cannot distinguish between multiple pathologies. The model 540 can only determine whether there is a region indicative of a pathology, e.g., severe diabetic retinopathy.

[0031] 3, the search module 120 searches for regions 410 in the image that, instead of showing only one type of lesion, show one of a plurality of different types of lesions in a variation of this exemplary embodiment, and searches for regions 410 in the image by processing the received image data with the learning algorithm. Thus, in the variation of this exemplary embodiment, the learning algorithm may be trained based on image data defining a plurality of images of the portion of an unhealthy eye, each having a corresponding one of the plurality of different types of lesions.

[0032] 6 is a schematic diagram illustrating how a modified learning algorithm 630 may be trained based on image data 604 defining a plurality of images of a retina of a healthy eye and image data 601, 602, 603 defining a plurality of images of a retina of an unhealthy eye, each having at least one region exhibiting a corresponding one of a plurality of different types of pathology. By way of example, image data 601 defines a plurality of images of the portion of an unhealthy eye having glaucoma, image data 602 defines a plurality of images of the portion of an unhealthy eye having moderate diabetic retinopathy, and image data 603 defines a plurality of images of the portion of an unhealthy eye having severe diabetic retinopathy.

[0033] The learning algorithm 630 is configured to learn from and make predictions based on input data by constructing a model 640 from an exemplary training set 600 of input data, including image data 604 defining a plurality of images of a retina of a healthy eye and image data 601, 602, 603 defining a plurality of images of a retina of an unhealthy eye. The plurality of images defined by image data 601-604 in the exemplary training set 600 may be collected by acquiring images of the retina of multiple subjects. More generally, each image relates to the same portion of the eye (or substantially the same portion of the eye 510, or a portion of the eye including the same portion) as the image defined by the received image data. Moreover, each image defined by image data 610-604 in the exemplary training set 600 is acquired by the ophthalmic imaging system 520, or the same type of ophthalmic imaging system operating in the same imaging modality.

[0034] The learning algorithm 630 may be a supervised learning algorithm. Thus, each example image of the portion of the eye 510 in the example training set 600 is associated with an indicator indicating whether the image is of a "healthy" or "unhealthy" eye portion, and if the image is of an "unhealthy" eye portion, a second indicator indicating which one of the pathologies is present in the image. The supervised learning algorithm analyzes the image data in the example training set 600 to create a model 640 that can be used to classify new (first seen) image data defining the image of the portion of the eye as one of, for example, "healthy", "unhealthy-glaucoma", "unhealthy-moderate diabetic retinopathy", and "unhealthy-severe diabetic retinopathy".

[0035] As will be apparent to one skilled in the art, the device 100 may be adapted to classify additional lesions by expanding the training set 600 to include, for each additional lesion, image data defining a plurality of images of the retina (or other portion) of an unhealthy eye having the lesion and the associated indices described above. For example, the training set 600 may be expanded to include image data defining a plurality of images of the portion of an unhealthy eye having a tumor and / or image data defining a plurality of images of the portion of an unhealthy eye having an edema. Furthermore, any of the image data 601, 602, 603 defining a plurality of images of the portion of an unhealthy eye may be replaced or substituted with image data defining a plurality of images of the portion of an unhealthy eye having a different lesion. A modified version of the model 640 may then be created based on the modified training set 600.

[0036] The supervised learning algorithm 630 may be, as in this exemplary embodiment, a neural network that automatically generates specific features without requiring any prior knowledge by processing the input data, such as the image data in the exemplary training set 600.

[0037] The neural network may be, as in this exemplary embodiment, a convolutional neural network, which is particularly suited for image and video recognition tasks.

[0038] As illustrated in FIG. 7, a convolutional neural network generally consists of an input layer, an output layer, and multiple hidden layers. Each of these layers consists of multiple artificial neurons (A-F in FIG. 7), and each layer may perform a different type of transformation on its input. Each artificial neuron may be connected to multiple artificial neurons in adjacent layers. The output of each artificial neuron may be calculated by some nonlinear function of the sum of the inputs to each artificial neuron. Artificial neurons and connections between artificial neurons typically have corresponding weights (WAD, WAE, etc. in FIG. 7) that determine the signal strength at a given connection. These weights are adjusted as learning progresses, and thus the output of the convolutional neural network is adjusted accordingly. Signals may flow from the first layer (the input layer) to the last layer (the output layer), traversing each layer multiple times.

[0039] The output of the neural network may be considered as a probability that the input image data contains the particular characteristic of the lesion, and the classification may, in this exemplary embodiment, include determining whether the output of trained model 640 exceeds a predefined threshold value, which may represent an acceptably low probability that the input image data contains the particular characteristic of the lesion, and thus a high probability that the subject's eye is healthy.

[0040] If the learning algorithm is a neural network, as in the present exemplary embodiment, the search module 120 may be configured to search for regions 410 indicative of the lesion in the image by decomposing the neural network, as part of process S12 of FIG. 3.

[0041] When image data defining the image 410 of the portion of the eye is input to the trained model 640, the trained model 640 classifies the image as either "healthy" or "unhealthy" with a particular pathology (i.e., the model 640 determines whether the input image data contains a particular characteristic of a pathology). A conventional convolutional neural network does not output an explanation for this classification. Thus, the convolutional neural network can be decomposed, i.e., processed, to determine which input variables of the input image data (i.e., the pixels of the image 400) are relevant to the output of the neural network. The input variables most relevant to the classification of the image 400 defined by the received image data as "unhealthy" with a pathology correspond to the region 410 (or regions) indicative of the pathology.

[0042] If the trained model 640 classifies the image 400 defined by the received image data as "healthy" (i.e., not containing any areas indicative of a pathology), then the search module 120 may be configured, as in the present exemplary embodiment, to not decompose the neural network.

[0043] In embodiments, such as this exemplary embodiment, where the neural network is a convolutional neural network, search module 120 may be configured to decompose the convolutional neural network by performing the process illustrated in the flow diagram of FIG. 8, as in this exemplary embodiment.

[0044] At S122 of FIG. 8, the search module 120 performs, for each of a number of different sections of the image 400, a process of masking the section of the image to generate a mask image.

[0045] At S124 of FIG. 8, the search module 120 performs a process for searching for the region within the mask image by processing image data defining the mask image using the learning algorithm for each of a number of different sections of the image 400.

[0046] In S126 of FIG. 8, the search module 120 performs a process to determine, for each of a number of different sections of the image 400, a difference between a first result which is the result of a search performed using the image data defining the mask image and a second result which is the result of a search performed using the received image data (i.e., the received image data in which the section of the image is not masked).

[0047] In S128 of FIG. 8, the search module 120 determines, as the location of the region 410 in the image 400, the location of the segment in which the determined difference is maximum.

[0048] Alternatively, if the neural network is a convolutional neural network, the search module 120 may:

[0049] (i) applying a Taylor decomposition to each layer of the neural network from the top layer of the neural network to the input layer of the neural network to determine the relationship of each input variable of the neural network to the output of the neural network;

[0050] (ii) determining locations to be recorded based on at least one section of the received image data that corresponds to a most relevant input variable of the neural network;

[0051] The method may be configured to decompose the convolutional neural network.

[0052] Taylor decomposition is a method of explaining the predictions of individual neural networks by decomposing the output of the neural network based on its input variables. In this method, an image 400 is represented as x={x pThe image 400 is considered as a set of pixel values ​​{x, y, z}. The function f(x) quantifies the presence in the image 400 of a certain type of object, here a region 410 indicative of a lesion. A function value f(x)=0 indicates the absence of an object, while a function value f(x)>0 indicates the presence of an object with some degree of certainty or amount. The Taylor decomposition method calculates for each pixel p in the image a relevance score R p (x), which indicates how much pixel p contributes to explaining the classification decision f(x) for image x. The relevance of each pixel is calculated using a function R(x) = {R p The location of the region can then be determined as the location of the most relevant pixel in the heat map.

[0053] As a further alternative, if the neural network is a convolutional neural network, the search module 120 may be configured to decompose the convolutional neural network by performing a deconvolution of the convolutional neural network. In particular, the convolutional neural network may be decomposed by training a multi-layer deconvolutional network that receives as input the output of the convolutional neural network and provides as output a probability map of the same size as the image input to the convolutional neural network indicating the probability of each pixel belonging to one of the predefined classes. The location of the region may thus be determined as the location of the pixel that has the highest probability of belonging to one of the predefined classes (i.e., has the highest probability of having one of the multiple different types of lesions).

[0054] Thus, in this exemplary embodiment, the search module 120 may be configured to search for a region 410 indicative of a lesion within the image by identifying pixels of the image 400 defined by the received image data that are most associated with the output of the neural network as the region 410 indicative of a lesion.

[0055] 3, in process S13, search module 120 determines whether an area in the image that is indicative of the lesion was found as a result of the search performed in process S12. If an area in the image that is indicative of the lesion was found as a result of the search performed in process S12, the process proceeds to S14. If not, the process ends.

[0056] In process S14 of Figure 3, the instruction generation module 130 determines the location in the image of the region 410 found by the search module 120. The location may be recorded, for example, by the search module 120 in the working memory 230 of the hardware configuration 200 illustrated in Figure 2, and may be read from the working memory by the instruction generation module 130 in S14.

[0057] In an embodiment such as the present exemplary embodiment in which the results of the process of the device 100 are displayed on a display device 215 as described herein, the determined location as illustrated in FIG. 4(a) may be at the center or other location of an overlaid visual element, such as a bounding box 440 as illustrated in FIG. 4(b), which is overlaid on a region 410 of the image 400 found to contain the lesion.

[0058] In process S16 of Figure 3, the instruction generation module 130 generates instructions for causing the ophthalmic measurement device 300 to perform the measurement on the portion of the eye using a reference point based on the determined position to set the position of the measurement on the portion of the eye and generate measurement data. The instructions generated in process S16 of Figure 3 may be provided to the ophthalmic measurement device 300 by any suitable means known to those skilled in the art.

[0059] In an embodiment such as the present example embodiment, the determined location is a location within image 400 of one or more groups of pixels that make up region 410. To determine a reference point that can be used to set the location of the measurement performed by ocular measurement device 300 on the portion of the eye, the location recorded by search module 120 at S12 can be converted into a corresponding set of one or more control parameters that operate ocular measurement device 300 to perform the measurement at substantially the same location in / on the eye as imaged by ocular imaging system 520. This can be done in one of a number of different ways.

[0060] For example, the instruction generation module 130 may use a mapping between pixel coordinate values ​​from an image acquired by the SLO and corresponding values ​​of the control parameters, which may be provided, for example, in the form of a look-up table or a function defined by a set of parameters, and the mapping may be determined by calibration using techniques known to those skilled in the art.

[0061] In process S18 of FIG. 3, the receiving module 110 receives the measurement data from the ophthalmic measurement device 300, after which the process ends.

[0062] The ocular measurement device 300 may be any type of device that allows for supplemental measurements of the subject's eye, and such a device may be used to verify that the area found by the search module 120 definitely (or with a high probability) contains the imaged lesion. The ocular measurement device 300 may, for example, measure the functional response of the subject's eye to a light stimulus in response to instructions generated by the instruction generation module 130 and send data representative of the measured functional response to the receiving module 110.

[0063] Alternatively, the ophthalmic measurement device 300 may be configured to perform an image capture process of a second imaging modality different from the first imaging modality (i.e., SLO in this exemplary embodiment) for imaging the area of ​​the retina as the measurement on the retina (or other selected portion of the eye as described above) using the aforementioned reference point for setting the location of the area of ​​the retina to be imaged in the image capture process as the location of the measurement. Thus, in this exemplary embodiment where an area in the image showing the pathology is found in S12 of Fig. 2, the instruction generation module 120 generates, in S16 of Fig. 3, an instruction for the ophthalmic measurement device 300 to perform the measurement on the portion of the eye, an instruction for the ophthalmic measurement device 300 to perform the image capture process using the reference point for setting the location of the area of ​​the portion of the eye to be imaged in the image capture process as the location of the measurement. The receiving module 110 then receives, in process S18 of Fig. 3, the image data of the second imaging modality acquired by the ophthalmic measurement device 300 as the measurement data. It should be noted that the functionality of ocular imaging system 520 and ocular measurement device 300 may be provided by a single multi-modality ocular imaging system capable of acquiring ocular images in two or more different imaging modalities (e.g., a combined SLO and OCT imaging system). Ocular imaging system 520 and said ocular measurement device may alternatively be provided as separate components.

[0064] The mapping described above may depend on the subject's eye being positioned and oriented in a predetermined manner relative to the ocular measurement device. This may be a good assumption in any practical application, but may not be true in all cases. If this assumption is not true, it may be preferable to determine the location of the region found by the search module 120 with respect to an anatomical feature (e.g., the fovea) that is recognizable in both the image acquired by the ocular imaging system 520 and the different modality image acquired by the ocular measurement device 300 of the present exemplary embodiment. The location of the region with respect to the location of the fovea in the image 400 may then be transformed into a location that corresponds to the location of the fovea in the image acquired by the ocular measurement device 300.

[0065] The reference point may be used to position the measurement relative to the portion of the eye in any suitable manner, for example, the eye measurement device 300 may be controlled to image (or, more generally, perform any other type of measurement) an area of ​​a predetermined size that is centered on a location of the subject's retina that corresponds to the reference point.

[0066] In the variation of the first exemplary embodiment described above, in which the search module 120 searches for a region 410 in the image that shows one of a plurality of different types of lesions by processing the received image data using the learning algorithm, the instruction generation module 130 executes a process of selecting, depending on the region 410 in the image 400 that shows one of the plurality of different types of lesions found by the search module 120, a corresponding one of a plurality of different types of imaging modalities to be used for performing the image capture process on the retina of the eye for one of the plurality of different types of lesions and as the second imaging modality. In the variation of the first embodiment, the instruction generation module 130 may, for example, refer to a look-up table (LUT) in which respective indices of the plurality of different types of lesions are stored in relation to a corresponding indices of one of the plurality of different types of imaging modalities, and select the corresponding one of the plurality of different types of imaging modalities to be used for performing the image capture process on the subject's retina using the one of the plurality of different types of lesions found by the search module 120 as a search key.

[0067] For example, if the one of the multiple different types of pathology is glaucoma, instruction generation module 130 may generate instructions to cause eye measurement device 300 to perform an OCT scan of a region of the retina of the subject's eye as the image capture process of the second imaging modality. Receiving module 110 then receives image data of the OCT scan as the measurement data in S18 of FIG. 3, and device 100 may process the image data of the OCT scan to obtain complementary data useful for detecting (and, optionally, estimating the severity of) glaucoma, i.e., measurements of the nerve fiber layer and / or optic disc of the retina of the eye.

[0068] As a further example, if the one of the plurality of different types of lesions is severe diabetic retinopathy, instruction generation module 130 may generate instructions to cause ocular measurement device 300 to perform an OCT scan of a region of the retina of the eye as the image capture process of the second imaging modality. Receiving module 110 then receives image data of the OCT scan as the measurement data in S18 of FIG. 3, and device 100 can process the image data of the OCT scan to obtain complementary data useful for detecting (and, optionally, estimating the severity of) severe diabetic retinopathy, namely, a measurement of macular thickness.

[0069] As another example, if the one of the plurality of different types of lesions is a tumor, the instruction generation module 130 may generate instructions to cause the ocular measurement device 300 to perform a high density OCT B-scan on a region of the retina of the eye as the image capture process of the second imaging modality. The receiving module 110 may then receive image data of the high density OCT B-scan as the measurement data in S18 of FIG. 3, which may be useful for detecting (and, optionally, estimating the size of) a tumor.

[0070] As a further example, if the one of the multiple different types of lesions is a fungal mass, the instruction generation module 130 may generate an instruction to cause the eye measurement device 300 to perform an OCT B-scan on a region of the retina of the eye as the image capture process of the second imaging modality. The receiving module 110 may then receive image data of the OCT B-scan as the measurement data in S18 of Fig. 3, which may be useful for detecting the presence of or estimating the size and / or number of fungal masses. This data may be useful, for example, for diagnosing early stage age-related macular degeneration.

[0071] Furthermore, if the one of the plurality of different types of lesions is edema or atrophy, the instruction generation module 130 may generate instructions to cause the ocular measurement device 300 to perform an OCT scan of a region of the retina of the eye as the image capture process of the second imaging modality. The receiving module 110 may then receive image data of the OCT scan as the measurement data in S18 of FIG. 3, which may be useful for detecting edema (or atrophy, as the case may be), and estimating its severity, if desired.

[0072] In the modification of the first exemplary embodiment described above, the search module 120 searches for an area 410 in the image showing one of a plurality of different types of lesions by processing the received image data using the learning algorithm, and the instruction generation module 130 executes a process of selecting, depending on the area 410 in the image 400 showing one of the plurality of different types of lesions found by the search module 120, a corresponding one of a plurality of different types of imaging modalities to be used for the one of the plurality of different types of lesions and for performing an image capture process on the retina of the eye as the second imaging modality. However, as mentioned above, the ocular measurement device 300 does not have to be configured to acquire an image of the retina (or, more generally, any other selected part of the eye), but may instead perform different types of measurements on the eye. Furthermore, the ocular measurement device 300 may be multi-modal in the sense that it is operable in a selected one of a plurality of different measurement modalities. In such a variant, the instruction generation module 130 may perform a process of selecting a corresponding one of a plurality of different types of measurement modalities for a measurement to be performed on the eye for the one of the plurality of different types of lesions in response to an area 410 in the image 400 showing one of the plurality of different types of lesions found by the search module 120, and generating an instruction for the eye measurement device 300 of the selected measurement modality to perform the measurement on the portion of the eye using a reference point for setting the position of the measurement with respect to the portion of the eye as the instruction to cause the eye measurement device 300 to perform the measurement on the portion of the eye.

[0073] In a second variant of the first exemplary embodiment, the ocular measurement device 300 may be an ocular imaging device of the same imaging modality as the ocular imaging system 520, which does not distinguish between different types of lesions as in the first embodiment. In this second variant, the receiving module 110 is configured to receive image data representing the result of imaging the portion of the eye using a first value of an imaging parameter, which is the imaging resolution, aperture size or wavelength used in the imaging. The instruction generating module 130 is configured to execute a process of generating, depending on an area 410 in the image 400 showing a lesion found by the searching module 120, an instruction for the ocular measurement device 300 to perform the measurement on the portion of the eye, using a second value of the imaging parameter for imaging the area of ​​the portion of the eye, different from the first value of the imaging parameter, and again using the reference point for setting the position of the area of ​​the portion of the eye to be imaged in the image capturing process as the position of the measurement. The receiving module 110 is configured to receive, as the measurement data from the eye measurement device 300, the image data representing the result of imaging the area of ​​the portion of the eye using the second value of the imaging parameter, and the received image data and the received measurement data are in the same imaging format.

[0074] For example, in the second variant, the ocular imaging system 520 may be configured to acquire image data defining the image by imaging the portion of the eye at a first imaging resolution, and the instruction generation module 130 may be configured to generate instructions to cause the ocular measurement device 300 to perform an image capture process at a second, higher imaging resolution to image the area in more detail, in response to an area within the image indicative of a lesion found by the search module 120.

[0075] [Second embodiment]

[0076] Fig. 9 is a schematic diagram of an apparatus 800 for searching for the presence of a lesion in an image of an eye portion acquired by an ocular imaging system (520 in Fig. 5) according to a second exemplary embodiment herein. The apparatus 800 of the second exemplary embodiment differs from the apparatus 100 of Fig. 1 in that the apparatus of Fig. 9 does not need to determine the location of an area indicative of a lesion in the received image 400.

[0077] The apparatus 800 includes a receiving module 810 configured to receive image data defining an image created by the ophthalmic imaging system 520, and a searching module 820 configured to search for the presence of at least one of a plurality of different types of lesions in the image. The apparatus 800 further includes an instruction generating module 830 configured to perform, in response to at least one of the plurality of different types of lesions found to be present in the image by the searching module 820, a process of selecting, for each of the at least one type of lesion found to be present in the image, a corresponding one of a plurality of different types of measurement modalities to be used to perform a measurement on the portion of the eye, and a process of generating, for each of the at least one type of lesion found to be present in the image, a corresponding instruction to cause an ophthalmic measurement device of the corresponding selected measurement modality to perform the measurement on the portion of the eye. The receiving module 810 is further configured to receive measurement data of the measurements performed by the ophthalmic measurement device of each selected measurement modality. Optionally, the apparatus 800 may include a display control signal generator 840 (this optional component is shown in dashed lines in FIG. 9 ) as in this exemplary embodiment.

[0078] The device 800 may be configured to search for the presence of any pathology (including, for example, glaucoma, moderate diabetic retinopathy, severe diabetic retinopathy, tumors, fungal masses, edema, and atrophy) as described above with respect to the first embodiment.

[0079] In this exemplary embodiment, the receiving module 810 is configured to receive image data defining an image of a portion of the retina of the eye generated by the ocular imaging system 520. However, in other exemplary embodiments, the received image data may define an image of a portion of the eye other than the retina, such as, for example, a portion of an anterior segment of the eye, or a portion of a posterior segment of the eye.

[0080] The ophthalmic imaging system 520 may be a scanning laser ophthalmoscope as in this exemplary embodiment, or alternatively, the ophthalmic imaging system may be any of the ophthalmic imaging systems described above with respect to the first embodiment.

[0081] The receiving module 810 may be configured to receive image data defining an image acquired by the ocular imaging system 520 by any of the means described above with respect to the first embodiment.

[0082] As will become more apparent from the following description of the operations performed by the device 800 of this exemplary embodiment, the device 800 automatically processes image data defining an image of the portion of the eye acquired by the ophthalmic imaging system 520, searches for the presence of at least one of a plurality of different types of lesions in the image, and, if at least one of the plurality of different types of lesions is found to be present in the image, selects, for each of the at least one type of lesion found to be present in the image, a corresponding one of a plurality of different types of measurement modalities to be used to perform a measurement on the portion of the eye. Corresponding instructions to an ophthalmic measurement device of the corresponding selected measurement modality to perform the measurement on the portion of the eye are then generated for each of the at least one type of lesion found to be present in the image. Thus, the device 800 acquires both the image suspected of containing a particular type of lesion and additional measurement data of a measurement modality related to the suspected type of lesion. These different but complementary types of information automatically acquired by the device 800 may enable rapid and reliable identification of eye images containing any lesion.

[0083] The apparatus 800 of FIG. 9 may be implemented by a signal processing hardware configuration such as that shown in FIG. 2, or by any other suitable means.

[0084] FIG. 10 is a flow diagram illustrating a process by which the device 800 of FIG. 9 searches for the presence of a lesion in an image of a portion of the eye acquired by the eye imaging system 520, according to a second exemplary embodiment herein.

[0085] In process S20 of FIG. 10, a receiving module 810 receives image data defining the image of the portion of the eye acquired by the SLO (as an example of an ophthalmic imaging system 520).

[0086] In process S22 of Figure 10, search module 820 searches for the presence of at least one of a plurality of different types of lesions in the images by processing the received image data with a learning algorithm trained on image data defining a plurality of images of a healthy eye and a plurality of image data of an unhealthy eye, each having a corresponding one of a plurality of different types of lesions. The learning algorithm may be trained to classify input image data defining images of the portion of the eye as either "healthy" or "unhealthy" and having a particular lesion, as described above with respect to Figure 6.

[0087] The device 800 may not record the location of regions indicative of lesions within the image, and thus the device 800 may not process the learning algorithm to determine which input variables of the input image data (i.e., pixels of the image defined by the received image data) were associated with an output (i.e., finding one of a plurality of different types of lesions to be present), as in this exemplary embodiment. Alternatively, in other embodiments, such processing may be performed by the device 800 as part of S22 of FIG. 10 to identify fiducial points, if desired.

[0088] In process S23, the search module 820 determines whether at least one of the multiple different types of lesions was found to be present in the image as a result of the search performed in process S22. If at least one of the multiple different types of lesions was found to be present in the image as a result of the search performed in process S22, the process proceeds to S24. If not, the process ends.

[0089] In process S24 of FIG. 10, in response to at least one of the multiple different types of lesions found to be present in the image by search module 820, instruction generation module 830 executes a process to select, for each of the at least one type of lesion found to be present in the image, a corresponding one of multiple different types of measurement modalities to be used to perform measurements on the portion of the eye.

[0090] The instruction generation module 830 may select a corresponding one of a plurality of different types of measurement modalities by any of the means described above with respect to the apparatus 100 of FIG.

[0091] In process S26 of FIG. 10, in response to at least one of the multiple different types of lesions found to be present in the image by search module 820, instruction generation module 830 executes a process to generate, for each of the at least one type of lesion found to be present in the image, corresponding instructions for an eye measurement device of the corresponding selected measurement modality to perform the measurement on the portion of the eye.

[0092] The instructions may be generated substantially as described above with respect to the apparatus 100 of Figure 1. However, in embodiments such as this exemplary embodiment in which the apparatus 800 does not search for areas indicative of a lesion, the instruction generation module 830 need not use a reference point based on the recorded location of an area indicative of a lesion to position the measurement relative to the portion of the eye. As part of process S26 of Figure 10, the instruction generation module 130 may, as in this exemplary embodiment, generate, for each of the at least one type of lesion found to be present in an image, corresponding instructions to the ophthalmic measurement device of the corresponding selected measurement modality to perform the measurement on the same portion of the eye as imaged by the ophthalmic imaging system.

[0093] In process S28 of Figure 10, a receiving module 810 receives measurement data for the measurements performed by the ophthalmic measurement device for each selected measurement modality. The receiving module 810 may receive the measurement data by any of the means described above with respect to the first embodiment.

[0094] Some of the above described embodiments are summarized in Examples E1 to E52 below.

[0095] E1. A computer-implemented method for searching for a region (410) indicative of a pathology within an image (400) of an eye portion acquired by an ophthalmic imaging system (520), the method comprising:

[0096] receiving (S10) image data defining said image (400);

[0097] searching (S12) for said region (410) in said image (400) by processing said received image data with a learning algorithm (530; 630) trained on image data (502; 604) defining a plurality of images of said portion of a healthy eye and on image data (601, 602, 603; 501) defining a plurality of images of said portion of an unhealthy eye, each image having at least one region indicative of said pathology; and

[0098] If a region (410) within the image (400) that is indicative of the lesion is found in the searching,

[0099] determining (S14) a location of said region (410) within said image (400);

[0100] generating (S16) instructions for causing an ophthalmic measurement device (300) to perform the measurement on the portion of the eye using a reference point based on the determined position for positioning the measurement on the portion of the eye and generating measurement data; and

[0101] receiving (S18) the measurement data from the ophthalmic measurement device (300);

[0102] A method comprising:

[0103] E2.A computer-implemented method according to E1, comprising:

[0104] Locating (S12) the region (410) in the image (400) comprises: locating a region (410) in the image (400) that is indicative of one of the plurality of different types of lesions by processing the received image data with the learning algorithm (630) that has been trained on image data (601, 602, 603) that defines a plurality of images of the portion of an unhealthy eye, each image having a corresponding one of the plurality of different types of lesions;

[0105] If a region (410) in the image (400) that shows one of the plurality of different types of lesions is found in the searching (S12),

[0106] the method further comprising selecting, for the one of the plurality of different types of lesions, a corresponding one of a plurality of different types of measurement modalities for a measurement to be performed on the eye;

[0107] The method includes generating (S16) instructions for causing the eye measurement device (300) of the selected measurement modality to perform the measurement on the portion of the eye using the reference point to set the position of the measurement on the portion of the eye as the instructions for causing the eye measurement device (300) to perform the measurement on the portion of the eye.

[0108] E3.A computer-implemented method according to E1, comprising:

[0109] the received image data is image data in a first imaging modality;

[0110] the ophthalmic measurement device (300) performs an image capture process of a second imaging modality for imaging an area within the portion of the eye, different from the first imaging modality, as the measurement for the portion of the eye, and obtains image data of the second imaging modality as the measurement data;

[0111] If a region (410) in the image (400) indicative of the lesion is found in the searching (S12), the method further comprises:

[0112] generating (S16) instructions for the ophthalmic measurement device (300) to perform the measurement on the portion of the eye, for the ophthalmic measurement device (300) to perform the image capture process using the reference point to set the position of an area within the portion of the eye to be imaged in the image capture process as the measurement location; and

[0113] receiving (S18) from the ophthalmic measurement device (300) as the measurement data image data of the second imaging modality defining an image of the area within the portion of the eye.

[0114] E4. A computer-implemented method according to E3, comprising:

[0115] Locating (S12) the region (410) within the image (400) comprises: locating a region (410) within the image (400) that is indicative of one of the plurality of different types of lesions by processing the received image data with the learning algorithm (630) that has been trained based on the image data (601, 602, 603) that defines a plurality of images of the portion of an unhealthy eye, each image having a corresponding one of a plurality of different types of lesions;

[0116] The method further includes, if a region (410) in the image (400) showing one of the plurality of different types of lesions is found in the searching (S12), selecting a corresponding one of a plurality of different types of imaging modalities for the one of the plurality of different types of lesions and as the second imaging modality to be used to perform the image capture process on the portion of the eye.

[0117] E5.A computer-implemented method according to E4, comprising:

[0118] If the one of the plurality of different types of lesions is glaucoma, the method includes generating (S16) instructions to cause the ocular measurement device to perform an optical coherence tomography (OCT) scan of the area within the portion of the eye as the image capture process of the second imaging modality and to obtain image data of the OCT scan as the measurement data;

[0119] If the one of the plurality of different types of lesions is severe diabetic retinopathy, the method includes generating (S16) instructions to cause the ocular measurement device to perform an optical coherence tomography (OCT) scan of a region of the retina of the eye as the image capture process of the second imaging modality and to obtain image data of the OCT scan as the measurement data;

[0120] If the one of the plurality of different types of lesions is a tumor, the method includes generating (S16) instructions to cause the ophthalmic measurement device to perform a high density optical coherence tomography (OCT) B-scan of the area within the portion of the eye as the image capture process of the second imaging modality and obtain image data of the high density OCT B-scan as the measurement data;

[0121] If the one of the plurality of different types of lesions is a bacterial mass, the method includes generating (S16) instructions to cause the ophthalmic measurement device to perform an Optical Coherence Tomography (OCT) B-scan of the area within the portion of the eye as an image capture process of the second imaging modality and obtain image data of the OCT B-scan as the measurement data;

[0122] If the one of the plurality of different types of lesions is edema or atrophy, the method includes generating (S16) instructions to cause the eye measurement device to perform an optical coherence tomography (OCT) scan of the area within the portion of the eye as an image capture process of the second imaging modality, and acquiring image data of the OCT scan as the measurement data.

[0123] A computer-implemented method as described in E6.E1, wherein if the area (410) in the image (400) indicative of the lesion is found in the searching, the method includes generating (S16) instructions for causing the ophthalmic measurement device to measure a functional response of the eye to a light stimulus using the reference point for setting a position of the measurement based on the determined position as the instructions for causing the ophthalmic measurement device to perform the measurement on the portion of the eye.

[0124] A computer-implemented method according to E7.E1, comprising:

[0125] the received image data represents a result of the imaging of the portion of the eye using a first value of an imaging parameter, the first value being one of an imaging resolution, an aperture size, and a wavelength used in imaging;

[0126] the ophthalmic measurement device is configured to perform an image capture process as the measurement for the portion of the eye using a second value of the imaging parameter for imaging an area within the portion of the eye, the second value being different from the first value of the imaging parameter, and to obtain as the measurement data image data representative of the imaging of the area using the second value of the imaging parameter, the received image data and the obtained image data being of the same imaging modality;

[0127] If a region (410) within the image (400) indicative of the lesion is found in the searching, the method further comprises:

[0128] generating (S16) instructions for the ophthalmic measurement device to perform the measurement on the portion of the eye, for the ophthalmic measurement device to perform the image capture process using the reference point to set a location of an area within the portion of the eye to be imaged in the image capture process as a location of the measurement; and

[0129] receiving (S18) from the ophthalmic measurement device as the measurement data, the image data representative of the result of imaging the area (410) within the portion of the eye using the second value of the imaging parameter;

[0130] A method comprising:

[0131] E8.A computer-implemented method according to any of E1 to E7, further comprising generating instructions for controlling a display unit (215) to display a display of the position of the region (410) within the image (400) of the portion of the eye and the received measurement data.

[0132] E9. The computer-implemented method of any of E1-E8, wherein the learning algorithm (530; 630) is a supervised learning algorithm.

[0133] E10.A computer-implemented method as described in E9, wherein the supervised learning algorithm includes a neural network, and the region (410) indicative of the lesion is located within the image (400) by decomposing the neural network.

[0134] E11. The computer-implemented method of E10, comprising:

[0135] The neural network is a convolutional neural network, and the neural network performs the following steps for each of a plurality of different partitions of the image (400) defined by the received image data:

[0136] a process of masking (S122) the section of the image (400) to generate a mask image;

[0137] searching for the region in the mask image by processing image data defining the mask image with the learning algorithm (S124);

[0138] determining (S126) a difference between a result of the search performed using the image data defining the mask image and a result of a search performed using the received image data; and

[0139] determining (S128) the location of the segment in which the determined difference is maximum as the location of the region (410) in the image (400).

[0140] E12. The computer-implemented method of E10, comprising:

[0141] The neural network is a convolutional neural network,

[0142] determining the association of each input variable of the neural network to the output of the neural network by applying a Taylor decomposition to each layer of the neural network, from the top layer of the neural network to the input layer of the neural network; and

[0143] determining a location of the region (410) within the image (400) based on at least one section of the received image data that corresponds to a most relevant input variable of the neural network.

[0144] E13. The computer-implemented method of E10, wherein the neural network is a convolutional neural network, and the convolutional neural network is decomposed by performing a deconvolution of the convolutional neural network.

[0145] E14. A computer-implemented method for searching for the presence of a lesion in an image (400) of an eye portion acquired by an ophthalmic imaging system (520), the method comprising:

[0146] receiving (S20) image data defining said image (400);

[0147] searching for the presence of at least one of the plurality of different types of lesions in the images (400) by processing the received image data with a learning algorithm (630) trained on image data defining a plurality of images (502) of a healthy eye and a plurality of images (601, 602, 603) of an unhealthy eye, each having a corresponding one of the plurality of different types of lesions; and

[0148] if at least one of the plurality of different types of lesions is found to be present in the image (400);

[0149] selecting (S24) for each of at least one type of lesion found to be present in the image (400) a corresponding one of a plurality of different types of measurement modalities to be used to perform measurements on the portion of the eye;

[0150] generating (S26) for each of the at least one type of lesion found to be present in the image (400) corresponding instructions to cause an ophthalmic measurement device (300) of the corresponding selected measurement modality to perform the measurement on the portion of the eye; and

[0151] receiving (S28) measurement data of the measurements performed by the ophthalmic measurement device (300) for each selected measurement modality;

[0152] A method comprising:

[0153] E15. The computer-implemented method of E14, comprising:

[0154] if at least one of the plurality of different types of lesions is found to be present in the image (400);

[0155] and for each of the at least one of the plurality of different types of lesions found to be present in the image, processing the received image data with the learning algorithm (630) to search for a corresponding region (410) in the image (400) indicative of the corresponding type of lesion, and recording the location of the corresponding region (410) in the image (400);

[0156] A method in which corresponding instructions are generated to cause the eye measurement device (300) of each selected measurement modality to perform the measurement on the eye using a reference point for positioning the measurement based on the corresponding recorded position.

[0157] E16. The computer-implemented method of E15, comprising:

[0158] if at least one of the plurality of different types of lesions is found to be present in the image (400);

[0159] For each of the at least one of the plurality of different types of lesions found to be present in the image (400),

[0160] a corresponding one of a plurality of different types of imaging modalities to be used for imaging the portion of the eye is selected as the corresponding one of the plurality of different types of measurement modalities;

[0161] corresponding instructions are generated for the ophthalmic measurement device (300) of the selected imaging modality to image the portion of the eye using reference points for positioning the area of ​​the eye to be imaged based on the corresponding recorded positions, as the corresponding instructions for the ophthalmic measurement device (300) of the selected measurement modality;

[0162] The method, wherein image data corresponding to the imaging performed by the ophthalmic measurement device (300) of the selected imaging modality is received as the measurement data of the measurement performed by the ophthalmic measurement device (300).

[0163] E17. The computer-implemented method of E16, comprising:

[0164] If one of the plurality of different types of lesions found to be present in the image (400) is glaucoma, the method further comprises generating (S26) the corresponding instructions for the ophthalmic measurement device (300) to image the portion of the eye in the selected imaging modality, instructions for the ophthalmic measurement device (300) to perform an Optical Coherence Tomography (OCT) scan of the area within the portion of the eye and to obtain image data of the OCT scan as the measurement data;

[0165] If one of the plurality of different types of lesions found to be present in the image (400) is severe diabetic retinopathy, the method further comprises generating (S26) the corresponding instructions for the ophthalmic measurement device (300) of the selected imaging modality to image the portion of the eye, instructions for the ophthalmic measurement device (300) to perform an Optical Coherence Tomography (OCT) scan of an area of ​​the retina of the eye and acquiring image data of the OCT scan as the measurement data;

[0166] If one of the plurality of different types of lesions found to be present in the image (400) is a tumor, the method further comprises generating (S26) the corresponding instructions for the ophthalmic measurement device (300) in the selected imaging modality to image the portion of the eye, instructions for the ophthalmic measurement device (300) to perform a high density optical coherence tomography (OCT) B-scan of the area within the portion of the eye and to obtain image data of the high density OCT B-scan as the measurement data;

[0167] If the one of the plurality of different types of lesions found to be present in the image (400) is a bacterial mass, the method further comprises generating (S26) the corresponding instructions for the ophthalmic measurement device (300) in the selected imaging modality to image the portion of the eye, instructions for the ophthalmic measurement device (300) to perform an Optical Coherence Tomography (OCT) B-scan of the area within the portion of the eye and to obtain image data of the OCT B-scan as the measurement data;

[0168] If one of the plurality of different types of lesions found to be present in the image (400) is edema or atrophy, the method includes generating (S26) instructions to cause the ocular measurement device (300) of the selected imaging modality to image the portion of the eye, the instructions causing the ocular measurement device (300) to perform an optical coherence tomography (OCT) scan of the area within the portion of the eye and acquiring image data of the OCT scan as the measurement data.

[0169] E18.A computer-implemented method as described in E14, wherein if at least one of the plurality of different types of lesions is found to be present in the image (400), at least one of the generated instructions is an instruction to cause an eye measurement device (300) of a selected measurement modality to measure a functional response of the eye to a light stimulus.

[0170] E19. A computer-implemented method according to any of E15 to E17, further comprising generating instructions for controlling a display unit (215) to display a display of the recorded position of the region (410) within the image (400) of the portion of the eye and the received measurement data.

[0171] E20. The computer-implemented method of any of E15 to E17, wherein the learning algorithm (630) is a supervised learning algorithm.

[0172] E21. A computer-implemented method as described in E20, wherein the supervised learning algorithm includes a neural network, and a region indicative of one of the plurality of different types of lesions found to be present in the image (400) is searched for in the image by decomposing the neural network.

[0173] E22. The computer-implemented method of E21, comprising:

[0174] The neural network is a convolutional neural network, and the neural network performs the following steps for each of a plurality of different partitions of the image (400) defined by the received image data:

[0175] a process of masking (S122) the section of the image (400) to generate a mask image;

[0176] searching for the region in the mask image by processing image data defining the mask image with the learning algorithm (S124);

[0177] determining (S126) a difference between a result of the search performed using the image data defining the mask image and a result of a search performed using the received image data; and

[0178] and determining (S128) the location of the segment in which the determined difference is maximum as the location to be recorded.

[0179] E23. A computer-implemented method according to E21, comprising:

[0180] The neural network is a convolutional neural network,

[0181] determining the association of each input variable of the neural network to the output of the neural network by applying a Taylor decomposition to each layer of the neural network, from the top layer of the neural network to the input layer of the neural network; and

[0182] determining locations to be recorded based on at least one section of the received image data that corresponds to a most relevant input variable of the neural network.

[0183] E24. The computer-implemented method of E21, wherein the neural network is a convolutional neural network, and the convolutional neural network is decomposed by performing a deconvolution of the convolutional neural network.

[0184] E25. A computer program which, when executed by a computer, causes the computer to carry out at least one of the methods recited in E1 to E24.

[0185] A computer-readable storage medium (250) storing the computer program according to E26.E25.

[0186] A signal (260) carrying a computer program according to E27.E25.

[0187] E28. An apparatus (100) for searching for an area indicative of a pathology in an image (400) of an eye portion acquired by an ophthalmic imaging system (520; 720), said apparatus (100) comprising:

[0188] a receiving module (110) configured to receive image data defining said image (400);

[0189] a search module (120) configured to search for said region within said image (400) by processing said received image data with a learning algorithm (530; 630) trained on image data (502; 604) defining a plurality of images of said portion of a healthy eye and on image data (601, 602, 603; 501) defining a plurality of images of said portion of an unhealthy eye, each image having at least one region indicative of said pathology; and

[0190] In response to a region (410) in the image (400) that represents the lesion found by the search module (120),

[0191] determining a location of the region (410) within the image (400); and

[0192] an instruction generation module (130) configured to execute a process of generating instructions to cause an ophthalmic measurement device (300) to perform the measurement on the portion of the eye using a reference point based on the determined position to set a position of the measurement on the portion of the eye and generate measurement data;

[0193] The apparatus (100), wherein the receiving module (110) is further configured to receive the measurement data from the ophthalmic measurement device (300).

[0194] The device (100) according to E29.E28,

[0195] the search module (120) is configured to search for a region (410) in the image (400) that is indicative of one of a plurality of different types of lesions by processing the received image data with the learning algorithm (630) trained on image data (601, 602, 603) defining a plurality of images of the portion of an unhealthy eye, each image having a corresponding one of a plurality of different types of lesions;

[0196] the instruction generation module (130) responding to a region (410) in the image (400) that shows one of the plurality of different types of lesions found by the search module (120),

[0197] selecting, for said one of said plurality of different types of lesions, a corresponding one of a plurality of different types of measurement modalities for a measurement to be performed on said eye; and

[0198] An apparatus (100) configured to execute a process of generating instructions to cause an eye measurement device (300) of the selected measurement modality to perform the measurement on the portion of the eye using the reference point for setting the position of the measurement on the portion of the eye as the instructions to cause the eye measurement device (300) to perform the measurement on the portion of the eye.

[0199] The device (100) according to E30.E28,

[0200] the receiving module (110) is configured to receive image data in a first imaging modality as the image data;

[0201] the instruction generation module (130) is configured to generate, in response to an area (410) in the image (400) indicative of the lesion found by the search module (120), as the instruction for causing the ophthalmic measurement device (300) to perform the measurement on the portion of the eye, instructions for causing the ophthalmic measurement device (300) to perform an image capture process of a second imaging modality for imaging the area in the portion of the eye, different from the first imaging modality, using the reference point for setting a position of the area in the portion of the eye to be imaged in the image capture process as a position of the measurement;

[0202] The apparatus (100), wherein the receiving module (110) is configured to receive, from the ocular measurement device (300), image data of the second imaging modality acquired by the ocular measurement device (300) as the measurement data.

[0203] The device (100) according to E31.E30,

[0204] the search module (120) is configured to search for a region (410) in the image (400) that is indicative of one of the plurality of different types of lesions by processing the received image data with the learning algorithm (630) that has been trained based on the image data (601, 602, 603) that defines a plurality of images of the portion of an unhealthy eye, each image having a corresponding one of a plurality of different types of lesions;

[0205] The apparatus (100), wherein the instruction generation module (130) is configured to execute a process for selecting, in response to a region (410) in the image (400) showing one of the plurality of different types of lesions found by the search module (120), a corresponding one of a plurality of different types of imaging modalities to be used for the one of the plurality of different types of lesions and for performing the image capture process on the portion of the eye as the second imaging modality.

[0206] The device (100) according to E32.E31,

[0207] the instruction generation module (130) is configured to generate instructions to cause the eye measurement device (300) to perform an optical coherence tomography (OCT) scan of the area within the portion of the eye as the image capture process of the second imaging modality if the one of the plurality of different types of lesions is glaucoma, and the receiving module (110) is configured to receive image data of the OCT scan as the measurement data;

[0208] the instruction generation module (130) is configured to generate instructions to cause the eye measurement device (300) to perform an optical coherence tomography (OCT) scan on a region of the retina of the eye as the image capture process of the second imaging modality if the one of the plurality of different types of lesions is severe diabetic retinopathy, and the receiving module (110) is configured to receive image data of the OCT scan as the measurement data;

[0209] the instruction generation module (130) is configured to generate instructions to cause the ophthalmic measurement device (300) to perform a high density optical coherence tomography (OCT) B-scan on the area within the portion of the eye as the image capture process of the second imaging modality if the one of the plurality of different types of lesions is a tumor, and the receiving module (110) is configured to receive image data of the high density OCT B-scan as the measurement data;

[0210] the instruction generation module (130) is configured to generate an instruction to cause the eye measurement device (300) to perform an optical coherence tomography (OCT) B-scan of the area within the portion of the eye as the image capture process of the second imaging modality if the one of the plurality of different types of lesions is a fungal mass, and the receiving module (110) is configured to receive image data of the OCT B-scan as the measurement data;

[0211] The apparatus (100), wherein the instruction generation module (130) is configured to generate instructions to cause the eye measurement device (300) to perform an optical coherence tomography (OCT) scan of the area within the portion of the eye as an image capture process of the second imaging modality if the one of the plurality of different types of lesions is edema, and the receiving module (110) is configured to receive image data of the OCT scan as the measurement data.

[0212] An apparatus (100) as described in E33.E28, wherein the instruction generation module (130) is configured to generate, in response to the area (410) in the image (400) indicating the lesion found by the search module (120), an instruction to cause the eye measurement device (300) to perform the measurement on the portion of the eye, the instruction to cause the eye measurement device (300) to measure a functional response of the eye to a light stimulus using the reference point for setting the position of the measurement based on the determined position.

[0213] The device (100) according to E34.E28,

[0214] the receiving module (110) is configured to receive image data representative of a result of the imaging of the portion of the eye using a first value of an imaging parameter, the first value being one of an imaging resolution, an aperture size, and a wavelength used in imaging;

[0215] the instruction generation module (130) is configured to execute a process for generating, in response to a region (410) in the image (400) indicative of the lesion found by the search module (120), instructions for causing the ophthalmic measurement device (300) to perform the measurement on the portion of the eye, instructions for causing the ophthalmic measurement device (300) to perform an image capture process using a second value of the imaging parameter that is different from the first value of the imaging parameter and to image the region within the portion of the eye using the reference point for setting a position of the region within the portion of the eye to be imaged in the image capture process as a position of the measurement;

[0216] The apparatus (100), wherein the receiving module (110) is configured to receive, as the measurement data from the eye measurement device (300), the image data representing the result of imaging the area within the portion of the eye using the second value of the imaging parameter, and the received image data and the received measurement data are of the same imaging modality.

[0217] An apparatus (100) according to any one of E35.E28 to E34, wherein the instruction generation module (130) is further configured to generate an instruction for controlling a display unit (215) to display a display of the determined position of the area (410) within the image (400) of the portion of the eye and the received measurement data.

[0218] The apparatus (100) according to any one of E36.E28 to E35, wherein the learning algorithm (630) is a supervised learning algorithm.

[0219] An apparatus (100) as described in E37.E36, wherein the supervised learning algorithm includes a neural network, and the search module (120) is configured to search for the region indicative of the lesion in the image by decomposing the neural network.

[0220] The apparatus (100) according to E38.E37, wherein the neural network is a convolutional neural network and the search module (120) comprises:

[0221] For each of a plurality of different sections of the image (400) defined by the received image data,

[0222] masking the portion of the image (400) to generate a mask image;

[0223] locating said region within said mask image by processing image data defining said mask image with said learning algorithm;

[0224] determining a difference between a result of the search performed using the image data defining the mask image and a result of a search performed using the received image data; and

[0225] An apparatus (100) configured to decompose the neural network by performing a process of determining the location of the partition in which the determined difference is maximum as the location of the region (410) in the image (400).

[0226] The device (100) according to E39.E37,

[0227] The neural network is a convolutional neural network, and the search module (120)

[0228] determining the association of each input variable of the neural network to the output of the neural network by applying a Taylor decomposition to each layer of the neural network, from the top layer of the neural network to the input layer of the neural network; and

[0229] 1. An apparatus (100) configured to decompose the convolutional neural network by determining a location of the region (410) within the image (400) based on at least one partition of the received image data that corresponds to a most relevant input variable of the neural network.

[0230] The apparatus (100) described in E40.E37, wherein the neural network is a convolutional neural network, and the search module (120) is configured to decompose the convolutional neural network by performing a deconvolution of the convolutional neural network.

[0231] E41. An apparatus (800) for searching for the presence of a pathology in an image (400) of an eye portion acquired by an ophthalmic imaging system (520), said apparatus (800) comprising:

[0232] a receiving module (810) configured to receive image data defining said image (400);

[0233] a search module (820) configured to search for the presence of at least one of the plurality of different types of lesions in the images (400) by processing the received image data with a learning algorithm (630) trained on image data defining a plurality of images (604) of a healthy eye and a plurality of images (601, 602, 603) of an unhealthy eye, each image having a corresponding one of the plurality of different types of lesions;

[0234] In response to at least one of the plurality of different types of lesions found to be present in the image by the search module (820),

[0235] selecting, for each of at least one type of lesion found to be present in the image (400), a corresponding one of a plurality of different types of measurement modalities to be used to perform measurements on the portion of the eye; and

[0236] and an instruction generation module (830) configured to execute a process for generating, for each of the at least one type of lesion found to be present in the image (400), a corresponding instruction for causing an ophthalmic measurement device (300) of the corresponding selected measurement modality to perform the measurement on the portion of the eye;

[0237] The apparatus (800), wherein the receiving module (810) is further configured to receive measurement data of the measurements performed by the ophthalmic measurement device (300) for each selected measurement modality.

[0238] E42. The apparatus (800) of E41, wherein the searching module (820), in response to finding that at least one of the plurality of different types of lesions is present in the image (400),

[0239] and further configured to perform, for each of the at least one of the plurality of different types of lesions found to be present in the image (400), a process of searching for a corresponding region (410) in the image (400) indicative of the corresponding type of lesion by processing the received image data using the learning algorithm (630), and a process of recording a location of the corresponding region (410) in the image (400);

[0240] The apparatus (800), wherein the instruction generation module (830) is configured to generate corresponding instructions for causing the eye measurement device (300) of each selected measurement modality to perform the measurement on the eye using a reference point for positioning the measurement based on the corresponding position.

[0241] E43. The apparatus (800) according to E42, wherein the command generation module (830) is adapted to: in response to at least one of the plurality of different types of lesions found to be present in the image (400) by the search module (820); and for each of the at least one of the plurality of different types of lesions found to be present in the image (400),

[0242] selecting a corresponding one of a plurality of different types of imaging modalities to be used for imaging the portion of the eye as the corresponding one of the plurality of different types of measurement modalities; and

[0243] generating, as the corresponding instructions to the ophthalmic measurement device (300) of the selected imaging modality, corresponding instructions for the ophthalmic measurement device (300) of the selected imaging modality to image the portion of the eye using reference points for positioning the area of ​​the eye to be imaged based on the corresponding recorded positions;

[0244] The device (800), wherein the receiving module (810) is configured to receive corresponding image data of the imaging performed by the ocular measurement device (300) of the selected imaging modality as measurement data of the measurement performed by the ocular measurement device (300).

[0245] The device (800) according to E44.E43,

[0246] the instruction generation module (830) is configured to generate, if one of the plurality of different types of lesions found to be present in the image (400) is glaucoma, instructions for causing the ophthalmic measurement device (300) to perform an optical coherence tomography (OCT) scan on the area within the portion of the eye as the corresponding instructions for causing the ophthalmic measurement device (300) in the selected imaging modality to image the portion of the eye, and the receiving module (810) is configured to receive image data of the OCT scan as the measurement data;

[0247] the instruction generation module (830) is configured to generate instructions for the ophthalmic measurement device (300) to perform an optical coherence tomography (OCT) scan on a region of the retina of the eye as the corresponding instructions for the ophthalmic measurement device (300) of the selected imaging format to image the portion of the eye if one of the plurality of different types of lesions found to be present in the image (400) is severe diabetic retinopathy, and the receiving module (810) is configured to receive image data of the OCT scan as the measurement data;

[0248] the instruction generation module (830) is configured to generate, if one of the plurality of different types of lesions found to be present in the image (400) is a tumor, instructions for causing the ophthalmic measurement device (300) to perform a high density optical coherence tomography (OCT) B-scan on the area within the portion of the eye as the corresponding instructions for causing the ophthalmic measurement device (300) in the selected imaging modality to image the portion of the eye, and the receiving module (810) is configured to receive image data of the high density OCT B-scan as the measurement data;

[0249] the instruction generation module (830) is configured to generate, if the one of the plurality of different types of lesions found to be present in the image (400) is a bacterial mass, instructions to cause the ophthalmic measurement device (300) to perform an optical coherence tomography (OCT) B-scan on the area within the portion of the eye as the corresponding instructions to the ophthalmic measurement device (300) in the selected imaging modality to image the portion of the eye, and the receiving module (810) is configured to receive image data of the OCT B-scan as the measurement data;

[0250] The apparatus (800), wherein the instruction generation module (830) is configured to generate instructions to cause the ocular measurement device (300) to perform an optical coherence tomography (OCT) scan of the area within the portion of the eye as the corresponding instruction to the ocular measurement device (300) of the selected imaging modality to image the portion of the eye if one of the plurality of different types of lesions found to be present in the image (400) is edema, and the receiving module (810) is configured to receive image data of the OCT scan as the measurement data.

[0251] An apparatus (800) as described in E45.E41, wherein the instruction generation module (830) is configured to generate instructions for causing the eye measurement device (300) of the selected measurement modality to measure the functional response of the eye to a light stimulus in response to at least one of the plurality of different types of lesions found to be present in the image by the search module (820).

[0252] E46. An apparatus (800) according to any of E42 to E44, wherein the instruction generation module (830) is further configured to generate an instruction for controlling a display unit (215) to display a display of the recorded position of the area (410) within the image (400) of the portion of the eye and the received measurement data.

[0253] E47. The apparatus (800) according to any one of E42 to E44, wherein the learning algorithm (630) is a supervised learning algorithm.

[0254] An apparatus (800) as described in E48.E47, wherein the supervised learning algorithm includes a neural network, and the search module (820) is configured to search for a region (410) indicative of one of the plurality of different types of lesions found to be present in the image (400) by decomposing the neural network.

[0255] The apparatus according to E49.E48, wherein the neural network is a convolutional neural network, and the search module (820) comprises:

[0256] For each of a plurality of different sections of the image (400) defined by the received image data,

[0257] masking the portion of the image (400) to generate a mask image;

[0258] locating said region within said mask image by processing image data defining said mask image with said learning algorithm;

[0259] determining a difference between a result of the search performed using the image data defining the mask image and a result of a search performed using the received image data; and

[0260] An apparatus configured to decompose the neural network by performing a process of determining as the location to be recorded the location of the partition in which the determined difference is maximum.

[0261] The apparatus (800) according to E50.E48, wherein the neural network is a convolutional neural network, and the search module (820) further comprises:

[0262] determining the association of each input variable of the neural network to the output of the neural network by applying a Taylor decomposition to each layer of the neural network, from the top layer of the neural network to the input layer of the neural network; and

[0263] An apparatus (800) configured to decompose the convolutional neural network by determining locations to be recorded based on at least one section of the received image data that corresponds to a most relevant input variable of the neural network.

[0264] An apparatus (800) as described in E51.E48, wherein the neural network is a convolutional neural network, and the search module (820) is configured to decompose the convolutional neural network by determining a deconvolution of the convolutional neural network.

[0265] E52. An apparatus for searching for areas indicative of a pathology within an image (400) of an eye portion acquired by an ophthalmic imaging system (520; 720), the apparatus (100) including a processor and a memory storing computer program instructions, the computer program instructions, when executed by the processor, causing the processor to perform at least one of the methods described in E1 to E24.

[0266] In the above description, the exemplary aspects have been described with reference to several exemplary embodiments. This specification should therefore be considered illustrative and not restrictive. Similarly, the figures in the drawings that highlight features and advantages of the exemplary embodiments are provided for illustrative purposes only. The configurations of the exemplary embodiments are sufficiently flexible and configurable that they can be used (and operated) in ways other than those shown in the accompanying drawings.

[0267] Exemplary software embodiments provided herein may be provided as computer programs, or software, such as one or more programs having instructions or sequences of instructions contained or stored in an article of manufacture, such as a machine-accessible or machine-readable medium, instruction storage device, or computer-readable storage device, each of which may be non-transitory, in one exemplary embodiment. The programs or instructions on said non-transitory machine-accessible or machine-readable medium, instruction storage device, or computer-readable storage device may be used to program a computer system or other electronic device. The machine or computer-readable medium, instruction storage device, and storage device may include, but is not limited to, floppy diskettes, optical disks, and magneto-optical disks, or other types of media / machine-readable media / instruction storage devices / storage devices suitable for storing or transmitting electronic instructions. The techniques described herein are not limited to any particular software configuration. They may be applicable in any computing or processing environment. As used herein, the terms "computer-readable," "machine-accessible medium," "machine-readable medium," "instruction storage device," and "computer-readable storage device" are intended to include any medium that can store, encode, or transmit instructions or sequences of instructions for execution by a machine, computer, or computer processor, causing the machine / computer / computer processor to perform any one of the methods described herein. Furthermore, in the art, software is typically spoken of in some form (e.g., program, procedure, process, application, module, unit, logic, etc.) that performs an action or produces a result. Such expressions are merely a shorthand way of stating that execution of the software by a processing system causes the processor to perform an action or produce a result.

[0268] Further, some embodiments may be implemented by the preparation of application specific integrated circuits, programmable gate arrays, or by interconnecting an appropriate network of conventional component circuits.

[0269] Some embodiments include a computer program product. The computer program product may be a storage medium(s), instruction storage(s), or storage device(s) having stored thereon or within instructions that can be used to control a computer or computer processor or to cause a computer or computer processor to perform any of the procedures of the exemplary embodiments described herein. The storage medium / instruction storage / storage device may include, by way of example and not limitation, optical disks, ROM, RAM, EPROM, EEPROM, DRAM, VRAM, flash memory, flash cards, magnetic cards, optical cards, nanosystems, molecular memory integrated circuits, RAID, remote data storage / archive / archive devices, and / or any other type of device suitable for storing instructions and / or data.

[0270] Some embodiments stored on any one of the computer readable medium(s), instruction storage device(s), storage device(s) include software for controlling the hardware of the system and for enabling the system or microprocessor to interact with a human user or other machines using the results of the exemplary embodiments described herein. Such software may include, but is not limited to, device drivers, operating systems, and user applications. Finally, such computer readable medium or storage device(s) further includes software for implementing the exemplary aspects of the present invention, as described above.

[0271] Included within the programming and / or software of the system are software modules for carrying out the procedures described herein. In some exemplary embodiments herein, the modules comprise software. However, in other exemplary embodiments herein, the modules comprise hardware or a combination of hardware and software.

[0272] While various exemplary embodiments of the present invention have been described above, it should be understood that they are provided by way of example, not limitation. It will be apparent to those skilled in the art that various changes in form and detail may be made thereto. Thus, the present invention should not be limited by any of the exemplary embodiments described above, but should be defined only by the following claims and their equivalents.

[0273] Moreover, the purpose of the Abstract is to enable patent offices, and the public generally, particularly scientists, engineers, and practitioners in the art who are not familiar with patent or legal terminology or language, to quickly ascertain the nature and substance of the technical disclosure of the present application upon reading thereof. The Abstract is not intended to be in any way limiting on the scope of the exemplary embodiments described herein. It will also be understood that the steps recited in the claims need not be performed in the order presented.

[0274] Although the specification contains details of many specific embodiments, these should not be considered as limitations on the scope of any invention or what may be claimed, but rather as descriptions of features specific to the specific embodiments described herein. Moreover, certain features that are described herein in the context of an individual embodiment may also be implemented in combination in a single embodiment. Conversely, various features that are described in the context of a single embodiment may also be implemented in multiple embodiments individually or in subcombinations in any suitable manner. Furthermore, even if features are described above as operating in a certain combination and initially claimed as such, one or more features from a claimed combination may, in some cases, be excluded from that combination, and the claimed combination may relate to a subcombination or a variation of the subcombination.

[0275] In some circumstances, multitasking or parallel processing may be advantageous. Furthermore, the separation of various components in the embodiments described above should not be understood as requiring such separation in all embodiments, and the described program components and systems should generally be understood as being capable of being integrated into a single software product or implemented within multiple software products.

[0276] Although several exemplary embodiments and implementations have been described above, it should be apparent that the above are provided by way of illustration and not limitation. In particular, while many of the examples described herein include specific combinations of device or software elements, those elements may be combined in other ways to achieve the same purpose. Operations, elements, and features described with respect to only one embodiment are not intended to be excluded from a similar role in other embodiments or implementations.

[0277] The apparatus and computer programs described herein may be embodied in other specific forms without departing from their characteristics. The above embodiments are illustrative rather than limiting of the described systems and methods. Thus, the scope of the apparatus and computer programs described herein is indicated by the appended claims rather than the above description, and all changes within the meaning and scope of the claims are intended to be encompassed within the scope of the claims.

Claims

1. 1. A computer-implemented method for searching for a region (410) indicative of a pathology in an image (400) of an eye portion acquired by an ophthalmic imaging system (520), the method comprising: Receiving (S10) image data defining said image (400); Searching for said region (410) in said image (400) by processing (S12) said received image data with a learning algorithm (530; 630) trained on image data (502; 604) defining a plurality of images of said portion of a healthy eye and on image data (601, 602, 603; 501) defining a plurality of images of said portion of an unhealthy eye, each image having at least one region indicative of said pathology; and If a region (410) within the image (400) indicative of the lesion is found in the searching, determining (S14) a location of said region (410) within said image (400); generating (S16) instructions for causing an ophthalmic measurement device (300) to perform the measurement on the portion of the eye using a reference point based on the determined position for positioning the measurement on the portion of the eye and generate measurement data; and Receiving (S18) the measurement data from the ophthalmic measurement device (300); Including, the received image data represents a result of imaging the portion of the eye using a first value of an imaging parameter, the first value being one of an imaging resolution, an aperture size, and a wavelength used in imaging; the ophthalmic measurement device (300) is configured to perform, as the measurement for the portion of the eye, an image capture process using a second value of the imaging parameter for imaging an area within the portion of the eye, the second value being different from the first value of the imaging parameter, and to obtain, as the measurement data, image data representative of the results of imaging the area using the second value of the imaging parameter, the received image data and the obtained image data being of the same imaging modality; If a region (410) within the image (400) indicative of the lesion is found in the searching, the method further comprises: generating (S16) instructions for the ophthalmic measurement device (300) to perform the measurement on the portion of the eye, for the ophthalmic measurement device (300) to perform the image capture process using the reference point to set the location of an area within the portion of the eye to be imaged in an image capture process as the location of the measurement; and receiving (S18) from the ophthalmic measurement device (300) as the measurement data, the image data representative of the result of imaging the area (410) within the portion of the eye using the second value of the imaging parameter. A computer implemented method.

2. 2. The computer-implemented method of claim 1, further comprising generating instructions for controlling a display unit (215) to display a display of the location of the region (410) within the image (400) of the portion of the eye and the received measurement data.

3. The computer-implemented method of claim 1 , wherein the learning algorithm (530; 630) is a supervised learning algorithm.

4. 4. The computer-implemented method of claim 3, wherein the supervised learning algorithm includes a neural network, and the region indicative of the lesion is located in the image by decomposing the neural network.

5. the neural network is a convolutional neural network; The neural network, For each of a plurality of different sections of the image (400) defined by the received image data, a process of masking (S122) the section of the image (400) to generate a mask image; searching for said region in a mask image by processing image data defining said mask image with said learning algorithm (S124); determining (S126) a difference between a result of the search performed using the image data defining the mask image and a result of a search performed using the received image data; and determining (S128) the location of the partition in which the determined difference is maximum as the location of the region (410) in the image (400); A first process comprising: determining the association of each input variable of the neural network to the output of the neural network by applying a Taylor decomposition to each layer of the neural network, from the top layer of the neural network to the input layer of the neural network; and determining a location of the region (410) within the image (400) based on at least one section of the received image data that corresponds to a most relevant input variable of the neural network; A second process comprising: Or, Deconvolving the convolutional neural network. a third process comprising:

5. The computer implemented method of claim 4, wherein the decomposition is performed by performing any one of the following:

6. An apparatus (100) for searching for areas indicative of a pathology in an image (400) of an eye part acquired by an ophthalmic imaging system (520; 720), said apparatus (100) comprising: a receiving module (110) configured to receive image data defining said image (400); a search module (120) configured to search for said region in said image (400) by processing said received image data with a learning algorithm (530; 630) trained on image data (502; 604) defining a plurality of images of said portion of a healthy eye and on image data (601, 602, 603; 501) defining a plurality of images of said portion of an unhealthy eye, each image having at least one region indicative of said pathology; and In response to a region (410) in the image (400) that represents the lesion found by the search module (120), determining a location of said region (410) within said image (400); and an instruction generation module (130) configured to execute a process of generating instructions for causing an ophthalmic measurement device (300) to perform the measurement on the portion of the eye using a reference point based on the determined position for setting a position of the measurement on the portion of the eye and generate measurement data; the receiving module (110) is further configured to receive the measurement data from the ophthalmic measurement device (300); the receiving module (110) is configured to receive image data representative of a result of the imaging of the portion of the eye using a first value of an imaging parameter, the first value being one of an imaging resolution, an aperture size, and a wavelength used in imaging; the instruction generation module (130) is configured to execute a process for generating, in response to a region (410) in the image (400) indicative of the lesion found by the search module (120), instructions for causing the ophthalmic measurement device (300) to perform the measurement on the portion of the eye, instructions for causing the ophthalmic measurement device (300) to perform an image capture process using a second value of the imaging parameter different from the first value of the imaging parameter and to image the region within the portion of the eye using the reference point for setting a position of the region within the portion of the eye to be imaged in the image capture process as a position of the measurement, the receiving module (110) is configured to receive, as the measurement data, from the ophthalmic measurement device (300), the image data representative of the result of imaging the area within the portion of the eye using the second value of the imaging parameter, and the received image data and the received measurement data are of the same imaging modality. Apparatus (100).

7. 7. The apparatus (100) of claim 6, wherein the instruction generation module (130) is further configured to generate instructions for controlling a display unit (215) to display the determined position of the region (410) within the image (400) of the portion of the eye and a display of the received measurement data.

8. The apparatus (100) of claim 6, wherein the learning algorithm (630) is a supervised learning algorithm.

9. 10. The apparatus of claim 8, wherein the supervised learning algorithm comprises a neural network, and the search module is configured to search for the region indicative of the lesion in the image by decomposing the neural network.

10. The neural network is a convolutional neural network, and the search module (120) For each of a plurality of different sections of the image (400) defined by the received image data, masking said section of said image (400) to generate a mask image; locating said region within said mask image by processing image data defining said mask image with said learning algorithm; determining a difference between a result of the search performed using the image data defining the mask image and a result of a search performed using the received image data; and determining the location of the region (410) in the image (400) in the section where the determined difference is maximum; A first process comprising: determining the association of each input variable of the neural network to the output of the neural network by applying a Taylor decomposition to each layer of the neural network, from the top layer of the neural network to the input layer of the neural network; and determining a location of the region (410) within the image (400) based on at least one section of the received image data that corresponds to a most relevant input variable of the neural network; A second process comprising: Or, Deconvolving the convolutional neural network. a third process comprising:

10. The apparatus (100) of claim 9, configured to decompose the neural network by performing any one of the following:

Images