Retinal Image Processing

Abstract

A device for generating an alert indicating the unreliability of a location of a landmark feature in a retinal image predicted by a machine learning algorithm, the device comprising: a receiver module that receives the predicted location; a probability indication determination module that uses the predicted location and a hybrid model to determine, for each of a plurality of retinal image categories, a corresponding probability indication indicating the probability that the retinal image belongs to the category, the hybrid model comprising a probability distribution of landmark feature locations and based on the determined location of the landmark feature in a retinal image having the plurality of retinal image categories; an outlier detection module that uses the probability indication to determine whether the retinal image is an outlier that does not belong to any of the categories; and an alert generator module that generates an alert indicating the unreliability of the predicted location when the retinal image does not belong to any of the categories.

Description

Technical Field

[0001] Example aspects herein relate generally to the field of eye image data processing systems, and more particularly to techniques for processing retinal images using machine learning algorithms.

[0002] background

[0003] Various ocular imaging systems (such as laser scanning ophthalmoscopes and fundus cameras) are typically used to obtain images of the retina of a subject's eye. The acquired retinal images can be examined by an ophthalmologist or other medical professional to assess the health of the retina. The acquired retinal images can also be automatically processed by image processing software for various purposes. For example, a machine learning algorithm such as a convolutional neural network (CNN) can be used to perform eye image classification (e.g., classifying an image as belonging to a first category of images including the left eye, or belonging to a second category of images including the right eye) using the location of retinal landmarks (e.g., optic disc or fovea) that have been predicted by the machine learning algorithm based on information in the retinal image. Such a machine learning algorithm is typically trained based on a data set of eye images, and is then used to perform predictions on new images.

[0004] Overview

[0005] According to a first example aspect of the present invention, there is provided a method for generating an alert indicating unreliability of a predicted position of a landmark feature that has been predicted by a machine learning algorithm in a retinal image of at least a portion of the retina of an eye, the method comprising: receiving a predicted position of a landmark feature that has been predicted by a machine learning algorithm in a retinal image; using the received predicted position of the landmark feature and a hybrid model to determine, for each of a plurality of predetermined categories of the retinal image, a corresponding probability indication indicating the probability that the retinal image belongs to the category, the hybrid model comprising a probability distribution for the position of the landmark feature and based on the determined position of the landmark feature in a retinal image of a set of retinal images having the plurality of predetermined categories; using the determined probability indication to determine whether the retinal image does not belong to any of the predetermined categories; and in response to determining that the retinal image does not belong to any of the predetermined categories, generating an alert indicating that the predicted position in the retinal image is unreliable.

[0006] According to a second exemplary aspect of the present invention, there is also provided a computer program comprising computer program instructions, which, when executed by a computer, causes the computer to perform the method according to the first exemplary aspect of the present invention. The computer program may be stored on a non-transitory computer readable recording medium, or the computer program may be carried by a signal.

[0007] According to a third example aspect of the present invention, there is also provided an apparatus for generating an alert indicating an unreliability of a predicted position of a landmark feature in a retinal image of at least a portion of a retina of an eye that has been predicted by a machine learning algorithm. The apparatus includes a receiver module configured to receive a predicted position of a landmark feature in a retinal image that has been predicted by a machine learning algorithm. The apparatus also includes a probability indication determination module configured to use the received predicted position of the landmark feature and a hybrid model to determine, for each of a plurality of predetermined categories of the retinal image, a corresponding probability indication indicating a probability that the retinal image belongs to the category, the hybrid model including a probability distribution of the position of the landmark feature and based on the determined position of the landmark feature in a retinal image belonging to a set of retinal images having the plurality of predetermined categories. The apparatus also includes: an outlier detector module configured to use the determined probability indication to determine whether the retinal image is an outlier that does not belong to any of the predetermined categories; and an alert generator module configured to generate an alert indicating that the predicted position in the retinal image is unreliable in response to the outlier detector module determining that the retinal image does not belong to any of the predetermined categories. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] Example embodiments of the present disclosure will now be explained in detail by way of non-limiting examples only, with reference to the accompanying drawings described below.Unless otherwise indicated, like reference numerals appearing in different figures of the drawings may represent the same element or a functionally similar element.

[0010] Figure 1 is a schematic diagram of an eye imaging system according to a first example embodiment herein, the eye imaging system comprising means for alerting a user to the unreliability of at least one predicted position of a landmark feature in a retinal image.

[0011] Figure 2 is a block diagram illustrating an example implementation of the apparatus of the first example embodiment in programmable signal processing hardware.

[0012] Figure 3 is a flow chart illustrating a computer-implemented method of alerting a user of the unreliability of at least one predicted position of a landmark feature in a retinal image, according to a first example embodiment.

[0013] Figure 4It is shown that in a first example embodiment, an acquired retinal image is processed by a machine learning algorithm to predict the positions of a first landmark feature and a second landmark feature in the retinal image, and the acquired retinal image shows the fovea as the first landmark feature of the retina and the optic disc as the second landmark feature of the retina.

[0014] Figure 5 The predicted positions of the fovea and optic disc in two different retinal images of the same area of ​​the retina generated in the first example embodiment are shown.

[0015] Figure 6 A probability distribution of the distance between the fovea and the optic disc is shown, which is based on measurements of this distance in a set of retinal images and is used in a first example embodiment.

[0016] Figure 7 A weighting function is shown that may be multiplied with the estimated distance metric to calculate a weighted distance metric according to example embodiments described herein.

[0017] Figure 8 is a flow chart illustrating a computer-implemented method of detecting unreliable predicted locations of landmark features in a retinal image by using a second estimated distance metric and a second probability distribution according to example embodiments herein.

[0018] Fig.9A An example retinal image is shown with an example of an estimated second distance metric superimposed thereon, the estimated second distance metric taking the form of a Euclidean distance between a predicted position of the fovea in the retinal image and a reference position in the retinal image.

[0019] Fig. 9B An example of a second probability distribution is shown, which can be used to determine the fovea in Fig.9A Reliability of predicted positions in example retinal images.

[0020] Fig.10 is a schematic diagram of an eye imaging system including means for detecting unreliability of a predicted position of a landmark feature in a retinal image according to a second example embodiment described herein.

[0021] Fig.11 It is shown that in a second example embodiment, the acquired retinal image is processed by a machine learning algorithm to predict the position of the landmark feature in the retinal image.

[0022] Fig.12is a flow chart illustrating a computer-implemented method used in a second example embodiment to detect unreliability in predicted positions of landmark features in retinal images using a mixture model.

[0023] Fig.13 An example of a hybrid model is shown that may be used in a second example embodiment to verify the reliability of the predicted positions of landmark features in a retinal image.

[0024] Fig.14 is a flow chart illustrating the processing performed in a first variation of the second example embodiment, in which the hybrid model is also based on the determined positions of landmark features in the retinal image of retinal images belonging to a failure class.

[0025] Fig.15 is a flowchart illustrating processing performed in a second variation of the second example embodiment, the processing utilizing a comparative metric to determine whether a retinal image belongs to one of predetermined categories of retinal images.

[0026] Detailed Description

[0027] Machine learning algorithms typically need to be trained on many retinal images to achieve an acceptable level of accuracy. Even with large datasets, 100% accuracy is rarely possible. For example, a machine learning algorithm may not be able to make accurate predictions about the locations of landmark features when important portions of a retinal image are occluded (e.g., due to blink events or imaging artifacts (e.g., eyelashes) that occur during imaging, or due to the presence of cataracts in the imaged eye). This failure, which typically goes unnoticed, may adversely affect the performance of (and therefore user confidence in) downstream data processing operations that rely on accurate landmark feature locations, such as auto-laterality routines for determining whether a retinal image is of the right eye or the left eye, disease pipelines for automatically diagnosing various eye diseases or disorders, and data processing operations for automatically annotating retinal images (among other things).

[0028] In view of the above, the inventors have designed computer-implemented techniques and apparatus for alerting a user of the unreliability of the predicted positions of one or more landmark features in a retinal image that have been predicted by a machine learning algorithm. The user can then respond to the alert, for example, by determining the landmark positions by other means (e.g., by examining the retinal image), or take other appropriate actions.

[0029] Example embodiments of such an apparatus and related computer-implemented techniques will now be described in more detail with reference to the accompanying drawings.

[0030] Figure 1 is a schematic diagram of an eye imaging system 10 according to a first example embodiment, the eye imaging system 10 being configured to acquire a retinal image of at least a portion of a retina 20 of an eye 30 of a subject. The eye imaging system 10 includes an image acquisition module 40 configured to acquire a retinal image. As in the present example embodiment, the image acquisition module 40 may take the form of a scanning laser ophthalmoscope (SLO), but may alternatively include a fundus camera or any other type of image acquisition device known to those skilled in the art that is capable of acquiring retinal data from the eye 30.

[0031] The eye imaging system 10 further includes a landmark position prediction module 50, which is configured to use at least one machine learning algorithm to predict a first position L1 of the first landmark feature in the retinal image and a second position L2 of the second landmark feature in the retinal image as predicted positions of the landmark features in the retinal image. The eye imaging system 10 further includes an apparatus 60, which is configured to alert a user of the eye imaging system 10 to the unreliability of at least one of the predicted positions of the landmark features. Figure 1 As shown, the apparatus 60 includes a receiver module 61, a distance metric estimation module 62, an outlier detector module 63, and an alert generator module 64. An example implementation of the apparatus 60 and the functionality of its illustrated component modules are described in more detail below.

[0032] Figure 2 200 according to an example embodiment of the present invention. In an example embodiment of the present invention, the signal processing device 200 may be included in one or more of the various components described herein and / or form one or more of the various components described herein (and is an example implementation of one or more of the various components described herein) in programmable signal processing hardware. As just one example, the signal processing device 200 may be included in Figure 1The signal processing device 200 includes an interface module 210 for receiving the predicted position of the landmark feature in the retinal image from the landmark position prediction module 50, and outputting a reminder to draw the user's attention to the determined unreliability of the predicted landmark position. The interface module 210 may also receive user input from one or more peripheral devices (e.g., one or more of a keyboard, a touchpad, or a mouse), and output the reminder to one or more output devices (e.g., a speaker and / or a computer monitor), which may form part of the eye imaging system 10 of the present example embodiment or part of the eye imaging system 1000 of the second example embodiment described below. The signal processing device 200 also includes a processor (CPU) 220, a working memory 230 (e.g., a random access memory), and an instruction storage device 240 storing a computer program 290 including computer-readable instructions, which, when executed by the processor 220 (in the case where the device 200 forms the device 60), cause the processor 220 to perform the processing operations of the device 60. The instruction storage device 240 may include a ROM (e.g., in the form of an electrically erasable programmable read-only memory (EEPROM) or flash memory) preloaded with computer-readable instructions. Alternatively, the instruction storage device 240 may include a RAM or similar type of memory, and the computer-readable instructions may be input to the instruction storage device 240 from a computer program product (e.g., a computer-readable storage medium 250 (e.g., a CD-ROM, etc.) or a computer-readable signal 260 carrying computer-readable instructions).

[0033] In this example embodiment, Figure 2 The combination of hardware components 270 shown in (including processor 220, working memory 230 and instruction storage device 240) is configured to perform the functions of receiver module 61, distance metric estimation module 62, outlier detector module 63 and reminder generator module 64 to be described below. In addition, in an example embodiment of the present invention, Figure 1 The landmark position prediction module 50 may include Figure 2 The signal processing hardware 200 and / or Figure 2 The signal processing hardware 200 is formed / implemented.

[0034] Figure 3is a flow chart illustrating a computer-implemented method according to a first example embodiment of the present invention, by which the device 60 processes a first position L1 of a first landmark feature in a retinal image and a second position L2 of a second landmark feature in the retinal image that have been predicted by the landmark position prediction module 50 as predicted positions of the landmark features in the retinal image to alert a user of the device 60 to the unreliability of at least one of the predicted positions of the landmark features. The landmark position prediction module 50 may use a common machine learning algorithm to predict both the first position L1 of the first landmark feature in the retinal image and the second position L2 of the second landmark feature in the retinal image, or alternatively, the landmark position prediction module 50 may use a first machine learning algorithm to predict the first position L1 of the first landmark feature in the retinal image and use a second machine learning algorithm different from the first machine learning algorithm to predict the second position L2 of the second landmark feature in the retinal image. After the image acquisition module 40 has acquired the retinal image, and once the landmark position prediction module 50 has made its predictions of the first position L1 of the first landmark feature in the retinal image and the second position L2 of the second landmark feature in the retinal image, the following reference may be performed: Figure 3 Describes the processing operation.

[0035] exist Figure 3 In step S10, the receiver module 61 receives as predicted positions a predicted first position L1 of a first landmark feature in the retinal image and a predicted second position L2 of a second landmark feature in the retinal image that have been predicted by at least one machine learning algorithm of the landmark position prediction module 50. For example, each landmark feature may be any anatomical feature of the retina that is located at substantially the same position in the left eye or the right eye of a plurality of different (human) subjects. In this example embodiment, the first landmark feature includes an image of the fovea in the retinal image, and the second landmark feature includes an image of the optic disc in the retinal image. However, the first landmark feature and the second landmark feature are not limited to these anatomical features, and may include other anatomical features imaged in the retinal image. Furthermore, as in the present embodiment, each predicted position of a landmark feature may be received as coordinates of the landmark feature in the retinal image, for example, as XY coordinates of a pixel of the retinal image that forms part of the image of the landmark feature.

[0036] exist Figure 3 In step S20, the distance metric estimation module 62 estimates a distance metric indicating the distance between the first landmark feature and the second landmark feature using the predicted first position and the predicted second position. An example form of the distance metric is described below.

[0037] exist Figure 3In step S30, the outlier detector module 63 uses data indicating a probability distribution of the distance between the first landmark feature and the second landmark feature to determine whether the estimated distance metric is an indication of an outlier that is outside a predetermined interval of the probability distribution, the probability distribution of the distance being obtained from measurements of the distance between the first landmark feature and the second landmark feature in a plurality of retinal images that are different from the retinal image. As in the present example embodiment, the indication may indicate whether the estimated distance metric is outside a predetermined interval of the probability distribution that includes at least one peak of the probability distribution. The set of retinal images different from the retinal image are images of substantially the same portion of the retina as the retinal image that was imaged (by the system 10) to obtain the retinal image for which position prediction is performed by the landmark position prediction module 50.

[0038] In step S40, Figure 3 In the case where the indication determined by the outlier detector module 63 in step S30 is an indication that the estimated distance metric is outside a predetermined interval of the probability distribution (e.g., predetermined interval (I)), the alert generator module 64 generates an alert indicating that at least one of the predicted positions of the landmark features in the retinal image is unreliable. The alert can be provided in any suitable or desired form for the user, for example, as a message or symbol displayed on a display device (such as a computer screen), and additionally or alternatively, as an audio signal, for example, to inform the user that at least one of the predicted positions in the retinal image is unreliable. On the other hand, in Figure 3 In the event that the indication determined by the outlier detector module 63 in step S30 is an indication that the estimated distance metric is not outside the predetermined interval (I) of the probability distribution, the alert generator module 64 does not generate an alert, and Figure 3 The processing is finished.

[0039] It should be noted that although the alert generator module 64 generates an alert (e.g., an alert in the form of a visual, audio, and / or tactile alert) when the outlier detector module 63 has determined that the estimated distance metric is outside the predetermined interval (I) of the probability distribution, the generation of the alert by the alert generator module 64 may be subject to one or more additional conditions. For example, the additional condition may be that a user of the eye imaging system 10 (e.g., via an input device such as a keyboard, touchpad, or mouse) provides instructions to the eye imaging system 10 to alert the detection of any outliers by the outlier detector module 63 after the multiple retinal images are acquired and processed by the eye imaging system 10 as described above. Alternatively, the additional condition may be that a predetermined number of retinal images have been acquired and processed by the eye imaging system 10 as described above. In both examples, the alert generator module having the above configuration may generate an identifier indicating a predicted position that has been determined by the device 10 to be unreliable, and / or the identifier identifies a retinal image from the acquired and processed retinal images that have been processed by the machine learning algorithm to produce an unreliable prediction, so that once all conditions are met, the user's attention may be alerted to one or more retinal images that need to be checked by the user.

[0040] Figure 4 An example of a retinal image 400 is shown, which shows the fovea 410 as an example of a first landmark feature of the retina, and the optic disc 420 as an example of a second landmark feature of the retina. The retinal image 400 is composed of Figure 1 The eye imaging system 10 of the present example embodiment shown acquires and is input to the landmark position prediction module 50, which is configured to use a machine learning algorithm to predict the position of the fovea 410 in the retinal image 400 and the position of the optic disc 420 in the retinal image 400. In the present example embodiment, the machine learning algorithm includes a convolutional neural network (CNN) configured to predict the positions of the fovea 410 and the optic disc 420 in the retinal image 400. However, any suitable machine learning algorithm may be used to perform the prediction. In addition, as in the present example embodiment, the retinal image 400 has an X dimension and a Y dimension, and the landmark position prediction module 50 may predict the XY coordinates of the fovea 410 in the retinal image 400 and the XY coordinates of the optic disc 420 in the retinal image 400 as the position L1 of the fovea 410 in the retinal image 400 and the position L2 of the optic disc 420 in the retinal image 400. Figure 4 , the predicted XY coordinates of the fovea 410 in the retinal image 400 are given by (F x , F y ) and the predicted XY coordinates of the optic disc 420 in the retinal image 400 are represented by ( x , O y)express.

[0041] Furthermore, as in the present example embodiment, the landmark position prediction module 50 may be configured to predict the location of the fovea 410 based on the predicted coordinates (F x , F y ) and the predicted coordinates (O) of the optic disc 420 in the retinal image 400 x , O y ), using a machine learning algorithm, classifies the retinal image 400 as belonging to one of a plurality of predetermined different categories. Thus, in this example embodiment, the alert generated by the alert generator module 64 indicates that the classification of the retinal image by the landmark position prediction module 50 is unreliable. As described above, the alert can be an audio alert generated by an audio signal generator (e.g., a computer speaker) of the eye imaging system 10 and / or a visual alert generated on a display device (e.g., a computer screen) of the eye imaging system 10, which can prompt the user to correct the classification of the retinal image by inspecting the retinal image on the display of the eye imaging system 10 and inputting the correct classification using an input device (e.g., a keyboard, mouse, or trackpad) of the eye imaging system 10. In this way, the landmark position prediction module 50 can automatically classify the retinal image using a machine learning algorithm, requiring the user to intervene by determining the correct classification and providing the determined classification only in the event that the landmark position prediction module 50 has generated an unreliable prediction of the landmark feature location and therefore made an unreliable classification.

[0042] As an example, the landmark position prediction module 50 of the present example embodiment is configured to classify the retinal image as belonging to one of two retinal image categories, namely, a first category of the retinal image category of the subject's left eye, or a second category of the retinal image category of the subject's right eye. However, the number of categories is not limited to two, and other types of classifications may also be performed. For example, in some example embodiments, the landmark position prediction module 50 may be additionally or optionally configured to use the predicted positions of the first landmark and the second landmark to classify the retinal image as a healthy eye or an unhealthy eye. In other example embodiments, different categories may involve different corresponding gaze directions of the eyes, such as central gaze, inferior gaze, superior gaze, nasal gaze, and temporal gaze, each of which may be specified for the right eye or the left eye. Therefore, in these example embodiments, the landmark position prediction module 50 may classify each retinal image being processed as belonging to one of 10 different categories.

[0043] As in this example embodiment, Figure 3The distance metric mentioned in steps S20, S30 and S40 may include the Euclidean distance between the first signature feature and the second signature feature. As an example, Figure 5 5 shows the predicted positions of the fovea and optic disc in two different retinal images 510 and 520 that have been processed by the landmark position prediction module 50. For the retinal image 510, the landmark position prediction module 50 predicts the coordinates of the fovea as (F x , F y ) and predict the coordinates of the optic disc as (O x , O y ), where the predicted coordinates are illustrated as “+” in the retinal image 510. In addition, the distance metric estimation module 62 estimates (F x , F y ) and (O x , O y ), which is represented as d1 in the retinal image 510. Similarly, for the retinal image 520, the landmark position prediction module 50 predicts the coordinates of the fovea as (F′ x , F′ y ) and predict the coordinates of the optic disc as (O′ x , O′ y ), where the predicted coordinates are illustrated as “+” in the retinal image 520. In addition, the distance metric estimation module 62 estimates (F′ x , F′ y ) and (O′ x , O′ y ) as a distance metric, which is illustrated as d2 in the retinal image 520.

[0044] As in this example embodiment, Figure 3 The probability distribution mentioned in step S30 can take the normal distribution N(μ, σ) of the Euclidean distance between the first signature feature and the second signature feature. 2 ), the Euclidean distance being obtained from the measurement of the Euclidean distance of a set of retinal images different from the retinal image (obtained by the system 10) for which the prediction is made. As an example, Figure 6 Based on the observation of the distance between the fovea and the optic disc in the retinal image set, the probability distribution of the Euclidean distance between the fovea and the optic disc in the retinal image is illustrated. As in this example embodiment, Figure 6The probability distribution in may be a ground-truth distribution, and the set of retinal images from which the probability distribution is derived may include a training data set of retinal images used to train a machine learning algorithm to perform predictions of landmark feature locations. However, the probability distribution is not necessarily based on the retinal images in the training data set, and any suitable set of retinal images may be used to obtain a probability distribution of the Euclidean distance between the fovea and the optic disc.

[0045] exist Figure 6 In the example of , the predetermined interval of the probability distribution for determining the reliability of the predicted position is illustrated as a predetermined interval I around the mean value μ of the normal distribution. Figure 6 Also illustrated is the value d1 (corresponding to Figure 5 The predicted position (Fx, F y ) and (O x , O y ) and the value d2 (corresponding to the distance between Figure 5 The coordinates (F′) in the retinal image 520 x , F′ y ) and (O′ x , O′ y )). Figure 6 In the example, for Figure 5 In the retinal image 510, the outlier detector module 63 uses Figure 6 The probability distribution in determines an indication that the estimated distance d1 is outside the predetermined interval I, and thus the alert generator module 64 generates an alert indicating that at least one of the predicted positions of the landmark features in the retinal image 510 is unreliable. However, for Figure 5 In the illustrated retinal image 520 , the estimated distance d2 falls within the predetermined interval I, and thus in this case, the alert generator module 64 does not generate an alert because the predicted position of the landmark feature in the retinal image 520 is determined to be reliable.

[0046] In some example embodiments, the predetermined interval I may be defined as a function of the standard deviation σ of the normal distribution. However, the predetermined interval may also be defined in other ways using any suitable method for outlier identification. For example, the predetermined interval may be defined as the range of distance values ​​for which the value of the probability distribution function (the probability distribution of step S30) exceeds a predetermined likelihood threshold. In other words, the predetermined likelihood threshold p th may be set, and the outlier detector module 63 may be configured to detect the outlier only when the value of the probability distribution function of the value of the estimated distance metric (between the predicted position of the fovea and the optic disc) is below a predetermined likelihood threshold p th Only then can the predicted positions of the optic disc and fovea be determined to be reliable.

[0047] Although reference Figure 5 and Figure 6 The described examples estimate the Euclidean distance between the predicted positions of the fovea and the optic disc, but it should be noted that any other suitable distance metric may be used. For example, in some example embodiments, the distance metric may alternatively provide a measure of the Manhattan distance between the position of the first landmark feature and the position of the second landmark feature. Furthermore, in example embodiments where the estimated distance metric includes the Manhattan distance, data indicative of a probability distribution of Manhattan distances obtained from measurements of the Manhattan distance between the first landmark feature and the second landmark feature in a set of retinal images other than the retinal image may be used to determine an indication of whether the estimated distance metric is outside a predetermined interval of the probability distribution.

[0048] In some example embodiments, an additional weighting factor may be applied to the estimated distance metric in order to emphasise outliers. More specifically, in some example embodiments, the distance metric may include a weighted distance between the first landmark feature and the second landmark feature, the weighted distance being the product of the distance between the first landmark feature and the second landmark feature and a weighting factor, the value of the weighting factor increasing as the absolute value of the difference between the distance and the distance value corresponding to the peak of the probability distribution increases. The distance metric weighted to calculate the weighted distance may be a Euclidean distance, in which case the probability distribution used by the outlier detector module 63 is a Euclidean distance between the first landmark feature and the second landmark feature obtained from a measurement of the Euclidean distance in a retinal image set different from the retinal image for which the position prediction is made. Alternatively, the distance weighted to calculate the weighted distance may be a Manhattan distance, in which case the probability distribution used by the outlier detector module 63 is a Manhattan distance between the first landmark feature and the second landmark feature obtained from a measurement of the Manhattan distance in a retinal image set different from the retinal image for which the position prediction is made.

[0049] Figure 7 A non-limiting example of a mapping function between |d-μ| and a weighting factor w that may be used to weight the estimated distance metric d (indicating the distance between the predicted position of the fovea and the predicted position of the optic disc) is shown, where μ represents the mean of the probability distribution and thus represents the distance value corresponding to the peak of the probability distribution. Figure 7 In the mapping function, Figure 5 The Euclidean distances d1 and d2 estimated from the retinal images 510 and 520 in FIG. 5 determine the corresponding weighting factors. Figure 7In the example of , for the retinal image 510 (associated with the estimated distance d1), the outlier detector module 63 determines the distance |d1-μ| and uses Figure 7 The mapping function in determines the corresponding weight w1. The outlier detector module 63 also applies the weight w1 to the estimated distance (e.g., by calculating the weighted metric w1|d1-μ|) and then determines whether w1|d1-μ| falls within Figure 6 An indication within a predetermined interval I of a probability distribution in (which represents the expected distribution of the Euclidean distance between the fovea and the optic disc).

[0050] like Figure 7 As shown, in the above alternative embodiment, similar processing is performed on the retinal image 520 by the distance metric estimation module 62 and the outlier detector module 63. First, a weighting factor w2 corresponding to the estimated Euclidean distance d2 (indicating the distance between the predicted position of the fovea and the predicted position of the optic disc with respect to the retinal image 520) is determined, and the distance metric estimation module 62 calculates the weighted distance metric w2|d2-μ|, and the outlier detector module 63 determines whether w2|d2-μ| falls within a predetermined interval I, so as to estimate the reliability of the predicted positions of the fovea and the optic disc of the retinal image 520. Figure 7 As shown, because |d1-μ| is greater than |d2-μ|, the value of the weight w1 applied to |d1-μ| is greater than the value of the weight w2 applied to |d2-μ|.

[0051] It should be noted that the calculation of the weighting factor w is not limited to using Figure 7 Rather, any suitable variation of weighting factors may be used, where the weighting factors increase as the absolute value of the difference between the estimated distance and the distance value corresponding to the peak of the probability distribution increases.

[0052] In the example embodiment described above, an estimated distance metric indicating the distance between the predicted position of the first landmark feature in the retinal image and the predicted position of the second landmark feature in the retinal image is used to determine the reliability of the predicted position. However, there is a possibility that the estimated distance metric falls within the predetermined interval of the probability distribution despite the predicted position of the landmark feature significantly deviating from the expected position of the landmark feature. In this case, the alert generator module 64 will fail to generate an alert indicating that at least one of the predicted positions of the landmark features in the retinal image is unreliable.

[0053] Figure 8 is a flow chart of optional processing steps that may be performed by apparatus 60 to address the above-described problem, and which may be combined with any of the previously described embodiments to improve the reliability of the determination of the predicted location of a landmark feature in a retinal image. Figure 8In step S110, the distance metric estimation module 62 also uses the predicted first position L1 (of the first landmark feature) to estimate the distance between the predicted first position and the reference position (eg, the reference position L ref ) between the second distance metric.

[0054] Then, in Figure 8 In step S120, the outlier detector module 63 also uses the first marker feature indicating the relationship between the first marker feature and the reference position L ref The method further comprises determining a second indication based on data of a second probability distribution of distances between the first and second retinal images, wherein the second probability distribution is based on measurements of distances in a set of retinal images different from the retinal image (being examined). The second indication is an indication of whether the estimated second distance metric is outside a predetermined interval I' of the second probability distribution, the predetermined interval I' of the second probability distribution including a peak of the second probability distribution.

[0055] exist Figure 8 In step S130, when the determined second indication is an indication that the estimated second distance metric is not within the predetermined interval I' of the second probability distribution, the reminder generator module 64 generates a reminder indicating that the first predicted position L1 in the retinal image 510 is unreliable as a reminder.

[0056] Fig.9A Shows the use of Figure 5 An example of a second distance metric for a retinal image 520. More specifically, Fig.9A It shows that at the coordinate (F′ x , F′ y ) at the predicted position L1 of the first landmark (fovea), at the coordinate (L ref_x , L ref_y ) at the reference position L ref , and in (F′ x , F′ y ) and (L ref_x , L ref_y ) between the Euclidean distance d3. Fig.9A and Fig. 9B In this example, the reference position (L ref_x , L ref_y ) takes the (0,0) coordinate of the retinal image, which is, for example, the lower left corner of the retinal image. However, the reference position is not limited in this regard and can be any position in the retinal image, provided that the same reference position is used for all retinal images in the retinal image set to obtain a second distribution of distances between the first landmark feature and the reference position.

[0057] Fig. 9B shows an example of a second probability distribution that can be used to determine Fig.9A The reliability of the predicted position L1 of the fovea in Fig. 9B In the second probability distribution, the form of the fovea and the coordinate (L ref_x , L ref_y ) at the reference position L ref The normal distribution N′(μ′,σ′) of the Euclidean distance measured between 2 ). Fig. 9B Also shown is the predetermined interval I' used in step S120, which is used by the outlier detector module 63 to determine the predicted position of the fovea (F' x , F′ y ) reliability. Fig. 9B In the example in FIG. 1 , the predetermined interval I′ is set to the value of the probability density function exceeding the predetermined likelihood threshold P′. th However, the predetermined interval I' is not limited in this respect and can be set using any suitable method known to those skilled in the art for performing outlier detection. Figure 6 The probability distribution N(μ,σ 2 ), from which N′(μ′,σ′) is derived 2 ) may include a set of retinal images used to train the machine learning algorithm used by landmark location prediction module 50. However, other retinal images of the same portion of the retina as in the retinal image may also be used.

[0058] exist Fig.9A and Fig. 9B In the example shown, the distance metric estimation module 62 estimates Fig.9A In the retinal image (F′ x , F′ y ) and (L ref_x , L ref_ y), as the second distance metric in step S110. Figure 8 In step S120, the outlier detector module 63 also determines that the estimated Euclidean distance d3 is located in the normal distribution N′(μ′, σ′ 2 ). Thus, the reminder generator module 64 generates, as a reminder, a reminder indicating that the first predicted position L1 in the retinal image is unreliable.

[0059] It should be noted that the second distance metric is not limited to the Euclidean distance, but may include the Manhattan distance between the first landmark feature and the reference position. In the case where the second distance metric is the Manhattan distance, the second probability distribution is a probability distribution of the Manhattan distance between the first landmark feature and the reference position of the first landmark feature obtained from the measurement results of the Manhattan distance in the retinal image set different from the retinal image.

[0060] In addition, in some example embodiments, the second distance metric may be weighted by a second weighting factor whose value increases as the absolute value of the difference between the estimated second distance metric and the distance value corresponding to the peak of the second probability distribution increases. Subsequently, the weighted second distance metric may be used to estimate the reliability of the first signature feature by determining whether the weighted second distance metric falls within a predetermined interval of the second probability distribution, in a manner similar to that described previously in Figure 7 The method described in the weighted distance metric.

[0061] It should be noted that in some example embodiments, the outlier detector module 63 may be configured to determine the reliability of the predicted first position L1 of the first landmark feature by using data of a third probability distribution indicating the position of the first landmark feature in the retinal image to determine a third indication, the third probability distribution being based on a set of retinal images different from the retinal image, the third indication being an indication of whether the predicted first position L1 is located outside a predetermined interval of the third probability distribution, the predetermined interval of the third probability distribution including a peak of the third probability distribution. In these example embodiments, in the case where the determined third indication is an indication that the predicted first position is not located within the predetermined interval of the third probability distribution, the alert generator module 64 generates an alert indicating that the predicted first position is unreliable. As in the previous example embodiments, the predicted position of the first landmark feature may be defined by coordinates representing the position of a pixel in the retinal image, and the third probability distribution may be, for example, a two-dimensional distribution of the observed positions of the first landmark feature in the set of retinal images. As in the previous example embodiments, any suitable method may be used to set the predetermined interval of the third probability distribution.

[0062] In About Figure 1 In the first example embodiment described in Figure 9, a distance metric indicating the distance between the predicted position of the first landmark feature and the predicted position of the second landmark feature is compared to the observed distribution to determine the reliability of the predicted position, and thereby determine the reliability of any subsequent retinal image classification processing performed using the predicted position.

[0063] In a second example embodiment described herein that provides an alternative to the first example embodiment, the reliability of a predicted location of a landmark feature in a retinal image is determined using a hybrid model obtained using the ground truth location of the landmark feature in a dataset of retinal images. The dataset includes retinal images of a plurality of predetermined categories, and the probability that the retinal image belongs to each of the predetermined categories is calculated using the predicted location of the landmark feature and the hybrid model to check the reliability of the predicted location.

[0064] Fig.10 is a schematic diagram of an eye imaging system 1000 according to a second exemplary embodiment of the present invention. Fig.10 In the embodiment of the present invention, the eye imaging system 1000 includes the Figure 1 In addition, the eye imaging system 1000 includes a landmark position prediction module 55 configured to use a machine learning algorithm to predict the position of at least one landmark feature in a retinal image of at least a portion of the retina 20 of the eye 30. In an example embodiment herein, the eye imaging system 1000 and / or the landmark position prediction module 55 may include in programmable signal processing hardware Figure 2 The signal processing device 200 comprises Figure 2 The signal processing device 200 is formed and / or implemented thereby.

[0065] Fig.11 An example of a retinal image 1200 is shown, and a fovea 1210 is shown as an example of a landmark feature of the retina 20. The retinal image 1200 is composed of Fig.10 The eye imaging system 1000 shown acquires and inputs to the landmark position prediction module 55, which uses a machine learning algorithm to predict the position of the fovea 1210 in the retinal image 1200. In the present example, the machine learning algorithm includes a convolutional neural network (CNN) configured to predict the position of the fovea 1210 in the retinal image 1200, but any suitable machine learning algorithm may be used to perform the prediction. In addition, as in the present example embodiment, the retinal image 1200 has an X dimension and a Y dimension, and the landmark position prediction module 55 may predict the XY coordinates of the fovea 1210 as the position L1 of the fovea 1210 in the retinal image 1200. Fig.11 , the predicted XY coordinates of the fovea 1210 in the retinal image 1200 are represented by (Fx, Fy).

[0066] Back to Fig.10, the eye imaging system 1000 also includes a device 1010 for generating an alert indicating the unreliability of the predicted location of the landmark feature in the retinal image. The device 1010 includes a receiver module 1020, a probability indication determination module 1030, an outlier detector module 1040, and an alert generator module 1050. An example implementation of the device 1010 and the functionality of its illustrated component modules are described in more detail below. In some example embodiments, the device 1010 may be used Figure 2 The signal processing device 200 is implemented.

[0067] Fig.12 1 is a flow chart illustrating a computer-implemented method performed in a second example embodiment by which apparatus 1010 generates an alert indicating the unreliability of a predicted position of a landmark feature in a retinal image that has been predicted by landmark position prediction module 55. As in this example, the landmark feature in the retinal image may include a fovea in the retinal image. However, the landmark feature is not limited in this regard and may be any other landmark feature as described above, such as an optic disc in a retinal image.

[0068] exist Fig.12 In step S210, the receiver module 1020 receives the predicted position of the landmark feature in the retinal image predicted by the landmark position prediction module 55 using the machine learning algorithm. As in the present example embodiment, the machine learning algorithm may include a convolutional neural network (CNN) configured to predict the position of the landmark feature in the retinal image. However, any suitable machine learning algorithm capable of feature recognition may be used to perform the prediction.

[0069] exist Fig.12 In step S220, the probability indication determination module 1030 uses the received predicted position of the landmark feature and a mixture model to determine, for each of a plurality of predetermined categories of the retinal image, a corresponding probability indication indicating the probability that the retinal image belongs to the category, the mixture model including a probability distribution of the position of the landmark feature and based on the determined position of the landmark feature in the retinal image of a set of retinal images belonging to a plurality of predetermined categories.

[0070] exist Fig.12 In step S230 , the outlier detector module 1040 uses the determined probability indication to determine whether the retinal image is an outlier that does not belong to any of the predetermined categories.

[0071] As in the present example embodiment, the outlier detector module 1040 may use the determined probability indications to determine whether the retinal image does not belong to any of the predetermined categories in step S230 by determining whether each of the determined probability indications exceeds a threshold value. In the event that none of the determined probability indications exceeds the threshold value, the outlier detector module 1040 determines that the retinal image does not belong to any of the predetermined categories. In addition, in the event that at least one of the determined probability indications exceeds the threshold value, the outlier detector module 1040 determines that the retinal image belongs to at least one of the predetermined categories.

[0072] In response to the outlier detector module 1040 determining that the retinal image does not belong to any of the predetermined categories, Fig.12 In step S240, the reminder generator module 1050 generates a reminder indicating that the predicted position in the retinal image is unreliable. The reminder can be provided to the user of the eye imaging system 1000 in any suitable or desired form, such as a message or symbol displayed on a display device (such as a computer screen of the eye imaging system 1000), and additionally or alternatively, such as an audio signal, to notify the user to pay attention to the predicted position in the retinal image is unreliable.

[0073] On the other hand, in the event that the probability indication determined by the outlier detector module 1040 in step S230 does not indicate that the retinal image is an outlier, the alert generator module 1050 does not generate an alert.

[0074] As in the present example embodiment, the landmark position prediction module 55 may also classify the retinal image as belonging to one of a plurality of predetermined categories based on the predicted position of the landmark feature in the retinal image. In addition, the reminder generated when the predicted position is determined to be unreliable may indicate that the classification of the retinal image by the machine learning algorithm is unreliable. In addition, as in the present example embodiment, the landmark position prediction module 55 may classify the retinal image as belonging to one of a plurality of predetermined categories by classifying the retinal image as a retinal image of the left eye or the right eye of the subject. However, the plurality of predetermined categories are not limited to the example of two categories for left / right eye classification, and other types of classification may be performed in addition or alternatively. For example, in other example embodiments, in addition or alternatively, the landmark position prediction module 55 may be configured to classify the retinal image as a healthy eye or an unhealthy eye using the predicted position of the landmark feature. In other examples, different categories may involve different corresponding gaze directions of the eye, such as central gaze, lower gaze, upper gaze, nasal gaze, temporal gaze, each of which may be specified for the right eye or the left eye. In addition, the machine learning algorithm is not necessarily configured to perform any retinal image classification, and may be used only to predict the position of the landmark feature.

[0075] Fig.13 An example of a probability density distribution 1300 of a mixture model that may be used to determine the reliability of a predicted location of a landmark feature (the fovea in this example) in a retinal image is shown according to an example implementation of the second example embodiment. Fig.13 In the example, the mixture model includes a two-dimensional Gaussian mixture model obtained using the actual or expected position of the fovea in a dataset of retinal images. The dataset of retinal images used in this example embodiment includes retinal images of two predetermined categories, namely, a left eye category and a right eye category. In addition, in this example embodiment, the position of the fovea is represented by the XY coordinates of the pixels of the retinal image including the landmark feature, wherein the retinal image includes an X dimension and a Y dimension, such as Fig.13 as shown in . Fig.13 The third Z dimension in represents the probability density value of the fovea being located at the corresponding XY coordinate position in the retinal image.

[0076] For a distribution C with K components i A multivariate Gaussian mixture model (or K clusters, each cluster can be modeled with a different Gaussian distribution), according to the variables The probability density function is given by:

[0077]

[0078] in represents the multivariate Gaussian distribution corresponding to the ith component distribution (or cluster) and is given by:

[0079]

[0080] in and∑ i Respectively represent the i-th component distribution C that forms the Gaussian mixture model i The mean and covariance matrix of i is the i-th component distribution C i The component distribution weights of

[0081] Given the parameters of the multivariate Gaussian mixture model, the data vector Belongs to the i-th component distribution C i The posterior probability of can be calculated using Bayes' theorem:

[0082]

[0083] For the above reference Fig.13 Describe the example Gaussian mixture model, the vector (variable) of equation (1) It can be understood as representing the position of the fovea in the retinal image, that is, the XY coordinates of the pixels included in the image of the fovea. In addition, the Gaussian mixture model includes two component distributions C1 and C2, where C1 is distributed as is characterized and corresponds to the retinal image of the left eye category, while C2 is distributed The two component distributions C1 and (C2) are combined using their corresponding weights φ1 and φ2 to form Fig.13 Gaussian mixture model. Therefore, Fig.13 As shown, the Gaussian mixture model shows two peaks corresponding to the respective peaks of two different Gaussian distributions, which respectively characterize the distribution of the observation position of the fovea in the left eye and the right eye. In this example, the expected maximization technique can be used to learn from the dataset of retinal images Fig.13 The expectation maximization technique is a well-known maximum likelihood estimation method for the parameters of the Gaussian mixture model in, and its details will not be further described here.

[0084] use Fig.13 As an example, in the mixed model Fig.12 In step S210, the receiver module 1020 receives the predicted XY coordinates of the fovea of ​​the retinal image that has been processed by the CNN-based machine learning algorithm running in the landmark position prediction module 55. Fig.12 In step S220, the probability indication determination module 1030 uses the predicted XY coordinates of the fovea and the Gaussian mixture model to determine a corresponding probability indication for each of the two predetermined categories (left eye category and right eye category), the probability indication indicating the probability that the retinal image belongs to that category. More specifically, the probability indication determination module 1030 of this example embodiment calculates the predicted XY coordinate position of the fovea using equation (3) above (using equation (3) represents the posterior probability that the component distribution C1 corresponding to the left eye category belongs to In addition, the probability indication determination module 1030 calculates the posterior probability that the predicted position belongs to the component distribution C2 corresponding to the right eye category.

[0085] exist Fig.12 In step S230, the outlier detector module 1040 uses the posterior probability and To this end, as in the present example embodiment, the outlier detector module 1040 may be configured to detect whether the retinal image is an outlier that does not belong to any of the predetermined categories. and The calculated value is consistent with the predetermined threshold probability p tFor comparison. or The value of p t In the case of , the outlier detector module 1040 determines that the retinal image does not belong to any of the two predetermined categories in this example, and then in step S240, the alert generator module 1050 generates an alert indicating that the predicted position of the fovea is unreliable. On the other hand, if and Any of the values ​​of t , then the outlier detector module 1040 determines that the retinal image does belong to the corresponding category, and the reminder generator module 1050 does not generate a reminder.

[0086] Although the Gaussian mixture model in the present example embodiment is formed using a data set including two categories of retinal images, the Gaussian mixture model may alternatively be formed by any number of retinal image categories, which may include, but are not limited to, the different classification categories previously described. In addition, although the present example embodiment uses a two-dimensional Gaussian mixture model, it should be noted that higher dimensional mixture models may also be used, as will be described in more detail below. In addition, although the present example embodiment uses a multivariate Gaussian mixture model, it should be noted that, for example, where the mixture model and subsequent classification are based on distance measurements related to the location of a landmark feature (e.g., distance relative to a second landmark feature, or distance relative to a reference location, as described with respect to the first example embodiment), a univariate mixture model may be used instead.

[0087] Fig.14 A first variation of the second example embodiment is shown that also utilizes a "failure class" of the retinal image to determine whether the predicted position of the fovea in the retinal image is unreliable. In particular, the inventors have discovered that in cases where predictions fail, such as where the retinal image being processed is obscured due to a blink event, imaging artifacts, etc., the machine learning algorithm can cluster its predicted positions around a particular region. In these cases, failed predictions (i.e., predicted positions of landmark features that have been determined by the device 1010 to be unreliable) can be used to create a separate "failure class" of the retinal image and can be characterized by a (e.g., Gaussian) distribution that can be used as a component distribution in a mixture model.

[0088] More specifically, in a first variation of the second exemplary embodiment, Fig.12 The hybrid model used in step S220 is also based on the determined position of the landmark feature in the retinal image of the retinal image belonging to the failure category, and the determined position is processed by a machine learning algorithm to predict the corresponding position of the landmark feature therein, thereby generating a predicted position that has been determined to be unreliable.

[0089] For example, one can perform Fig.12 , and marking (e.g., by saving a retinal image file identifier, etc.) only those retinal images whose predicted positions have been determined to be unreliable for subsequent processing to obtain retinal images of the failure class. Fig.13 The example mixture model of , using the additional failure class of retinal images is equivalent to having a distribution of An additional component distribution C3 is introduced into Fig.13 The Gaussian mixture model in and ∑3 represent the mean and covariance matrix of the component distribution C3 corresponding to the failure category, respectively. The probability density function of the Gaussian mixture model including component distributions C1, C2, and C3 is given by:

[0090]

[0091] exist Fig.14 In step S310, the probability indication determination module 1030 ( Fig.10 ) uses the received predicted positions of the landmark features and the hybrid model to determine, for the failure class, a corresponding probability indication indicating the probability that the retinal image belongs to the failure class. As an example, assuming that the hybrid model has a probability density function of equation (4), the probability indication determination module 1030 in this example calculates the predicted position The posterior probability As an indication of probability, where:

[0092]

[0093] exist Fig.14 In step S315, the outlier detector module 1040 determines whether the determined probability indication for the failure category is higher than the determined probability indications for the plurality of predetermined categories. If the determined probability indication for the failure category is higher than the determined probability indications for the plurality of predetermined categories, then Fig.14 In step S320A, the outlier detector module 1040 determines that the retinal image belongs to the failure category (and therefore does not belong to any of the predetermined categories). Using the mixture model of equation (4) as an example of a mixture model that is also based on the failure distribution, the probability (indicator of probability of component distribution C1 of the left eye category) and the probability (the probability indication of the component distribution C2 representing the right eye category) is calculated in step S220 as:

[0094]

[0095] and

[0096]

[0097] if The calculated value is greater than and , the outlier detector module 1040 of the variation of the second exemplary embodiment determines (at step S320A) that the retinal image belongs to the failure category and therefore does not belong to any of the predetermined categories.

[0098] On the other hand, if the probability indication determined for the failure class is not higher than the probability indications determined for the plurality of predetermined classes, the outlier detector module 1040 of the variation of the second example embodiment performs the following steps: Fig.14 In step S320B, it is determined that the retinal image belongs to at least one of the predetermined retinal image categories. As an example, referring to the hybrid model defined by equation (4), if The calculated value exceeds The outlier detector module 1040 can determine that the retinal image belongs to the left eye category if The calculated value exceeds If the calculated value of , the outlier detector module 1040 can determine that the retinal image belongs to the right eye category.

[0099] Fig.15 is a flowchart showing a process performed in a second variation of the second example embodiment, which also uses a hybrid model that is also based on the determined positions of landmark features in retinal images of retinal images belonging to a failure class, the retinal images being processed by a machine learning algorithm to predict corresponding positions of the landmark features therein, thereby producing predicted positions that are determined to be unreliable.

[0100] exist Fig.15 In step S410, the probability indication determination module 1030 of the second variant uses the received predicted positions of the landmark features and the hybrid model to determine a probability indication for the failure category, the probability indication indicating the probability that the retinal image belongs to the failure category. Fig.14 The same as step S310 in.

[0101] However, compared to the first variation of the second exemplary embodiment, the probability indication determination module 1030 of the second variation of the second exemplary embodiment does not directly compare the probability indication about the failure category with the probability indication calculated about the predetermined category to determine whether the retinal image does not belong to any of the predetermined categories. Fig.15In step S415, the probability indication determination module 1030 of the second variation determines a corresponding value of a comparison metric for each of the plurality of predetermined categories, the corresponding value of the comparison metric being based on the probability indication determined for the failure category and the probability indication determined for the predetermined category.

[0102] In some examples, the probability indication determination module 1030 may be configured to determine, in step S415, a corresponding value of a ratio of the probability indication determined for the failure category and the probability indication determined for the predetermined category as a value of the comparison metric for each of the plurality of predetermined categories. For example, referring to equations (5), (6), and (7), which may be used to calculate corresponding probability indications for the failure category (C3), the left eye category (C1), and the right eye category (C2), for the probability indications determined by the vector variable Representing the predicted position of the fovea, the comparison metric for the left eye category can be calculated as the ratio Similarly, for the vector variable Representing the predicted position of the fovea, the comparison metric for the right eye category can be calculated as the ratio However, it should be noted that the comparison metric used in step S415 is not limited to the ratio of the probability indication determined for the failure category and the probability indication determined for the predetermined category. For example, in an alternative implementation of the comparison metric in step S415, the probability indication determination module 1030 may alternatively determine a corresponding value based on the difference between the probability indication determined for the failure category and the probability indication determined for the predetermined category as the value of the comparison metric for each of the plurality of predetermined categories.

[0103] exist Fig.15 In step S417, the outlier detector module 1040 of the second variant determines whether all values ​​of the comparison metrics determined for the plurality of predetermined categories exceed a predetermined threshold. If the outlier detector module 1040 determines that all values ​​of the comparison metrics determined for the plurality of predetermined categories exceed the predetermined threshold, the outlier detector module 1040 of the second variant Fig.15 On the other hand, if the outlier detector module 1040 determines that not all of the values ​​of the comparison metrics determined for the plurality of predetermined categories exceed the predetermined threshold, the outlier detector module 1040 of the second variant determines that the retinal image belongs to the failure category and therefore does not belong to any of the predetermined categories in step S420A. Fig.15 In step S420B, it is determined that the retinal image belongs to at least some of the predetermined categories.

[0104] In the above-described second exemplary embodiment and its variations, for ease of explanation, the predicted landmark position is considered to be the predicted position of a single landmark feature (i.e., the position of the fovea in the retinal image). Since the predicted position is received as a two-dimensional XY coordinate pair representing the position of a pixel in the retinal image including the image of the fovea, the hybrid model in these exemplary embodiments takes the form of a two-dimensional hybrid model. However, as illustrated below, the hybrid model is not limited to two dimensions and may have a dimension greater than 2.

[0105] In reference Fig.10 and Fig.12 In another (third) variation of the second example embodiment described, the receiver module 1020 of the third variation may receive predicted positions of multiple landmark features in a retinal image, and the device 1010 of the variation may determine whether at least one of the predicted positions is unreliable. More specifically, in the third variation, the receiver module 1020 may be configured to receive, as the predicted positions of the landmark features in the retinal image, a predicted position of a first landmark feature in a retinal image and a predicted position of a second landmark feature in a retinal image that have been predicted by a machine learning algorithm. For example, as in the previous example, the first landmark feature may be the fovea in the retinal image, and the second landmark feature may be the optic disc in the retinal image.

[0106] In addition, the probability indication determination module 1030 of the third variant can be configured to use the received predicted position of the first landmark feature, the received predicted position of the second landmark feature, and the mixture model to determine, for each of the plurality of predetermined categories, a corresponding probability indication indicating the probability that the retinal image belongs to the category, the mixture model also including a probability distribution of the position of the second landmark feature and also based on the determined position of the second landmark feature in the retinal image belonging to the set of retinal images. As an example, assuming that the predicted position of each of the first landmark feature and the second landmark feature is represented by a two-dimensional XY coordinate pair, then the mixture model of the fact probability distribution including four coordinate variables representing the observed position of the fovea and the optic disc will be a four-dimensional Gaussian mixture model, which can be written in the same general form given in equation (1). In addition, given the parameters characterizing each component distribution (corresponding to each predetermined category), the probability indication indicating the probability that the retinal image belongs to each of the predetermined categories can be calculated using equation (3) in the same manner as described in the previous example.

[0107] Furthermore, the reminder generator module 1050 of the third variation may be configured to generate, as a reminder, a reminder indicating that at least one of the predicted position of the first landmark feature in the retinal image or the predicted position of the second landmark feature in the retinal image is unreliable.

[0108] In an example embodiment described herein, an apparatus for generating an alert indicating the unreliability of a landmark position prediction made by a machine learning algorithm forms part of an eye imaging system having an image acquisition module and a landmark position prediction module. However, the apparatus described need not necessarily constitute part of an eye imaging system, but may instead be provided as an independent data processing device, such as a PC or the like, which is configured to process the predicted positions of landmark features of a retinal image. In such an application, the alert generator module of the above-described example embodiment may generate an identifier indicating a predicted position determined by the apparatus to be unreliable, and / or identifying a retinal image that has been processed by the machine learning algorithm to produce an unreliable prediction, whereby the alert generator module may use the identifier to generate an alert for a user that draws the user's attention to one or more predicted positions and / or retinal images that need to be checked by the user.

[0109] Example aspects described herein avoid limitations caused by the unreliability of computer-implemented machine learning techniques for predicting the locations of landmark features in retinal images, particularly limitations rooted in computer technology. Such computer-implemented machine learning techniques may provide inaccurate predictions of retinal landmark locations in certain circumstances, such as when a large portion of the retinal image being processed by the machine learning algorithm is occluded due to a blink event, imaging artifacts, etc. Inaccurate predictions may adversely affect the performance of downstream processing operations, the success of which depends on the accuracy of the landmark feature location predictions. By way of example only, downstream processing operations, such as automatic laterality routines for determining whether a retinal image is of the right eye or the left eye, disease pipelines for automatically diagnosing various eye diseases or disorders, and data processing operations for automatically annotating retinal images (among others), may suffer / be adversely affected if the locations of retinal landmark features cannot be accurately predicted. On the other hand, the example systems, apparatus, computer programs, and methods described herein automatically detect inaccurate and / or unreliable predictions of landmark feature locations (e.g., such as those due to occlusions in retinal images processed by machine learning algorithms) to enable correction and / or substantial avoidance of inaccurate predictions, thereby compensating for occlusions, improving the accuracy of machine learning predictions, and also improving the performance and results of downstream processing operations. Thus, by virtue of the above-mentioned capabilities of the example aspects described herein that are rooted in computer technology, the example aspects described herein improve computers and computer processing / functionality, and also improve the fields of at least retinal image processing and data processing, and also mitigate and / or substantially avoid additional computer processing and memory requirements that would otherwise be required to perform downstream operations based on inaccurate landmark feature predictions.

[0110] In the foregoing description, example aspects have been described with reference to several example embodiments. Therefore, the description should be regarded as illustrative rather than restrictive. Similarly, the drawings shown in the accompanying drawings highlighting the features and advantages of the example embodiments are presented for illustrative purposes only. The architecture of the example embodiments is sufficiently flexible and configurable that it can be utilized in ways other than those shown in the accompanying drawings.

[0111] In an example embodiment, the software embodiments of the examples presented herein may be provided as computer programs or software, such as one or more programs having instructions or instruction sequences included or stored in a product (e.g., a machine-accessible or machine-readable medium, an instruction storage device, or a computer-readable storage device, each of which may be non-transitory). Programs or instructions on non-transitory machine-accessible media, machine-readable media, instruction storage devices, or computer-readable storage devices may be used to program a computer system or other electronic device. Machine-readable media or computer-readable media, instruction storage devices, and storage devices may include, but are not limited to, floppy disks, optical disks, and magneto-optical disks or other types of media / machine-readable media / instruction storage devices / storage devices suitable for storing or transmitting electronic instructions. The techniques described herein are not limited to any particular software configuration. They may be applied in any computing or processing environment. The terms "computer-readable", "machine-accessible medium", "machine-readable medium", "instruction storage device", and "computer-readable storage device" used herein shall include any medium capable of storing, encoding, or transmitting instructions or instruction sequences to be executed by a machine, a computer, or a computer processor and causing a machine / computer / computer processor to perform any of the methods described herein. Furthermore, it is common in the art to refer to software in one form or another (e.g., program, procedure, process, application, module, unit, logic, etc.) as taking an action or causing a result. Such expressions are merely a shorthand way of stating that execution of the software by a processing system causes the processor to perform an action to produce a result.

[0112] Some embodiments may also be implemented by the preparation of application specific integrated circuits, field programmable gate arrays, or by interconnecting an appropriate network of conventional component circuits.

[0113] Some embodiments include computer program products. A computer program product may be one or more storage media, instruction storage devices, or storage devices on or in which instructions that can be used to control or cause a computer or computer processor to perform any process of the example embodiments described herein are stored. Storage media / instruction storage devices / storage devices may include, by way of example and without limitation, optical disks, ROMs, RAMs, EPROMs, EEPROMs, DRAMs, VRAMs, flash memory, flash memory cards, magnetic cards, optical cards, nanosystems, molecular memory integrated circuits, RAIDs, remote data storage / archiving / warehousing devices, and / or any other type of device suitable for storing instructions and / or data.

[0114] Some implementations stored on any of one or more computer-readable media, instruction storage devices (multiple instruction storage devices), or storage devices (multiple storage devices) include hardware for controlling the system and software for enabling the system or microprocessor to interact with a human user or other mechanism using the results of the example embodiments described herein. Such software may include, without limitation, device drivers, operating systems, and user applications. Ultimately, as described above, such computer-readable media or storage devices also include software for performing example aspects of the present invention.

[0115] Software modules for implementing the processes described herein are included in the programming and / or software of the system. In some example embodiments herein, the modules include software, but in other example embodiments herein, the modules include hardware or a combination of hardware and software.

[0116] Although a number of example embodiments of the present invention have been described above, it should be understood that they are presented as examples rather than limitations. It will be apparent to those skilled in the relevant art that various changes may be made in form and detail. Therefore, the present invention should not be limited to any of the above example embodiments, but should only be defined in accordance with the appended claims and their equivalents.

[0117] Furthermore, the purpose of the Abstract is to enable patent offices and the public in general, and especially scientists, engineers, and practitioners in the art who are not familiar with patent or legal terminology or wording, to quickly determine the nature and essence of the technical disclosure of the present application based on a cursory inspection. The Abstract is not intended to be limiting in any way with respect to the scope of the example embodiments presented herein. It should also be understood that any process recited in the claims need not be performed in the order presented.

[0118] Although this specification contains many specific embodiment details, these should not be understood as limitations on the scope of any invention or content that may be claimed, but should be understood as descriptions of features specific to the specific embodiments described herein. Certain features described in this specification in the context of separate embodiments may also be implemented in combination in a single embodiment. Conversely, the various features described in the context of a single embodiment may also be implemented individually or in any suitable sub-combination in multiple embodiments. In addition, although features may be described above as acting in a specific combination and even initially claimed as such, one or more features from the claimed combination may be deleted from the combination in some cases, and the claimed combination may be directed to a sub-combination or a variant of the sub-combination.

[0119] In some cases, multitasking and parallel processing may be advantageous. In addition, the separation of the various components in the above-described embodiments should not be understood as requiring such separation in all embodiments, and it should be understood that the described program components and systems can generally be integrated together in a single software product or packaged into multiple software products.

[0120] Now that some illustrative embodiments and embodiments have been described, it is apparent that the foregoing embodiments are illustrative rather than restrictive and have been given by way of example. Specifically, although many of the examples presented herein relate to specific combinations of devices or software elements, these elements may be combined in other ways to achieve the same purpose. Actions, elements, and features discussed only in conjunction with one embodiment are not intended to be excluded from similar roles in the embodiments or other embodiments.

[0121] The devices described herein may be implemented in other specific forms without departing from their characteristics. The foregoing embodiments are illustrative rather than limiting of the described systems and methods. Therefore, the scope of the devices described herein is indicated by the appended claims rather than the foregoing description, and therefore changes that fall within the meaning and range of equivalents of the claims are included therein.

[0122] Aspects of the present disclosure may be implemented in one or more of the following embodiments:

[0123] Item 1): A method of generating an alert indicating the unreliability of a predicted position of a landmark feature that has been predicted by a machine learning algorithm in a retinal image (1200) of at least a portion of a retina (20) of an eye (30), the method comprising:

[0124] receiving (S210) the predicted position of the landmark feature in the retinal image (1200) that has been predicted by the machine learning algorithm;

[0125] using (S220) the received predicted position of the landmark feature and a mixture model (1300) to determine, for each of a plurality of predetermined classes of retinal images, a corresponding probability indication indicating a probability that the retinal image (1200) belongs to that class, the mixture model comprising a probability distribution for the position of the landmark feature and based on the determined position of the landmark feature in retinal images of a set of retinal images having the plurality of predetermined classes;

[0126] using (S230) the determined probability indication to determine whether the retinal image (1200) does not belong to any of the predetermined categories; and

[0127] In response to determining that the retinal image (1200) does not belong to any of the predetermined categories, an alert is generated (S240) indicating that the predicted position in the retinal image (1200) is unreliable.

[0128] Item 2): The method according to item 1), wherein the determined probability indication is used to determine whether the retinal image (1200) does not belong to any of the predetermined categories by:

[0129] determining whether each of the determined probability indications exceeds a threshold;

[0130] In the event that none of the determined probability indications exceeds the threshold, determining that the retinal image (1200) does not belong to any of the predetermined categories; and

[0131] In case at least one of the determined probability indications exceeds the threshold, it is determined that the retinal image (1200) belongs to at least one of the predetermined categories.

[0132] Item 3): The method according to Item 1), wherein:

[0133] The hybrid model (1300) is also based on the determined location of the landmark feature in the retinal image of the retinal image belonging to the failure category, the retinal image of the retinal image belonging to the failure category being processed by the machine learning algorithm to predict the corresponding location of the landmark feature in the retinal image of the retinal image belonging to the failure category, thereby generating a predicted location determined to be unreliable,

[0134] The method further comprises determining, for the failure class, a probability indication indicating a probability that the retinal image (1200) belongs to the failure class using (S310) the received predicted positions of the landmark features and the hybrid model,

[0135] In case the probability indication determined for the failure category is higher than the probability indications determined for the plurality of predetermined categories, determining (S320A) that the retinal image (1200) belongs to the failure category, and

[0136] In case the probability indication determined for the failure category is not higher than the probability indications determined for the plurality of predetermined categories, it is determined (S320B) that the retinal image (1200) belongs to one of the predetermined retinal image categories.

[0137] Item 4): The method according to Item 1), wherein:

[0138] The hybrid model (1300) is also based on the determined location of the landmark feature in the retinal image of the retinal image belonging to the failure category, the retinal image of the retinal image belonging to the failure category being processed by the machine learning algorithm to predict the corresponding location of the landmark feature in the retinal image of the retinal image belonging to the failure category, thereby generating a predicted location determined to be unreliable,

[0139] The method further comprises determining, for the failure class, a corresponding probability indication indicating a probability that the retinal image (1200) belongs to the failure class using (S410) the received predicted positions of the landmark features and the hybrid model,

[0140] for each predetermined category of the plurality of predetermined categories, determining (S415) a corresponding value of a comparison metric, the corresponding value of the comparison metric being based on the probability indication determined for the failure category and the probability indication determined for the predetermined category,

[0141] In case the values ​​of the comparison metrics determined for the plurality of predetermined categories exceed a predetermined threshold, determining (S420A) that the retinal image (1200) belongs to the failure category, and

[0142] In case at least one of the values ​​of the comparison metrics determined for the plurality of predetermined categories does not exceed a predetermined threshold, it is determined (S420B) that the retinal image (1200) belongs to one of the predetermined retinal image categories.

[0143] Item 5): A method according to Item 4), wherein a corresponding value of a ratio of a probability indication determined for the failure category to a probability indication determined for the predetermined category is determined as a value of the comparison metric for each of the plurality of predetermined categories.

[0144] Item 6): A method according to Item 4), wherein a corresponding value based on the difference between the probability indication determined for the failure category and the probability indication determined for the predetermined category is determined as the value of the comparison metric for each of the multiple predetermined categories.

[0145] Item 7): The method according to any one of Items 1) to 6), wherein:

[0146] Receiving the predicted position of the landmark feature in the retinal image (1200) comprises receiving a predicted position of a first landmark feature in the retinal image (1200) and a predicted position of a second landmark feature in the retinal image (1200) that have been predicted by the machine learning algorithm,

[0147] the received predicted position of the first landmark feature, the received predicted position of the second landmark feature and the mixture model (1300) are used to determine, for each of the plurality of predetermined categories, a respective probability indication indicating a probability that the retinal image belongs to that category, the mixture model further comprising a probability distribution for the position of the second landmark feature and further based on the determined position of the second landmark feature in the retinal images belonging to the set of retinal images, and

[0148] The alert is generated in response to determining that the retinal image (1200) does not belong to any of the predetermined categories, the alert indicating that at least one of the predicted position of the first landmark feature in the retinal image or the predicted position of the second landmark feature in the retinal image is unreliable.

[0149] Item 8): A method according to Item 7), wherein the first landmark feature is the fovea (1210) in the retinal image (1200), and the second landmark feature is the optic disc in the retinal image (1200).

[0150] Item 9): The method according to any one of Items 1) to 8), further comprising:

[0151] The machine learning algorithm is used to predict the location of the landmark feature in the retinal image (1200).

[0152] Item 10): The method according to Item 9), wherein:

[0153] The machine learning algorithm further classifies the retinal image (1200) as belonging to one of the plurality of predetermined categories based on the predicted location of the landmark feature in the retinal image (1200), and

[0154] The alert is generated when the predicted position is determined to be unreliable, the alert indicating that classification of the retinal image (1200) by the machine learning algorithm is unreliable.

[0155] Item 11): A method according to Item 10), wherein classifying the retinal image (1200) as belonging to one of the multiple predetermined categories includes classifying the retinal image (1200) as a retinal image of the left eye or a retinal image of the right eye of the subject.

[0156] Item 12): A method according to any one of Items 1) to Item 11), wherein the machine learning algorithm comprises a convolutional neural network configured to predict the position of the landmark feature in the retinal image (1200).

[0157] Item 13): A computer program comprising computer program instructions, which, when executed by a computer, cause the computer to perform a method according to at least one of items 1) to 12).

[0158] Item 14): A device (1010) for generating an alert indicating the unreliability of a predicted position of a landmark feature that has been predicted by a machine learning algorithm in a retinal image of at least a portion of a retina (20) of an eye (30), the device comprising:

[0159] a receiver module (1020) configured to receive the predicted position of the landmark feature in the retinal image (1200) that has been predicted by the machine learning algorithm;

[0160] a probability indication determination module (1030) configured to determine, for each of a plurality of predetermined categories of retinal images, a corresponding probability indication indicating a probability that the retinal image (1200) belongs to that category using the received predicted position of the landmark feature and a mixture model comprising a probability distribution for the position of the landmark feature and based on the determined position of the landmark feature in retinal images of a set of retinal images having the plurality of predetermined categories;

[0161] an outlier detector module (1040) configured to use the determined probability indication to determine whether the retinal image (1200) is an outlier that does not belong to any of the predetermined categories; and

[0162] An alert generator module (1050) configured to generate an alert in response to the outlier detector module (1050) determining that the retinal image (1200) does not belong to any of the predetermined categories, the alert indicating that the predicted position in the retinal image (1200) is unreliable.

[0163] Item 15): The apparatus (1010) according to Item 14), wherein the outlier detector module (1050) is configured to use the determined probability indication to determine whether the retinal image (1200) does not belong to any of the predetermined categories by:

[0164] determining whether each of the determined probability indications exceeds a threshold;

[0165] In the event that none of the determined probability indications exceeds the threshold, determining that the retinal image (1200) does not belong to any of the predetermined categories; and

[0166] In case at least one of the determined probability indications exceeds the threshold, it is determined that the retinal image (1200) belongs to one of the predetermined categories.

[0167] Item 16): The device (1010) according to Item 14), wherein:

[0168] The hybrid model is also based on the determined location of the landmark feature in the retinal image of the retinal image belonging to the failure category, the retinal image of the retinal image belonging to the failure category being processed by the machine learning algorithm to predict a corresponding location of the landmark feature in the retinal image of the retinal image belonging to the failure category to produce a predicted location determined to be unreliable,

[0169] The probability indication determination module (1030) is further configured to determine, for the failure category, a corresponding probability indication indicating a probability that the retinal image (1200) belongs to the failure category using the received predicted positions of the landmark features and the hybrid model,

[0170] The outlier detector module (1040) is configured to determine that the retinal image (1200) belongs to the failure category if the probability indication determined for the failure category is higher than the probability indications determined for the plurality of predetermined categories, and

[0171] The outlier detector module (1040) is configured to determine that the retinal image (1200) belongs to one of the predetermined retinal image categories if the probability indication determined for the failure category is not higher than the probability indications determined for the plurality of predetermined categories.

[0172] Item 17): The device (1010) according to Item 14), wherein:

[0173] The hybrid model is also based on the determined location of the landmark feature in the retinal image of the retinal image belonging to the failure category, the retinal image of the retinal image belonging to the failure category being processed by the machine learning algorithm to predict a corresponding location of the landmark feature in the retinal image of the retinal image belonging to the failure category to produce a predicted location determined to be unreliable,

[0174] The probability indication determination module (1030) is further configured to determine, for the failure category, a probability indication indicating a probability that the retinal image (1200) belongs to the failure category using the received predicted positions of the landmark features and the hybrid model,

[0175] The probability indication determination module (1030) is further configured to determine, for each predetermined category of the plurality of predetermined categories, a corresponding value of a comparison metric, the corresponding value of the comparison metric being based on the probability indication determined for the failure category and the probability indication determined for the predetermined category,

[0176] The outlier detector module (1040) is configured to determine that the retinal image (1200) belongs to the failure category if the value of the comparison metric determined for the plurality of predetermined categories exceeds a predetermined threshold, and

[0177] The outlier detector module (1040) is configured to determine that the retinal image (1200) belongs to one of the predetermined retinal image categories if at least one of the values ​​of the comparison metrics determined for the plurality of predetermined categories does not exceed a predetermined threshold.

[0178] Item 18): An apparatus (1010) according to Item 17), wherein the outlier detector module (1040) is configured to determine a corresponding value of a ratio of a probability indication determined for the failure category to a probability indication determined for the predetermined category as a value of the comparison metric with respect to each of the plurality of predetermined categories.

[0179] Item 19): An apparatus (1010) according to Item 17), wherein the outlier detector module (1040) is configured to determine a corresponding value based on a difference between a probability indication determined for the failure category and a probability indication determined for the predetermined category as a value of the comparison metric with respect to each of the plurality of predetermined categories.

[0180] Item 20): The device (1010) according to any one of Items 14) to 19), wherein:

[0181] The receiver module (1020) is configured to receive, as the predicted positions of the landmark features in the retinal image (1200), predicted positions of the first landmark features in the retinal image (1200) and predicted positions of the second landmark features in the retinal image (1200) that have been predicted by the machine learning algorithm,

[0182] The probability indication determination module (1030) is configured to use the received predicted position of the first landmark feature, the received predicted position of the second landmark feature and the hybrid model (1300) to determine, for each of the plurality of predetermined categories, a corresponding probability indication indicating a probability that the retinal image (1200) belongs to that category, the hybrid model also comprising a probability distribution for the position of the second landmark feature and further based on the determined position of the second landmark feature in the retinal images belonging to the set of retinal images, and

[0183] The alert generator module (1050) is configured to generate, as the alert, an alert indicating that at least one of the predicted position of the first landmark feature in the retinal image (1200) or the predicted position of the second landmark feature in the retinal image (1200) is unreliable.

[0184] Item 21): An apparatus (1010) according to Item 20), wherein the first landmark feature is a fovea (1210) in the retinal image (1200), and the second landmark feature is an optic disc in the retinal image (1200).

[0185] Item 22): The device (1010) according to any one of Items 14) to 21), further comprising:

[0186] A landmark position prediction module (55) is configured to use the machine learning algorithm to predict the position of the landmark feature in the retinal image (1200).

[0187] Item 23): The device (1010) according to Item 22), wherein:

[0188] The landmark position prediction module (55) is configured to classify the retinal image (1200) as belonging to one of the plurality of predetermined categories based on the predicted position of the landmark feature in the retinal image (1200) using the machine learning algorithm, and

[0189] The alert generator module (1050) is configured to generate, as the alert, an alert indicating that the classification of the retinal image (1200) by the machine learning algorithm is unreliable.

[0190] Item 24): An apparatus (1010) according to Item 23), wherein the landmark position prediction module (55) is configured to classify the retinal image (1200) as belonging to one of the multiple predetermined categories by classifying the retinal image (1200) as a retinal image of the left eye or a retinal image of the right eye of the subject.

[0191] Item 25): An apparatus (1010) according to any one of Items 14) to 24), wherein the machine learning algorithm comprises a convolutional neural network configured to predict the position of the landmark feature in the retinal image (1200).

[0192] Item 26): An eye imaging system (1000) for acquiring a retinal image of at least a portion of a retina (20) of an eye (30), comprising:

[0193] An image acquisition module (40) configured to acquire the retinal image (510);

[0194] a landmark location prediction module (55) configured to use a machine learning algorithm to predict the location of landmark features in the retinal image; and

[0195] The device (1010) according to any one of items 14) to 25), is used to generate an alert indicating the unreliability of the position of the landmark feature in the retinal image predicted by the landmark position prediction module (55).

Claims

1. An eye imaging system (10) for acquiring a retinal image (510) of at least a portion of a retina (20) of an eye (30), comprising: An image acquisition module (40) configured to acquire the retinal image (510); a landmark position prediction module (55) configured to use a machine learning algorithm to predict a first position (L1) of a first landmark feature in the retinal image (510) and a second position (L2) of a second landmark feature in the retinal image (510) as predicted positions of the landmark features in the retinal image (510); and Means (60) for alerting a user of the eye imaging system (10) to the unreliability of at least one of the predicted positions of the landmark features, the means (60) comprising: a receiver module (61) configured to receive the predicted position (L1, L2) of the landmark feature; a distance metric estimation module (62) configured to estimate a distance metric indicative of a distance between the first landmark feature and the second landmark feature using the predicted positions (L1, L2) of the landmark features; An outlier detector module (63) is configured to use an outlier detector module (63) indicating a distance (d Fov-OD ), determining an indication of whether the estimated distance metric lies outside a predetermined interval (I) of the probability distribution, the probability distribution being obtained from measurements of the distance in a set of retinal images different from the retinal image (510), the predetermined interval (I) of the probability distribution including a peak of the probability distribution; and An alert generator module (64) is configured to generate an alert if the determined indication is an indication that the estimated distance metric is outside a predetermined interval (I) of the probability distribution, the alert indicating that at least one of the predicted positions of the landmark feature in the retinal image (510) is unreliable.

2. The eye imaging system (10) according to claim 1, wherein: The distance metric comprises one of a Euclidean distance and a Manhattan distance between the first landmark feature and the second landmark feature, and The outlier detector module (63) is configured to determine an indication of whether the estimated distance metric is outside a predetermined interval (I) of the probability distribution using data indicating a probability distribution of one of the Euclidean distance and the Manhattan distance between the first landmark feature and the second landmark feature, wherein the probability distribution of one of the Euclidean distance and the Manhattan distance between the first landmark feature and the second landmark feature is obtained from measurements of one of the Euclidean distance and the Manhattan distance in a set of retinal images different from the retinal image (510).

3. An eye imaging system (10) according to claim 1 or claim 2, wherein: The distance metric includes a weighted distance between the first landmark feature and the second landmark feature, wherein the weighted distance is the product of the distance between the first landmark feature and the second landmark feature and a weighting factor (w1), and the value of the weighting factor increases with the increase of the absolute value of the difference between the distance and the distance value corresponding to the peak of the probability distribution.

4. The eye imaging system (10) according to any one of claims 1 to 3, wherein: The distance metric estimation module (62) is further configured to estimate a second distance metric using the predicted first position (L1), the second distance metric indicating a second distance between the predicted first position (L1) and the reference position (L ref ), The outlier detector module (63) is further configured to use an indication of the difference between the first landmark feature and the reference position (L ref ) fov-ref ), determining a second indication indicating whether the estimated second distance metric is outside a predetermined interval (I′) of the second probability distribution, the predetermined interval (I′) of the second probability distribution including a peak of the second probability distribution, the second probability distribution being based on a retinal image different from the retinal image (510) centered between the first landmark feature and the reference location (L ref ) between fov-ref ) and The alert generator module (64) is configured to generate, as the alert, an alert indicating that the predicted first position in the retinal image (510) is unreliable, if the determined second indication indicates that the estimated second distance metric is not within a predetermined interval (I′) of the second probability distribution.

5. An eye imaging system (10) according to any preceding claim, wherein: The landmark position prediction module (50) is further configured to classify the retinal image (510) into one of a plurality of predetermined different categories based on the predicted positions of the landmark features in the retinal image (510) using the machine learning algorithm, and The alert generator module (64) is also configured to generate, as the alert, an alert further indicating that the classification of the retinal image (510) by the machine learning algorithm is unreliable.

6. The eye imaging system (10) according to claim 5, wherein: The landmark position prediction module (50) is configured to classify the retinal image (510) into one of the plurality of predetermined different categories by classifying the retinal image (510) as a retinal image of a left eye of the subject or a retinal image of a right eye of the subject.

7. An eye imaging system (10) according to any preceding claim, wherein: The first landmark feature is the fovea in the retinal image (510), and the second landmark feature is the optic disc in the retinal image (510).

8. A device (60) for alerting a user to the unreliability of at least one of the predicted positions of a landmark feature in a retinal image (510) of at least a portion of a retina (20) of an eye (30), wherein the user is a user of the device (60), the device (60) being configured to process a first position (L1) of a first landmark feature in the retinal image (510) and a second position (L2) of a second landmark feature in the retinal image (510) that have been predicted by a machine learning algorithm as the predicted positions of the landmark features in the retinal image (510), the device (60) comprising: a receiver module (61) configured to receive the predicted position (L1, L2) of the landmark feature; a distance metric estimation module (62) configured to estimate a distance metric indicative of a distance between the first landmark feature and the second landmark feature using the predicted positions (L1, L2) of the landmark features; an outlier detector module (63) configured to determine, using data indicative of a probability distribution of the distance between the first landmark feature and the second landmark feature, an indication of whether the estimated distance measure lies outside a predetermined interval (I) of the probability distribution, the probability distribution being obtained from measurements of the distance in a set of retinal images other than the retinal image (510), the predetermined interval (I) of the probability distribution including a peak of the probability distribution; and An alert generator module (64) is configured to generate an alert if the determined indication is an indication that the estimated distance metric is outside a predetermined interval (I) of the probability distribution, the alert indicating that at least one of the predicted positions of the landmark feature in the retinal image (510) is unreliable.

9. A method for generating an alert indicating unreliability of at least one of the predicted positions of a landmark feature in a retinal image (510) of at least a portion of a retina (20) of an eye (30), wherein the method is for processing a first position (L1) of a first landmark feature in the retinal image (510) and a second position (L2) of a second landmark feature in the retinal image (510) that have been predicted by a machine learning algorithm as the predicted positions of the landmark features in the retinal image (510), the method comprising: Receiving (S10) the predicted position (L1, L2) of the landmark feature; using (S20) the predicted position (L1, L2) of the landmark feature to estimate a distance metric indicative of a distance between the first landmark feature and the second landmark feature; The distance (d Fov-OD ), determining (S30) whether the estimated distance metric is outside a predetermined interval (I) of the probability distribution, the probability distribution being an indication of the distance (d ) between the first landmark feature and the second landmark feature in a retinal image different from the retinal image (510) Fov-OD ), the predetermined interval (I) of the probability distribution includes the peak value of the probability distribution; as well as In case the determined indication is an indication that the estimated distance metric is outside a predetermined interval (I) of the probability distribution, an alert is generated (S40), the alert indicating that at least one of the predicted positions of the landmark feature in the retinal image (510) is unreliable.

10. The method according to claim 9, wherein: The distance metric comprises one of a Euclidean distance and a Manhattan distance between the first landmark feature and the second landmark feature, and An indication of whether the estimated distance metric lies outside a predetermined interval (I) of the probability distribution is determined using data indicating a probability distribution of one of the Euclidean distance and the Manhattan distance between the first landmark feature and the second landmark feature, the probability distribution of one of the Euclidean distance and the Manhattan distance between the first landmark feature and the second landmark feature being obtained from measurements of one of the Euclidean distance and the Manhattan distance in a set of retinal images different from the retinal image (510).

11. The method according to claim 9 or claim 10, wherein: The distance metric includes a weighted distance between the first landmark feature and the second landmark feature, wherein the weighted distance is the product of the distance between the first landmark feature and the second landmark feature and a weighting factor (w1), and the value of the weighting factor increases with the increase of the absolute value of the difference between the distance between the first landmark feature and the second landmark feature and the distance value corresponding to the peak of the probability distribution.

12. The method according to any one of claims 9 to 10, further comprising: A second distance metric is estimated using the predicted first position (L1), the second distance metric indicating a second distance between the predicted first position (L1) and a reference position (L ref ), Using the indication between the first marker feature and the reference position (L ref ) between fov-ref ), determining a second indication indicating whether the estimated second distance metric is outside a predetermined interval (I′) of the second probability distribution, the predetermined interval (I′) of the second probability distribution including a peak of the second probability distribution, the second probability distribution being based on a distance between the first landmark feature and the reference location (L ) in a retinal image different from the retinal image (510); ref ) between fov-ref ) and In case the determined second indication is an indication that the estimated second distance measure is outside a predetermined interval (I') of the second probability distribution, an alert is generated as the alert indicating that the predicted first position in the retinal image (510) is unreliable.

13. The method according to any one of claims 9 to 12, further comprising: The machine learning algorithm is used to predict a first position (L1) of the first landmark feature in the retinal image (510) and a second position (L2) of the second landmark feature in the retinal image (510).

14. The method according to claim 13, wherein: The machine learning algorithm further classifies the retinal image (510) as belonging to one of a plurality of predetermined different categories based on the predicted location of the landmark feature in the retinal image (510), and The method includes generating, as the alert, an alert that further indicates that classification of the retinal image (510) by the machine learning algorithm is unreliable.

15. The method according to claim 14, wherein: Classifying the retinal image (510) as belonging to one of the plurality of predetermined different categories includes classifying the retinal image (510) as a retinal image of a left eye of a subject or a retinal image of a right eye of the subject.

16. The method according to any one of claims 9 to 15, wherein: The first landmark feature is the fovea in the retinal image (510), and the second landmark feature is the optic disc in the retinal image (510).

17. The method according to any one of claims 9 to 16, wherein: The machine learning algorithm includes a convolutional neural network configured to predict the location of the landmark feature in the retinal image (510).

18. A non-transitory computer-readable storage medium comprising computer program instructions which, when executed by a computer, cause the computer to perform the method according to at least one of claims 9 to 17.

Images