Systems and methods for detecting age-related macular degeneration

Abstract

A method for detecting the presence of an AMD in an input eye is described. In an example, an input eye image corresponding to an input eye is obtained. Once an input eye image is obtained, the input eye image may undergo a pre-processing step, such as cropping. Thereafter, the input eye image may be processed based on the view analysis model to select a macular centralized view image. The input eye image may then be processed based on the quality estimation model to explicit the quality of the input eye image. Once the input eye image is clarified to be acceptable based on a quality criterion, the input eye image may be processed based on an AMD detection model to obtain eye characteristic information to detect the presence of AMD and binary classification of the input eye image as one of an AMD positive eye and an AMD negative eye.

Description

Background Art

[0001] Eyes are the most vulnerable organs of the human body, and in order to prevent them from being affected by different visual disorders, special care is required. One of the methods of providing special care is to screen eyes regularly to diagnose different visual disorders. This screening or diagnosis can prevent abnormalities such as blurred vision, different color perceptions and more that may be caused by any developing eye diseases (such as age-related macular degeneration (AMD)). AMD is the main cause of serious, irreversible visual impairment that usually develops in the elderly. If not detected in the early stages of AMD, the impact of AMD may be expanding day by day. Usually, AMD is diagnosed by examining the retina of the eye to detect the presence and characteristics of tiny yellow deposits called drusen under the retina. However, existing tests do not provide automatic detection of AMD with accurate results for large-scale population screening at minimal cost. BRIEF DESCRIPTION OF THE DRAWINGS

[0002] Systems and / or methods according to examples of the present subject matter are now described with reference to the accompanying drawings, in which:

[0003] Figure 1A-1B A training system for training an age-related macular degeneration (AMD) detection model pipeline according to an example is illustrated;

[0004] Figure 2 An AMD detection system for detecting the presence of AMD in an eye of a subject according to one example is illustrated;

[0005] Figure 3 A method for training an AMD detection model pipeline according to an example is illustrated; and

[0006] Figures 4A-4B A method for detecting the presence of AMD in an input eye image based on a trained AMD detection model pipeline according to an example is illustrated. DETAILED DESCRIPTION

[0007] The eye is the most used sensory organ of the five senses of the human body and the eye perceives the majority of information about the world. The eye includes a retina at its back, which, when illuminated with light, has photoreceptors that convert the light into electrical signals. These electrical signals travel from the retina through the optic nerve to the brain for further processing. This electrical signal is then processed by the brain to create a visual feed or perception of surrounding objects, which we see as an image or video.

[0008] Individuals may suffer from different visual impairments. The example of this visual impairment may include but is not limited to blurred vision (refractive error), age-related macular degeneration, glaucoma, cataract, diabetic retinopathy, etc. A kind of such visual impairment is age-related macular degeneration (AMD). AMD is a common eye disease, and is the main cause of vision loss in the elderly aged 50 years and above. AMD damages the macula lutea, which is a small dot near the center of the retina and is the part responsible for sharp, central vision in the eye, which is indispensable for activities (such as reading, driving and recognizing faces) that mainly focus on visual details.

[0009] AMD is characterized by the change of subretinal existence of drusen (tiny yellow deposits) and retinal pigment epithelium (RPE), which may cause serious vision loss.The example of some additional characteristic effects that may be caused by AMD to human eyes includes but is not limited to the yellow deposits called drusen, retinal pigment epithelium (RPE) abnormalities, such as hypopigmentation or hyperpigmentation, the map-like atrophy of RPE, choroidal neovascularization (exudative, wet), serous and / or hemorrhagic detachment of sensory retina or RPE, subretinal and RPE fibrovascular hyperplasia or discoid scar.AMD may be caused by many factors, for example, aging, race, smoking, heredity etc. The damage caused by AMD may be irreversible, but correct and timely AMD diagnosis can prevent the progress of AMD.

[0010] The disease is usually diagnosed by examining the retina and detecting the above-mentioned characteristic effects on the retina. Examples of conventional methods include but may not be limited to optical coherence tomography (OCT), fluorescein angiography, fundus photography, indocyanine green, fundus autofluorescence, microperimetry, and adaptive optics. However, the process of diagnosing AMD may be complicated and requires professional medical practitioners and equipment, which may not be easily available in all medical care environments (particularly in rural areas).

[0011] Understandably, the presence of such highly specialized medical practitioners and equipment is limited to tertiary care centers, which are far from the rural population, which is the highest in India. In order to screen large populations at minimal cost, there is a need for a system for automated detection of AMD with just the right configuration to reduce the cost and operation time of such a system.

[0012] Describe the method for the existence of age-related macular degeneration (AMD) in the eyes of a subject based on certain eye characteristics detection. In one example, obtain the input eye image to be screened to detect AMD. The input eye image can be the image of the eyes of the subject being screened. This input eye image can be stored in a database repository, or can be captured in real time by a camera device. In another example, it is also possible to obtain an input eye image set for screening an image of each eye (that is, left eye and right eye) of the subject, for example. In this case, the collective analysis of these images is used to determine the existence of AMD.

[0013] Once the input eye image is obtained, a preprocessing step is performed on the input eye image. In an example, the purpose of the preprocessing step is to make the input eye image compatible with further processing stages. For example, the preprocessing step is a cropping operation, and unnecessary parts of the input eye image are removed via the cropping operation. Further proceeding, the input eye image is processed to clarify the view of the input eye image. Examples of various possible views of the input eye image include, but are not limited to, a temporal view, a nasal view, a disc-centered view, a macula-centered view, a top view, and a bottom view. In addition, in an example, in the case of a set of input eye images, the view type of each of the input eye images can be clarified.

[0014] Once the view type of the input eye image is clarified, a determination is made to check whether the view type is a macular-centered view. For example, if the clarified view type is a macular-centered view, the input eye image can be used for further processing. On the other hand, if the input eye image is not a macular-centered view, the user or subject can be prompted by displaying a visual indicator on the display device indicating an instruction to capture another input eye image. Thereafter, the input eye image is processed to identify the presence of AMD in the subject's eye. However, some additional processing operations may also be performed before identifying the presence of AMD, and one such operation is quality assessment.

[0015] In an example, the input eye image is processed to evaluate the quality of the input eye image to determine a quality score. Based on the quality score, if the quality score of the input eye image is determined to be greater than a threshold score, the input eye image can be passed for identification of the presence of AMD. On the other hand, if the determined quality score is less than the threshold score, the user or subject can be prompted to capture another input eye image by displaying a visual indicator on the display.

[0016] Once it is clear that the input eye image is acceptable based on the quality standard, it is further processed to identify eye characteristic information. In an example, the eye characteristic information corresponds to multiple eye image characteristics that indicate the presence or absence of AMD in the eyes of the subject individually or in combination. Examples of eye image characteristics include, but are not limited to, the size, area, color and quantity of drusen at the rear of the retina, the size, area, color and quantity of damage at the retinal pigment epithelium (RPE) level, the size, area, color and quantity of drusen above the retinal pigment epithelium (RPE) level, other RPE changes, choroidal neovascularization or its suggestive features, geographic atrophy, discoid scar or its combination.

[0017] Subsequently, based on the identified eye characteristic information, the AMD detection model detects the presence of AMD and performs a binary classification of the input eye image as one of AMD positive (AMD eye) and AMD negative (non-AMD eye). It can be noted that although limited examples of eye image characteristics indicating the presence or absence of AMD are described above, other such examples will still be within the scope of the present subject matter. In one example, as described subsequently, the eye characteristic information can be used as a measurement parameter for clarifying the presence of AMD.

[0018] It can be noted that the existence of AMD thus determined can be used to provide further referral for treatment or other interventions that may be needed.For example, a test result indicating AMD diagnosis can be generated.Except for the test result of AMD existing in the input eye image, a visual output can also be generated.In one example, the visual output can be in the form of an activation map.The activation map thus obtained can indicate or highlight the abnormal area in the input eye image, and the abnormal area represents the input eyes being designated as these districts or significant areas that are called AMD eyes.These and other aspects have been discussed in further detail later in this specification.

[0019] It can be noted that the above-mentioned determination involving view evaluation, quality evaluation, obtaining eye characteristic information, detecting the existence of AMD can involve various models, such as view analysis model, quality estimation model and AMD detection model. In one example, each of the above models is a model based on machine learning. In an example, the machine learning model can be a deep learning model. Although described as a unique or separate model, the view analysis model, the quality estimation model and the AMD detection model can be implemented as an AMD detection model pipeline for detecting AMD in the eyes of a subject. It can also be noted that, without departing from the scope of this subject, the AMD detection system including multiple machine learning algorithms (such as view analysis model, quality estimation model and AMD detection model) also includes an analysis engine, which performs one or more intermediate functions, such as a cropping operation, to the input eye image.

[0020] The machine learning model in the AMD detection model pipeline can be trained based on various training data. For example, the view analysis model can be trained based on training images with different views (for example, the image has a temporal view, a macula view, a optic disc centered view, a top view, a bottom view and a nasal view). Such training enables the view analysis model to identify the view type of the input eye image. Similarly, the quality estimation model can be trained based on various training images with various resolutions, contrasts, sharpness or other such attributes. In a similar manner, the AMD detection model can be trained based on training images associated with AMD and training images without AMD or not associated with AMD.

[0021] The AMD detection model can also be trained based on training eye characteristic information, which can be obtained by clinical history, comprehensive eye examination and research mode, and the research mode includes but is not limited to optical coherence tomography, visual field, intraocular pressure measurement, corneal thickness measurement, etc. In an example, the AMD detection model includes two sub-models, that is, a binary classification model trained to detect the presence or absence of AMD and a categorized classification model trained to classify eye images into various categories (such as healthy eyes, early AMD eyes, mid-term AMD eyes and late AMD eyes). In an example, the categorized classification model can only be used during training to supplement the accuracy of the detection of AMD by the binary classification model. Although training has been described in the context of view analysis model, quality estimation model and AMD detection model, this similar training process can be carried out for other models that can be implemented in the AMD detection model pipeline. Such processing will still fall within the scope of this theme without limitation.

[0022] The present method overcomes the above-mentioned technical disadvantages. For example, the above method can be implemented in a single device for effective AMD screening. Since no professional equipment or skills are required, the system implementing the present method is mobile, cost-effective and accurate for the purpose of AMD detection. For example, the implementation system allows screening without expert knowledge and can be performed on the portable retinal camera itself while ensuring the accuracy of the desired and functional level.

[0023] The descriptions provided above and the examples discussed further in the current description are merely exemplary. For example, some examples may have been described in which only one image or multiple images are considered during the training or reasoning phase. However, the current method may also be used for other instances or situations without departing from the scope of the present subject matter.

[0024] Reference Figures 1-4 illustrate the manner in which an AMD detection model pipeline is trained and used to predict the presence of AMD in an input eye image. Although aspects of the described system can be implemented in any number of different electronic devices, environments, and / or implementations, examples are described in the context of the following (one or more than one) example devices. It can be noted that the drawings of the subject matter shown here have illustrative purposes and should not be construed as limiting the scope of the claimed subject matter.

[0025] Fig. 1 illustrates a training system 102 for training a model present in an AMD detection model pipeline including a processor or memory (not shown). In an example, the training system 102 (referred to as system 102) can be coupled to a repository 104 by a network 106 communication. The repository 104 can also include training information 108. The training information 108 can include a first set of training eye images and a second set of training eye images captured from different sides and angles with different views. In an example, these multiple training eye images are images previously captured while manually screening subjects annotated to the corresponding AMD category.

[0026] In an example, the first set of training eye images may include a large data set of eye images captured from different cameras or normal pre-existing cameras with normal quality. In an example, the first set of training eye images includes eye images collected while conducting a survey or any health plan survey including images from different geographic regions of the world to provide general learning to the AMD detection model pipeline. On the other hand, the second set of training eye images includes a small data set of images captured from target cameras with different image qualities. In an example, the second set of training eye images includes images from a specific geographic region to personalize the AMD detection model pipeline for the region corresponding to a specific race and genotype.

[0027] In another example, together with multiple images, training information 108 can also include training eye characteristic information for each of multiple training images and corresponding AMD categories for representing the severity of AMD in each of the training images. The training eye characteristic information corresponds to multiple training eye image characteristics. The examples of multiple training eye image characteristics include but are not limited to the size, area, color and quantity of drusen at the rear of the retina, the size, area, color and quantity of the damage at the retinal pigment epithelium (RPE) level, the size, area, color and quantity of drusen above the retinal pigment epithelium (RPE) level, other RPE changes, choroidal neovascularization or its suggestive features, geographic atrophy, discoid scar or its combination.

[0028] In another example, the training eye image sets (i.e., the first set of training eye images and the second set of training eye images) can each include images in groups, where each group includes images of a single eye. Although the training information 108 is depicted as being obtained from a single repository (such as repository 104), it can also be obtained from multiple other sources without departing from the scope of the present subject matter. In this case, each of such multiple repositories can be interconnected by a network (such as network 106).

[0029] The network 106 may be a private network or a public network, and may be implemented as a wired network, a wireless network, or a combination of wired and wireless networks. The network 106 may also include a collection of separate networks that are interconnected and used as a single large network (such as the Internet). Examples of such separate networks include, but are not limited to, a Global System for Mobile Communications (GSM) network, a Universal Mobile Telecommunications System (UMTS) network, a Personal Communications Service (PCS) network, a Time Division Multiple Access (TDMA) network, a Code Division Multiple Access (CDMA) network, a Next Generation Network (NGN), a Public Switched Telephone Network (PSTN), a Long Term Evolution (LTE), and an Integrated Services Digital Network (ISDN).

[0030] The system 102 may also include instructions 110 and a training engine 112. In an example, the instructions 110 are retrieved from a memory and executed by a processor included in the system 102. The training engine 112 may be implemented as a combination of hardware and programming (e.g., programmable instructions) to implement various functions. In the examples described herein, this combination of hardware and programming may be implemented in several different ways. For example, the programming for the training engine 112 may be executable instructions, such as instructions 110. Such instructions may be stored on a non-transitory machine-readable storage medium, which may be directly or indirectly (e.g., via a networking component) coupled to the system 102. In an example, the training engine 112 may include processing resources (e.g., a single processor or a combination of multiple processors) to execute such instructions. In this example, the non-transitory machine-readable storage medium may store instructions (such as instructions 110) that implement the training engine 112 when executed by the processing resources. In other examples, the training engine 112 may be implemented as an electronic circuit.

[0031] The instructions 110, when executed by the processing resource, cause a training engine 112 to train a model pipeline, such as an AMD detection model pipeline 114, based on the training information 108. The instructions 110 may be executed by the processing resource for training the AMD detection model pipeline 114 based on the training information 108. The system 102 may also include a first set of (one or more) training eye images 116, a second set of (one or more) training eye images 118, training eye characteristic information 120, and an AMD category 122. In an example, the system 102 can obtain training information 108 corresponding to multiple eyes from the repository 104, and the information is related to information stored as a first set of (one or more) training eye images 116, a second set of (one or more) training eye images 118, and an AMD category 122 in the system 102 (in the further description of the present invention, the first set of (one or more) training eye images 116 and the second set of (one or more) training eye images 118 are collectively referred to as (one or more) training eye images 116, 118).

[0032] As previously described, the AMD detection model pipeline 114 (referred to as the detection model pipeline 114) may also include multiple machine learning models. Examples of such machine learning models include deep learning models. For purposes of illustration, as an example, a current method for detecting the presence of AMD has been described in which different steps are performed using one or more deep learning models. Although this example has been described with respect to a deep learning model, the foregoing method may also be implemented using other machine learning models. It may also be noted that any description provided in conjunction with a deep learning model is applicable to other machine learning models without limitation and without departing from the scope of the present subject matter. For the sake of brevity, such an example is not described. In conjunction with Figure 1B Further described is the manner in which training of multiple models within the detection model pipeline 114 can be performed.

[0033] Figure 1B 1 depicts example deep learning models that can be implemented within the detection model pipeline 114. In one example, the detection model pipeline 114 can include a view analysis model 124, a quality estimation model 126, and an AMD detection model 128. It can be noted that the detection model pipeline 114 can also include other deep learning models (such as Figure 1B ). It may also be the case that one or more models can be implemented to perform a combination of one or more functions. Such variations and combinations will still be examples of the subject matter without limitation.

[0034] With respect to training the view analysis model 124, the training eye image(s) 116, 118 may be used, wherein the training eye image(s) 116, 118 may include images with different views (e.g., temporal view, nasal view, disc-centered view, macula-centered view, top view, and bottom view). To train the quality estimation model 126, the training eye image(s) 116, 118 may include images with higher resolution, contrast, clarity, or other such attributes. The quality estimation model 126 is trained to assess the quality of the input eye images so that images with low quality may be discarded and only good quality images with higher quality are considered for further processing.

[0035] The AMD detection model 128 may then be trained based on the (one or more) training eye images 116, 118. In an example, while training, the training engine 112 may first process the AMD detection model 128 based on the (one or more) training eye image first set 116, wherein the AMD detection model 128 identifies the training eye characteristic information 120 corresponding to a plurality of eye image characteristics within the (one or more) training eye image first set 116. Examples of the plurality of eye image characteristics include, but are not limited to, the size, area, color, and number of drusen at the posterior retina, the size, area, color, and number of lesions at the retinal pigment epithelium (RPE) level, the size, area, color, and number of drusen above the retinal pigment epithelium (RPE) level, other RPE changes, choroidal neovascularization or suggestive features thereof, geographic atrophy, disciform scars, or a combination of these characteristics, etc. It may be noted that the eye image characteristics disclosed above are exemplary, and different characteristics based on the type of images present in each of the training eye image sets may be used.

[0036] It can be noted that once trained based on (one or more than one) training eye image first set 116, AMD detection model 128 is able to generally classify eyes. However, since (one or more than one) training eye image first set 116 does not include images captured by target camera devices of different qualities, AMD detection model 128 may miss specific features of eyes. For example, certain characteristics that only appear in the eyes of people in certain geographic areas. Therefore, such training enables AMD detection model 128 to widely predict the health category of input eye images. Therefore, in order to train AMD detection model 128 to more accurately predict output, (one or more than one) training eye image second set 118 is used to further train or fine-tune AMD detection model 128.

[0037] For example, while further training or fine-tuning, the training engine 112 can process the AMD detection model 128 based on the second set 118 of (one or more) training eye images, wherein the AMD detection model 128 identifies the training eye characteristic information 120 corresponding to the multiple eye image characteristics within the second set 118 of (one or more) training eye images. It can be noted that once trained based on the second set 118 of (one or more) training eye images, the AMD detection model 128 is now able to classify the eyes at a personalized level. In addition to the training images and the corresponding characteristic information, the AMD detection model 128 can also be trained based on the AMD categories 122 associated with each training image. In the example, the AMD category 122 represents the state of the corresponding training eye image. Therefore, based on the AMD category 122 (i.e., AMD positive or AMD negative), the training engine 112 accordingly identifies and learns the eye characteristic information corresponding to the training eye image of AMD positive.

[0038] In an example, the AMD detection model 128 includes two sub-models, i.e., a binary classification model trained to detect the presence or absence of AMD and a categorized classification model trained to classify eye images into various stages (such as healthy eyes, early AMD eyes, mid-stage AMD eyes, and late AMD eyes). In an example, the categorized classification model can be used only during training to supplement the accuracy of the binary classification model in detecting AMD.

[0039] As will be discussed later, the view analysis model 124, the quality estimation model 126, and the AMD detection model 128, when trained, can be used to perform various tasks sequentially or simultaneously, based on which the presence of AMD in the subject's eye can be determined. In addition, as also described above, the training of the view analysis model 124, the quality estimation model 126, and the AMD detection model 128 can be performed in any order and at different times. As can be appreciated, although one or more common training data sets can be used, the training of any one deep learning model in the detection model pipeline 114 is independent of the training of another model.

[0040] Once trained, the AMD detection model 128 can be used to determine or predict the presence of AMD in an input eye image. For example, the trained AMD detection model 128 identifies specific eye image characteristic information related to the input eye image to determine an AMD category indicating the presence of AMD in the input eye image. Once the AMD category of the input eye image is determined, a treatment suitable for that category is recommended or determined to cure or prevent enhancement of visual impairment.

[0041] Combination Figure 2Further described is the manner in which the detection model pipeline can be used to detect AMD in the eye of a subject.

[0042] Figure 2 A clinical environment 200 with an age-related macular degeneration (AMD) detection system 202 is illustrated, and the age-related macular degeneration (AMD) detection system 202 is used to determine the AMD category to which an input eye image 204 of a subject 206 may belong. In an example, the AMD detection system 202 (referred to as system 202) is one of a mobile phone, a tablet computer, and any other portable computing device. In an example, the portable computing device attached to the system 202 can capture a retinal image of the subject's eye. The input eye image 204 can be an image of the eye of a subject 206 being screened for AMD diagnosis. In an example, the input eye image 204 is a retinal image. In an example, the system 202 can analyze multiple eye image characteristics of the input eye image 204 based on a trained detection model pipeline 114.

[0043] Similar to system 102, system 202 may also include instructions 208 and an investigation engine 210. In an example, instructions 208 are extracted from a memory and executed by a processor included in system 202. Investigation engine 210 may be implemented as a combination of hardware and programming (e.g., programmable instructions) to implement various functions. In the examples described herein, this combination of hardware and programming may be implemented in several different ways. For example, the programming of investigation engine 210 may be executable instructions, such as instructions 208. Such instructions 208 may be stored on a non-transitory machine-readable storage medium, which may be directly or indirectly (e.g., via a networking component) coupled to system 202. In an example, investigation engine 210 may include processing resources (e.g., a single processor or a combination of multiple processors) to execute such instructions. In this example, a non-transitory machine-readable storage medium may store instructions (such as instructions 208) that implement investigation engine 210 when executed by a processing resource. In other examples, investigation engine 210 may be implemented as an electronic circuit.

[0044] In one example, the investigation engine 210 can utilize the trained detection model pipeline 114 to determine whether AMD is present in the subject's eye based on processing of the input eye image 204 of the subject 206. Figure 1A-1B The method discussed is used to train the detection model pipeline 114. In addition, as mentioned above, the detection model pipeline 114 may also include a trained view analysis model 124, a quality estimation model 126, and an AMD detection model 128.

[0045] The system 202 may also include (one or more) input eye images 212, view types 214, eye characteristic information 216, detection results 218, and activation maps 220. It may be noted that the aforementioned data elements are generated by the investigation engine 210 using the detection model pipeline 114 and in response to the execution of the (one or more) instructions 208. These aspects and further details will be discussed in the following paragraphs.

[0046] In operation, an input eye image may be obtained, such as an input eye image 204 of an eye of a subject 206 being screened for the presence of AMD. For example, the input eye image 204 may be captured by any image sensing subsystem that may be present within the system 202. In an example, the image sensing subsystem may be a retinal camera device that is mounted on the system 202 itself or may be removably integrated with the system 202. In another example, instead of having a single input eye image, a collection of input eye images may be obtained, in which there are images corresponding to individual eyes.

[0047] Once the input eye image 204 is obtained, it is stored in the system 202 as (one or more than one) input eye image 212 (referred to as the input eye image 212). Thereafter, the investigation engine 210 may perform a pre-processing step on the input eye image 212. The purpose of the pre-processing step is to make the input eye image 212 compatible with further processing stages and to remove unnecessary parts of the input eye image 212. Specifically, the pre-processing step includes a cropping operation, and the unnecessary parts of the input eye image 212 are removed via the cropping operation.

[0048] Continuing further, the input eye image 212 is processed to evaluate the view of the input eye image 212. In one example, the investigation engine 210 can utilize the trained view analysis model 124 of the detection model pipeline 114 to clarify the view type (such as the view type 214) of the input eye image 212. In an example, the trained view analysis model 124 evaluates various features of the input eye image 212 to determine the view of the input eye image. Examples of various possible views of the input eye image 212 include, but are not limited to, a temporal view, a nasal view, a disc-centered view, a macula-centered view, a top view, and a bottom view. Furthermore, in one example, in the case of a set of input eye images, the view type of each input eye image can be clarified.

[0049] Once the view type 214 of the input eye image 212 is ascertained, it is determined whether the ascertained view type 214 is a macular-centered view. If the ascertained view type 214 is a macular-centered view, the input eye image 212 may be processed by the investigation engine 210 for further analysis. In an example, if the input eye image is not a macular-centered view, the user or subject 206 may be prompted to capture another input eye image 212 by displaying an indicator on a display of the system 202, or the user or subject 206 may choose to continue with the initially captured or obtained input eye image 212. In the event that a set of input eye images is obtained, an image having a macular-centered view is selected for each eye of the subject 206 among other images included in the set of input eye images, and the image having a macular-centered view is designated as the set of input eye images.

[0050] Continuing further, the input eye image 212 is processed using the trained detection model pipeline 114 to assess the quality of the input eye image 212. In one example, the investigation engine 210 can utilize the trained quality estimation model 126 of the detection model pipeline 114 to determine the quality score of the input eye image 212. In an example, the quality score depicts an acceptance level of the input eye image. For example, images with higher quality scores are accepted, and images with lower quality scores are discarded. In an example, high quality images are used because these images include clearer feature details.

[0051] Returning to this example, once the quality score of the input eye image 212 is determined, if the quality score of the input eye image is greater than the threshold score, the input eye image 212 can be processed by the survey engine 210 using the AMD detection model 128 to detect the presence of AMD in the subject's eye. In another example, if the quality score is less than the threshold score, the user or subject 206 can be prompted to capture another input eye image 212 by displaying an indicator on the display of the system 202, or the user or subject 206 can choose to continue with the input eye image 212 initially captured or obtained. Both examples are supplementary and therefore have no impact on the scope of the present subject. It can be understood that the quality of the explicit input eye image 212 can depend on various features or attributes of the input eye image 212 as detected by the quality estimation model 126. It can be noted that in the example, without departing from the scope of the present subject, the user or subject 206 can choose to continue subsequent processing based on the input eye image 212 without evaluating the quality of the image.

[0052] The input eye image 212 (once determined to be acceptable or as the case may be) may be further processed by the survey engine 210 using the trained detection model pipeline 114 to identify eye characteristic information, such as eye characteristic information 216 of the input eye image 212. In one example, the survey engine 210 may utilize the trained AMD detection model 128 of the detection model pipeline 114 to identify the eye characteristic information 216 of the input eye image 212. In an example, the eye characteristic information 216 corresponds to a plurality of eye image characteristics that, individually or in combination, indicate the presence or absence of AMD in the subject's eye.

[0053] To this end, the survey engine 210 may use the AMD detection model 128 to identify one or more eye image characteristics. Examples of eye image characteristics include, but are not limited to, the size, area, color, and number of drusen behind the retina, the size, area, color, and number of lesions at the retinal pigment epithelium (RPE) level, the size, area, color, and number of drusen above the retinal pigment epithelium (RPE) level, other RPE changes, choroidal neovascularization or suggestive features thereof, geographic atrophy, disciform scarring, or a combination thereof.

[0054] Based on the eye characteristic information 216 thus determined using the trained AMD detection model 128, the investigation engine 210 may further process the eye characteristic information 216 based on the AMD detection model 128 to determine a detection result, such as a detection result 218 of the input eye image 212 corresponding to the subject's eye. In an example, the detection result 218 indicates the absence or presence of AMD within the input eye image 212 of the subject's eye. In another example, in the case of multiple input eye images, the investigation engine 210 may determine the detection result 218 indicating the absence or presence of AMD in the subject's eye as a whole by considering the input eye images corresponding to the respective eyes of the subject 206. Based on the detection result 218, the investigation engine 210 may classify the input eye image 212 as one of an AMD-positive eye and an AMD-negative eye.

[0055] It can be noted that the test result 218 thus determined can be used to provide further referral for treatment or other interventions that may be needed. For example, the test result 218 can indicate the diagnosis of AMD. Based on the state represented by the test result 218, appropriate action can be taken. Although it is described as being obtained by processing the above-mentioned example of the eye image characteristics included in the eye characteristic information 216, without departing from the scope of this theme, the detection of the presence of AMD can be performed by considering any other eye image characteristics. This example will still fall within the scope of this theme without any limitation.

[0056] In addition, the investigation engine 210 may also generate an activation map 220 to be displayed on a display device of the system 202. In an example, the activation map 220 depicts significant areas within the input eye image 212 that triggered the detection of AMD within the input eye image 212 of the subject 206. In addition, the displayed activation map may also be used by a medical practitioner to identify areas where disease has occurred.

[0057] In another example, the system 202 may be connected via a network ( Figure 2 The network may be a private network or a public network, and may be implemented as a wired network, a wireless network, or a combination of wired and wireless networks, and may be similar to network 106 (e.g., Figure 1A All of the above disclosed steps that may be performed by the investigation engine 210 of the system 202 may be implemented or performed by a central computing server on behalf of the system 202 to reduce the computational burden on the edge of the network.

[0058] Figure 3 An example method 300 for training a detection model pipeline according to an example of the present subject matter is illustrated. The order in which the above method is described is not intended to be construed as a limitation, and some of the described method blocks can be combined in a different order to implement the method or an alternative method.

[0059] In addition, the above method can be implemented in suitable hardware, computer-readable instructions, or a combination thereof. The steps of this method can be performed by a system under the instruction of machine-executable instructions stored on a non-transitory computer-readable medium, or by a dedicated hardware circuit, a microcontroller, or a logic circuit. For example, the method can be performed by a training system such as system 102. In implementation, the method can be performed under "as a service" delivery (wherein the system 102 operated by the provider receives the programmable code). In this document, some examples are also intended to cover non-transitory computer-readable media (e.g., digital data storage media) that are computer-readable and encode computer-executable instructions, wherein the instructions perform some or all of the steps of the above method.

[0060] In an example, method 300 can be implemented by system 102 for training several deep learning models based on training information (such as training information 108). At box 302, training information for training a detection model pipeline can be obtained. For example, system 102 can obtain training information 108. Training information 108 can be obtained from a repository (such as repository 104). Training information 108 may include a first set of training eye images captured from different sides and angles with different views and a second set of training eye images. In an example, these multiple training eye images are these images of corresponding AMD categories previously captured while manually screening a subject with annotations.

[0061] In an example, (one or more than one) first set of training eye images may include a large data set of eye images captured from different cameras with normal quality or normal pre-existing cameras. In an example, (one or more than one) first set of training eye images includes eye images collected while conducting a survey or any health program survey including images from different geographic regions of the world to provide general learning to a detection model pipeline (such as AMD detection model pipeline 114). On the other hand, a second set of training eye images includes a small data set of images captured from target cameras with different image qualities. In an example, the second set of training eye images includes images from a specific geographic region to personalize the AMD detection model pipeline 114 for that region.

[0062] In another example, together with multiple images, training information 108 may include training eye characteristic information for each of multiple training images and corresponding AMD categories for representing the severity of AMD in each of the training images. The training eye characteristic information corresponds to multiple training eye image characteristics. The examples of multiple training eye image characteristics include but are not limited to the size, area, color and quantity of drusen at the rear of the retina, the size, area, color and quantity of the damage at the retinal pigment epithelium (RPE) level, the size, area, color and quantity of drusen above the retinal pigment epithelium (RPE) level, other RPE changes, choroidal neovascularization or its suggestive features, geographic atrophy, discoid scar or its combination.

[0063] In another example, the sets of training eye images (ie, the first set of training eye images and the second set of training eye images) may each include images in groups, where each group includes an image of a single eye.

[0064] At block 304, a view analysis model present within the detection model pipeline may be trained. In one example, the training engine 112 may train the view analysis model 124 using the training eye image(s) 116, 118, wherein the training eye image(s) 116, 118 may include images with different views (e.g., temporal view, nasal view, disc-centered view, macula-centered view, top view, and bottom view). Thus, the view analysis model 124 is trained to understand the view of the input eye image.

[0065] At block 306, a quality estimation model of the detection model pipeline may be trained. For example, the training engine 112 of the system 102 may train a quality estimation model 126 based on (one or more than one) training eye images 116, 118, which may include images with higher resolution, different kinds of contrast, sharpness, or other such attributes. The quality estimation model 126 is trained to assess the quality of input eye images so that images with low quality may be discarded and only good quality images with higher quality are considered for further processing.

[0066] At box 308, the AMD detection model of the detection model pipeline can be trained. For example, while training the AMD detection model 128, the training engine 112 can first process the AMD detection model 128 based on the first set of (one or more than one) training eye images 116, wherein the AMD detection model 128 identifies the training eye characteristic information 120 corresponding to multiple eye image characteristics within the first set of (one or more than one) training eye images 116. Examples of multiple eye image characteristics include, but are not limited to, the size, area, color, and number of drusen at the rear of the retina, the size, area, color, and number of lesions at the retinal pigment epithelium (RPE) level, the size, area, color, and number of drusen above the retinal pigment epithelium (RPE) level, other RPE changes, choroidal neovascularization or its suggestive features, geographic atrophy, discoid scars, or a combination of these features, etc. It can be noted that the eye image characteristics disclosed above are exemplary, and different characteristics based on the type of images present in each of the training eye image sets can be used.

[0067] It can be noted that once trained based on (one or more than one) training eye image first set 116, AMD detection model 128 is able to generally classify eyes. However, since (one or more than one) training eye image first set 116 does not include images captured by target camera devices of different qualities, AMD detection model 128 may miss specific features of eyes. For example, certain characteristics that only appear in the eyes of people in certain geographic areas. Therefore, such training enables AMD detection model 128 to widely predict the health category of input eye images. Therefore, in order to train AMD detection model 128 to more accurately predict output, (one or more than one) training eye image second set 118 is used to further train or fine-tune AMD detection model 128.

[0068] At box 310, the AMD detection model is then trained based on the second set of training eye images. For example, while further training or fine-tuning, the training engine 112 can process the AMD detection model 128 based on (one or more than one) training eye image second set 118, wherein the AMD detection model 128 identifies the training eye characteristic information 120 corresponding to the multiple eye image characteristics in the (one or more than one) training eye image second set 118. It can be noted that once trained based on (one or more than one) training eye image second set 118, the AMD detection model 128 is now able to classify the eyes at the personalized level. In addition to the training images and the corresponding characteristic information, the AMD detection model 128 can also be trained based on the AMD categories 122 associated with each training image. In the example, the AMD category 122 represents the state of the corresponding training eye image. Therefore, based on the AMD category 122 (i.e., AMD positive or AMD negative), the training engine 112 correspondingly identifies and learns the eye characteristic information corresponding to the training eye image of AMD positive.

[0069] In an example, the AMD detection model 128 includes two sub-models, i.e., a binary classification model trained to detect the presence or absence of AMD and a categorized classification model trained to classify eye images into various stages (such as healthy eyes, early AMD eyes, mid-stage AMD eyes, and late AMD eyes). In an example, the categorized classification model can be used only during training to supplement the accuracy of the binary classification model in detecting AMD.

[0070] In an example, once trained, the detection model pipeline 114 can be used to classify an input eye image as one of AMD positive and AMD negative. Figures 4A-4B Further described are method steps involved in classifying an input eye image as one of AMD positive and AMD negative.

[0071] Figures 4A-4B An example method 400 is illustrated for classifying an input image into one of AMD positive (AMD eye) and AMD negative (non-AMD eye) categories. Figure 3 , the order in which the above method is described is not intended to be construed as limiting, and some of the described method blocks may be combined in a different order to implement the method or an alternative method. Based on the present method as described in the context of the example method 400, eye image characteristics of an input eye image are analyzed based on the trained detection model pipeline 114.

[0072] In addition, the above method 400 can be implemented in suitable hardware, computer-readable instructions, or a combination thereof. The steps of this method can be performed by the system under the instructions of machine-executable instructions stored on a non-transient computer-readable medium, or by a dedicated hardware circuit, a microcontroller, or a logic circuit. For example, the method can be performed by an AMD detection system such as system 202. In implementation, the method can be performed under an "as a service" delivery model (wherein a system 202 operated by a provider receives programmable code). In this article, some examples are also intended to cover non-transient computer-readable media (e.g., digital data storage media) that are computer-readable and encode computer-executable instructions, wherein the instructions perform some or all of the steps of the above method.

[0073] At block 402, an input eye image is obtained. For example, the input eye image 204 may be captured by any image sensing subsystem that may be present within the system 202. In an example, the image sensing subsystem may be a retinal camera device that is mounted on the system 202 itself or may be removably integrated with the system 202. In another example, instead of having a single input eye image, a set of input eye images may be obtained, in which there are images corresponding to individual eyes.

[0074] At block 404, a cropping operation may be performed on the input eye image thus obtained. For example, once the input eye image 204 (or the input eye image 212) is obtained, the survey engine 210 may perform a preprocessing step on the input eye image 212. The purpose of the preprocessing step is to make the input eye image 212 compatible with further processing stages and to remove unnecessary parts of the input eye image 212. Specifically, the preprocessing step includes a cropping operation, and the unnecessary parts of the input eye image 212 are removed via the cropping operation.

[0075] At block 406, a view type of the input eye image is determined by processing the input eye image. For example, the investigation engine 210 may utilize the trained view analysis model 124 of the detection model pipeline 114 to determine a view type (such as the view type 214) of the input eye image 212. In an example, the trained view analysis model 124 evaluates various features of the input eye image 212 to determine a view of the input eye image. Examples of various possible views of the input eye image 212 include, but are not limited to, a temporal view, a nasal view, a disc-centered view, a macula-centered view, a top view, and a bottom view. Furthermore, in an example, in the case of a set of input eye images, the view type of each of the input eye images may be determined.

[0076] At box 408, a determination is made to clarify whether the view type of the input eye image is a macular-centered view. For example, if the clarified view type 214 is a macular-centered view, the input eye image 212 can be further processed by the survey engine 210 using the detection model pipeline 114 as will be described in a later step (the "yes" path from box 408). However, if the input eye image 212 is not a macular-centered view, the subject 206 can be prompted to capture or obtain another input eye image 212 by displaying an indicator on the display of the system 202 (the "no" path from box 408), or the subject 206 can choose to continue with the initially captured or obtained input eye image 212. In the event that a set of input eye images is obtained, an image with a macular-centered view is selected for each eye of the subject 206 among the other images included in the set of input eye images, and the image is designated as the set of input eye images.

[0077] At block 410, the input eye image may be further processed based on the quality estimation model to determine its acceptability. For example, the investigation engine 210 may utilize the trained quality estimation model 126 of the detection model pipeline 114 to determine a quality score of the input eye image 212. In an example, the quality score depicts an acceptance level of the input eye image. For example, images with higher quality scores are accepted, and images with lower quality scores are discarded. In an example, high quality images are used because these images include clearer feature details.

[0078] At box 412, it is determined whether the determined quality score of the input eye image is acceptable. For example, if the quality score of the input eye image is determined to be greater than a threshold score, the input eye image 212 can be processed by the survey engine 210 using the AMD detection model to detect the presence of AMD in the subject's eye (the "yes" path from box 412). However, if the determined quality score is less than the threshold score, the user or subject 206 can be prompted to capture another input eye image 212 by displaying an indicator on the display of the system 202 (the "no" path from box 412), or the user or subject 206 can choose to continue with the input eye image 212 initially captured or obtained. These two examples are complementary and therefore have no impact on the scope of the present subject. It can be understood that the quality of the explicit input eye image 212 can depend on various features or attributes of the input eye image 212 as detected by the quality estimation model 126. It can be noted that in the example, without departing from the scope of the present subject, the user or subject 206 can choose to continue subsequent processing based on the input eye image 212 without evaluating the quality of the image.

[0079] At block 414, the input eye image may be further processed based on the AMD detection model to identify eye characteristic information of the input eye image. For example, the survey engine 210 may utilize the trained AMD detection model 128 of the detection model pipeline 114 to identify eye characteristic information 216 of the input eye image 212. In an example, the eye characteristic information 216 corresponds to a plurality of eye image characteristics that individually or in combination indicate the presence or absence of AMD in the subject's eye.

[0080] To this end, the survey engine 210 may use the AMD detection model 128 to identify one or more eye image characteristics. Examples of eye image characteristics include, but are not limited to, the size, area, color, and number of drusen behind the retina, the size, area, color, and number of lesions at the retinal pigment epithelium (RPE) level, the size, area, color, and number of drusen above the retinal pigment epithelium (RPE) level, other RPE changes, choroidal neovascularization or suggestive features thereof, geographic atrophy, disciform scarring, or a combination thereof.

[0081] At box 416, the determined eye characteristic information can be further processed based on the AMD detection model to determine a detection result indicating the presence or absence of AMD in the subject's eye. For example, the survey engine 210 can further process the eye characteristic information 216 based on the AMD detection model 128 to determine a detection result 218 of the input eye image 212 corresponding to the subject's eye. In the example, the detection result 218 represents the absence or presence of AMD within the input eye image 212 of the subject's eye. In another example, in the case of multiple input eye images, the survey engine 210 can determine the detection result 218 indicating the absence or presence of AMD in the subject's eye as a whole by considering the input eye images corresponding to the respective eyes of the subject 206. Based on the detection result 218, the survey engine 210 classifies the input eye image 212 as one of an AMD-positive eye and an AMD-negative eye.

[0082] It can be noted that the test result 218 thus determined can be used to provide further referral for treatment or other interventions that may be needed. For example, the test result 218 can indicate the diagnosis of AMD. Based on the state represented by the test result 218, appropriate action can be taken. Although it is described as being obtained by processing the above-mentioned example of the eye image characteristics included in the eye characteristic information 216, without departing from the scope of this theme, the detection of the presence of AMD can be performed by considering any other eye image characteristics. This example will still fall within the scope of this theme without any limitation.

[0083] In addition, the investigation engine 210 may also generate an activation map 220 to be displayed on a display device of the system 202. In an example, the activation map 220 depicts significant areas within the input eye image 212 that triggered the detection of AMD within the input eye image 212 of the subject 206. In addition, the displayed activation map may also be further used by a medical practitioner to identify areas where disease has occurred.

[0084] Although examples of the present disclosure have been described in language specific to structural features and / or methods, it should be understood that the appended claims are not necessarily limited to the specific features or methods described. Rather, the specific features and methods are disclosed and described as examples of the present disclosure.

Claims

1. A system, include: processor; as well as a survey engine coupled to the processor to: obtaining an input eye image corresponding to an eye of a subject, wherein the subject is being screened for the presence of age-related macular degeneration (AMD); Using an AMD detection model pipeline, wherein the AMD detection model pipeline is trained based on training information, the training information comprising training images associated with the AMD and training eye characteristic information corresponding to a plurality of training eye image characteristics of respective ones of the training images, wherein, in the case of feeding the input eye image, wherein, when using the AMD detection model, the AMD detection model is used to: determining a view type of the input eye image; identifying eye characteristic information corresponding to a plurality of input eye image characteristics from the input eye image in a case where the view type is determined to be a macula-centered view; and A detection result indicating the presence of AMD in the eye of the subject is generated based on the eye characteristic information.

2. The system according to claim 1, in, The survey engine is used to: A cropping operation is performed on the input eye image to remove irrelevant parts of the input eye image.

3. The system according to claim 1, in, The AMD detection pipeline is used to: In case the view type of the input eye image is a view other than a macular-centered view, the input eye image is discarded.

4. The system according to claim 1, in, The survey engine is used to: obtaining a plurality of input eye images, wherein the plurality of input eye images comprises a first set of eye images corresponding to a left eye and a second set of eye images corresponding to a right eye, wherein respective ones of the plurality of input eye images correspond to a different view of an eye of the subject; Feeding the plurality of input eye images into the AMD detection model pipeline, wherein in the case of feeding the AMD detection model pipeline is for: determining a view of each of the plurality of input eye images; Based on the determined views, specifying an image with a macula-centered view as an input eye image for each eye; identifying, for each eye, eye characteristic information corresponding to the designated image; and A detection result indicating the presence of AMD in the subject's eyes is generated in combination based on the eye characteristic information of the respective eyes.

5. The system according to claim 1, in, The AMD detection model pipeline includes multiple deep learning models, which are selected from the group consisting of view analysis models, quality estimation models, and AMD detection models.

6. The system according to claim 1, in, The survey engine is used to: performing a quality estimation of the input eye image to generate a quality score; In a case where it is determined that the quality score is greater than a threshold quality score, determining a view type of the input eye image; or In case it is determined that the quality score is less than the threshold quality score, discarding the input eye image; as well as The user is prompted to obtain or capture a new input eye image.

7. The system according to claim 1, in, The multiple input eye image characteristics include: size, area, color and number of drusen at the posterior retina, size, area, color and number of lesions at the retinal pigment epithelium, i.e., RPE level, size, area, color and number of drusen above the retinal pigment epithelium, i.e., RPE level, other RPE changes, choroidal neovascularization or its suggestive features, geographic atrophy, disciform scars or a combination thereof.

8. The system according to claim 1, in, The survey engine is also used to: An activation map is generated that highlights abnormal regions in the input eye image, wherein the highlighted regions have resulted in designation of the input eye as an AMD-positive eye.

9. A method, include: obtaining training information including a first set of training eye images and a second set of training eye images, wherein each of the training eye images is associated with a corresponding training eye image characteristic and an age-related macular degeneration (AMD) category; training an AMD detection model pipeline based on the first set of training eye images, corresponding training eye characteristic information, and associated AMD categories, wherein the training eye characteristic information corresponds to a plurality of training eye image characteristics; and The AMD detection model pipeline is then trained based on the second set of training eye images, corresponding training eye characteristic information, and associated AMD categories.

10. The method according to claim 9, in, The multiple training eye image characteristics include: size, area, color and number of drusen at the posterior retina, size, area, color and number of lesions at the retinal pigment epithelium, i.e., RPE level, size, area, color and number of drusen above the retinal pigment epithelium, i.e., RPE level, other RPE changes, choroidal neovascularization or its suggestive features, geographic atrophy, disciform scars or a combination thereof.

11. The method according to claim 9, in, The first set of training eye images comprises a large dataset of training eye images captured from a camera device having pre-existing image capture capabilities, wherein the AMD detection model pipeline is used to learn eye characteristic information of a general population with AMD when trained using the first set of training eye images.

12. The method according to claim 9, in, The second set of training eye images comprises a small dataset of images from a specific geographic area captured from a target camera device having specific image capture capabilities, wherein the AMD detection model pipeline is personalized for a specific population with AMD when trained using the second set of training eye images.

13. The method according to claim 9, in, The AMD detection model pipeline includes multiple deep learning models, which are selected from the group consisting of view analysis models, quality estimation models, and AMD detection models.

14. The method according to claim 9, in, The method includes evaluating the quality of input eye images through a trained AMD detection model pipeline to discard images with unacceptable quality.

15. The method according to claim 9, in, The method includes determining the view type of the input image through a trained AMD detection model pipeline to only accept images with a macula-centered view, wherein the input eye image has one of a temporal view, a nasal view, a disc-centered view, a macula-centered view, a top view, and a bottom view.

Images