Method and system for training machine learning network and processing images

Abstract

The present disclosure relates to a computer-based method of training a machine learning network to predict progression or other aspects of a disease state of a subject and a method of processing an image using the training method. A training image is synthesized by superimposing the first image on a second image of the same anatomical region and recording its disease progression status. At least an initial layer of a machine learning model is trained on a first image training set comprising a number of such synthetic training images. At least some of the pre-trained initial layers of the machine learning model are migrated to another machine learning model and the subsequent layers are trained. Another training set is used that includes a plurality of composite images of the same anatomical region of the same subject at different times and having associated disease progression states.

Description

Technical Field

[0001] The present invention relates to a method for training a machine learning network and processing images, in particular images of anatomical regions of a subject captured at different points in time. Background Art

[0002] Significant advances have been made in medical imaging technologies that can provide unique visual representations of different tissues and organs in the body and provide insights into normal and abnormal states of a subject's body. These imaging technologies include methods such as X-ray, computed tomography (CT), positron emission tomography (PET), magnetic resonance imaging (MRI), digital mammography, and optical coherence tomography (OCT). With the increase in imaging modalities and based on the rapid development of artificial intelligence, clinicians' monitoring and diagnosis of a subject's disease state are increasingly supported by machine learning technologies.

[0003] However, using machine learning techniques such as convolutional neural networks (CNNs) to detect disease progression or differences between images faces several challenges. First, it is difficult to obtain a large number of high-quality medical images due to privacy issues and the need for expert annotation. These images are essential for training CNNs, and the scarcity of this data may limit the performance of the algorithm. In addition, medical images may be noisy or have differences due to different imaging techniques, which complicates the training process.

[0004] Second, class imbalance in the dataset can lead to biased models that perform poorly at detecting the most important classes, such as disease progression. Disease progression often involves subtle changes in images that can be difficult for CNNs to detect, making it challenging to develop reliable models that generalize well to unseen data. Another challenge is the lack of interpretability of CNNs. As “black box” models, it is difficult to understand how CNNs make their decisions, making it difficult for medical professionals to trust their predictions and limiting their adoption in clinical settings.

[0005] Finally, training CNNs on large medical image datasets can be computationally intensive, requiring specialized hardware like GPUs and large amounts of memory. This can pose a challenge for researchers who have limited access to such resources.

[0006] One application of machine learning techniques being developed in the medical imaging modality is the interpretation of retinal images of patients with AMD. Age-related macular degeneration (AMD) is the leading cause of blindness worldwide. AMD is a degenerative disease that primarily affects the central portion of the retina (the macula) and can lead to severe vision loss if not detected and treated promptly. AMD is generally classified into two main types: “dry” AMD – and “wet” or exudative neovascular AMD. Wet AMD is caused by abnormal blood vessel growth in the choroid (the vascular middle layer of the eye wall) that often extends into the subretinal space with blood and fluid leaking into the area. If untreated, it can eventually lead to scarring and permanent vision loss.

[0007] Early diagnosis and treatment using methods such as photodynamic therapy or drugs that block abnormal growth of retinal blood vessels (e.g., anti-vascular endothelial growth factor, or anti-VEGF, therapy) can prevent or delay serious complications (such as blindness) and improve health outcomes.

[0008] AMD can be diagnosed based on a combination of reported changes in vision, fundus appearance (such as blood in the macula or evidence of macular thickening), injected dye-based tests (such as fundus fluorescein angiography or indocyanine green angiography), or optical coherence tomography (OCT) scans. OCT plays an increasingly important role in the diagnosis and monitoring of AMD.

[0009] OCT scans use light to quickly obtain multiple cross-sectional images of the eye. Typically, multiple scans are performed on the macula, such as a rectangular volume scan (containing approximately 30-50 separate horizontal scans) and / or a radial scan (containing approximately 6-8 scans arranged radially about the center of the macula). Trained professionals (such as ophthalmologists) examine these images, looking for characteristic anatomical changes to determine if AMD is present, as well as looking for any changes in the disease state in patients with existing AMD. There has been some progress in using machine learning (deep learning) to diagnose AMD based on a comparison of disease OCT images to normal OCT images, but technology has not yet been developed to monitor changes in the disease state (whether it is stable, improving, or worsening) in patients who have previously had AMD.

[0010] Once a patient is diagnosed with AMD, they will need to visit an eye clinic regularly (approximately 4-6 months for stable patients, more frequently if the disease is unstable) for disease monitoring to determine if the treatment plan is successful. Multiple OCT scans are typically performed at each visit.

[0011] The traditional method of monitoring patients with AMD requires manually reviewing current and previous OCT scans and determining if there is evidence of disease progression. Although acquiring multiple images for each patient is relatively quick and can be performed remotely in a well-equipped clinic, it takes a significant amount of time for healthcare professionals to interpret and review these images.

[0012] Unfortunately, AI has not yet been used to interpret changes in disease status in patients who have been diagnosed with AMD due to issues such as demanding training images, computational power requirements, and model complexity.

[0013] For example, one approach involves using two AI models in succession, where the first model is used to segment the lesion region, while the other AI model is used to assess progression and perform risk assessment. Unfortunately, the first model may not segment the correct region associated with the disease progression state, and the segmentation error will affect the classification. It should be understood that this approach is computationally expensive, both steps require high-quality data, and the model's classification cannot be explained to clinicians.

[0014] Another approach is to use a "twin neural network," which consists of two or more identical subnetworks that have the same configuration, the same parameters, and the same weights, and are used to find similarities in inputs by comparing feature vectors. In the application of twin networks to detect disease progression, in order to determine whether there is a positive or negative difference with a reference image, three models are required to work simultaneously, which is computationally expensive and time-consuming.

[0015] Therefore, there is a need to provide a system that solves or at least partially improves some of the above-mentioned problems and disadvantages of the prior art. Summary of the invention

[0016] The features and advantages of the present disclosure will be set forth in the following description, and in part will be obvious from the description, or may be learned by practicing the principles disclosed herein. The features and advantages of the present disclosure may be realized and obtained by the instruments and combinations particularly pointed out in the appended claims.

[0017] In one aspect of the disclosure, there is provided a computer-based method of training a machine learning network to predict the progression or other aspects of a disease state in a subject; the method comprising:

[0018] synthesizing a training image by superimposing a first image of an anatomical region of a subject on a second image of the same anatomical region;

[0019] recording the assignment of a disease progression state to the synthetic training image;

[0020] pre-training at least an initial layer of a machine learning model on a first image training set comprising a plurality of synthetic training images; wherein the pre-training comprises, for each synthetic training image, comparing a plurality of regions of the first image to corresponding regions of the second image and the disease progression states assigned thereto;

[0021] migrating at least a pre-trained initial layer of the machine learning model and training subsequent layers thereof on at least another training set comprising a plurality of synthetic images of a subject,

[0022] Wherein each of the plurality of composite images in the another training set comprises a superposition of a first image of an anatomical region of a subject and a second image of the same anatomical region of the same subject at a different time and has an associated disease progression state assigned.

[0023] The first training set may include at least one synthetic training image including a second image acquired from the same anatomical region of a different subject.

[0024] The first training set may include at least one synthetic training image comprising a second image acquired from the same anatomical region of the same subject at a different time than the first image.

[0025] At least some of the composite training images of the first training set may include first and second images selected such that there are no signs of disease progression, and the composite image formed therefrom is recorded as being stable.

[0026] Optionally, the first image and the second image in the first synthetic image training set may be the same image, and wherein the synthetic image is recorded as being stable.

[0027] Advantageously, the first image and the second image may be selected such that there are visible changes in the second image indicative of disease progression, and the composite image formed therefrom has a disease progression state assigned thereto.

[0028] Optionally, the first image and the second image may be selected such that there is a visible change in the second image indicating improvement, and the disease state of the composite image formed therefrom is assigned as improved.

[0029] There may be visible morphological differences between the first image and the second image in the first synthetic image training set that indicate a change in disease state; wherein whether the disease state is assigned as improvement or progression depends on the sequence of the images.

[0030] Preferably, the analyzing of each of the superimposed images comprises generating a comparison score indicative of a similarity between each of a plurality of regions of the first image and a corresponding region of the second image.

[0031] The disease progression status may be selected from the group consisting of stable, improved, progressive, indeterminate.

[0032] Advantageously, the disease may be Age-Related Macular Degeneration (AMD), and the images in the first training set and in the further training set are Optical Coherence Tomography (OCT) images of a region of at least a part of the retina.

[0033] Alternatively, the disease may be selected from the group comprising age-related macular degeneration (AMD), choroidal neovascularization, and Alzheimer's disease or pancreatic cancer.Preferably, the choroidal neovascularization may include idiopathic choroidal neovascularization, pachychoroid-associated neovascularization and myopic choroidal neovascularization.

[0034] On the other hand, a method of image processing is provided; comprising using the method described herein to train a machine learning network; acquiring a first set of multiple images of an anatomical region of the subject; acquiring a second set of multiple images of the same anatomical region of the same subject after a predetermined time interval; combining images in the first set of images with corresponding images in the second set of images, and analyzing the combined images using the trained machine learning network to generate one or more scores for the combined images indicating possible progression or other aspects of the subject's disease state.

[0035] Advantageously, the method comprises analysing a plurality of regions of the first image and a corresponding plurality of regions in the second image to determine for each region a score indicative of the likely progression or other aspect of the subject's disease state.

[0036] Preferably, the one or more scores generated are scores indicative of each of a likelihood of progression, a likelihood of improvement, or a likelihood of stabilization of the subject's disease state.

[0037] Optionally, the method may further comprise overlaying a visible color on at least some areas of the combined image, wherein the visible color is determined according to the score of the area.

[0038] Advantageously, each of the plurality of possible predetermined ranges of said one or more scores is assigned a different visible colour.

[0039] The method may further include generating a color image based on color aggregating the image.

[0040] The method may further include generating an overall score for each of the likelihood of progression, likelihood of improvement, or likelihood of stabilization of the overall image based on a comparison between an image in the first set of images and a corresponding image in the second set of images.

[0041] In another aspect, a data processing system is provided, comprising: a processor; and a computer-readable medium storing executable instructions for causing the processor to perform a method as described herein for training a machine learning network to predict progression of a subject's disease state or other aspects.

[0042] In yet another aspect, a memory device is provided for storing instructions which, when executed on a processor of a data processing system, cause the data processing system to train a machine learning network to predict the progression or other aspects of a disease state of a subject by performing the steps described. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to describe the method by which the above and other advantages and features of the present disclosure can be obtained, a more particular description of the principles briefly described above will be presented by reference to specific embodiments of the present disclosure shown in the accompanying drawings. It should be understood that these drawings depict only exemplary embodiments of the present disclosure and, therefore, should not be considered to limit the scope of the present disclosure. The principles herein will be described and illustrated with additional specificity and detail through the use of the accompanying drawings.

[0044] Preferred embodiments of the present disclosure will be explained in further detail below by way of examples and with reference to the accompanying drawings, in which:-

[0045] FIG. 1A(i) depicts an exemplary conventional infrared image of the retina (particularly the macula region) of a subject's left eye.

[0046] FIG. 1A(ii) is an exemplary single OCT line scan (corresponding to the highlighted green line with arrows as depicted in FIG. 1A(i)) showing a cross-section of the retina at that location.

[0047] FIG. 1B(i) depicts another exemplary infrared image of the left retina of a patient's macular region, with lines depicting locations where radial OCT scans were performed.

[0048] FIG. 1B(ii) depicts an exemplary single OCT line scan in a radial series (corresponding to the highlighted green line with an arrow in FIG. 1B(i) ), showing a cross-section of the subject's retina at that location.

[0049] FIG. 1C (i) (ii) (iii) depicts exemplary conventional CT images of a pancreatic region of a subject taken at a first time point.

[0050] Figure 1D Depicted are exemplary conventional CT images of a subject's brain taken at serial intervals.

[0051] Figure 2A Depicted is a conventional arrangement in which a pair of line scans are obtained from radial scans from the current visit and the previous visit of the same eye of the same patient at exactly the same location on the macula (in this case, the vertical line scans highlighted in green). The pair of images is displayed by a computer system for review by the ophthalmologist during consultation for disease monitoring.

[0052] Figure 2B Depicted is an exemplary OCT radial scan in a conventional arrangement where a computer system displays a pair of line scans (horizontal line scans highlighted in green) obtained from the same eye of the same patient at the current visit and the previous visit at the same location on the macula for review by an ophthalmologist during a consultation.

[0053] Figure 2C Another exemplary OCT radial scan in a conventional arrangement is depicted in which a pair of line scan images (highlighted in green) are obtained from radial scans from the current and previous visits at the same location on the macula in the same eye of the same patient and displayed by a computer system for review by an ophthalmologist during a consultation.

[0054] Figure 2D Another exemplary OCT radial scan in a conventional arrangement is depicted in which a pair of line scan images (highlighted in green) are obtained from radial scans from the current and previous visits at the same location on the macula in the same eye of the same patient and displayed by a computer system for review by an ophthalmologist during a consultation.

[0055] Figure 3A Depicted is a schematic overview of the two main training phases of a machine learning model according to an embodiment of the present disclosure.

[0056] Figure 3B An exemplary schematic representation of the superposition process of images is depicted.

[0057] Figure 3C The results of the trained second model on a hold-out validation set are described.

[0058] Figure 3D Performance / recall curves for a held-out validation set are plotted.

[0059] Figure 4A Depicts the embodiment according to the present disclosure Figure 3A Schematic overview of a trained machine learning model applied to image processing.

[0060] Figure 4BDepicted are matrix representations indicating the performance of the trained model as tested on a small validation set of patient data for AMD progression and three trained experts.

[0061] Figure 5A Depicted are exemplary scores and pairs of images from which such scores are derived, showing disease progression that may have produced such scores; wherein the scores are generated by embodiments of the systems and methods of the present disclosure.

[0062] Figure 5B Depicted are exemplary scores that may be generated by the systems and methods of the present disclosure and pairs of images from which the scores are derived.

[0063] Figure 5C Depicted are exemplary scores that may be generated by embodiments of the systems and methods of the present disclosure and pairs of images from which the scores are derived.

[0064] Figure 6 Depicted is an exemplary schematic process for generating a color overlay on an image of a subject's retina based on a score reflecting the evaluation of a trained model described herein.

[0065] Fig. 7A Depicted is an exemplary representation of a sample output report for a subject scan with an overall impression of being stable.

[0066] Figure 7B An exemplary representation of a sample output report for a subject scan with an overall impression of progression is depicted.

[0067] Figure 7C Depicted is an exemplary representation of a sample output report of a subject scan where the overall impression is of progression although some sections show improvement.

[0068] Fig.7D Depicted is an exemplary representation of a sample output report of a subject's scan where the overall impression is mostly stable, but there is one area of ​​progression and an adjacent area of ​​uncertain progression.

[0069] The images of FIG. 7E(i)-(iv) depict exemplary color overlay images of corresponding individual volume slices of the retina from subjects at the first visit and at a later visit.

[0070] Figure 8 Depicted is an exemplary schematic representation of a computer network in which the machine learning models described herein may operate. DETAILED DESCRIPTION

[0071] Various embodiments of the present disclosure are discussed in detail below. Although specific implementations are discussed, it should be understood that this is only for illustrative purposes. Those skilled in the relevant art will recognize that other components and configurations may be used without departing from the spirit and scope of the present disclosure.

[0072] The disclosed technology addresses the need in the art for methods and systems for training machine learning networks and for processing images, particularly images of anatomical regions of a subject, to detect progression or other aspects of a disease state in the subject.

[0073] FIG. 1A(i) depicts an exemplary image 10 of a patient's retina, particularly the macula region, depicting a plurality of lines 12 upon which an optical coherence tomography (OCT) scan is to be performed.

[0074] The green lines above depict the multiple lines where an Optical Coherence Tomography (OCT) scan will be performed. This is an exemplary OCT rectangular "volume" scan where multiple horizontal cross-sectional scans (25 in this example) are performed.

[0075] It should be understood that each line depicts a single OCT scan acquired for a patient at each visit. In the described example, 25 OCT volume scans will be acquired for each eye of the patient at each visit.

[0076] FIG. 1A(ii) is an exemplary OCT "volume" scan image 14 of one of the cross sections, showing the retinal surface.

[0077] FIG. 1B(i) depicts another exemplary image 20 of a patient's retina, in this case with lines depicting locations where radial OCT scans were performed.

[0078] It should be understood that 6 radial scans are performed, generally centered on the fovea, which is known to contain the highest concentration of cones and is responsible for central high-resolution vision. In order to enable subsequent scans to be performed at the same location where the image was acquired, the radial OCT scans are generally numbered in ascending order of clockwise position, starting at the 12 o'clock position.

[0079] Figure 1B(ii) depicts Figure 2B (i) One of the cut-out exemplary OCT radial scan images 22 showing a cross section of a subject's retina. In this radial scan, a subretinal scar can be seen and is marked with an arrow 24.

[0080] Thus, an OCT scan performed on each eye of each patient at each visit may typically include 6 sagittal cross-sectional images and 25-30 cross-sectional images of a "volume" or rectangular scan.

[0081] Figure 1C depicts a CT image of a pancreatic neuroendocrine tumor in the head of a 53-year-old man. CT images of the axial arterial phase (i) and the portal venous phase (ii) and (iii) show a well-defined mass (asterisk) surrounded by a high-density of blood vessels with a central cystic area. Only the MPD (arrow) is slightly dilated due to external compression of the tumor, rather than the tumor originating from the duct. Histopathology showed a pancreatic endocrine tumor. These images are taken from publicly available images cited in the following article:

[0082] Elbanna, KY, Jang, HJ., and Kim, TK. "Imaging diagnosis and staging of pancreatic ductal adenocarcinoma: a comprehensive review," Insights into Imaging, 11, 58 (2020). https: / / doi.org / 10.1186 / s13244-020-00861-y

[0083] The above is an example of another type of image that can be used with the overlay techniques of the present disclosure; wherein these images can be combined with each other, or with corresponding similar regions of the same or different patients at different times, as further described herein with reference to training a machine learning model for detecting AMD progression.

[0084] Similarly, Figure 1D Depicted are CT images of the same subject used to assess the progression of early Alzheimer's disease in the subject. T1-weighted (T1W) images of temporal, parieto-occipital, and frontal views of two typical cases at baseline and two years are shown. By comparing the T1W images of the same subject over two years, the area indicated by the red arrow has significant atrophy. The typical MCI stable case showed no progressive atrophy over two years, while the MCI-to-AD case showed increased width of the right choroidal fissure and temporal horn (temporal view), enlarged lateral ventricles (parieto-occipital view), and increased frontal atrophy (frontal view). This image is from the following reference Figure 6Zhao L, Luo Y, Lew D, Liu W, Au L, Mok V, Shi L. “Risk estimation before progression to mild cognitive impairment and Alzheimer's disease: an ADresemblance atrophy index.” Aging (Albany NY). 2019 Aug 29;11:6217-6236. https: / / doi.org / 10.18632 / aging.102184

[0085] This figure is another example of images that can be used with the overlay techniques of the present disclosure; wherein these images can be combined with each other, or with corresponding similar regions of the same or different patients at different times, as further described herein with reference to training a machine learning model for detecting AMD progression.

[0086] Returning to the known and FIG. 2A to FIG. 2D Retinal images captured by OCT are depicted differently in the Figure 1. The main signs of progression of age-related macular degeneration visible in OCT scans are new / increased intraretinal fluid (IRC), new / increased subretinal fluid (SRF), new / enlarged pigment epithelial detachment (PED), or increased macular thickness / new subretinal lesions.

[0087] like Figure 2A As depicted, a conventional arrangement is shown in which a pair of images obtained from a current visit and a previous visit of the retina of the same patient are displayed by a computer system for review by an ophthalmologist during a consultation. Typically, as depicted, these images include a retinal image 32a and a historical OCT scan image 32b at the top 32 and a recently corresponding retinal image 34a and OCT scan image 34b for a particular volumetric portion / radial scan portion.

[0088] These are interpreted by the ophthalmologist in consultation to provide insight into potential progression of the patient's AMD disease status between the scan dates. As shown, there is no change in these views and therefore no change in disease status, with subretinal scarring being the only significant clinical feature visible. This case is in stable condition.

[0089] In such Figure 2BIn the particular view depicted and marked by arrow 40, the patient's condition has improved, and a reduction in the pigment epithelial detachment (RPED) can be seen when comparing the top OCT image to the most recent OCT radial scan image below. Because this example shows a reduction in lesion size, it is considered an improvement in AMD status.

[0090] like Figure 2C Depicted, depicts another example OCT radial scan in a conventional arrangement, wherein, as marked by arrow 44, the size of the intraretinal cysts visible in the top scan obtained at the most recent visit has increased compared to the image obtained for the same OCT radial scan at the previous visit. This shows an example of mild AMD progression (increase in intraretinal fluid, manifested as an increase in the number and size of intraretinal cysts). The ophthalmologist, upon reviewing these images, will most likely further manage or treat the patient based on the worsening / progression of the disease state.

[0091] Finally, if Figure 2D Depicted, depicts another example OCT radial scan in a conventional arrangement, where new subretinal and intraretinal lesions 46 are visible in the OCT scan from the most recent visit at the bottom of the image; new evidence of retinal pigment epithelial detachment (RPED) 48, new intraretinal fluid (IRF) 52, and new subretinal fluid (SRF) 50 are visible in the fovea compared to the image obtained from the same OCT radial scan at the previous visit. This indicates significant progression of AMD, and therefore, the ophthalmologist, when reviewing these images, would likely further manage or treat the patient based on the progression of the disease.

[0092] refer to Figure 3A , depicts a schematic overview 100 of two main training phases of an embodiment of a method according to the present disclosure.

[0093] As depicted, in a first step 110, various images of different subjects are acquired from the same general anatomical region, which images may be combined as described below to form a first training data set. It is not important whether the images are of the same subject or different subjects, as the images are generally used to help the model learn the structure of images used in subsequent training processes. In this way, the model can be trained to recognize images that are generally progressive, stable, or improving.

[0094] The following description refers to the training of a machine learning network for detecting progression or other aspects of AMD, but the same process is also used for another network analysis based on a machine learning network.

[0095] In an exemplary application, these images are OCT scans similar to those depicted in FIGS. 1A-1B , but it should be understood that these images may be images of the pancreas or brain or some other anatomical region of the subject without limitation; as shown in FIGS. 1C and Figure 1D Depicted.

[0096] Image 112a is at a first point in time t＝1 Photographed subject S x Schematic representation of an OCT scan image of the macula. Image 112b is at a second time point t＝2 The same subject S was photographed x Schematic representation of an OCT scan image of the macula.

[0097] Image 112c is at the first time point t＝1 Different subjects photographed z Schematic representation of an OCT scan image of the macula. Image 112d is at a second time point t＝2 Photographed subject S z Schematic representation of an OCT scan image of the macula. Similarly, image 112e is a first time point t＝1 Photographed subject S y Schematic representation of an OCT scan image of the macula.

[0098] These images may include normal images, images clearly showing significant choroidal neovascularization. In an exemplary embodiment, the following number of images of the macula of different subjects were obtained from a publicly available dataset. As is typical in training AI models, a portion of the dataset was isolated and then used to evaluate the AI, as described in the table below.

[0099]

[0100] Table 1 - Dataset structure

[0101] Next, in step 120, a plurality of training images are synthesized, for example, by superimposing a first image of an anatomical region of a subject on a second image of the same anatomical region. Figure 3B Depicted is an exemplary schematic representation of such a superposition process using images for AMD progression assessment. It should be understood that such a combination of images can be represented as changing the size by concatenating or joining together multiple two-dimensional images on the z-axis, for example, by changing the size from [20, 224, 224, 3] to [20, 224, 224, 6].

[0102] When generating the first corpus of synthetic training images in an exemplary embodiment for detecting AMD progress, each normal image is paired with a subsequent normal image to form a normal image pair. Further, each image showing choroidal neovascularization (CNV) is paired with itself to form a stable image pair. In addition, normal images are paired with CNV images to form a progress image pair. Advantageously, progress image pairs can be created from different subjects, which increases overall variation and increases the robustness of the training model.

[0103] Thus, the first training set includes at least one synthetic training image, the synthetic training image including a second image acquired from the same anatomical region of a different subject. It also includes at least one synthetic training image, the synthetic training image including a second image acquired from the same anatomical region of the same subject at a different time than the first image.

[0104] The training set also includes a composite image formed from images selected such that there are no signs of disease progression, and the composite image formed therefrom is recorded as being stable.The same images may be used and the resulting overlaid composite image may be recorded as being stable.

[0105] Alternatively, the images may be selected such that there are visible changes in subsequent images indicative of disease progression, and the composite image disease state formed therefrom is assigned as progression.

[0106] There may be visible changes in the second image indicating improvement, and the disease state of the composite image formed therefrom is assigned as improved.

[0107] It will be appreciated that there are visible morphological differences between the first image and the second image in the first synthetic image training set that indicate a change in disease state; whether the disease state is assigned as improvement or progression depends on the sequence of the images.

[0108] Advantageously, the disease progression state can be selected from the group consisting of stable, improved, progressive, uncertain, but it should be understood that such a state can also be expanded or changed according to the determined clinically relevant outcome. For example, if further refinement is required, additional sub-states can be added to the above states.

[0109]

[0110] Table 2 - Synthetic images used for pre-training initial layers

[0111] In this way, images are schematically depicted as 124a, 124b, 124c, each comprising a pair of images. In an exemplary arrangement, composite image 124a may include two copies of image 112a; while composite image 124b may include image 112c and image 112d; and composite image 124c may include images 112c and 112e. It should be understood that in order to obtain the number of training images outlined in Table 2 above, many such image combinations need to be synthesized.

[0112] Advantageously, each of these composite images can be assigned a disease progression state by a clinician at the time of examination or according to the business rules outlined above. This is represented by the labels 126a, 126b, 126c depicted at step 120 in the figure.

[0113] exist Figure 3B In the example depicted, these images are OCT images of the macula of the same subject at different visits, but it will be understood that the images used may be images of anatomical regions other than the eye; and these images may be from the same patient at the same visit; or indeed different patients at different times or other combinations as have been described herein.

[0114] Next, at least an initial layer of a machine learning model is pre-trained on this first image training set comprising a plurality of synthetic training images in step 130. This pre-training includes, for each synthetic training image, comparing a plurality of regions of the first image with corresponding regions of the second image and the disease progression states assigned thereto.

[0115] Thus, the first corpus of synthetic images is used to train the first model so that it is generally able to identify different progression states of a disease from superimposed images.

[0116] Advantageously, convolutional neural networks are used in this process. In an exemplary embodiment, an XresNet with 50 layers is used, which is a modification of the Res-Net 50 model. Advantageously, the first layer is modified to accept dual channel images. Other modifications include the use of the Ranger optimizer and the use of Max-Blur pooling layers instead of the usual max pooling layers in CNNs. Each layer, including the XRes-Net, uses discriminative learning rates, which are selected based on an iterative process of finding the best learning rate for each layer where the gradient of the loss is highest.

[0117] The first model achieved an AUROC of 0.90 on the synthetic dataset. The validation set for this model was created from the synthetic dataset itself. However, since the purpose of this model is not to find progression between images from the same subject, validating it on the collected dataset is not relevant.

[0118] like Figure 3A As depicted, a further image training dataset or second image corpus is created in steps 140 and 150 .

[0119] For ease of presentation, only images acquired and training images created for one subject (Subject A) are shown; however, it will be understood that images of many different subjects taken at different times will be utilized.

[0120] As depicted in step 140, 30 images of subject A at various positions of the AMD scan are obtained at a first time 142 and a second time 144, labeled as images I1 ... I 30 It should be understood that more or fewer images may be acquired at different times without departing from the scope of the present disclosure.

[0121] As schematically shown in step 150, stacked training images of the same OCT scans of the same patient at different times are created; and the image pairs are assigned disease state or progression labels 156a, 156b based on manual inspection by a clinician.

[0122]

[0123] Table 3 - Images of the same subject at different times before enhancement

[0124] To improve the robustness of the subsequent model, image enhancement is performed on these image pairs, as further described below. It should be understood that various other types of enhancements may be performed without departing from the scope of the present invention, and the enhancement steps described subsequently are merely exemplary and non-limiting. For the purpose of external validation, 10% of the initial images are removed as an external validation set before enhancement, which is a typical case in AI model validation used in the art.

[0125] An exemplary enhancement process performed includes the following:

[0126] 1. Random contrast: 0.75*original value*1.25

[0127] 2. Random brightness: 0.75*original value*1.25

[0128] 3. Random flip:

[0129] 4. Elastic local deformation

[0130] 5. Elastic shear: 10 degrees vertically and horizontally

[0131]

[0132] Table 4 - Images of the same subject at different times after augmentation used in the second training set.

[0133] Finally, in step 160, the initial layers of the model trained on the first synthetic image training set are transferred to and reused in the second model; which is then trained on the second set of labeled synthetic images.

[0134] It should be understood that in order to achieve optimal weight initialization, the parameters between models must be consistent.

[0135] The Resnet 50 model is used; it is modified to accept six channels instead of 3. Other modifications include:

[0136] 1. Input layer

[0137] The 64 7×7 convolutional layers are modified to 56 3×3 convolutional layers because this is more computationally efficient.

[0138] 2. Downsampling path A

[0139] The stride of 1×1 convolution is changed from 2 to 1, and the stride of 3×3 convolution is changed to 2, retaining more information.

[0140] 3. Downsampling in Path B

[0141] Replacing the layer with average pooling 2 with a stride of 2 and using a 1×1 convolutional layer preserves more information.

[0142] 4. Max Pooling Layer

[0143] All max pooling layers are converted to max-blur-pool layers to increase regularization and anti-aliasing effects

[0144] The model hyperparameters are adjusted to maximize the learning speed and learning rate as follows:

[0145] 1. Optimizer = Modified Adam + Look Ahead (Ranger), Momentum = 0.9

[0146] 2. Label smoothing = 0.1

[0147] 3. Loss = Classification Cross Entropy

[0148] 4. Batch size = 64

[0149] 5. Number of periods = 3

[0150] In the example training process, the following steps were performed (batch size was reduced to 32)

[0151] 1.5 epochs, where the range of discriminative LRs (each layer has a different LR) is cyclically varied from 1e-5 to 1e-3;

[0152] 2.5 epochs, where the range of discriminative LRs (each layer has a different LR) is cyclically varied from 1e-7 to 1e-5;

[0153] 3.5 epochs, where the range of discriminative LRs (each layer has a different LR) is cyclically varied from 1e-9 to 1e-7.

[0154] The performance of the model trained on the above dataset is as follows: training accuracy is about 97%; validation accuracy is about 95%; external validation accuracy is about 99%.

[0155] Figure 3C The results of the trained second model for the holdout set are listed in a tabular format.

[0156] It should be understood that these figures show that for images excluded from the training set and manually labeled; the average validation accuracy was 99%; the area under the ROC curve was >0.99, indicating that the trained model achieved an excellent trade-off between true positive rate and false positive rate when analyzing the held-out images (unenhanced) in the validation set. The model ran in less than 5 seconds on each set of patient images.

[0157] Figure 3D The precision-recall curve calculated by plotting the precision and recall of the classifier at various thresholds is presented in a graphical format.

[0158] Figure 4A An exemplary process is depicted in which a trained machine learning network can be used to process images from the same subject (Subject B) and generate a score indicative of the likely progression or other aspects of the subject's disease state.

[0159] As depicted; in step 210, a plurality of images (in this example, 15 images) are obtained from subject B at a first time 212 and a second time 214, labeled as images I1 ... I 30 .

[0160] It should be understood that more or fewer images may be obtained at different times without departing from the scope of the present disclosure.

[0161] These images are provided as described herein according to reference FIG. 3A to FIG. 3D The model trained by the process outlined in step 220 of . An image of an anatomical region of the subject obtained at a first time is combined with an image of the same anatomical region of the subject obtained at a second time and processed by the trained machine learning model. The regions of these combined images are analyzed for similarity to assess the likelihood of stability, progression, or improvement; and individual scores are generated for these regions.

[0162] The model is then configured to generate a report 232; as depicted in step 230, the report includes scores and representations thereof that indicate the likely progression or other aspects of the subject's disease state based on the provided images.

[0163] Below is an extract of exemplary data based on image comparison results, in this case for another set of subject images that have already been analyzed, but the principles apply. As depicted, each image is compared to an image from the same location and three predictions are given, each independent of the other. It should be understood that an image may have areas of stability, improvement, and progression at the same time.

[0164]

[0165] Table 5 - Exemplary image comparison result data excerpt

[0166] Thus, analyzing the combined image using the machine learning network includes analyzing a plurality of regions of the first image and a corresponding plurality of regions in the second image to determine for each region a score indicative of a likely progression or other aspect of the subject's disease state.

[0167] refer to Figure 4B , depicts a matrix summarizing the agreement between the evaluation of the trained model and the consensus of three ophthalmologist experts who independently reviewed AMD images captured for 10 patients. The matrix results represent the consensus among clinicians, which has been normalized due to label imbalance.

[0168] As will be appreciated by those skilled in the art, for consensus between two assessments, the relevant statistical test is to calculate Cohen's kappa. For consensus between more than two subjects, the statistical test is Fleiss's kapp. In general, higher numbers mean more consensus. Initial results show a comparison of the trained model predictions with the most frequently occurring labels for a given image.

[0169] For example, if two out of three ophthalmologists agreed on a prediction, it was considered final.

[0170] In this case, the external validation data used is as follows.

[0171] ·Ten subjects were randomly selected from an ophthalmology hospital, with one eye from each subject.

[0172] About 30 image pairs per subject.

[0173] Marking was performed by three senior visual retina specialists with at least 15 years of experience.

[0174] Labels are a consensus among three labelers.

[0175] The overall accuracy of the AI ​​model with consensus was 0.96, while the overall accuracy of the consensus among individual ophthalmologists ranged from 0.86 to 0.93.

[0176] In practice, the overall progression status is predicted based on a combination of 30-40 cross-sectional image pairs per eye, so that the trained machine learning model will be the most robust and accurate at the level of each patient.

[0177] So, as depicted, the matrix represents how accurate the labels from the AI ​​are compared to each ophthalmologist individually. The second row of the matrix represents the accuracy of the labels between ophthalmologists, which basically shows how the labels for a particular image vary depending on the interpretation of the clinicians, and there is not a perfect consensus among the clinicians themselves. The uncertain dataset is the dataset that contains images where these ophthalmologists did not agree among themselves.

[0178] In one aspect of the present disclosure, the output can be provided as three indicative scores calculated by the model based on the analyzed images. These scores can be determined by the model independently of each other and provide the ophthalmologist with all the information needed to understand the overall prediction provided.

[0179] FIG. 5A to FIG. 5C Depicted are exemplary scores and the types of images from which they are derived.

[0180] It should be understood that in another exemplary aspect of the present disclosure, these scores for multiple image scans of the same eye of the same patient at different times may also be obtained by, for example, 7A to 7D .

[0181] Figure 5A Depicted are exemplary scores that can be used to assist clinicians in gaining confidence in the system. Patient image from the most recent visit (bottom) and corresponding image from the previous visit (top). The score reflects the disease state observed in a visual comparison of the top and bottom images (OCT radial / volume images of AMD in this case) of the same patient at different visits.

[0182] These scores are independent statistical calculations of the model derived probabilities of improvement, progression, and stability. These statistical measures provide an empirical estimate of the reliability of the predictions that can be used by healthcare providers at the time of examination. Advantageously, as depicted; an indicative prediction can also be displayed; which is represented as an overall score for each of the likelihood of progression, likelihood of improvement, or likelihood of stability for the overall image derived from a comparison between an image in the first set of images and a corresponding image in the second set of images.

[0183] By inspection Figure 5AIt can be seen that the statistical measure of the determined probability of progression is the highest, which is consistent with the visual comparison of a healthcare professional (such as an ophthalmologist experienced in examining such images). For this case, the score for this case indicates a high probability of progression of the AMD disease state.

[0184] Similarly, in Figure 5B In the example, the statistical measures determined for the probability of improvement, the probability of progression, and the probability of stability can be presented. In this case, it can be seen that the determined probability of improvement is the highest, approximately 0.8. A healthcare professional can intuitively interpret this by comparing the historical images of the upper part with the most recent images of the lower part of the same patient. Specifically, this example shows a score and image with a higher probability of improvement.

[0185] Finally, in Figure 5C , an exemplary image showing a mixed situation is depicted, where the displayed probabilities of improvement and progression are approximately equal. This type of score can reflect the image shown in the figure, where features that show improvement (a decrease in detectable change associated with progression) and features that show progression in both images (an increase in detectable change) are circled.

[0186] If a subject's scores are associated with these types of paired images, manual review by an ophthalmologist may be necessary to determine possible progression based on clinical experience.

[0187] Figure 6 An exemplary schematic process 300 for generating a colored overlay on an image based on a score reflecting the evaluation of a trained model described herein is depicted. The images shown are colored based on similarity or other analysis of the same anatomical region of the same subject to indicate the progression or other aspects of the disease being monitored.

[0188] In an exemplary embodiment of the present disclosure, what is being monitored is the progression of AMD, and the images 312a-o may be volumetric images of the macula, such as FIG. 2A to FIG. 2D Those depicted in .

[0189] Those skilled in the art will appreciate that the images and diseases monitored may include the progression of other diseases of the retina, such as choroidal neovascularization (e.g., idiopathic CNV, thick choroid-related neovascularization, or potentially myopic CNV) or other disease states in other anatomical regions, as shown in FIG. 1C or Figure 1D Those depicted in .

[0190] like Figure 6As described in , an image of an anatomical region of a subject can be acquired and superimposed on another image of the same anatomical region of the subject at a different time. Therefore, in step 310, multiple superimposed images of the same anatomical region of the subject are obtained.

[0191] In step 320, the evaluation for each cross section may be reduced to a plurality of slices 322a-o having a similarity "score." These slices may then be reassembled in step 330 into a single two-dimensional image 332 representing the respective score values.

[0192] In the composite image 332 of the subject's retina formed from these slices, regions in the image (such as exemplary regions 333a-e) can be individually colored to reflect regions having specific values ​​within a predetermined range. These values ​​can be colored based on the scores evaluated for the probability of improvement, probability of progression, and probability of stability, thereby providing the clinician with insight into which regions are of particular importance in the evaluation of the overall score during the examination.

[0193] It should be understood that the machine learning model of the present disclosure receives subject images in batches, with one batch equal to the total number of 2D slices of the 3D images including the two time points. Each slice in the batch is paired with a corresponding slice at a different time point. The model evaluates the images in pairs, and each 3D image pair ultimately provides a set of 3 scores for each pair of images in the batch.

[0194] 7A to 7D Depicted is an exemplary retinal cross section obtained from the same subject (representing the current scan) for a retinal image for detecting AMD progression in an exemplary application of the systems and methods of the present disclosure. Based on the assessment scores for each metric analyzed, a coloring technique has been applied to assist healthcare professionals in conducting an examination.

[0195] The coloring applied is exemplary, with blue representing stable (no progression), green (improvement), and red (progression or worsening of disease state). The images are generated by superimposing a visible color on at least some areas of the combined image, and the visible color is determined based on the similarity score of the area determined by the model. It should be understood that each of a plurality of possible predetermined ranges of one or more scores is assigned a different visible color, and the selected color may be changed without departing from the scope of the present disclosure.

[0196] exist 7A to 7D, these volume scans were performed on April 26, 2023, on different subjects; and the previous scan was performed on October 3, 2022, but it should be understood that such dates are exemplary and are dictated by patient availability and specified follow-up frequency. Advantageously, the color scans are depicted for the volume images, taking into account the number of adjacent scans. It should be understood that similar (multiple) images can also be generated for radial scans if more radial scans are obtained close together.

[0197] Advantageously, the initial score is based on a set of three independent scores for each pair, which represent predetermined labels, namely, stable, progressing, improving. The overall label is determined according to a specified rule, where:

[0198] 1) If two or more consecutive progress labels appear with high confidence, the image can be labeled as progress;

[0199] 2) If two or more consecutive progress labels appear with low confidence, the image can be labeled as uncertain progress;

[0200] 3) The progression label takes precedence over other labels because it is more clinically important.

[0201] The confidence here refers to the magnitude of the label and the secondary label.

[0202] It can be seen that for Fig. 7A The scan in gives a prediction that the disease state is stable; as reflected by the stable score of 100% and the other scores of 0%.

[0203] However, in Figure 7B In the AMD scan, the disease state of the marker is progression. Given the importance of signs of progression in AMD, although stable and improved have the same score, the business rules have dictated that in this case, if there are more than 2 consecutive scans with the highest probability of progression, the overall disease state of the marker is progression despite other areas showing stability. It can be seen that the area in the center of the scan is colored red; this indicates that changes in these areas have occurred that indicate progression.

[0204] exist Figure 7C In the 2018 image, despite the presence of improved areas (depicted in green), signs of progression were still detected (23%, colored in red), with more than 2 consecutive scans showing the highest probability of progression. Therefore, the overall impression is progression. This is clinically relevant because in this example, even though the central part of the macula has shown improvement in the disease, a new lesion has appeared next to the old lesion in the upper left corner, which, to a trained ophthalmologist, means progression of the disease.

[0205] refer to Fig.7D, this conclusion is uncertain because although most scans are stable (92%), there is 1 scan that shows the highest probability of progression, and the scan next to it shows the highest probability of uncertain progression. Given the impact of disease progression, business rules have been defined to ensure a very high sensitivity for disease progression, so this arrangement is marked as uncertain and then undergoes further review.

[0206] In the described examples; the color scan may be a volume scan, but it will be appreciated that given enough radial scans, a similar color image may be generated.

[0207] The images of Fig. 7E (i)-(iv) depict exemplary color overlay images of various volume slices, in which areas of progression (red), stability (blue), and improvement (green) have been colored. These areas visually indicate the scores assessed for these areas of the overlay image of the volume "slice" of the patient's retinal volume scan at their most recent visit compared to their previous visit. In this way, these images can provide a "slice-by-slice" individual comparison, highlighting the progression or other aspects of the disease state in the volume scan.

[0208] It should be understood that alternative colorings from other predefined categories may be used without departing from the scope of the present disclosure. 7A to 7D When one or more scan lines in the composite image depicted in are provided, images such as those depicted in Figures 7E(i)-(iv) can be provided.

[0209] like Figure 8 As depicted, an exemplary schematic representation and process occurring in a system 300 in which the systems and methods of the present disclosure may be implemented is shown. The training details and process of the AI ​​model have been disclosed above.

[0210] In the exemplary embodiment outlined below, the disease monitored is AMD, but it should be understood that, without departing from the present disclosure, similar whole systems can be used for the acquisition, processing, display and inspection of patient images. For the convenience of providing suitable images to the trained model of the present disclosure, it is necessary to customize pre-existing systems (such as the system to be implemented outlined below). It should be understood that the following is exemplary and other arrangements are also possible.

[0211] As depicted in step 1 (S1), the OCT device 302 acquires images of the subject's (multiple) retinas. The authenticated user enters his ID to open a viewer to browse the acquired OCT images and select appropriate images for study, dragging and dropping the acquired OCT images of a particular subject into the viewer program. Advantageously, the program client viewer running on the computer terminal 303 enables the authorized person 304 to match the acquired images with the time point of the previous day and perform a preliminary verification of the acquired images.

[0212] Exemplary validation may include ensuring that left eye images will only be compared with left eye images and right eye images will only be compared with right eye images, that the compared images belong to the same patient, that the comparison is performed on the same series of images at different time points, that the OCT type of the compared images (e.g., volume scan to volume scan, radial scan to radial scan), and the anatomical reference points of the compared images are also the same.

[0213] As depicted; the validation gateway 306 performs a check of the image metadata and, assuming the metadata is validated, transmits the image to the central gateway 310 in step (S2).

[0214] Optionally, these images may also be stored in a local hospital data storage device 308, preferably in accordance with PACS protocols. It should be understood that other appropriate protocols may be utilized without departing from the present disclosure. (Prior to implementation of the improved analysis system of the present disclosure, storage in a local hospital data storage device 308 was typically used.)

[0215] Then in step 3 (S3), the central gateway can advantageously trigger the backend server 320 via web service integration (such as REST API or the like). The backend server 320 is configured to store the image ID and related metadata including identification information in the data storage device 330 and store the image on the file server 332 in step 4 (S4).

[0216] The backend server 320 generates an AI image analysis request to the AI ​​backend server 340 in step 5 (S5) and transmits necessary image recognition information (including image storage location information).

[0217] In step 6 (S6), the AI ​​backend server 340 obtains the image file from the file server 332 based on the provided identification information and pre-processes the image file. Advantageously, the pre-processing may include converting the DICOM image to an image format and normalizing it with an average pixel value; and adjusting the image size to fit the specifications of the AI ​​model.

[0218] In step 7 (S7), the backend AI server 340 provides the preprocessed image to the AI ​​model. Advantageously, the AI ​​model can be executed on the AI ​​backend server 340 that triggers the AI ​​model 350 (but for ease of understanding, it is depicted separately in the figure). As discussed herein, the current image and the corresponding previous image from the same subject will be superimposed with the combined images of other volumes and radial scans of the same retina of the same patient to form a new image for analysis. These combined images are then evaluated for disease progression. As discussed herein, the final OCT progression results will be evaluated based on the outcome of each individual comparison result of each image.

[0219] In step 8 (S8), the AI ​​model 350 is executed on the image on the GPU 342 of the AI ​​backend server 340, and the following is generated: 7A to 7D The report depicted. The results are typically available within 1-2 minutes. Advantageously, in an exemplary embodiment, the GPU used for this processing is an A30 GPU card, which has an average prediction time of 2-3 minutes for approximately 60 images provided for one patient.

[0220] In step 9 (S9), the image analysis results and any supporting reports for the particular subject are sent to the backend server 340.

[0221] In step 10 ( S10 ), these images and corresponding metadata as well as any evaluation scores generated by the AI ​​model are returned to the authorized user 304 accessing the computer terminal 303 .

[0222] For cases indicated by the AI ​​model as “progression,” subjects may be called back for an urgent appointment and examined by an ophthalmologist.

[0223] Advantageously, the methods of the present disclosure enable clinicians to review base images based on which the trained model has provided indicative scores associated with disease state or lack of disease progression. Because the methods of the present disclosure present the acquired image(s) to the clinician without distortion or change in size or scaling, the clinician can review the images with confidence that the images have not changed.

[0224] Furthermore, the less stringent requirements for training images, in particular the ability to use images from the same or different subjects (in the first training set) significantly increases the number of images readily available for training. Segmentation errors are avoided, and the methods of the present disclosure are computationally less demanding and relatively fast.

[0225] Importantly, by adding a colored overlay, various features in the region that contribute significantly to the score are highlighted, making them easily interpretable by clinical staff who are not trained as experienced ophthalmologists. However, it is expected that all scores indicating disease progression will be reviewed by an experienced professional as part of the patient's management plan and possible treatment options explored.

[0226] In the exemplary application of the present disclosure in monitoring the progress of AMD in patients, it will be recognized that the technology of the present disclosure is particularly effective. Neovascular AMD (nAMD) patients need long-term frequent outpatient visits, imaging studies and repeated injections of expensive anti-VEGF drugs. This then leads to an increase in the cost of health care providers, including medicine, consultation time, imaging studies, outpatient visits and labor costs. It is predicted that each nAMD patient will need at least 6-7 visits and 7-8 injections to maintain vision within the first year after diagnosis. This has had a huge impact on health care providers. In addition, in the long run, the vision loss caused by AMD will bring high social costs, cause a decline in quality of life, lose independence and employability, and increase the risk of suffering from mental illness and premature death.

[0227] Like many other developed countries, public hospitals in Hong Kong face the challenge of providing care for the growing number of AMD patients with limited budgets and staff. Patients may not receive adequate treatment for a variety of reasons, including the cost of medications, fear of repeated injections, poor prognosis in advanced disease, and long waiting times for appointments with retinal specialists and injections. This disclosure addresses the urgent need to free up critical resources and increase capacity to ensure that those in need receive timely treatment.

[0228] It should be appreciated that, from an analytical perspective, the advantage of having a model that can compare images from the same subject is that it reduces the impact of disease progression and anatomical morphological variations between subjects. This can be contrasted with a traditional two-step model, where the first step requires segmentation of the region of interest and then determination of disease severity based on that segmentation. It should be appreciated that patients with the same disease may have very different presentations and progressions, and may also respond differently to the same treatment. Therefore, classification models may not be able to effectively capture these individual differences. Therefore, the techniques of the present disclosure provide a more accurate comparison between images from the same patient. At the same time, it avoids the problems associated with a lack of longitudinal datasets for training, which is typically a problem with this approach.

[0229] "Computer readable media" or memory devices are information storage media that can be accessed by a computer using an available or customized interface. Examples include memory (e.g., ROM or RAM, flash memory, etc.), optical storage media (e.g., CD-ROM), magnetic storage media (computer hard drives, floppy disks, etc.), punch cards, and many other media that are commercially available. Information can be transferred between a system of interest and a computer, or between a computer and a computer readable medium to store or access stored information. The transfer can be electrical, or can be done by other available methods, such as IR links, wireless connections, etc.

[0230] "System instructions" are a set of instructions that can be executed in part or in whole by a system. Typically, the instruction set exists as system software.

Claims

1. A computer-based method for training a machine learning network to predict the progression or other aspects of a disease state in a subject; the method comprising: synthesizing a training image by superimposing a first image of an anatomical region of a subject on a second image of the same anatomical region; recording the assignment of a disease progression state to the synthetic training image; pre-training at least an initial layer of a machine learning model on a first image training set comprising a plurality of synthetic training images; wherein the pre-training comprises, for each synthetic training image, comparing a plurality of regions of the first image to corresponding regions of the second image and the disease progression states assigned thereto; migrating at least a pre-trained initial layer of the machine learning model and training subsequent layers thereof on at least another training set comprising a plurality of synthetic images of a subject, Wherein each of the plurality of composite images in the another training set comprises a superposition of a first image of an anatomical region of a subject and a second image of the same anatomical region of the same subject at a different time and has an associated disease progression state assigned.

2. A computer-based method of training a machine learning network to predict the progression or other aspects of a subject's disease state according to claim 1, wherein: The first training set includes at least one synthetic training image including a second image acquired from a same anatomical region of a different subject.

3. A computer-based method of training a machine learning network to predict the progression or other aspects of a subject's disease state according to claim 1 or claim 2, wherein: The first training set includes at least one synthetic training image including a second image acquired from the same anatomical region of the same subject at a different time than the first image.

4. A computer-based method of training a machine learning network to predict the progression or other aspects of a disease state in a subject according to any one of the preceding claims, wherein: At least some of the composite training images of the first training set include first and second images selected such that there is no evidence of disease progression, and the composite image formed therefrom is recorded as being stable.

5. A computer-based method of training a machine learning network to predict the progression or other aspects of a disease state in a subject according to any one of the preceding claims, wherein: The first image and the second image in the first synthetic image training set are the same image, and wherein the synthetic image is recorded as being stable.

6. The computer-based method of claim 1 for training a machine learning network to predict the progression or other aspects of a subject's disease state, wherein: The first image and the second image are selected such that there are visible changes in the second image indicative of disease progression, and the composite image formed therefrom has a disease progression state assigned thereto.

7. The computer-based method of claim 1 for training a machine learning network to predict progression or other aspects of a subject's disease state, wherein: The first image and the second image are selected such that there is a visible change in the second image indicating improvement, and the disease state of the composite image formed therefrom is assigned as improvement.

8. The computer-based method of claim 1 for training a machine learning network to predict progression or other aspects of a subject's disease state, wherein: There is a visible morphological difference between the first image and the second image in the first synthetic image training set that indicates a change in disease state; wherein whether the disease state is assigned as improvement or progression depends on the sequence of the images.

9. The computer-based method of claim 1 for training a machine learning network to predict progression or other aspects of a subject's disease state, wherein: The analyzing of each of the superimposed images includes generating a comparison score indicating a similarity between each of a plurality of regions of the first image and a corresponding region of the second image.

10. The computer-based method of claim 1 for training a machine learning network to predict progression or other aspects of a subject's disease state, wherein: The disease progression status is selected from the group consisting of stable, improved, progressive, and indeterminate.

11. The computer-based method of claim 1 for training a machine learning network to predict progression or other aspects of a subject's disease state, wherein: The disease is age-related macular degeneration (AMD), and the images in the first training set and the further training set are optical coherence tomography (OCT) images of a region of at least a portion of a retina.

12. The computer-based method of claim 1 for training a machine learning network to predict progression or other aspects of a subject's disease state, wherein: The disease is selected from the group consisting of age-related macular degeneration (AMD), choroidal neovascularization, and Alzheimer's disease or pancreatic cancer.

13. An image processing method; include Using the method of any one of claims 1 to 12 to train a machine learning network; acquiring a first plurality of images of an anatomical region of the subject; acquiring a second plurality of images of the same anatomical region of the same subject after a predetermined time interval; An image in the first set of images is combined with a corresponding image in the second set of images, and the combined image is analyzed using the trained machine learning network to generate one or more scores indicative of a likely progression or other aspect of a disease state of the subject for the combined image.

14. The image processing method according to claim 13, wherein: Analyzing the combined image using the machine learning network includes analyzing a plurality of regions of the first image and a corresponding plurality of regions in the second image to determine a score for each region indicative of a likely progression or other aspect of the subject's disease state.

15. The image processing method according to claim 13, wherein: The one or more scores generated are scores indicative of each of a likelihood of progression, a likelihood of improvement, or a likelihood of stabilization of the subject's disease state.

16. The image processing method of claim 13, further comprising overlaying a visible color on at least some areas of the combined image, wherein: The visible color is determined based on the score of the area.

17. The image processing method according to claim 16, wherein: Each of a plurality of possible predetermined ranges of the one or more scores is assigned a different visible color.

18. The image processing method according to claim 17, comprising generating a color image by color aggregation of the image.

19. The method for determining the progression or other aspects of a subject's disease state according to claim 13, further comprising generating an overall score for each of the likelihood of progression, likelihood of improvement, or likelihood of stabilization of the overall image based on a comparison between an image in the first set of images and a corresponding image in the second set of images.

20. A data processing system comprising: processor; as well as A computer-readable medium storing executable instructions for causing the processor to perform a method for training a machine learning network to predict progression of a subject's disease state or other aspects according to any one of claims 1-12.

21. A data processing system comprising: processor; as well as A computer-readable medium storing executable instructions, wherein the instructions are used to cause the processor to execute the image processing method according to any one of claims 13-19.

22. A memory device storing instructions which, when executed on the processor of a data processing system, cause the data processing system to train a machine learning network to predict the progression or other aspects of a subject's disease state by performing the steps of any one of claims 1-12.

23. A memory device storing instructions which, when executed on the processor of a data processing system, cause the data processing system to process an image according to any one of claims 13 to 19.

Images