Method and system for identifying neovascular age-related macular degeneration lesions

Abstract

The present application belongs to the field of image recognition, and provides a method and system for identifying neovascular age-related macular degeneration lesions, which comprises collecting and preprocessing the original macular degeneration image of a patient; based on the preprocessed macular degeneration image, a pre-trained classification convolutional neural network is used to detect the linear markers of the slice image of the macular degeneration image; based on the preprocessed macular degeneration image, a pre-trained full convolution network is used for segmentation to determine the block markers of the macular degeneration image; the length and segmentation ratio of the linear markers are extracted; the area, diameter and average reflection intensity of the block markers are determined; based on the length and segmentation ratio of the linear markers and the area, diameter and average reflection intensity of the block markers, a trained random forest model is used to identify the neovascular age-related macular degeneration lesions. The complete rate of the linear markers and the area and diameter of the block markers are simultaneously identified and quantified.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of image recognition, and particularly relates to a method and system for identifying lesions of neovascular age-related macular degeneration. BACKGROUND

[0002] The statements in this section merely provide background information related to the application and do not necessarily constitute prior art.

[0003] Neovascular age-related macular degeneration (nAMD) is the main cause of blindness in people over the age of 50, can affect both eyes of patients in turn or at the same time, and is related to age, heredity, environment, oxidative stress, immunity and other factors. nAMD is prone to cause irreversible central visual impairment in a short period of time, and the incidence significantly increases with age. The main pathological features of nAMD patients are choroidal or retinal neovascularization, which leads to changes such as retinal edema, exudation, hemorrhage or subretinal fibrosis plaques in the macular area. The early symptoms of nAMD are not obvious, and patients may have slight central visual distortion or black spots. If the vision of the healthy eye is good, it is usually difficult to detect, but as the disease progresses, once the macular edema or hemorrhage occurs, it can cause the patient's central vision to drop sharply or even blindness, which seriously affects the quality of life and self-care ability of elderly patients. Therefore, screening and accurate diagnosis of nAMD in people over the age of 50 is of great importance to individuals, families and even the whole society.

[0004] Traditional examination methods for nAMD include color fundus photography, but it is difficult to diagnose and accurately assess the patient's condition by color fundus photography alone. Indocyanine green angiography (ICGA) and fundus fluorescein angiography (FFA) are the gold standard for diagnosing neovascularization in the macular area of nAMD patients, but both are invasive examinations that require intravenous injection of contrast agents. Not only is there a risk of infection, but there are also contraindications for patients with severe liver and kidney dysfunction who contain iodine contrast agents, and they cannot be used as routine ophthalmic examinations for repeated use. Optical coherence tomography angiography (OCTA) can be used to observe blood vessels at each layer of the fundus and more clearly display the abnormal neovascularization morphology in the macular area. However, OCTA examination is very sensitive to signal quality, and local signal intensity is lower, which is more likely to cause dark areas and affect the judgment of lesions. Moreover, OCTA cannot show the leakage and accumulation of liquid, which is very important for determining, differentiating different lesions and assessing disease activity.

[0005] As the most widely used key examination in the clinical diagnosis and treatment of nAMD, SD-OCT is a fast, safe and non-invasive imaging method, which can diagnose and monitor disease activity to guide the development of further treatment plan. The high-resolution retinal tissue cross-sectional images obtained by OCT technology realize the non-invasive and quantitative detection of its in vivo morphology, which can directly show the lesions closely related to the diagnosis and prognosis of nAMD, such as intraretinal fluid, subretinal fluid, retinal pigment epithelial layer detachment, subretinal high reflection, etc., and has an irreplaceable advantage in the diagnosis and follow-up of the disease. However, the current prediction of refractory nAMD mainly focuses on the segmentation of fluid area and high reflection point, ignoring the ellipsoid zone (EZ) and external limiting membrane (ELM) which are crucial to the occurrence and development of nAMD. The integrity of EZ and ELM reflects the function of photoreceptor cells, and at the same time, is significantly related to the relief of fluid accumulation and visual prognosis of patients treated with anti-VEGF. However, due to the presence of macular edema, effusion and hemorrhage, it is very difficult to analyze them.

[0006] However, the current domestic and foreign researches mainly focus on predicting the visual prognosis of nAMD, and the latest researches have proved that visual decline is only the final outcome caused by repeated disease activity, and is closely related to individual differences of patients and whether the lesion involves the fovea of the macula, which cannot represent the activity of the disease. In addition, the existing prediction model of visual prognosis requires comprehensive clinical data, such as age, gender, multi-modal image data, and even genotype, which is difficult to realize in real-world clinical application due to its complexity. SUMMARY

[0007] In order to solve the above problems, the present application provides a method and system for identifying lesions of neovascular age-related macular degeneration, which uses an algorithm based on convolutional neural network and full convolution network to successfully identify and quantify the integrity of linear markers EZ and ELM, as well as the area and diameter of block markers IRF and SRF, and the area, diameter and reflectivity of PED and SHRM on the SD-OCT image. Further based on the quantitative data obtained by automatic segmentation, a prediction model is constructed by using random forest algorithm to predict the disease activity after 1 year of treatment, which successfully realizes the identification and prediction of refractory nAMD at the initial stage of treatment according to non-invasive SD-OCT image. Due to its simplicity, it can assist clinicians with insufficient experience to develop individualized treatment plans, and is easy to be widely promoted in clinical practice, and its high efficiency has great clinical application prospect.

[0008] According to some embodiments, the first aspect of the present application provides a new blood vessel age-related macular degeneration lesion identification system, which adopts the following technical solutions:

[0009] The new blood vessel age-related macular degeneration lesion identification system comprises:

[0010] A data processing module configured to collect and pre-process original macular degeneration images of a patient;

[0011] A linear marker detection module configured to perform slice segmentation based on the pre-processed macular degeneration images, and detect linear markers of the slice images of the macular degeneration images using a pre-trained classification convolutional neural network according to the slice images of the macular degeneration images;

[0012] A block marker segmentation module configured to perform segmentation using a pre-trained full convolutional network based on the pre-processed macular degeneration images, and determine block markers of the macular degeneration images;

[0013] A quantitative feature extraction module configured to extract biomarkers of the linear markers of the slice images of the macular degeneration images, and extract the length and segmentation ratio of the linear markers; and determine the area, diameter and average reflection intensity of the block markers;

[0014] A lesion identification module configured to identify new blood vessel age-related macular degeneration lesions using a trained random forest model based on the length and segmentation ratio of the linear markers and the area, diameter and average reflection intensity of the block markers.

[0015] Further, the collecting and pre-processing of the original macular degeneration images of the patient are specifically as follows:

[0016] The original macular degeneration images of the patient are collected;

[0017] Based on the original macular degeneration images, a spectral domain optical coherence tomography image of a macular region of the retina with the fovea as the scanning center is obtained;

[0018] The spectral domain optical coherence tomography image is cropped to obtain a region of interest;

[0019] The image of the region of interest is subjected to regularization and normalization processing;

[0020] The pre-processed macular degeneration images are obtained.

[0021] Further, the classification convolutional neural network comprises four downstream modules, each of which comprises, in sequence, a convolutional layer, a batch normalization layer and a ReLU activation layer, a convolutional layer, a batch normalization layer and a ReLU activation layer, a max pooling layer and a classifier.

[0022] Further, based on the pre-processed macular degeneration image, the slice is segmented, and according to the slice image of the macular degeneration image, the linear markers of the slice image of the macular degeneration image are detected by using the pre-trained classification convolutional neural network, comprising:

[0023] Based on the pre-processed macular degeneration image, the slice is segmented;

[0024] According to the slice image of the macular degeneration image, the slice image is sequentially input into the pre-trained classification convolutional neural network from left to right, and the detection result corresponding to the multiple slice images is obtained, that is, the linear marker of the slice image of the macular degeneration image;

[0025] The detection results of the multiple slices are sequentially averaged and fused to obtain a fused prediction result.

[0026] Further, the full convolutional network comprises a backbone network layer and a segmentation head.

[0027] Further, the area, diameter and average reflection intensity of the block-shaped marker are determined by using a graph convolution algorithm before being input into the trained random forest model, and the relationship between each block-shaped structure is determined, specifically:

[0028] The block-shaped marker is intraretinal fluid, subretinal fluid, retinal pigment epithelial detachment and subretinal hyperreflective material;

[0029] The intraretinal fluid, subretinal fluid, retinal pigment epithelial detachment and subretinal hyperreflective material are defined as nodes N, and the nodes are connected by learnable edges E;

[0030] The area, diameter and average reflection intensity of the block-shaped marker are denoted as F, and the dimension of F is (H, W, 5), H and W represent the length and width of the original image, and 5 represents the output dimension, the 0th dimension represents the background class, and the 1st, 2nd, 3rd and 4th dimensions represent the intraretinal fluid, subretinal fluid, retinal pigment epithelial detachment and subretinal hyperreflective material, respectively;

[0031] The features f1, f2, f3 and f4 in the 1st, 2nd, 3rd and 4th dimensions are taken out, and the dimension is (H, W, 1) as the value of the node N, and the relationship of each category is fused by using the graph convolution GAT algorithm, and the output result of GAT is f1', f2', f3' and f4';

[0032] The original background class f0 is superimposed with the output result f'n of GAT to obtain the final output result f', and the dimension is (H, W, 5).

[0033] Furthermore, the linear markers include the retinal ellipsoid band and the external retinal membrane; the blocky markers include intraretinal fluid, subretinal fluid, retinal pigment epithelium detachment, and subretinal high-reflectivity substances.

[0034] According to some embodiments, a second aspect of the present invention provides a method for identifying neovascular age-related macular degeneration lesions, employing the following technical solution:

[0035] Methods for identifying neovascular age-related macular degeneration lesions include:

[0036] Acquire raw macular degeneration images of the patient and perform preprocessing;

[0037] Based on the preprocessed macular degeneration image, segments are made. Based on the segments of the macular degeneration image, a pre-trained classification convolutional neural network is used to detect linear markers in the segments of the macular degeneration image.

[0038] Based on the preprocessed macular degeneration image, a pre-trained fully convolutional network is used for segmentation to determine the blocky landmarks in the macular degeneration image;

[0039] Biomarkers were extracted from the retinal ellipsoid and external retinal membrane of slice images of macular degeneration, including the length and segmentation ratio of linear markers; the area, diameter, and average reflectance of block markers were determined.

[0040] Based on the length and segmentation ratio of linear markers, as well as the area, diameter, and average reflectance of block markers, a trained random forest model is used to identify neovascular age-related macular degeneration lesions.

[0041] According to some embodiments, a third aspect of the present invention provides a computer-readable storage medium.

[0042] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps in the method for identifying neovascular age-related macular degeneration lesions as described in the second aspect above.

[0043] According to some embodiments, a fourth aspect of the present invention provides a computer device.

[0044] A computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the program, performs the steps in the method for identifying neovascular age-related macular degeneration lesions as described in the second aspect above.

[0045] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0046] This invention utilizes algorithms based on convolutional neural networks and fully convolutional networks to successfully identify and quantify the integrity rate of linear markers EZ and ELM, as well as the area and diameter of blocky markers IRF and SRF, and the area, diameter, and reflectivity of PED and SHRM on SD-OCT images. Furthermore, based on the quantified data obtained from automatic segmentation, a predictive model is constructed using the random forest algorithm to predict disease activity one year after treatment. This successfully identifies and predicts refractory nAMD based on non-invasive SD-OCT images during the initial stages of diagnosis and treatment. Due to its simplicity, it can assist inexperienced clinicians in developing individualized treatment plans and is easily and widely promoted in clinical practice. Its high efficiency has enormous potential for clinical translational applications. Attached Figure Description

[0047] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention.

[0048] Figure 1 This is a flowchart of the method for identifying neovascular age-related macular degeneration lesions according to an embodiment of the present invention;

[0049] Figure 2 This is a schematic diagram of AUC for predicting 1-year disease activity of neovascular age-related macular degeneration according to an embodiment of the present invention;

[0050] Figure 3 This is a flowchart of the model for the method of identifying neovascular age-related macular degeneration lesions as described in an embodiment of the present invention. Detailed Implementation

[0051] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0052] It should be noted that the following detailed description is illustrative and intended to provide further explanation of the invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0053] It should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the scope of exemplary embodiments according to the invention. As used herein, the singular form is intended to include the plural form as well, unless the context clearly indicates otherwise. Furthermore, it should be understood that when the terms "comprising" and / or "including" are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof.

[0054] Where there is no conflict, the embodiments and features in the embodiments of the present invention can be combined with each other.

[0055] Terminology Explanation:

[0056] 1. Neovascular age-related macular degeneration (nAMD) is one of the leading causes of severe irreversible vision loss in people over 50 years of age in my country and economically developed countries. Due to the rapid development of neovascularization in the macula, photoreceptors are severely damaged, leading to irreversible visual impairment and significantly impacting the quality of life and self-care ability of elderly patients. There is no unified standard for refractory nAMD. Generally, it is considered acceptable if, after at least 3 months of continuous anti-VEGF drug treatment, persistent retinal effusion is still visible on OCT images, with no significant change or even an increase.

[0057] 2. Intravitreal injection of anti-vascular endothelial growth factor (VEGF) is a first-line treatment for nAMD, with approximately 30-35% of patients experiencing significant visual improvement, i.e., an increase in visual acuity of 15 or more letters.

[0058] 3. Spectral-domain optical coherence tomography (SD-OCT) is a non-contact, non-invasive imaging technique that uses low-coherence interference light to perform tomographic scanning of biological tissues. Widely used in ophthalmology, it can obtain high-resolution macular tomographic images in vivo and perform quantitative analysis. It is a crucial non-invasive examination method for diagnosing and monitoring disease activity in the clinical treatment of nAMD, displaying "imaging biomarkers" closely related to disease activity and prognosis, such as intraretinal fluid, subretinal fluid, retinal pigment epithelium detachment, and subretinal hyperreflectance.

[0059] Example 1

[0060] This embodiment provides a system for identifying neovascular age-related macular degeneration lesions, including:

[0061] The data processing module is configured to acquire raw macular degeneration images of the patient and perform preprocessing.

[0062] The linear marker detection module is configured to segment the preprocessed macular degeneration image into slices, and detect linear markers in the slice images of the macular degeneration image using a pre-trained classification convolutional neural network based on the slice images of the macular degeneration image.

[0063] The block marker segmentation module is configured to segment the pre-processed macular degeneration image using a pre-trained fully convolutional network to identify block markers in the macular degeneration image.

[0064] The quantization feature extraction module is configured to extract biomarkers from linear markers in slice images of macular degeneration images, extracting the length and segmentation ratio of linear markers; and determining the area, diameter, and average reflectance of block markers.

[0065] The lesion identification module is configured to identify neovascular age-related macular degeneration lesions based on the length and segmentation ratio of linear markers and the area, diameter, and average reflectance of block markers using a trained random forest model.

[0066] Furthermore, the acquisition and preprocessing of the patient's original macular degeneration images specifically includes:

[0067] Acquire original images of the patient's macular degeneration;

[0068] Based on the original macular degeneration image, a spectral domain optical coherence tomography image of the macular region with the fovea of ​​the retina as the scanning center is obtained;

[0069] The region of interest is obtained by cropping the spectral domain optical coherence tomography image;

[0070] The image of the region of interest is then regularized and normalized.

[0071] The preprocessed image of macular degeneration is obtained.

[0072] Furthermore, the classification convolutional neural network consists of four downstream modules, each of which includes a convolutional layer, a batch normalization layer and a ReLU activation layer, a convolutional layer, a batch normalization layer and a ReLU activation layer, a maximum set layer and a classifier connected in sequence.

[0073] Furthermore, based on the preprocessed macular degeneration image, segments are created. Then, based on these segments, a pre-trained classification convolutional neural network is used to detect linear markers in the segments, including:

[0074] Segmentation and slicing based on preprocessed macular degeneration images;

[0075] Based on the slice images of macular degeneration, they are sequentially input into a pre-trained classification convolutional neural network from left to right to obtain the detection results corresponding to multiple slice images, namely the linear markers of the slice images of macular degeneration.

[0076] The detection results of multiple slices are fused by moving average in sequence to obtain the fused prediction result.

[0077] Furthermore, the fully convolutional network consists of a backbone network layer and a segmentation head.

[0078] Furthermore, before the area, diameter, and average reflectance of the block markers are input into the trained random forest model, the relationship between the various block structures is determined using a graph convolution algorithm, specifically:

[0079] The block-shaped markers are intraretinal fluid, subretinal fluid, retinal pigment epithelium detachment, and subretinal high reflectivity substances.

[0080] Intraretinal fluid, subretinal fluid, retinal pigment epithelium detachment, and subretinal hyperreflective material are defined as nodes N, and nodes are connected by learnable edges E.

[0081] The area, diameter, and average reflectance of the block marker are denoted as F. The dimensions of F are (H, W, 5), where H and W refer to the length and width of the original image, and 5 refers to the output dimension. The 0th dimension represents the background class, and the 1st, 2nd, 3rd, and 4th dimensions represent intraretinal fluid, subretinal fluid, retinal pigment epithelium detachment, and subretinal high reflectance material, respectively.

[0082] Extract the features f1, f2, f3, and f4 of dimensions 1, 2, 3, and 4, respectively, and use them as the values ​​of node N in the dimension (H, W, 1). Substitute them into the graph convolution GAT algorithm to fuse the relationships between the categories. The output of GAT is f1', f2', f3', and f4'.

[0083] The original background class f0 is superimposed with the result f'n output by GAT to obtain the final output result f', which has dimensions (H, W, 5).

[0084] Furthermore, the linear markers include the retinal ellipsoid band and the external retinal membrane; the blocky markers include intraretinal fluid, subretinal fluid, retinal pigment epithelium detachment, and subretinal high-reflectivity substances.

[0085] Example 2

[0086] like Figure 1 As shown, this embodiment provides a method for identifying neovascular age-related macular degeneration lesions, including:

[0087] Acquire raw macular degeneration images of the patient and perform preprocessing;

[0088] Based on the preprocessed macular degeneration image, segments are made. Based on the segments of the macular degeneration image, a pre-trained classification convolutional neural network is used to detect linear markers in the segments of the macular degeneration image.

[0089] Based on the preprocessed macular degeneration image, a pre-trained fully convolutional network is used for segmentation to determine the blocky landmarks in the macular degeneration image;

[0090] Biomarkers were extracted from the retinal ellipsoid and external retinal membrane of slice images of macular degeneration, including the length and segmentation ratio of linear markers; the area, diameter, and average reflectance of block markers were determined.

[0091] Based on the length and segmentation ratio of linear markers, as well as the area, diameter, and average reflectance of block markers, a trained random forest model is used to identify neovascular age-related macular degeneration lesions.

[0092] Specifically, such as Figure 3 As shown, the method described in this embodiment specifically includes:

[0093] Newly diagnosed nAMD patients were included, and SD-OCT images of the macular region of the retina with the fovea of ​​the retina as the scanning center were obtained.

[0094] Preprocess the obtained SD-OCT image: crop the region of interest (ROI) of the original OCT image, reshape it into an image with a height of 256 and a width of 1600, and regularize the image to the range of minimum-maximum normalization.

[0095] Linear markers EZ and ELM were detected using a classification convolutional neural network (CNN) model. Biomarker extraction was performed on EZ and ELM, with a focus on length ratio and the proportion of the target problem in all segments.

[0096] The OCT image is segmented into slices, and the presence of corresponding tissue is detected in each slice. For example... Figure 3 As shown, the CNN model in this embodiment consists of four downstream modules. Each module is composed of stacked convolutional layers, batch normalization layers, and ReLU activation layers, followed by a maximal pooling layer. Conv is the convolutional layer, BN is the normalization layer, ReLU is the activation layer, MAX-pooling is the maximal pooling layer, and FC is the classifier.

[0097] The downstream module extracts abstract features from medical images and downsamples the images into tensors. Two fully connected layers are then used to reduce feature dimensionality before predicting the presence of the target. Detection of EZ and ELM within the segmentation volume is defined as a positive prediction. For EZ and ELM structures, the image data is sliced ​​from left to right and fed into the neural network for prediction. We designed a moving average fusion method; specifically, the prediction p' of the current block depends not only on the current model result p, but also on the model results p'(n-1) and p'(n-2) of the previous two blocks, i.e., p' = 0.7*p + 0.2*p'(n-1) + 0.1*p'(n-2). The continuity of EZ and ELM is determined based on the fused prediction result.

[0098] like Figure 3 As shown, in the processing branch for blocky markers, this embodiment uses a fully convolutional network (FCN) to segment the IRF, SRF, PED, and SHRM of the blocky structure, focusing more on the shape, location, and reflectivity of the tissue. The FCN network consists of a backbone network and a segmentation head. The backbone network structure is based on ResNet, and the performance of the backbone network greatly affects the accuracy of the segmentation results. We chose an efficient network that balances accuracy and effectiveness. Due to the limitation of training samples, the backbone network was pre-trained on ImageNet, which accelerated the training speed and provided strong prior knowledge for image understanding. For the neural network of blocky markers, an improved FCN was used. Before entering the random forest model in the final output, a graph convolutional layer was inserted to describe the relationship between IRF, SRF, PED, and SHRM.

[0099] Strictly speaking, IRF, SRF, PED, and SHRM are defined as nodes N, and nodes are connected by learnable edges E.

[0100] The final feature layer of the input image after passing through the FCN is extracted and denoted as F. The dimensions of F are (H, W, 5), where H and W represent the length and width of the original image, and 5 represents the output dimension. Dimension 0 represents the background class, and dimensions 1, 2, 3, and 4 represent IRF, SRF, PED, and SHRM, respectively. It should be noted that this final feature extraction refers to quantized feature extraction, which involves labeling the tissue categories of the OCT image and samples, and annotating the area and diameter of the blocky markers IRF and SRF, and the area, diameter, and reflectance of PED and SHRM.

[0101] Extract the features f1, f2, f3, and f4 of dimensions 1, 2, 3, and 4, respectively, and use them as the values ​​of node N in the dimension (H, W, 1). Substitute them into the graph convolution GAT algorithm to fuse the relationships between the categories. The output of GAT is f1', f2', f3', and f4'.

[0102] The original background class f0 and f'n are superimposed to obtain the final output result f' with dimensions (H, W, 5). Finally, argmax(f') is used to obtain the network's prediction result for the block structure.

[0103] Features were extracted from SD-OCT images of patients before and 1 month, 3 months and 12 months after anti-VEGF treatment, including: the integrity rate of EZ and ELM, the area and diameter of block markers IRF and SRF, and the area, diameter and reflectance of PED and SHRM.

[0104] The relative importance of individual variables was assessed by measuring the decrease in model predictive accuracy after each variable was input. Independent models were constructed from different longitudinal series of SD-OCT data (baseline, baseline to month 1, baseline to month 3). One-year disease activity was independently confirmed by two retinal specialists based on OCT and OCTA results. Type 0 represents stable disease with complete disappearance of retinal fluid on SD-OCT; Type 1 represents persistent disease activity: persistent fluid exudation, unresolved hemorrhage, or progressive fibrosis; Type 2 represents cure: disappearance of macular pathological lesions (SHRM, PED) and absence of any type of retinal fluid.

[0105] Automatic segmentation results based on extracted SD-OCT features and a random forest approach were used to construct a predictive model for the one-year disease activity of nAMD in the eye. This predictive model can efficiently predict refractory nAMD using baseline SD-OCT image features.

[0106] The baseline model had an area under the receiver operating characteristic (AUC) of 0.948, accuracy of 0.871, sensitivity of 0.834, and specificity of 0.932. The 3-month model (containing data from baseline to month 3) demonstrated the best predictive ability, with an AUC of 0.980, accuracy of 0.930, sensitivity of 0.920, and specificity of 0.962. In the 3-month model, the predictions for prognoses 0 (stable), 1 (PDA), and 2 (cured) were all accurate, with AUCs of 0.982, 0.987, and 0.971, respectively.

[0107] like Figure 2 As shown, separate models are constructed based on different SD-OCT data (baseline, baseline to month 1, baseline to month 3).

[0108] Type 0 (stable), Type 1 (PDA), and Type 2 (cured) were accurately predicted in the 3-month model.

[0109] Table 1 Performance metrics of the prediction model

[0110] Baseline 1 month post-treatment 1 month post-treatment AUC 0.950 0.966 0.980 Accuracy 0.871 0.912 0.930 Sensitivity 0.833 0.885 0.920 Specificity 0.932 0.953 0.962

[0111] Example 3

[0112] This embodiment provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps in the method for identifying neovascular age-related macular degeneration lesions as described in Embodiment 2 above.

[0113] Example 4

[0114] This embodiment provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements the steps in the method for identifying neovascular age-related macular degeneration lesions as described in Embodiment 2 above.

[0115] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of hardware embodiments, software embodiments, or embodiments combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage and optical storage) containing computer-usable program code.

[0116] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0117] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0118] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0119] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc.

[0120] While the specific embodiments of the present invention have been described above in conjunction with the accompanying drawings, this is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art without creative effort based on the technical solutions of the present invention are still within the scope of protection of the present invention.

Claims

1. A system for identifying neovascular age-related macular degeneration lesions, characterized in that, include: The data processing module is configured to acquire raw macular degeneration images of the patient and perform preprocessing. The linear marker detection module is configured to segment the preprocessed macular degeneration image into slices, and detect linear markers in the slice images of the macular degeneration image using a pre-trained classification convolutional neural network based on the slice images of the macular degeneration image. The linear markers include the retinal ellipsoid band and the external retinal membrane; The block marker segmentation module is configured to segment the pre-processed macular degeneration image using a pre-trained fully convolutional network to identify block markers in the macular degeneration image. The quantization feature extraction module is configured to extract biomarkers from linear markers in slice images of macular degeneration, extracting the length and segmentation ratio of the linear markers; determining the area, diameter, and average reflectance of block markers. Before being input into the trained random forest model, the area, diameter, and average reflectance of the block markers are further determined using a graph convolution algorithm to establish relationships between the various block structures. Specifically, the block markers are intraretinal fluid, subretinal fluid, retinal pigment epithelium detachment, and subretinal high-reflectance substances; these are defined as nodes N, connected by learnable edges E; the area of ​​the block markers... The product, diameter, and average reflectance are denoted as F, with dimensions (H, W, 5). H and W refer to the length and width of the original image, while 5 refers to the output dimension. Dimension 0 represents the background class, and dimensions 1, 2, 3, and 4 represent intraretinal fluid, subretinal fluid, retinal pigment epithelium detachment, and subretinal high reflectance material, respectively. The features f1, f2, f3, and f4 of dimensions 1, 2, 3, and 4 are extracted and used as the values ​​of node N with dimensions (H, W, 1). These are then substituted into the graph convolution GAT algorithm to fuse the relationships between the categories. The GAT outputs f1', f2', f3', and f4'. The original background class f0 is superimposed with the GAT output f'n to obtain the final output f', with dimensions (H, W, 5). The process involves segmenting the preprocessed macular degeneration image into slices, and then using a pre-trained classification convolutional neural network to detect linear markers in these slices. The steps include: segmenting the preprocessed macular degeneration image into slices; sequentially inputting each slice from left to right into the pre-trained classification convolutional neural network to obtain detection results for multiple slices, which are the linear markers of the macular degeneration image slices; and then fusing the detection results of multiple slices using a moving average in sequence to obtain a fused prediction result. The lesion identification module is configured to identify neovascular age-related macular degeneration lesions based on the length and segmentation ratio of linear markers and the area, diameter, and average reflectance of block markers using a trained random forest model.

2. The system for identifying neovascular age-related macular degeneration lesions as described in claim 1, characterized in that, The process of acquiring and preprocessing the patient's original macular degeneration images specifically involves: Acquire original images of the patient's macular degeneration; Based on the original macular degeneration image, a spectral domain optical coherence tomography image of the macular region with the fovea of ​​the retina as the scanning center is obtained; The region of interest is obtained by cropping the spectral domain optical coherence tomography image; The image of the region of interest is then regularized and normalized. The preprocessed image of macular degeneration is obtained.

3. The system for identifying neovascular age-related macular degeneration lesions as described in claim 1, characterized in that, The classification convolutional neural network consists of four downstream modules. Each downstream module includes a convolutional layer, a batch normalization layer and a ReLU activation layer, a convolutional layer, a batch normalization layer and a ReLU activation layer, a maximum set layer and a classifier, which are connected in sequence.

4. The system for identifying neovascular age-related macular degeneration lesions as described in claim 1, characterized in that, The fully convolutional network consists of a backbone network layer and a segmentation head.

5. A method for identifying neovascular age-related macular degeneration lesions, characterized in that, include: Acquire raw macular degeneration images of the patient and perform preprocessing; Based on the preprocessed macular degeneration image, segments are made. Based on the segments of the macular degeneration image, a pre-trained classification convolutional neural network is used to detect linear markers in the segments of the macular degeneration image. The linear markers include the retinal ellipsoid band and the external retinal membrane; Based on the preprocessed macular degeneration image, a pre-trained fully convolutional network is used for segmentation to determine the blocky landmarks in the macular degeneration image; Biomarker extraction was performed on the retinal ellipsoid and external retinal membrane of sliced ​​images of macular degeneration. The length and segmentation ratio of linear markers were extracted. The area, diameter, and average reflectance of block markers were determined. Before being input into the trained random forest model, the area, diameter, and average reflectance of the block markers were further determined using a graph convolution algorithm to establish relationships between the various block structures. Specifically, the block markers were defined as intraretinal fluid, subretinal fluid, retinal pigment epithelium detachment, and subretinal high-reflectance material. Intraretinal fluid, subretinal fluid, retinal pigment epithelium detachment, and subretinal high-reflectance material were defined as nodes N, connected by learnable edges E. The area, diameter, and average reflectance of the block markers were denoted as F. The dimensions are (H, W, 5), where H and W refer to the length and width of the original image, and 5 refers to the output dimension. Dimension 0 represents the background class, and dimensions 1, 2, 3, and 4 represent intraretinal fluid, subretinal fluid, retinal pigment epithelium detachment, and subretinal hyperreflective material, respectively. The features f1, f2, f3, and f4 of dimensions 1, 2, 3, and 4 are extracted and used as the values ​​of node N with dimensions (H, W, 1). These are then substituted into the graph convolutional GAT algorithm to fuse the relationships between the categories. The GAT outputs f1', f2', f3', and f4'. The original background class f0 is superimposed with the GAT output f'n to obtain the final output f', with dimensions (H, W, 5). The detection results of multiple slices are then fused using a moving average in sequence to obtain the fused prediction result. The process involves segmenting the preprocessed macular degeneration image into slices, and then using a pre-trained classification convolutional neural network to detect linear markers in these slices. The steps include: segmenting the preprocessed macular degeneration image into slices; sequentially inputting each slice from left to right into the pre-trained classification convolutional neural network to obtain detection results for multiple slices, which are the linear markers of the macular degeneration image slices; and then fusing the detection results of multiple slices using a moving average in sequence to obtain a fused prediction result. Based on the length and segmentation ratio of linear markers, as well as the area, diameter, and average reflectance of block markers, a trained random forest model is used to identify neovascular age-related macular degeneration lesions.

6. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the program implements the steps in the method for identifying neovascular age-related macular degeneration lesions as described in any one of claims 5.

7. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps in the method for identifying neovascular age-related macular degeneration lesions as described in any one of claims 5.

Images