Multi-source age-related macular degeneration analysis system based on a fusion deep learning model

Abstract

The application provides a multi-source age-related macular degeneration analysis system based on a fusion deep learning model, comprising: a preprocessing module, used for acquiring a data source, performing preprocessing on the data source, and obtaining preprocessing data, wherein the data source is used for representing multi-source age-related macular degeneration picture data; a classification prediction module, used for performing feature extraction on the preprocessing data to obtain feature data, performing fusion on the feature data to obtain fusion features, and performing classification on the fusion features to obtain an age-related macular degeneration analysis result. The application has the beneficial effect that, compared with a complete picture level black box classification, the technical solution of the application improves clinical interpretability, is compatible with AMD analysis of multiple data sources, and solves the problems of a small amount of positive data and labeled data and different data formats.

Description

Technical Field

[0001] This invention relates to the field of computer technology, and in particular to a multi-source age-related macular degeneration analysis system based on a fusion deep learning model. Background Technology

[0002] Existing techniques for analyzing age-related macular degeneration consume significant resources and have low accuracy.

[0003] Age-related macular degeneration (AMD) is an eye disease associated with symmetrical central chorioretinopathy and is one of the most common causes of vision loss in older adults. Globally, approximately 9.7% of blindness cases are related to AM. Etiology involves age, genes, habits, and environment. Analysis based on fundus images is the primary solution for AMD detection. Manual detection of AMD is extremely time-consuming. Due to the complexity of fundus analysis, the lack of ophthalmologists, and the scarcity of fundus equipment, early-stage AMD patients are not easily detected, especially in developing regions. Artificial intelligence (AI) technology has been widely applied in ophthalmology, and deep learning (DL) and machine learning (ML) algorithms have proven effective in the early screening and prognosis of fundus diseases. It demonstrates advanced advantages and opportunities for AMD detection with limited resources. In recent years, classification and segmentation algorithms for ophthalmic images have received increasing attention.

[0004] However, existing technologies still face theoretical and practical challenges in applying artificial intelligence to ophthalmology, including issues such as clinical data collection, ethical assessment, optimization of fundus image algorithms, augmentation of labeled data, extraction of key image features, and normalization and fusion of multi-source analysis images. Furthermore, fundus images used for assisted analysis of AMD are highly diverse, commonly including conventional color fundus photography (RCFP), fundus autofluorescence (FAF), infrared imaging (IR), spectral domain optical coherence tomography (OCT), optical coherence tomography angiography (OCT-A), scanning laser fundus examination (SLO), and ultra-wide-angle fundus imaging (UWF). AMD analysis based on fundus images is complex, making it difficult to guarantee recognition accuracy, and AMD features are not readily apparent. Existing age-related macular degeneration analysis technologies are resource-intensive and have low accuracy. Summary of the Invention

[0005] The main objective of this invention is to propose a multi-source age-related macular degeneration analysis system based on a fusion deep learning model, thereby improving the accuracy of multi-source age-related macular degeneration analysis.

[0006] One aspect of the present invention provides a multi-source age-related macular degeneration analysis system based on a fusion deep learning model, characterized in that it comprises:

[0007] The preprocessing module is used to acquire a data source, perform preprocessing on the data source, and obtain preprocessed data. The data source is used to characterize multi-source age-related macular degeneration image data.

[0008] The classification and prediction module is used to perform feature extraction on the preprocessed data to obtain feature data; perform fusion on the feature data to obtain fused features; and classify the fused features to obtain the analysis results of age-related macular degeneration.

[0009] According to the multi-source age-related macular degeneration analysis system based on the fusion deep learning model, the multi-source age-related macular degeneration image data includes at least one of conventional color fundus photography, fundus autofluorescence, spectral domain optical coherence tomography, and ultra-wide-angle imaging.

[0010] According to the aforementioned multi-source age-related macular degeneration analysis system based on a fusion deep learning model, the preprocessing module includes:

[0011] The image enhancement process is used to perform image enhancement processing on the multi-source age-related macular degeneration images, and then to perform salt-and-pepper noise filtering processing to obtain a filtered image. The image enhancement processing includes at least one of image rotation, data generation based on conditional generative adversarial networks, and noise reduction.

[0012] Furthermore, the filtered image is subjected to the corresponding preprocessing based on the data type of the multi-source age-related macular degeneration image data.

[0013] According to the multi-source age-related macular degeneration analysis system based on a fusion deep learning model, the preprocessing of conventional color fundus photography includes:

[0014] The image size of a standard color fundus photograph was adjusted to 512×512, and a segmented standard color fundus photograph dataset was generated using a connected component segmentation algorithm.

[0015] According to the aforementioned multi-source age-related macular degeneration analysis system based on a fusion deep learning model, the preprocessing of fundus autofluorescence includes:

[0016] The dataset of fundus autofluorescence is subjected to binary processing, followed by rotation processing.

[0017] According to the multi-source age-related macular degeneration analysis system based on a fusion deep learning model, the preprocessing of spectral domain optical coherence tomography includes:

[0018] The images from spectral domain optical coherence tomography were rotated and subjected to SPN noise reduction.

[0019] According to the aforementioned multi-source age-related macular degeneration analysis system based on a fusion deep learning model, the preprocessing of the ultra-wide-angle lens includes:

[0020] The ultra-wide-angle image is preprocessed using a deep learning model to rotate the region of interest and add additive noise. The region of interest is defined as the area centered at the fovea and with a radius equal to the distance from the center of the optic disc to the fovea.

[0021] According to the multi-source age-related macular degeneration analysis system based on a fusion deep learning model, the feature extraction of the classification prediction module includes:

[0022] Used to obtain image types from multi-source age-related macular degeneration image data via one-hot encoding;

[0023] The contrast, dissimilarity, homogeneity / inverse dissimilarity, energy, correlation and ASM energy of image textures in multi-source age-related macular degeneration image data were obtained, and texture features were extracted using the SCRICKITS-IMAGE function.

[0024] Cluster labels are determined by using at least one of the hierarchical clustering algorithms HC, ResNet-Kmeans, and support vector machine, with the number of clusters obtained from the unsupervised maximum likelihood model and the metric of intra-cluster error variance.

[0025] The image features are obtained by downsampling the preprocessed data using a deep learning model.

[0026] According to the multi-source age-related macular degeneration analysis system based on a fusion deep learning model, the classification prediction module performs fusion on the feature data to obtain fused features, including:

[0027] The texture features, clustering labels, and image features are combined into a matrix to obtain a feature matrix, and classification labels are added to the missing positions in the feature matrix.

[0028] According to the multi-source age-related macular degeneration analysis system based on a fusion deep learning model, the feature classification module classifies the fusion features to obtain age-related macular degeneration analysis results, including:

[0029] The feature matrix is ​​predicted using a deep learning network, and the results of age-related macular degeneration analysis are determined based on the classification labels.

[0030] Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0031] The above and / or additional aspects and advantages of the present invention will become apparent and readily understood from the description of the embodiments taken in conjunction with the following drawings, in which:

[0032] Figure 1 This is a schematic diagram of a multi-source age-related macular degeneration analysis system based on a fusion deep learning model, according to an embodiment of the present invention.

[0033] Figure 2 This is a schematic diagram of AMD image preprocessing according to an embodiment of the present invention.

[0034] Figure 3 This is a schematic diagram of different batches of images and SNP removal processed images according to an embodiment of the present invention.

[0035] Figure 4 This is a schematic diagram of the CFS process for OCT data in an embodiment of the present invention.

[0036] Figure 5 This is a schematic diagram of RCFP image rotation according to an embodiment of the present invention.

[0037] Figure 6 This is a schematic diagram illustrating the determination of cluster identifiers based on intra-cluster error variance in an embodiment of the present invention.

[0038] Figure 7 This is a schematic diagram of UWF ROI extraction according to an embodiment of the present invention.

[0039] Figure 8 This is a schematic diagram of ROI data enhancement based on SPN and UWF according to an embodiment of the present invention.

[0040] Figure 9 This is a schematic diagram of the structure of VGG16 for AMD detection according to an embodiment of the present invention.

[0041] Figure 10 This is a schematic diagram of the AMD classification process according to an embodiment of the present invention.

[0042] Figure 11 This is a schematic diagram illustrating the relationship between SSE and clusters in HC and ResNet-K Means according to embodiments of the present invention. Detailed Implementation

[0043] The embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings. Throughout the description, the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions. In the following description, suffixes such as "module," "part," or "unit" used to denote elements are used only for the purpose of illustrative purposes and have no specific meaning in themselves. Therefore, "module," "part," or "unit" can be used interchangeably. Terms such as "first," "second," etc., are used only to distinguish technical features and should not be construed as indicating or implying relative importance, or implicitly indicating the number of indicated technical features, or implicitly indicating the sequential relationship of the indicated technical features. In the following description, the consecutive reference numerals for method steps are for ease of review and understanding. Adjusting the implementation order of steps, in conjunction with the overall technical solution of the present invention and the logical relationship between the various steps, will not affect the technical effect achieved by the technical solution of the present invention. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and should not be construed as limiting the present invention.

[0044] Terminology Explanation: Age-related macular degeneration (AMD) includes regular color fundus photography (RCFP), fundus autofluorescence (FAF), spectral-domain optical coherence tomography (OCT), and ultra-widefield fundus (UWF).

[0045] For example, refer to Figure 1 It discloses a schematic diagram of a multi-source age-related macular degeneration analysis system based on a fusion deep learning model, which includes:

[0046] The preprocessing module is used to acquire data sources, perform preprocessing on the data sources, and obtain preprocessed data. The data sources are used to characterize multi-source age-related macular degeneration image data.

[0047] In some embodiments, multi-source age-related macular degeneration image data includes at least one of conventional color fundus photography, fundus autofluorescence, spectral-domain optical coherence tomography, and ultra-wide-angle imaging.

[0048] In some embodiments, the AMD image data may be in the form of a dataset, an image, etc.

[0049] In some embodiments, reference Figure 2 It illustrates the processing procedures for different image types, specifically including:

[0050] (1) Preprocessing of OCT Images. In this embodiment, the original dataset contains 3,000 (Alzheimer's disease) and 29,347 (normal) digits. Image augmentation processing is performed on the AMD images, including image rotation, data generation based on Conditional Generative Adversarial Network (CGAN), and noise reduction. GGAN is an effective data augmentation method, especially for medical image generation. Compared with ordinary GAN, the generator is more efficient due to the addition of information about the classification label y. As shown in the formula below, the objective function of the minimax game between the generator and the discriminator is the key to the execution of CGAN. Salt-and-pepper noise (SPN) filtering is applied to the generated images. Finally, the size of the OCT images is adjusted to 512×512. The OCT database is processed using the correlation-based feature selection (CFS) algorithm to generate a segmented OCT database.

[0051]

[0052] Where D is the discriminative model and G is the generative model.

[0053] Specifically, refer to Figure 3 The parameters of the CGAN model on OCT images were set as follows: batch size 32, number of channels 3, latent vector size 100 epochs, generator loss rate 0.05, discriminator loss rate 0.0025, and Adam Op-timizers Beta1 0.5. The input and output image sizes were both 256×256. 3000 AMD images were used as the training dataset. Cross-validation was performed on the trained model. The generator and discriminator losses were 0.1122 and 0.0352, respectively. The model training took 11454.39294 seconds, and generating a new image took 0.0186 seconds. Figure 3 Examples of generating batches of 1, 60, 120, 180, 240, and 300 images and SNP removal processing images are shown.

[0054] In some embodiments, reference Figure 4 Images in the OCT database are processed by the CFS algorithm to extract key features and output to the OCT segmentation database. The CFS model has been applied to various medical image processing applications (49, 50). Figure 4 As shown, sub-image A is the original image, B is the image related to image feature recognition, and C is the segmented OCT image. The shape features of the epiretinal membrane and choroid of the macula are extracted (the white part is shown in C). (2) Preprocessing of FAF data.

[0055] In some embodiments, preprocessing includes performing binary processing on the FAF dataset and rotating the dataset.

[0056] Specifically, refer to Figure 5 The diagram shown illustrates the preprocessing results of FAF data. Figure 5 In the image, (A), (B), and (C) represent the original image, the binary-processed image, and the rotated image, respectively. This implementation of the FAF dataset contains 1947 AMD images and 2874 normal images. Drusen and CNV are visible in the AMD images. Symptoms of CNV are difficult to detect in OCT, RCFP, and UWF, especially in late-stage AMD with abnormal RPE. However, the fluorescent control medium may cause allergic reactions in the eyes, posing a potential threat to patients. Binary processing was performed on the FAF dataset. Rotation was then performed on the dataset.

[0057] (3) Preprocessing of AMD RCFP images.

[0058] In some embodiments, the preprocessing of AMD RCFP images includes rotation processing and SPN noise reduction processing. Figure 6 The example demonstrates RCFP image rotation processing. Figure 8 Examples A and B in the diagram illustrate SPN noise enhancement (data augmentation).

[0059] (4) UWF image preprocessing.

[0060] Specifically, the accuracy of UWF images is low because key features are easily ignored by the model. Furthermore, it places high demands on computational and space resources. To improve the accuracy of deep learning model training and save time and resources, this embodiment extracts the Region of Interest (ROI) of UWF for the AMD classification task. This ROI is defined as the region centered at the fovea with a radius equal to the distance from the center of the optic disc to the fovea. Preprocessing based on SNP roles for rotation and additive noise performance is analyzed on AMD UWF images.

[0061] The classification and prediction module is used to perform feature extraction on the preprocessed data to obtain feature data; perform fusion on the feature data to obtain fused features; and classify the fused features to obtain the analysis results of age-related macular degeneration.

[0062] In some embodiments, the classification prediction module includes feature extraction, feature confusion, and AMD classification, wherein feature extraction includes the following features:

[0063] Image type: One-hot encoding is used to determine the image type of OCT, segmented OCT, FAF, RCFP, and UWF.

[0064] Image texture features include contrast, dissimilarity, homogeneity / inverse dissimilarity, energy (the sum of squares of the elements in the gray-level co-occurrence matrix, a measure of the stability of gray-level changes in image texture, reflecting the uniformity of gray-level distribution and texture coarseness; a large energy value indicates that the current texture is a regular and relatively stable texture), correlation, and angular second-moment (ASM) energy. Texture features are extracted using the SCRICKITS-IMAGE function.

[0065] Clustering labels, including at least one of hierarchical clustering (HC), ResNet-Kmeans, and support vector machine clustering algorithms, are used. The number of clusters in an unsupervised maximum likelihood model is determined by a metric of within-cluster error variance (SSE). For example, refer to... Figure 3 The vertical axis represents SSE, and the horizontal axis represents the number of cluster labels in SVM. This model uses 12 as the target number of clusters.

[0066] Image features are obtained through downsampling in the last layer of deep learning. The deep learning model is not limited to VGG16.

[0067] In some embodiments, reference Figure 9 This example illustrates the structure of VGG16 for AMD detection. The feature matrix is ​​constructed by adding "0"s at missing positions. Texture features (first row), cluster labels (second row), and features extracted from the last layer of VGG16 (last row) are combined into the matrix.

[0068] Specifically, this embodiment uses two unsupervised maximum likelihood methods: hierarchical clustering and support vector machines (SVMs), both important algorithms for image clustering. The advantage of unsupervised learning methods is that they save time and resources. Data-driven algorithms can be a good choice for feature extraction. However, when it comes to medical analysis, they are uninterpretable and uncontrollable. They may fail to cluster into specific required classes, exhibiting low levels of trust and efficiency. In this embodiment, (100, 150, and 200) AMD images and (100, 150, and 200) normal images are randomly selected. The images are clustered into two classes using HC, ResNet-K Means (ResNet for feature extraction, K-Means for classification), and SVMs, and the method is determined to have the highest accuracy in detecting AMD. Three algorithms are also used for feature extraction by clustering digits into an optimal number of classes. The parameters of the optimal clustering algorithm are determined by the sum of squared errors within the cluster (SSE). The parameter of HC is denoted as (0.645 * tree.Distance). The parameters of ResNet-K mean are labeled as follows (resized to 224*224, n_Clusters=2). The parameters of support vector machine are labeled as follows (grayscale, direction=12, block_norm='L1', pixel_els_per_cell=[8,8], cell_per_block=[4,4], visualization=FALSE, Transform_SQRT=True).

[0069] Supervised learning methods using VGG16 and ResNet are applied to AMD classification. Compared to unsupervised maximum likelihood methods, the classification algorithm achieves higher classification accuracy. However, this is time-consuming and resource-intensive. In medical analysis, the VGG16 model has 13 convolutional layers, 3 fully connected layers, and 5 pooling layers. Figure 5 The architecture of RESNET is the same as that of VGG16, except that RESNET adds skip layers between convolutional layers.

[0070] In some embodiments, AMD classification reference Figure 10 The combined feature matrix is ​​input into a deep learning network, not limited to multi-layer perceptron neural networks (MLPs), and is predicted as two categories: AMD (labeled as 1) and Normal (labeled as 0).

[0071] In some embodiments, the MLP-based fusion method is divided into three parts. For the first part of feature extraction, the cluster label features are determined by an unsupervised learning method, where the number of clusters parameter for the Support Vector Machine is 2, while the parameters for HC and ResNet-K mean are determined by the inflection points of the SSE-cluster graph.

[0072] Reference Figure 11 The diagram illustrates the relationship between SSE and clusters in HC and ResNet-K Means. For ROIs of OCT, segmented OCT, FAF, RCFP, and UWF, HC's classification results are 5, 4, 7, and 8, while ResNet-K Mean's results are 6, 9, 4, 8, and 8. The original UWF image has no inflection points, validating the necessity of UWF-based region of interest extraction for AMD recognition.

[0073] For example, referring to Table 1, which shows the accuracy, sensitivity, specificity, AUC, and test time (per image) of the multi-model approach, and referring to Table 2, which provides examples of image-normalized fusion feature matrices used to represent AMD detection results, the normalized fusion feature matrix for an image has a dimension of 15×2048. It includes image types such as OCT, segmentation OCT, FAF, RCFP, and UWF; texture features such as image contrast, dissimilarity, homogeneity, energy, correlation, and ASM; clustering labels from HC, ResNet-K-means, and support vector machines; and image features extracted by ResNet.

[0074] Accuracy, sensitivity, specificity, AUC, and test time (per image) are the metrics used to evaluate the algorithm. Unsupervised and supervised learning were performed on the OCT, segmented OCT, FAF, RCFP, and UWF databases, respectively. Experiments were conducted on the proposed MLP-based multi-data source fusion method, and comparisons were made with other methods. The unsupervised dataset was determined by randomly selecting 400 images from the database, 50% of which were associated with AMD. The dataset for the supervised learning method was split in a 3:1 ratio for both training and validation. 400 images (50% AMD) were selected for testing on each dataset.

[0075] The mean results for accuracy, sensitivity, specificity, AUC, and test time for all data types based on HC, support vector machine, and ResNet-K mean were 53.57%, 56.02%, 89%, 0.5467 and 0.0133, 72.10%, 80.00%, 80.20%, 0.8667 and 0.099, and 59.26%, 58.49%, 48.58%, 0.73986 and 0.119, respectively.

[0076] The mean values ​​for accuracy, sensitivity, specificity, AUC, and test time for all data types based on VGG16 and RES-Net were 0.81964, 0.80752, 0.83122, 0.88804, and 0.127, respectively, and 0.59262, 0.58488, 0.48584, 0.73986, and 0.11882, respectively.

[0077] The average results for all data types based on unsupervised and supervised learning models and the proposed method are 63%, 66%, 64%, 0.69 and 0.11, 80%, 77%, 0.77, 0.84 and 0.14, and 95%, 93%, 0.92, 0.94 and 0.09, respectively.

[0078] The accuracy, sensitivity, specificity, AUC, and test time of this method were 94.06%, 94.22%, 93.26%, 0.9607, and 0.074, respectively; 99%, 98.41%, 96.33%, 0.9823, and 0.065; 90.11%, 87.06%, 88.98%, 0.9401, and 0.096; 93.76%, 93.76%, 93.77%, 0.8924, and 0.121; 98.12%, 97.06%, and 95.47%; 0.9908 and 0.087; and 97.4%, 90.1%, 86.24%, 0.884, and 0.0886, respectively.

[0079] In AMD detection based on OCT, segmented OCT, Faf, RCFP, UWF, and a comprehensive database, this method achieves the highest accuracy and lowest processing time. The unsupervised model is the fastest model for this task. Compared to supervised and unsupervised methods, the supervised model has advantages in accuracy, sensitivity, specificity, and AUC. AMD detection based on the segmented OCT dataset achieves the highest accuracy compared to other data sources. Tables 1 and 2 are shown below:

[0080]

[0081]

[0082]

[0083] Table 1

[0084]

[0085] Table 2

[0086] According to embodiments of the present invention, the technical solution of the present invention has at least the following beneficial effects: fusion of multiple data sources; transformation of unstructured data to structured data; fusion of multiple machine learning and deep learning models; improved performance of artificial intelligence classification algorithms while saving time and computing resources; compared with completely image-level black-box classification, which first extracts feature values, the embodiments of the present invention achieve improved clinical interpretability, are compatible with AMD analysis of multiple data sources, and solve the problems of small amounts of positive and labeled data and inconsistent data formats. Different processing difficulties and features of ordinary color photographs, OCT, FFA, and ultra-wide-angle fundus color photographs are addressed separately, and ultimately, effective feature fusion for AMD is achieved.

[0087] In some alternative embodiments, the functions / operations mentioned in the block diagrams may not occur in the order shown in the operation diagrams. For example, depending on the functions / operations involved, two consecutively shown blocks may actually be executed substantially simultaneously, or the blocks may sometimes be executed in reverse order. Furthermore, the embodiments presented and described in the flowcharts of this invention are provided by way of example to provide a more comprehensive understanding of the technology. The disclosed methods are not limited to the operations and logic flows presented herein. Alternative embodiments are contemplated in which the order of various operations is altered and sub-operations described as part of a larger operation are executed independently.

[0088] Furthermore, although the invention has been described in the context of functional modules, it should be understood that, unless otherwise stated, one or more of the described functions and / or features may be integrated into a single physical device and / or software module, or one or more functions and / or features may be implemented in a separate physical device or software module. It is also understood that a detailed discussion of the actual implementation of each module is unnecessary for understanding the invention. Rather, given the properties, functions, and internal relationships of the various functional modules in the apparatus disclosed herein, the actual implementation of the module will be understood within the scope of conventional skill of an engineer. Therefore, those skilled in the art can implement the invention as set forth in the claims using ordinary techniques without excessive experimentation. It is also understood that the specific concepts disclosed are merely illustrative and not intended to limit the scope of the invention, which is determined by the full scope of the appended claims and their equivalents.

[0089] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, essentially, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0090] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device.

[0091] More specific examples of computer-readable media (a non-exhaustive list) include: electrical connections (electronic devices) having one or more wires, portable computer disk drives (magnetic devices), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Furthermore, computer-readable media can even be paper or other suitable media on which the program can be printed, since the program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in computer memory.

[0092] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0093] In the description of this specification, references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0094] Although embodiments of the invention have been shown and described, those skilled in the art will understand that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the claims and their equivalents.

[0095] The above is a detailed description of the preferred embodiments of the present invention, but the present invention is not limited to the embodiments described. Those skilled in the art can make various equivalent modifications or substitutions without departing from the spirit of the present invention, and these equivalent modifications or substitutions are all included within the scope defined by the claims of this application.

Claims

1. A multi-source age-related macular degeneration analysis system based on a fusion deep learning model, characterized in that, include: The preprocessing module is used to acquire a data source, perform preprocessing on the data source, and obtain preprocessed data. The data source is used to characterize multi-source age-related macular degeneration image data. The classification and prediction module is used to perform feature extraction on the preprocessed data to obtain feature data; The feature data is fused to obtain fused features; The fusion features are classified to obtain the analysis results of age-related macular degeneration; The feature extraction of the classification prediction module includes: obtaining image types from multi-source age-related macular degeneration (AMD) image data through one-hot encoding; obtaining contrast, dissimilarity, homogeneity / inverse discrepancy, energy, and correlation of image textures from multi-source AMD image data, and extracting texture features using the SCRICKITS-IMAGE function; determining cluster labels by using at least one of hierarchical clustering (HC), ResNet-Kmeans, and support vector machine clustering algorithms, and determining the number of clusters in an unsupervised maximum likelihood model using a metric of intra-cluster error variance; and using a deep learning model with the preprocessed data as input, obtaining image features through the last layer downsampling of the deep learning model. The deep learning model includes VGG16; The process of fusing the feature data to obtain fused features includes: combining the texture features, the clustering labels, and the image features extracted from the last layer of VGG16 into a matrix to obtain a feature matrix; and adding classification labels to the missing positions in the feature matrix to obtain the fused features.

2. The multi-source age-related macular degeneration analysis system based on a fusion deep learning model according to claim 1, characterized in that, The multi-source age-related macular degeneration image data includes at least one of conventional color fundus photography images, fundus autofluorescence images, spectral domain optical coherence tomography images, and ultra-wide-angle images.

3. The multi-source age-related macular degeneration analysis system based on a fusion deep learning model according to claim 2, characterized in that, The preprocessing module includes: The multi-source age-related macular degeneration image data is subjected to image enhancement processing, followed by salt-and-pepper noise filtering to obtain a filtered image. The image enhancement processing includes at least one of image rotation, data generation based on conditional generative adversarial networks, and noise reduction. Furthermore, the filtered image is subjected to the corresponding preprocessing based on the data type of the multi-source age-related macular degeneration image data.

4. The multi-source age-related macular degeneration analysis system based on a fusion deep learning model according to claim 3, characterized in that, The preprocessing of the conventional color fundus photographic image includes: The size of the standard color fundus photography images was adjusted to 512×512, and a segmented standard color fundus photography dataset was generated using a connected component segmentation algorithm.

5. The multi-source age-related macular degeneration analysis system based on a fusion deep learning model according to claim 3, characterized in that, The preprocessing of the fundus autofluorescence image: The dataset of fundus autofluorescence images is subjected to binary processing, followed by rotation processing.

6. The multi-source age-related macular degeneration analysis system based on a fusion deep learning model according to claim 3, characterized in that, The preprocessing of the spectral domain optical coherence tomography image includes: The spectral domain optical coherence tomography images were subjected to rotation processing and SPN noise addition processing.

7. The multi-source age-related macular degeneration analysis system based on a fusion deep learning model according to claim 3, characterized in that, The preprocessing of the ultra-wide-angle image includes: The ultra-wide-angle image is preprocessed using a deep learning model to rotate the region of interest and add additive noise. The region of interest is the area centered at the fovea and with a radius equal to the distance from the center of the optic disc to the fovea.

8. The multi-source age-related macular degeneration analysis system based on a fusion deep learning model according to claim 1, characterized in that, The classification and prediction module classifies the fused features to obtain age-related macular degeneration analysis results, including: The feature matrix is ​​predicted using a deep learning network, and the results of age-related macular degeneration analysis are determined based on the classification labels.

Images