Image coarse label AMD classification method based on prototype multi-instance learning framework

Abstract

The invention discloses an image coarse label AMD classification method of a prototype-based multi-instance learning framework. The prototype-based multi-instance learning framework (PGMIL) comprising a prototype module (PM), a prototype attention guiding module (PGA) and a prototype driving normalization module (PDN) is constructed. According to the method, global feature representation is extracted from normal samples, and then a feature prototype library is constructed; through continuous optimization based on a momentum updating mechanism, general characteristics of normal samples are accurately extracted, and a prototype module is established; then, by comparing an abnormal sample with a prototype and calculating an attention mechanism based on an abnormal score, the identification capability of the PGMIL framework model for a potential abnormal region is significantly improved. Besides, in the training process of the PGMIL framework model, comparison loss and diversity loss are introduced into a loss function, so that the PGMIL framework model is helped to more accurately distinguish different types of abnormal samples.

Description

Technical Field

[0001] This invention relates to an ADM classification method, and more particularly to an image coarse label AMD classification method based on a prototype-based multi-instance learning framework. Background Technology

[0002] Age-related macular degeneration (AMD) is a major cause of irreversible vision loss in older adults. This condition is triggered by a complex interplay of genetic, metabolic, and environmental factors, primarily affecting key retinal tissues such as photoreceptor cells and the retinal pigment epithelium (RPE). AMD can be classified into two types: dry and wet. Dry AMD is more common, while wet AMD progresses rapidly and is characterized by choroidal neovascularization (CNV), ultimately leading to vision loss. Early detection of wet AMD is crucial for effective intervention.

[0003] Deep learning has shown great potential in medical image processing, particularly in optical coherence tomography (OCT) image analysis. However, current deep learning methods applied to OCT image analysis still face several challenges: only a small fraction of the numerous OCT slices acquired during diagnosis contain useful information. Furthermore, traditional deep learning analysis methods struggle to handle the fluctuations in OCT image quality and the complexity of lesion structures, especially when only coarse labels are available. Since coarse labels cannot provide precise judgment criteria, the difficulty of accurate and efficient AMD classification analysis of OCT images becomes even more pronounced. Therefore, how to efficiently extract features from multiple OCT images has become a key challenge for AMD classification. Summary of the Invention

[0004] To address the shortcomings of the aforementioned technologies, this invention provides an image coarse label AMD classification method based on a prototype-based multi-instance learning framework.

[0005] To address the above technical problems, the technical solution adopted in this invention is: an image coarse label AMD classification method based on a prototype-based multi-instance learning framework, comprising the following steps: S1. Construct a dynamically updated normal prototype library through feature extraction and dimensionality reduction methods; S2. Using the prototype library, introduce an anomaly-guided attention mechanism with a normalization strategy to refine the features of the prototype. S3. Construct the prototype-based multi-instance learning framework PGMIL model, which includes the fusion prototype module PM, the prototype-guided attention module PGA, and the prototype-driven normalization module PDN. S4. Perform multi-task loss training on the prototype-based multi-instance learning framework PGMIL model until the optimal model parameters are obtained. S5. Use the PGMIL model with the optimal parameters to perform AMD classification analysis on the input OCT image.

[0006] Preferably, in step S1, the construction of the normal prototype library includes the following process: 1) Prototype Library Construction and Initialization: OCT images with normal retinal structure distribution are acquired as normal samples. Global features are extracted from these normal samples. A gated attention mechanism is used to calculate the attention weight for each image. Based on the attention weights... Each normal OCT image is filtered, and the top k images with the highest scores are retained to create a candidate feature set. ; 2) Dynamic update mechanism of normal prototype library: calculate the cosine similarity between normal input features and each prototype feature in the standard prototype set. The standard prototype set is obtained by concatenating the features of the first k images of normal samples after dimensionality reduction. Continuously track the frequency of occurrence of each standard prototype feature, and sort and filter the high-frequency features, as shown in the following formula: For any normal input feature vector f, its relationship with each prototype feature is calculated. cosine similarity ,here, This represents the similarity in semantic space between the normal input feature vector f and the j-th prototype feature. It's the operation that takes the j that maximizes the value of the function that follows. This represents the set of prototype indices that are closest to the input feature vector f; Let f be the mapping function; The update coefficient decreases with each training round. The normalized weights are obtained by applying softmax to the similarity scores within the first k images. This represents the similarity between input f and the r-th prototype in set j*. It is the feature vector of the r-th prototype in set j* during the t-th training round. For set The prototype index in This is the transpose of the input feature vector; During the momentum update phase, the matched prototype features The normal prototype library is dynamically updated by integrating new feature information through an exponential weighting strategy. This is to preserve some of the original information from the prototype. This is to incorporate the current input information; abnormal features are stored in a buffer for negative sample modeling or auxiliary discrimination.

[0007] Preferably, the gated attention mechanism calculates the activation value through two nonlinear transformations. and gate vector The attention score is obtained after Hadamard product aggregation. Then, attention was scored. Normalization is performed to obtain the attention weights for each image. : In the formula, This represents the Sigmoid activation function. and For learnable transformation matrices, The transpose of the linear transformation matrix used to score attention. This indicates the score for each instance in each package.

[0008] Preferably, in step S2, the anomaly-guided attention mechanism of the normalization strategy includes a prototype-guided attention mechanism and a prototype-driven normalization scheme. The prototype-guided attention mechanism quantifies the abnormality of each OCT image by measuring the deviation between the prototype-guided attention module PGA and the learned normal feature prototype, and guides attention to potential abnormal areas. The prototype-driven normalization scheme normalizes features through the prototype-driven normalization module PDN and recalibrates depth features using prototype statistics, thereby enhancing the contrast between normal and abnormal structures in OCT scan images from the same patient.

[0009] As a preferred approach, the prototype-guided attention mechanism involves: first, calculating the base attention weights for each feature. The base attention weights are obtained and normalized by content transformation and gating transformation; Then, an additional bootstrapping branch is introduced to interact with the normal prototype library and calculate the contrast score. This measures the deviation between the current input and the normal distribution. Where V and U are linear transformation matrices in the attention network, σ represents the sigmoid activation function, and w represents the weight vector. This represents the transpose of the weight vector w. Indicates in set Take the maximum value within the range; Represents each instance Normalized base attention weights It is the contrast score, and bank represents the set of common prototype features P initialized during the prototype library construction and initialization phase; A lightweight gating branch is added to adjust the fusion of basic attention and guided attention, allowing dynamic weighting between the two to ultimately obtain the fused attention weights. : in, Input features Perform embedding transformation, Let f be the gated embedding and projection matrix, σ be the sigmoid activation function, g(x) be the gate coefficients, D be the gated hidden dimension, and L be the input feature dimension, i.e., the length of f. For the original attention weights, This refers to the attention weights after fusion.

[0010] As a preferred embodiment, the prototype-driven normalization strategy is as follows: First, calculate the features of the input training data. and all prototype features in the prototype library The cosine similarity matrix between them is then used for subsequent feature normalization: in, The mean of the prototype set, The standard deviation of the prototype set, This indicates that the prototype corresponding to index j is being traversed. Calculate their arithmetic mean. This represents the feature of the i-th input training data. Represents the j-th prototype feature. This represents the original z-score normalized result. This represents the truncated normalized feature. This represents the final normalized feature after linear fusion with the original features. The fusion weights are normalized and then the cosine similarity is used to construct the similarity distribution.

[0011] Preferably, in step S3, the constructed PGMIL model obtains the final multi-instance features by linearly mixing the prototype-guided attention weighting result and the prototype-driven normalization result, and then performs weighted summation to obtain the package-level representation for classification. in, Original features This represents the features after processing by the attention mechanism. This represents the joint features resulting from the fusion of attention mechanisms and normalization. To integrate attention weights, The normalized features are referenced from the prototype. These are learnable parameters.

[0012] Preferably, in step S4, the total loss function of the PGMIL model multi-task loss training is... , by contrast loss Diversity loss With basic cross-entropy loss The result, obtained by combining these elements, is expressed as follows: in, and Representing the current features respectively Scaled mean cosine similarity between the normal and abnormal prototype libraries; It is a small constant used to ensure numerical stability; Represents the normalized feature set The pairwise similarity matrix; , Let B be the weight hyperparameter, B be the number of dormitory samples compared, and e be the natural exponent exp(·). N is a small, numerically stable constant used to calculate the number of instances for diversity loss.

[0013] This invention discloses a prototype-based multi-instance learning framework for image coarse label AMD classification. It innovatively proposes a prototype-based multi-instance learning framework (PGMIL) that can identify discriminative abnormal regions with only coarse bag-level labels. By integrating dynamic prototype optimization, anomaly-guided attention mechanism and auxiliary loss function, this framework significantly improves feature discrimination ability without pixel-level annotation, greatly improving the accuracy and efficiency of ADM classification of OCT images. Attached Figure Description

[0014] Figure 1 This is a comparison diagram between traditional clinical diagnosis of macular degeneration and the PGMIL framework of this invention.

[0015] Figure 2 This is an architecture diagram of the PGMIL network of this invention. Detailed Implementation

[0016] The present invention will now be described in further detail with reference to the accompanying drawings and specific embodiments.

[0017] like Figure 1 As shown, Figure 1 The left side shows the clinical diagnostic process for age-related macular degeneration (AMD), which is cumbersome and highly dependent on the doctor's judgment. To address the challenges of current AMD image analysis, this invention proposes a coarse-label AMD classification method based on the prototype-based multi-instance learning framework (PGMIL), such as... Figure 1 As shown on the right, this method uses multiple sets of OCT images of patients as training samples to input into the PGMIL model, and simulates the doctor's guidance mode through a prototype library to accurately focus on abnormal areas for diagnosis.

[0018] First, this invention innovatively constructs a prototype-based multi-instance learning framework (PGMIL). This framework employs multi-instance learning technology, requiring only patient-level labeled data, to identify key feature samples that determine disease diagnosis from coarse-grained labeled OCT images, and achieve multi-class prediction of age-related macular degeneration (AMD). Simultaneously, this framework constructs a prototype-guided segmentation scheme focusing on abnormal regions, thereby significantly simplifying the originally complex manual segmentation and labeling process.

[0019] like Figure 2 As can be seen from the prototype-based multi-instance learning framework (PGMIL) network architecture, it mainly consists of three core modules and one model optimization module. The three core modules are the prototype module (PM), the prototype-guided attention module (PGA), and the prototype-driven normalization module (PDN).

[0020] Among them, the prototype module PM: captures the distribution of normal retinal structure by initializing a set of potential normal feature prototypes. These normal feature prototypes are dynamically updated during training by exponential moving average, serving as a global prior for cross-sample shared structural consistency modeling. Prototype-guided attention module PGA: Quantifies the abnormality of each OCT image by measuring the deviation from the learned normal feature prototype, and guides attention to potential abnormal areas. Prototype-Driven Normalization (PDN) module: Normalizes features and recalibrates depth features using prototype statistics, enhancing the contrast between normal and abnormal structures in OCT scans from the same patient.

[0021] In addition, the model optimization module employs auxiliary loss and progressive optimization strategies for iterative training of PGMIL. It designs a contrastive loss to maximize inter-class margin, a diversity loss to maintain the dispersion of prototypes, and a curriculum-based training plan to gradually improve attention quality and prototype reliability.

[0022] Furthermore, the construction process of the aforementioned prototype-based multi-instance learning framework (PGMIL) model, and the specific steps for calling the optimal prototype-based multi-instance learning framework (PGMIL) model for image coarse label AMD classification, include the following: S1. Build a dynamically updated normal prototype library; In medical image classification, the differences between patients make it difficult to identify consistent normal features. To address this issue, this invention constructs a dynamically updated normal prototype library through a prototype module PM. This library provides stable and universal prior knowledge of normal retinal structure, which helps to align and distinguish features in the current sample.

[0023] First, OCT images with normal retinal structure distribution are acquired as normal samples, and global features are extracted from these normal samples. Second, the normal prototype library is dynamically updated based on a momentum update mechanism to extract common features from the normal samples. These common features include: locating abnormal regions in layer thickness / interlayer ratio through anomaly scores; statistically analyzing local texture and tissue heterogeneity through dimensionality reduction and attention mechanisms; and judging interface irregularities / edge curvature through comparative scores. Specifically: 1) Prototype Library Construction and Initialization: To create an initial normal prototype library, this invention first extracts global features from all normal OCT images and determines their importance through an image-level attention mechanism. Each training set... It contains N normal images, where, This represents the global feature vector of image i.

[0024] A gated attention mechanism is used to calculate the attention weight for each image, highlighting images with more significant structural features. This mechanism calculates the activation value through two nonlinear transformations. and gate vector The attention score is obtained after Hadamard product aggregation. Then, attention was scored. Normalization is performed to obtain the attention weights for each image. : In the formula, This represents the Sigmoid activation function, and For learnable transformation matrices, The transpose of the linear transformation matrix used to score attention. This indicates the score for each instance in each package; Finally, based on attention weights Images in each normal image packet are filtered, and the top k images with the highest scores are retained to create a candidate feature set. For large datasets, the K-means algorithm is applied. The goal of clustering is to minimize the objective function. Where f is a candidate feature. Let m be the cluster center. The global feature mean of all candidate features. The penalty for controlling the deviation of cluster centers from the global distribution, where M is the number of clusters.

[0025] 2) Prototype library dynamic update mechanism: In each training step, the normal input features and the standard prototype set are calculated. Cosine similarity between prototype features The representativeness of the samples is ensured by selecting the most similar prototype features. The standard prototype set is obtained by selecting the mid_feat (dimensionality-reduced features) of the top-k patches (the top k image patches with the highest scores) of each image according to weights, and concatenating the top-k mid_feat of all normal samples; if clustering is added, K-means is performed on the set, and the cluster centers are taken as the more compact prototype pK.

[0026] By continuously tracking the frequency of occurrence of each standard prototype feature, high-frequency features can be sorted and filtered, as shown in the following formula: For any normal input feature vector f, its relationship with each prototype feature is calculated. cosine similarity Here, f represents the features of all normal samples during initialization. This represents the similarity in semantic space between the normal input feature vector f and the j-th prototype feature. It is an operation that takes the j that maximizes the value of the function that follows. This represents the set of prototype indices that are closest to the input feature vector f; The update coefficient decreases with each training round. The normalized weights are obtained by applying softmax to the similarity scores within the first k images. This represents the similarity between input f and the r-th prototype in set j* (the set of indices of the closest prototypes). It is the feature vector of the r-th prototype in set j* during the t-th training round. For set The prototype index in This is the transpose of the input feature vector.

[0027] Next, in the momentum update phase, the matched prototype features The update is performed by integrating new feature information using an exponential weighting strategy; This is to preserve some of the original information from the prototype. This is to incorporate elements of the current input information; the learning rate is dynamically adjusted to accelerate early adaptation and stabilize the prototype later. Finally, anomalous features are stored in a buffer for negative sample modeling or auxiliary discrimination. This process balances knowledge accumulation and adaptation to new data, enhancing feature support for anomaly detection.

[0028] S2. Using the prototype library, introduce an anomaly-guided attention mechanism with a normalization strategy to refine the features of the prototype. To improve the ability of prototype-based multi-instance learning framework (PGMIL) models to detect anomalies and reduce domain differences among patients, this invention introduces a guided attention mechanism that uses a prototype library and normalization strategy to enhance feature aggregation and representation learning.

[0029] 1) Prototype-guided attention mechanism: In multiple instance learning (MIL), different samples contribute differently to classification. This invention extends the traditional gating attention framework by integrating guidance information from a conventional prototype library, focusing on anomalous regions that deviate from the conventional pattern.

[0030] The Prototype-guided Attention Module (PGA) first calculates the base attention weight for each feature. The base attention weights are derived from content transformation and gating transformation and normalized. Unlike the attention used to select the top-k during prototype construction, these are used for instance-level aggregation. Then, an additional bootstrapping branch is introduced to interact with the normal prototype library and calculate the contrast score. This measures the deviation between the current input and the normal distribution. In this context, V and U are linear transformation matrices in the attention network, σ represents the Sigmoid activation function, and w represents the weight vector. Represents each instance The normalized base attention weights (calculated using the same method as in the prototype initialization phase), and It is the contrast score, and bank represents the set of common prototype features P initialized during the prototype library construction and initialization phase. This represents the transpose of the weight vector w. Indicates in set Take the maximum value within the range; Next, a lightweight gating branch is added to adjust the fusion of basic attention and guided attention, allowing for dynamic weighting between the two, ultimately resulting in the fused attention weights. ; in, Input features Perform embedding transformation, Let f be the gated embedding and projection matrix, σ be the Sigmoid activation function, g(x) be the gate coefficient (adaptive fusion coefficient), D be the gated hidden dimension, and L be the input feature dimension (the length of f). For the original attention weights, This refers to the attention weights after fusion.

[0031] In summary, this mechanism enables prototype-based multi-instance learning framework (PGMIL) models to pay more attention to anomalous regions by utilizing guidance from normal prototype libraries.

[0032] 2) Prototype-Driven Normalization Scheme: Significant differences exist between patients in optical coherence tomography (OCT) images, which can easily lead to neighborhood shift during training. To enhance feature consistency and discriminative power, this invention proposes a feature normalization strategy based on a normalized sample library, as follows: First, calculate the features of the input training data. and all prototype features in the prototype library The cosine similarity matrix between them, here, The features of all training data during the training process are then normalized. in, The mean of the prototype set, The standard deviation of the prototype set, This indicates that the prototype corresponding to index j is being traversed. Calculate their arithmetic mean. This represents the i-th input feature. Represents the j-th prototype feature. This represents the original z-score normalized result. This represents the truncated normalized feature. This represents the final normalized feature after linear fusion with the original features. The fusion weight (fixed at 0.8 in this experiment) is normalized and then the cosine similarity is calculated to form a similarity distribution. The model can use this root to correct the input features and make them closer to the original feature direction.

[0033] S3. Construct the prototype-based multi-instance learning framework PGMIL model, which includes the fusion prototype module PM, the prototype-guided attention module PGA, and the prototype-driven normalization module PDN. Specifically, the process involves linearly mixing the prototype-guided attention weighted result with the prototype-driven normalization result to obtain the final multi-instance feature, followed by weighted summation to obtain a package-level representation for classification. in, This represents the features after processing by the attention mechanism. This represents the joint features resulting from the fusion of attention mechanisms and normalization. Original features To integrate attention weights, The normalized features are referenced from the prototype. These are learnable parameters.

[0034] S4. Perform multi-task loss training on the prototype-based multi-instance learning framework PGMIL model until the optimal model parameters are obtained. To improve the PGMIL model's ability to detect subtle anomalies, two auxiliary loss functions were introduced during training to enhance the discriminative power and robustness of the feature representations. Contrastive Loss This is used to measure the similarity between the current feature and the normal / abnormal prototype library, while diversity loss... This approach limits feature diversity by calculating the average of the off-diagonal elements of the similarity matrix, preventing features from collapsing in a single direction. These losses are similar to the basic cross-entropy loss. By combining these, the enhanced total loss function is formed. : in, and Representing the current features respectively Scaled mean cosine similarity between the normal and abnormal prototype libraries; It is a small constant used to ensure numerical stability; Represents the normalized feature set The pairwise similarity matrix; , Let B be the weight hyperparameter, B be the number of dormitory samples compared, and e be the natural exponent exp(·). N is a small, numerically stable constant used to calculate the number of instances for which diversity loss occurs (e.g., the total number of instances within a bag or the current batch).

[0035] The training objective is to make the total loss function Minimize until the optimal model parameters of the PGMIL model are obtained.

[0036] S5. Use the PGMIL model with the optimal parameters to perform AMD classification analysis on the input OCT image.

[0037] The specific process is as follows: A set of OCT images from the same patient is used as a whole to input the optimal PGMIL model. First, following the feature extraction and dimensionality reduction methods in S1, key features are extracted from each complete OCT image and their dimensions are compressed to form a low-dimensional feature set that is easy to process. Next, the dynamically updated and stable normal prototype library in S1 is called. Through the prototype-driven normalization module PDN in S2, these low-dimensional features are corrected by referencing the feature distribution of the normal prototype library, reducing the impact of individual differences between different patients and making the features more consistent. Finally, the prototype-guided attention module PGA in S2 processes the corrected features, first calculating the basic attention for each image feature through a gating attention mechanism. The importance of weights is assessed, and the degree of deviation from the normal structure is determined by combining the similarity between features and normal prototypes. Finally, a lightweight gating dynamic balance is used to obtain the final weights that can focus on abnormal features. Then, following the strategy in S3, the normalized features and the features weighted by the final weights are fused together, and all fused image features are summed to obtain the comprehensive features of the entire patient image package. Finally, this comprehensive feature is input into the classifier trained by the multi-task loss in S4, which outputs the classification probability distribution. The class with the highest probability is taken as the final AMD classification result, and the confidence level of the result is given. The entire process relies entirely on the model modules and optimal parameters built in S1-S4 to complete the classification.

[0038] In summary, this invention extracts global feature representations from normal samples to construct a feature prototype library. Through continuous optimization based on a momentum update mechanism, it accurately extracts common features of normal samples and establishes prototype modules. Subsequently, by comparing abnormal samples with the prototypes and calculating an attention mechanism based on abnormality scores, the ability of the PGMIL framework model to identify potential abnormal regions is significantly improved. Furthermore, during the training process of the PGMIL framework model, contrastive loss and diversity loss are introduced into its loss function to help the PGMIL framework model more accurately distinguish different types of abnormal samples.

[0039] The image coarse label AMD classification method of the prototype-based multi-instance learning framework of the present invention will be evaluated and further explained below in conjunction with the deployment of the prototype-based multi-instance learning framework of the present invention on the terminal.

[0040] 1. Evaluation Indicators To comprehensively evaluate the performance of the proposed method, three commonly used metrics are employed: accuracy, F1 score, and recall. These metrics together constitute a comprehensive evaluation system for the model's performance on different tasks.

[0041] 2. Implementation and Training Details The experiment was conducted using four NVIDIA RTX 3090 graphics cards. Training consisted of two phases: first, image features were extracted using a ResNet50 network, and the features for each patient were saved in package-level PKL format; then, the feature library was pre-trained using a `bank_warmup_epoch` to construct an initial normal sample feature library; finally, joint training was performed. The Adam optimizer was used, with an initial learning rate of 0.0001. Feature processing controlled the fusion weights between the original and normalized features using the `alpha_mode` parameter. The loss function integrated cross-entropy loss, contrastive loss, and diversity loss. Each training batch contained 16 samples, each consisting of 3 sub-bags; each sub-bag selected 18 instances, and the MaxMinS strategy was used to classify the samples into easy and difficult classes. The same grouping strategy was used in the validation phase. The final model was selected based on the highest F1 score on the validation set. The test metric, after excluding outliers, was the average of the last 50 training epochs.

[0042] The evaluation results on different datasets are as follows: Where, accuracy Acc = Recall rate Precision F1 score TP indicates a true positive, TN indicates a true negative, FP indicates a false positive, and FN indicates a false negative.

[0043] 3. Complete process of PGMIL network call and AMD classification The process of building the PGMIL network architecture: (1) Data input and preprocessing (offline): Pre-trained backbone networks such as ResNet50 are used to extract patch-level features from the original OCT images and store them in pkl format (each image corresponds to an N×C dimension feature matrix). This format is compatible with the input requirements of the subsequent load_data function and provides basic data for the model's subsequent feature processing.

[0044] (2) Basic module initialization: The core basic modules of the initialized model include the DimReduction module, which maps high-dimensional patch features (e.g., 1024-dimensional) to low-dimensional features (e.g., 512-dimensional); the first-stage Classifier_1fc classifier for sub-packet / aggregate feature classification; the gated attention module (Attention) that supports bank guidance (controlled by the use_bank_guidance parameter); the Attention_with_Classifier or a custom UClassifier for secondary classification; and the EnhancedNormalBank module, which is used to build and maintain the normal prototype library (initially initialized through initial_bank, and subsequently updated dynamically by EMA). Each module corresponds to a core component in Model.network.

[0045] (3) Construction of the Normal ROI Bank: For samples labeled NORMAL_CLASS in the training set, the patch features of each image are first converted into mid_feat (N×mDim dimension) using the DimReduction module. Then, the Attention module is used to calculate the unnormalized (isNorm=False) attention score. After softmax sorting, the top-k (set by the params.normal_topk parameter) most representative patches are selected. The mid_feat of these top-k patches are summarized and concatenated to obtain the initial normal_roi_bank (M×mDim dimension). Finally, the EnhancedNormalBank module is initialized using this initial library (which stores key data such as bank, abnormal_bank, and counter).

[0046] (4) Normalization / Prototype-Driven Modules (NormalizationMethods): Implement various normalization methods based on normal_bank: z-score, cosine, prototype, etc., to normalize or map mid_feat to a statistical space consistent with the prototype.

[0047] (5) Dynamic feature fusion mechanism (alpha_mode): A dynamic alpha calculation strategy (alpha = min (1.0, epoch / bank_warmup_epoch)) is adopted. The normalized feature (normalized_feat) and the attention-weighted feature (attention_feat) are dynamically fused through the formula combined = alpha * normalized_feat + (1 - alpha) * attention_feat, so as to balance the contribution of features at different stages.

[0048] (6) Multi-instance grouping and hard case mining (MaxMinS): The patch index of each image is randomly divided into numGroup sub-bags. Within each sub-bag, the tAA score after softmax is calculated using Attention to obtain the attention-weighted feature (attention_feat). Then, the patch-level class activation probability and uncertainty (entropy) are calculated using get_cam_1d. The samples are sorted according to combined_scores = w1 * cam_score + w2 * uncertainty. The first num_hard (hard examples) and the last num_easy (easy examples) are selected to form a sub-bag for secondary training / distillation. Finally, the selected instances are concatenated to obtain slide_pseudo_feat (as input to UClassifier).

[0049] (7) Loss and Optimization: The first-level loss is optimized by optimizer0 using the DimReduction, Attention, and Classifier_1fc modules. If the training epoch reaches bank_warmup_epoch and contrastive learning is enabled, enhanced_loss = CE + λ1 * contrastive + λ2 * diversity is used; otherwise, only cross-entropy loss (CE) is used. The second-level loss is optimized by optimizer1 using the UClassifier module, employing cross-entropy loss (CE, with gSlidePred and slide label as inputs). This is combined with gradient clipping and MultiStepLR learning rate scheduling strategies. The CurriculumScheduler divides the learning process into initial / middle / final stages by epoch, dynamically adjusting the hard ratio and learning rate factor to achieve a learning progression from easy to difficult. Furthermore, during each subgroup processing (or by update_freq / epoch frequency), the enhancedNormalBank.update module updates the most similar prototype (bank[idx] = (1-w)*bank[idx] + w *) for the mid_feat of normal samples using EMA or a weighted method. (new_feat), while accumulating abnormal features into abnormal_bank to support subsequent contrastive loss calculation.

[0050] The complete process of using the optimal PGMIL for ADM classification: The prerequisites are that the optimal model weights, including classifier, dim_reduction, attention, and UClassifier, have been trained and saved, and the test set pkl file has been prepared.

[0051] (1) Environment preparation: Load the model structure consistent with the training phase (including Classifier_1fc, DimReduction, Attention, UClassifier), load the optimal model weight file through torch.load, and call the load_state_dict function to distribute the weights to the corresponding modules. At the same time, set normal_bank to the final EnhancedNormalBank at the end of training (or reinitialize EnhancedNormalBank with normal_roi_bank saved during training) to ensure that the inference environment is consistent with the training environment.

[0052] Data loading: Use the load_data function to load the test set data and obtain SlideNames (image names), FeatureList (patch feature list), and Label (label). Each test sample corresponds to an N×C dimension patch feature matrix (feat).

[0053] (2) Single sample processing: Perform the following operations on each test image in sequence: First, send the feature to the dimReduction module (the feature needs to be transferred to the target device first) to obtain the midFeat of N×mDim dimension; then call the Attention module to calculate the attention score (AA), the parameter iswarm is set to True / False according to whether bank_warmup is completed, and after obtaining the N-dimensional score, it is normalized by softmax; if normalization is enabled and warmup has been completed, normalize midFeat by the corresponding method in NormalizationMethods to obtain midFeat_norm; complete feature fusion according to combined_feat = alpha * midFeat_norm + (1-alpha) * (midFeat * AA.unsqueeze (-1)); then divide the N instance indices into numGroups randomly or fixedly according to the same strategy as training, and for each subgroup: extract the combined_feat_sub and the corresponding tAA_sub (AA [index_sub]) of the instances in the group, and calculate traceFeats = combined_feat_sub * tAA_sub is summed to obtain tattFeat_tensor (1×mDim dimension), which is then fed into classifier to obtain sub_pred (used for optional patch-level scores and CAM calculation). Then, based on the distillation strategy during training (such as MaxMinS), the top-k instances within the group are selected to construct slide_d_feat.

[0054] (3) Secondary fusion prediction: After concatenating all the slide_d_feat of the subgroups, we send them into UClassifier to obtain gSlidePred. After softmax processing, we obtain the probability of each group. We take the mean of all group probabilities as the final probability distribution of the image (slide). We use the argmax function to determine the final predicted label from the probability distribution.

[0055] (4) Post-processing and index calculation: Collect the prediction results (gPred_1) and true labels (gt_1) of the entire test set, and call the eval_metric function to calculate evaluation indicators such as Accuracy, Recall, F1, and AUC; if a stability strategy is adopted, take the test results of the last few training rounds, remove outliers using the IQR method, and calculate the average value as the evaluation index of the final report.

[0056] In summary, the present invention has the following technical advantages: (i) A multi-instance learning framework is constructed for AMD classification, overcoming the limitation of existing methods that rely on fine-grained annotations. This method can automatically extract discriminative features from multiple images, and its performance is significantly better than traditional methods.

[0057] (ii) A dynamic feature prototype modeling technique is proposed, which updates the prototype parameters in real time during the training process, providing a reference for the identification of abnormal regions.

[0058] (iii) A prototype-guided contrastive learning method based on category prototypes and anomaly score measurements was developed to enhance the model’s discriminative ability by optimizing the classification loss function.

[0059] The above embodiments are not intended to limit the present invention, and the present invention is not limited to the examples given above. Any changes, modifications, additions or substitutions made by those skilled in the art within the scope of the technical solution of the present invention are also within the protection scope of the present invention.

Claims

1. An image coarse label AMD classification method based on a prototype-based multi-instance learning framework, characterized in that: Includes the following steps: S1. Construct a dynamically updated normal prototype library through feature extraction and dimensionality reduction methods; S2. Using the prototype library, introduce an anomaly-guided attention mechanism with a normalization strategy to refine the features of the prototype. S3. Construct the prototype-based multi-instance learning framework PGMIL model, which includes the fusion prototype module PM, the prototype-guided attention module PGA, and the prototype-driven normalization module PDN. S4. Perform multi-task loss training on the prototype-based multi-instance learning framework PGMIL model until the optimal model parameters are obtained. S5. Use the PGMIL model with the optimal parameters to perform AMD classification analysis on the input OCT image.

2. The image coarse label AMD classification method based on the prototype-based multi-instance learning framework according to claim 1, wherein: In step S1, the construction of the normal prototype library includes the following process: 1) Prototype library construction and initialization: collect OCT images with normal retinal structure distribution as normal samples, extract global features from normal samples, calculate attention weight of each image using gated attention mechanism, and select the top k images with the highest scores to create a candidate feature set Filter each normal OCT image, and retain the top k images with the highest scores to create a candidate feature set ; 2) Dynamic update mechanism of normal prototype library: calculate the cosine similarity between normal input features and each prototype feature in the standard prototype set. The standard prototype set is obtained by concatenating the features of the first k images of normal samples after dimensionality reduction. Continuously track the frequency of occurrence of each standard prototype feature, and sort and filter the high-frequency features, as shown in the following formula: where, for any normal input feature vector f, the cosine similarity between it and each prototype feature is computed where, represents the similarity between the normal input feature vector f and the jth prototype feature in the semantic space, is the operation of taking the j that maximizes the following function value, represents the prototype index set that is most similar to the input feature vector f; is the mapping function for f; is the update coefficient that decreases with the training round, is the normalized weight obtained by the similarity in the first k image range by softmax, represents the similarity between the input f and the rth prototype in the set j*, is the feature vector of the rth prototype in the set j* at the tth training round, is the prototype index in the set , and is the transpose of the input feature vector. In the momentum updating stage, the matched prototype features The new feature information is integrated by an exponential weighting strategy to update the normal prototype library dynamically, To retain part of the original information of the prototype, To integrate the current input information; the abnormal features are stored in the buffer for negative sample modeling or auxiliary discrimination.

3. The image coarse label AMD classification method based on a prototype-based multi-instance learning framework according to claim 2, characterized in that: The gating attention mechanism calculates the activation value through two nonlinear transformations and the gating vector , and obtains the attention score after the Hadamard product aggregation , and then normalizes the attention score to obtain the attention weight of each image : wherein, denotes a Sigmoid activation function, and is a learnable transformation matrix, is a linear transformation matrix transpose of the attention score, denotes the score of each instance in each bag.

4. The image coarse label AMD classification method based on the prototype-based multi-instance learning framework according to claim 1, wherein: In step S2, the anomaly-guided attention mechanism of the normalization strategy includes a prototype-guided attention mechanism and a prototype-driven normalization scheme. The prototype-guided attention mechanism quantifies the abnormality of each OCT image by measuring the deviation between the prototype-guided attention module PGA and the learned normal feature prototype, and guides attention to potential abnormal areas. The prototype-driven normalization scheme normalizes features through the prototype-driven normalization module PDN, and recalibrates depth features using prototype statistics to enhance the contrast and differentiation between normal and abnormal structures in OCT scan images from the same patient.

5. The image coarse label AMD classification method based on a prototype-based multi-instance learning framework according to claim 4, characterized in that: Prototype-guided attention mechanism: First, calculate the base attention weight for each feature. The base attention weights are obtained and normalized by content transformation and gating transformation; Then, an additional bootstrapping branch is introduced to interact with the normal prototype library and calculate the contrast score. This measures the deviation between the current input and the normal distribution. Where V and U are linear transformation matrices in the attention network, σ represents the sigmoid activation function, and w represents the weight vector. This represents the transpose of the weight vector w. Indicates in set Take the maximum value within the range; Represents each instance Normalized base attention weights It is the contrast score, and bank represents the set of common prototype features P initialized during the prototype library construction and initialization phase; A lightweight gating branch is added to adjust the fusion of basic attention and guided attention, allowing dynamic weighting between the two to ultimately obtain the fused attention weights. : in, Input features Perform embedding transformation, Let f be the gated embedding and projection matrix, σ be the sigmoid activation function, g(x) be the gate coefficients, D be the gated hidden dimension, and L be the input feature dimension, i.e., the length of f. For the original attention weights, This refers to the attention weights after fusion.

6. The image coarse label AMD classification method based on a prototype-based multi-instance learning framework according to claim 4, characterized in that: The prototype-driven normalization strategy is as follows: First, calculate the features of the input training data. and all prototype features in the prototype library The cosine similarity matrix between them is then used for subsequent feature normalization: in, The mean of the prototype set, The standard deviation of the prototype set, This indicates that the prototypes corresponding to index j are traversed, and their arithmetic mean is calculated. This represents the feature of the i-th input training data. Represents the j-th prototype feature. This represents the original z-score normalized result. This represents the truncated normalized feature. This represents the final normalized feature after linear fusion with the original features. The fusion weights are normalized and then the cosine similarity is used to construct the similarity distribution.

7. The image coarse label AMD classification method based on a prototype-based multi-instance learning framework according to claim 1, characterized in that: In step S3, the constructed PGMIL model obtains the final multi-instance features by linearly mixing the prototype-guided attention weighted results with the prototype-driven normalization results, and then performs weighted summation to obtain the bag-level representation for classification. in, Original features This represents the features after processing by the attention mechanism. This represents the joint features resulting from the fusion of attention mechanisms and normalization. To integrate attention weights, The normalized features are referenced from the prototype. These are learnable parameters.

8. The image coarse label AMD classification method based on a prototype-based multi-instance learning framework according to claim 1, characterized in that: In step S4, the total loss function for multi-task loss training of the PGMIL model is... , by contrast loss Diversity loss With basic cross-entropy loss The result, obtained by combining these elements, is expressed as follows: in, and Representing the current features respectively Scaled mean cosine similarity between the normal and abnormal prototype libraries; It is a small constant used to ensure numerical stability; Represents the normalized feature set The pairwise similarity matrix; , Let B be the weight hyperparameter, B be the number of dormitory samples compared, and e be the natural exponent exp(·). N is a small, numerically stable constant used to calculate the number of instances for diversity loss.

Images