Pathological image classification method and system based on teacher-student network and feature layering

Abstract

The invention discloses a pathological image classification method based on a teacher-student network and feature layering, and relates to the technical field of pathological image analysis. The method comprises the following steps: firstly, performing tissue region extraction and slicing on a full-slice pathological image; constructing a teacher model composed of a feature extraction module, an embedding module, a gating attention module and an aggregation module, calculating attention scores of instances by using the teacher model, and dynamically dividing the instances into a significant group, a potential group and a trivial group according to the attention scores; and then constructing a student model, rejecting trivial groups, endowing significant group instances with pseudo labels and executing merging and shuffling operations to carry out instance-level training, carrying out feature aggregation on potential group instances to carry out group-level training, and optimizing the student model through a joint loss function. According to the method, atypical information with diagnostic value in a potential group is fully utilized through a layering strategy, and the recognition capability and generalization performance of the model on complex pathological features are enhanced while background noise is removed.

Description

Technical Field

[0001] This invention relates to the field of pathological image analysis, specifically to a weakly supervised classification method and system for pathological images based on teacher-student networks and feature hierarchies. Background Technology

[0002] Accurate classification of whole-slice pathological images (WSI) is a core component of computer-aided pathological diagnosis. Weakly supervised multiple instance learning (MIL) has become the mainstream technology in this field because it only requires WSI-level labels to train the model, avoiding tedious instance-level annotation. However, existing weakly supervised multiple instance learning methods still have many technical challenges in WSI classification:

[0003] 1. Existing technologies cannot fully utilize potential positive feature information: Traditional attention-based MIL methods rely excessively on instances (patches) with significant pathological features, often ignoring "latent" positive instances with inconspicuous features but still containing diagnostic information, resulting in the model's incomplete utilization of complex pathological features.

[0004] 2. Existing technologies struggle to remove redundant noise while preserving key information: Existing Top-k selection or threshold filtering methods attempt to suppress salient features but are prone to information loss; while simple multi-branch networks are prone to pattern redundancy or branch marginalization, making it difficult to effectively extract useful information while removing background noise (such as normal tissues and blank areas).

[0005] 3. Lack of effective instance-level supervision signals under weak supervision: When only bag (i.e., the entire WSI) level labels are available, existing models have difficulty obtaining accurate instance-level classification capabilities, which limits the model's ability to locate lesions and generalize.

[0006] 4. Insufficient generalization ability and robustness of the model: Due to the heterogeneity of pathological images, existing methods are prone to overfitting significant features when faced with data from different sources or feature distributions, which can lead to a decline in generalization performance.

[0007] Therefore, those skilled in the art have provided a weakly supervised processing method and system for pathological images based on teacher-student networks and feature hierarchies to address the problems mentioned in the background art. Summary of the Invention

[0008] The purpose of this invention is to provide a weakly supervised pathological image classification method and system based on teacher-student networks and feature hierarchies, so as to solve the problems mentioned in the background art.

[0009] To achieve the above objectives, the present invention provides the following technical solution:

[0010] A weakly supervised processing method for pathological images based on teacher-student networks and feature hierarchies includes the following steps:

[0011] Step S1. Data Preprocessing: Process the input whole-slice pathological images The tissue region is extracted and segmented into multiple image patches of equal size to obtain an instance set. ;

[0012] Step S2. Build and train the teacher model (TeacherModel);

[0013] Step S21. Model Construction: Construct the model consisting of the feature extraction module ( ), feature embedding module ( Gated attention module ( ), Feature aggregation module ( The teacher model consists of a classifier and a package-level classifier.

[0014] Step S22. Feature Extraction (FETM): Extract image features using a pre-trained deep convolutional neural network or feature extraction model, and map the image features into a first-dimensional feature vector through an adaptive average pooling layer (AdaptiveAvgPool2d). ;

[0015] Step S23. Feature Embedding (FEBM): The feature vector is input into the feature embedding module, and sequentially passes through a one-dimensional batch normalization (BatchNorm1d), a fully connected layer (Linear), a ReLU activation function, and an L2 normalization layer to compress the feature dimension to 512 dimensions, thus obtaining the embedded features. ;

[0016] Step S24. Attention Scoring: Input the embedded features into the gating attention module Attention scores for each instance are calculated using fully connected layers and Sigmoid / Tanh activation functions. This score reflects the importance of the instance to the final classification;

[0017] Step S25. Feature Aggregation and Classification: The attention score of a bag is... and embedded features enter Obtain packet-level feature representation ,Will Input classifier Train the model and save the best-performing model as the teacher model;

[0018] Step S26. Model Training: Repeat the above steps on the entire training set until the model converges or reaches the preset stopping condition, and save the best performing model as the teacher model.

[0019] Step S3. Pathological Feature Stratification

[0020] Step S31. Feature Extraction: Extract image features using a pre-trained deep convolutional neural network or feature extraction model; map the image features into a first-dimensional feature vector using an adaptive average pooling layer (AdaptiveAvgPool2d). ;

[0021] Step S32. Feature Embedding: Embed all instance features in a WSI Input the feature embedding module of the teacher model trained in step S2 To obtain embedded features ;

[0022] Step S33. Scoring and Ranking: All embedded features in a WSI are scored and ranked. Input the gated attention module of the teacher model trained in step S2 Obtain the attention score for each instance. Attention scores are calculated for all instances within each bag (WSI) of the training set and sorted from highest to lowest.

[0023] Step S34. Dynamic Grouping: Set the hyperparameter ratio (Based on a preset ratio threshold), instances within a package are divided into a Significant Group (SG), a Latent Group (LG), and a Trivial Group according to the sorting results:

[0024] Step S35. Iterative processing: Divide all instances within each WSI in the training set into three groups: saliency group, latent group, and trivial group.

[0025] Step S4. Build and train the student model (StudentModel)

[0026] Step S41. Student Model Construction: The student model consists of a feature embedding module ( Gated attention module ( ), Feature aggregation module ( ), instance-level classifier Group-level classifier composition;

[0027] Step S42. Feature Embedding: Embed the features of a salient group instance in a WSI. and instance features of potential groups Input student model The saliency group embedding features were obtained respectively. and latent group feature embedding .

[0028] Step S43. Instance-level training: Merge all saliency groups in a training batch into a large instance library. And assign pseudo-labels to instances in the library and train instance-level classifiers. ;

[0029] Step S44. Group-level training: Training on latent group features Through the embedded module Obtaining embedded features Then, using the gating attention module ( ) and feature aggregation module ( Generate group-level features In pseudo-tags Post-training group level classifier .

[0030] Step S5: Inference and Prediction: The trained model is used to achieve accurate classification of whole-slice pathological images (WSI). Standardization and two-dimensional prediction fusion are used to ensure the reliability of the classification results. The specific optimized steps are as follows:

[0031] Step S51. Data preprocessing: Use the OTSU thresholding method to remove the background and retain the tissue area; cut into non-overlapping instances of a preset uniform size; complete pixel normalization;

[0032] Step S52. Teacher Model Feature Filtering Output: Input the preprocessed instances into the teacher model, extract feature vectors sequentially, compress them into embedded features; calculate instance attention scores, filter salient and latent group features according to preset hyperparameter ratios, and remove trivial groups;

[0033] Step S53. Student model group-level prediction S_G: The latent group features are optimized by embedding to generate group-level features, which are input into the group-level classifier and output as group-level prediction results S_G;

[0034] Step S54. Student model instance-level prediction S_I: Input salient group instances into the instance-level classifier to obtain the prediction result of a single instance; calculate the mean of all instance prediction results and output the instance-level prediction result S_I to reduce the random error of a single instance;

[0035] Step S55. Fusion Prediction (S_G+I): Using an equal-weighted averaging strategy, the group-level prediction result S_G and the instance-level prediction result S_I are averaged to obtain the final WSI classification probability distribution. The category with the highest probability is the final classification result.

[0036] As a further aspect of the present invention: the data preprocessing and feature extraction in step S1 further includes image segmentation and slicing, obtaining whole slice pathological images (WSI), removing the background using the OTSU thresholding method, and segmenting the region containing tissue into non-overlapping small patches (i.e., instances), with the size set to a preset uniform size.

[0037] As a further aspect of the present invention: the method for dividing the three groups in step S34 further includes:

[0038] Significant Group (SG): The top group with the highest scores Instances, containing the most discriminative pathological patterns, yield the following instance feature set: ;

[0039] Latent Group (LG): Top scorers in the middle Instances contain information that is not typical but has value for data mining; the resulting set of instance features is... ;

[0040] Trivial Group: The group with the lowest remaining score. Instances (usually background or normal tissue) are discarded during subsequent training of student models.

[0041] As a further aspect of the present invention, step S4, instance-level training (for the salient group), includes the following steps:

[0042] Step S401. Pseudo-tag assignment: Assign the package (WSI) tags Assign a significant group to each instance within this package .

[0043] Step S402. Merging: Merge all salient group instances from different packages in a training batch to form a large instance library (LG).

[0044] Step S403. Shuffling: Randomly shuffle the merged instance library to generate "unordered salient groups (DLG)". This step breaks the boundaries of the packs, increases the diversity of the samples, and forces the model to learn the features of the instances themselves rather than the contextual dependencies of the packs.

[0045] Step S404. Classification Training: Input the instances from the DLG into the instance-level classifier. Calculate instance-level loss.

[0046] As a further aspect of the present invention, the group-level training (for potential groups) in step S4 includes the following steps:

[0047] Step S411. Group Feature Aggregation: Instances within potential groups Using the attention module of the student model ( ) and aggregation module ( Generate group-level features .

[0048] Step S412. Assigning pseudo-tags: Assigning tags to the package Assign group-level features.

[0049] Step S413. Classification Training: Input group-level features into the group-level classifier Calculate group-level loss.

[0050] Step S414. Joint Loss Calculation: Use MarginLoss to optimize both instance-level classification and group-level classification tasks simultaneously. The total loss is the sum of the two parts.

[0051] A weakly supervised pathological image classification method and system based on teacher-student networks and feature hierarchies includes:

[0052] Data preprocessing module: used for WSI background removal, tissue region segmentation, and generation of preset uniform size instances;

[0053] Feature extraction module: Based on the pre-trained ResNet50 model (truncated to Stage 3), extracts 1024-dimensional feature vectors of instances;

[0054] Teacher model training module: includes feature extraction module ( ), feature embedding module ( Gated attention module ( ), Feature aggregation module ( A package-level classifier is used to generate attention scores and complete the training of the teacher model;

[0055] The pathological feature stratification module is used to receive the attention score and embedding features output by the teacher model, divide them into significant, latent, and trivial groups according to the proportion, and remove the trivial group.

[0056] Student model training module: includes feature embedding module ( Gated attention module ( ), Feature aggregation module ( ), instance-level classifier Group-level classifier Two-layer collaborative optimization is achieved through instance-level training and group-level training of "merging and shuffling";

[0057] Fusion Inference Module: Used to calculate the instance-level prediction mean and group-level prediction results, and output the final WSI classification result after fusion.

[0058] As a further aspect of the present invention: the feature extraction module uses an adaptive average pooling layer. Map instance features to feature vectors of fixed dimensions.

[0059] As a further aspect of the present invention: the total loss of the student model training module is the sum of the instance-level loss and the group-level loss, and MarginLoss is used for joint optimization.

[0060] Compared with the prior art, the beneficial effects of the present invention are:

[0061] 1. This invention divides instances into significant, potential, and trivial groups through a pathological feature stratification strategy. It makes full use of information in the potential group that has atypical features but diagnostic value, solves the problem of the incomplete utilization of complex pathological features in the existing technology, and improves the model's ability to identify various pathological patterns.

[0062] 2. This invention removes redundant noise by eliminating trivial group instances, while retaining effective information of salient and latent groups through the teacher-student network architecture, thus achieving a balance between noise removal and information retention and avoiding the problems of information loss or pattern redundancy in traditional methods.

[0063] 3. In student model training, this invention provides effective instance-level supervision signals for weakly supervised scenarios by assigning pseudo-labels to salient group instances and performing "merging and shuffling" processing, thereby improving the model's instance-level classification ability and lesion localization accuracy.

[0064] 4. This invention employs a two-layer collaborative training strategy and a fusion prediction mechanism, which enhances the model's adaptability to the heterogeneity of pathological images, effectively reduces the risk of overfitting, and improves the model's generalization ability and robustness, enabling it to maintain good performance on data from different sources or with different feature distributions. Attached Figure Description

[0065] Figure 1 This is the overall flowchart of the PFS-MIL of the present invention;

[0066] Figure 2 This is a flowchart of the feature extraction module and the feature embedding module in this invention;

[0067] Figure 3 This is a flowchart illustrating the workflow of the gating attention module and the feature fusion module in this invention. ;

[0068] Figure 4This is a flowchart of the pseudo-labeling, merging, and shuffling operations in this invention. Detailed Implementation

[0069] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0070] Please see Figures 1-4 In this embodiment of the invention, the specific workflow of the weakly supervised processing method and system for pathological images based on teacher-student networks and feature hierarchies is as follows:

[0071] Step S1. Data Preprocessing

[0072] For each WSI (Wide Sample Image), the data preprocessing module performs the following steps: The OTSU thresholding method is used to remove background information, automatically determining the optimal threshold and effectively separating background and tissue regions. Subsequently, the effective tissue-containing region is segmented into non-overlapping small patches, with a patch size set to 224×224. This size ensures sufficient pathological information while meeting the input requirements of subsequent feature extraction models, ultimately yielding a large number of instance samples to provide a data foundation for subsequent model training. Each WSI is considered a bag. ,Include One instance, namely The bag's label is .

[0073] Step S2. Teacher Model Training Phase

[0074] Step S21. Model Building

[0075] The teacher model is constructed according to step S21. This model consists of a feature extraction module ( ), feature embedding module ( Gated attention module ( ), Feature aggregation module ( It consists of a classifier and a package-level classifier, and the modules work together to achieve the mapping from instance features to package-level classification results.

[0076] Step S22. Feature Extraction (FETM)

[0077] The preprocessed 224×224 patch is input into the feature extraction module. This module uses a pre-trained, fixed-weight ResNet50 model (truncated to Stage 3) as the feature extractor. The ResNet50 model, pre-trained on a large-scale image dataset, possesses powerful feature extraction capabilities. Truncating to Stage 3 preserves more low- and mid-level features, making it suitable for fine-grained feature extraction from pathological images. After processing by the convolutional layers of the ResNet50 model, the patch yields multi-scale feature maps. Then, an adaptive average pooling layer (AdaptiveAvgPool2d) maps the feature maps to uniform 1024-dimensional feature vectors. For the bag The extracted feature set Represented as:

[0078]

[0079] Step S23. Feature Embedding (FEBM)

[0080] Then, the feature vector Input feature embedding module ( Perform feature dimension compression and optimization, such as Figure 2 As shown, the feature embedding module sequentially performs one-dimensional batch normalization (BatchNorm1d) on the feature vectors, which accelerates model convergence and improves model stability. Next, a fully connected layer (Linear) maps the 1024-dimensional feature vectors to 512 dimensions. Then, a ReLU activation function is used to introduce a non-linear transformation, enhancing the model's expressive power. Finally, an L2 normalization layer normalizes the features to obtain the embedded features. This embedded feature retains key information while reducing computational complexity, facilitating subsequent attention scoring calculation. The specific process is as follows:

[0081]

[0082]

[0083]

[0084] In the formula, Represents the input matrix, This represents the regularized matrix after regularization. This represents the projection matrix after projection. Let T denote the normalized matrix, and T denote the transpose matrix.

[0085] Step S24. Attention Score (GAM)

[0086] embed features Input Gated Attention Module ( ),like Figure 3 As shown, the gated attention module transforms the embedded features through a fully connected layer, and then uses a Sigmoid / Tanh activation function to calculate the attention score for each instance. The Sigmoid activation function has an output range of [0,1], and the Tanh activation function has an output range of [-1,1]. The appropriate activation function can be selected based on the actual training results. Attention score. The score reflects the importance of the corresponding instance to the final WSI classification result; a higher score indicates that the instance contains richer discriminative pathological information. For an instance... Its attention score The calculation formula is:

[0087]

[0088] In the formula, ⊙ represents element-wise multiplication, sigm is the sigmoid activation function, and tanh is the tanh activation function.

[0089] Step S25. Feature Aggregation and Classification (FAM): The attention scores of a bag (i.e., all instances corresponding to a WSI) are aggregated. and embedded features Simultaneously input the feature aggregation module ( ), Feature aggregation module ( Instance-level features are aggregated into package-level feature representations through weighted summation. Attention score As weights, important instances are given a larger proportion in the package-level features. This is how package-level features are represented. Input packet classifier The classifier uses a fully connected layer and a Softmax activation function, outputting the probability distribution of WSI belonging to each category. For example... The weighted fusion yields the package-level features. Later used in classifiers The prediction process is as follows:

[0090]

[0091]

[0092]

[0093]

[0094] Step S26. Model Training: During training, the MarginLoss function is used to calculate the loss value between the predicted result and the true label. The parameters of the teacher model are updated through the backpropagation algorithm (the weights of the feature extraction module are fixed, and only the parameters of the feature embedding module, gating attention module, feature aggregation module, and bag-level classifier are updated). The above steps are repeated on the entire training set for 200 rounds of training. After each round of training, the classification accuracy of the model on the validation set is calculated, and the model with the best performance on the validation set is saved as the final teacher model. This model can accurately evaluate the importance of each instance, providing a reliable basis for subsequent pathological feature stratification.

[0095] Step S3: Pathological Feature Stratification

[0096] Steps S31-S32: Using the trained teacher model, repeat steps S22-S23 to extract the embedding features of all bags in the training set. .

[0097] Step S33. Scoring and Ranking: The embedded features are then... Input the gated attention module GAMᵗ of the teacher model to obtain the attention score for each instance. Then, all instances within the WSI are sorted from highest to lowest attention score to obtain a sorted index matrix. :

[0098]

[0099] Step S34. Dynamic Grouping: Set the hyperparameter ratio In this embodiment, a grouping ratio of 1%:59%:40% is selected: the top 1% of instances with the highest scores are assigned to the salient group (SG), which contains the most discriminative pathological patterns and is the core basis for WSI classification; the middle 59% of instances (i.e., instances from the 1%+1th to the 60th percentile) are assigned to the latent group (LG), which, although their pathological features are not typical, still contain some diagnostic value and effectively supplement the features of the salient group; the remaining 40% of instances with the lowest scores are assigned to the trivial group, which typically consists of background areas or normal tissue and does not contain valuable pathological diagnostic information. These instances are removed during subsequent student model training to reduce noise interference. (Saliency group) and potential group The calculation process is as follows:

[0100]

[0101]

[0102] Following the above method, all instances within each WSI in the training set are grouped to ensure that all instances are divided into corresponding groups, providing structured data input for the two-layer collaborative training of the student model.

[0103] Step S4: Build and train the student model

[0104] Step S41. Student Model Construction: The student model consists of a feature embedding module ( Gated attention module ( ), Feature aggregation module ( ), instance-level classifier Group-level classifier Composition. The student model shares some module structures with the teacher model (such as the feature aggregation module), but has independent parameters. Through guidance from the teacher model and self-training, it achieves better classification performance.

[0105] Step S42. Feature Embedding: Obtaining salient group instance features after stratifying pathological features. and potential group instance features Input the feature embedding module of the student model respectively ( The structure of this module is similar to the feature embedding module of the teacher model. Consistent with each other, the data undergoes batch normalization, a fully connected layer, a ReLU activation function, and L2 normalization to obtain salient group embedding features. and latent group embedding features .

[0106]

[0107]

[0108] In the formula, Represents the embedding matrix of saliency groups. This represents the latent group embedding matrix.

[0109] Step S43. Instance-level training

[0110] Instance-level training is performed on salient group instances, and the specific steps are as follows:

[0111] Step S401. Pseudo-label assignment: Since the dataset only provides WSI-level labels, the package... tags Directly assign it to each instance of the significant group within the package. This provides supervision signals for instance-level training;

[0112] Step S402. Merging: In each training batch, salient group instances from all different WSI packages within that batch are merged to form a unified large instance library (LG), breaking the boundary limitations of individual WSI packages.

[0113]

[0114] Step S403. Shuffling: Shuffling the merged large instance library. Perform a random shuffling operation to generate an "unordered saliency group (DLG)," such as Figure 4 As shown. This operation increases sample diversity, avoids the model learning contextual dependencies within the WSI package, forces the model to focus on learning the pathological features of the instances themselves, and improves the model's generalization ability; the process of generating the order significant group (DLG) is as follows:

[0115]

[0116]

[0117] In the formula, , It is a permutation matrix with rows randomly shuffled.

[0118] Step S404. Classification Training: Input the instances from the DLG into the instance-level classifier. The classifier outputs the predicted label for each instance, and the instance-level loss is calculated using the MarginLoss function. .

[0119]

[0120]

[0121]

[0122] In the formula, m and s are hyperparameters, representing the classification margin and scaling factor, respectively, and bt is the size of the set of all salient groups DLG in a training batch.

[0123] Step S44. Group-level training

[0124] Group-level training is performed on potential group instances, and the specific steps are as follows:

[0125] Step S411. Group Feature Aggregation: Embed features for latent group instances within each WSI package. The gating attention module input to the student model ( The attention score for each instance is calculated, and then the feature aggregation module (FAM) weights and aggregates the features of all instances within a potential group into group-level features. The attention score serves as the weight, highlighting the contribution of important instances within the group. The calculation process for group-level features is as follows:

[0126]

[0127]

[0128] Step S412. Pseudo-tag assignment: Assign the tag to the WSI package. Assign corresponding group-level features ;

[0129] Step S413. Classification Training: Group-level features Input group classifier Output group-level prediction results Group-level loss is calculated using the MarginLoss function. ;

[0130]

[0131]

[0132] In the formula, b is the training batch size.

[0133] Step S414. Joint Loss Calculation: Using MarginLoss as the joint loss function, the total loss is calculated. By updating all trainable parameters of the student model (including the parameters of the feature embedding module, gating attention module, feature aggregation module, instance-level classifier, and group-level classifier) ​​through the backpropagation algorithm, the collaborative optimization of instance-level training and group-level training is achieved, thereby improving the overall performance of the model.

[0134] Step S5. Reasoning and Prediction Stage:

[0135] Step S51. Data preprocessing: Use the OTSU thresholding method to remove the background and retain the tissue area; cut into non-overlapping instances of a preset uniform size; complete pixel normalization;

[0136] Step S52. The teacher model outputs salient group instance features and latent group instance features: The preprocessed instances are input into the teacher model, and feature vectors are extracted and compressed into embedded features in sequence; the instance attention score is calculated, and salient group and latent group features are selected according to the preset hyperparameter ratio, and trivial groups are removed;

[0137] Step S53. For salient group instances, input the instance-level classifier of the student model. The prediction result for each instance is obtained, and the average of the prediction results for all instances is calculated as the instance-level prediction result. The calculation process is as follows:

[0138]

[0139] Step S54. For potential group instances, generate group-level features through the gating attention module and feature aggregation module of the student model, and input them into the group-level classifier Pᵝ to obtain the group-level prediction results. ;

[0140] Step S55. The fusion inference module averages the instance-level prediction result S_I and the group-level prediction result S_G to obtain the final WSI classification result. :

[0141]

[0142] Experimental setup and performance metrics

[0143] Preprocessing

[0144] The experiment first transformed the WSI into a series of image patches. Specifically, OTSU was used as the image segmentation technique to perform thresholding segmentation on the grayscale WSI, and a segmentation mask was output. Based on the mask, the experiment preserved the tissue regions and removed the background regions. Within the tissue regions, the patches were cut in a non-overlapping order of 224×224 pixels to avoid redundancy and overlap of information.

[0145] experiment

[0146] The training and test set split ratio was set to 4:1. During the experiments, we repeated the process 20 times (i.e., four rounds of 5-fold cross-validation) to ensure the stability of the results. All models were trained and evaluated on the same training and test sets, and the model with the best performance (highest accuracy) on the training set was selected for performance evaluation on the test set. All experimental results reported in this paper are the average of these 20 experiments to ensure the reliability and reproducibility of the results.

[0147] train

[0148] We employed the Ranger optimizer, which optimizes the model parameters with a constant learning rate and constant weight decay of 1e-1 and 1e-5, respectively. In the feature embedding module, and They are 1024 and 512 respectively. In GAM, The value is 128. In feature grouping, the hyperparameter ratio can be adjusted according to the dataset; this embodiment preferably uses 128. : In FAM, The value was set to 0.3. In the loss function, m was set to 0.4 and s was set to 64. The entire training process consisted of 200 epochs, with each mini-batch size set to 64. All experiments were conducted using a GTX3080Ti.

[0149] index

[0150] To comprehensively evaluate the performance of teacher and student models, accuracy, F1 score, and area under the ROC curve (AUC) were used as evaluation metrics. For multi-class classification, AUC was calculated using macro-averaging.

[0151] The proposed PFS-MIL was experimentally evaluated on three lung cancer datasets—TCGA-lung, CPCAC, and HCH-lung—and compared with several state-of-the-art MIL methods. These comparisons included not only classic nonparametric pooling strategies such as max pooling and mean pooling, but also advanced attention-based models, specifically ABMIL, CLAM-SB, CLAM-MB, TransMIL, DTFD-MIL, ILRA-MI, ACMIL, and DGR-MIL. The experimental comparison results are shown in Table 2.

[0152] Comparison of PFS-MIL and other models in WSI classification on three datasets:

[0153]

[0154] In the table, the bold black text indicates the maximum value of the column.

[0155] As shown in the table above, the proposed PFS-MIL achieved the highest accuracy scores across the three datasets. Compared to traditional pooling techniques (i.e., max pooling and mean pooling), attention-based methods have significant advantages, primarily because the attention mechanism allows the model to focus more on features that have a significant impact on classification decisions, rather than simply averaging all features or selecting the maximum value. Specifically, on the TCGA-LUNG dataset, ABMIL's accuracy is 3.41 higher than max pooling. However, attention-based models often rely on specific instances during prediction, which can lead to potential bias issues. To address this, DTFD-MIL introduces the concept of pseudo-bags, while ACMIL, DGRMIL, and CLAM-MB propose multi-branch attention mechanisms. These methods effectively mitigate this problem and demonstrate superior performance. For example, on the CPTAC dataset, DTFD-MIL's accuracy is 3.25 higher than ABMIL, while ACMIL's accuracy is 3.08 higher than ABMIL. Our proposed method, PFS-MIL, differs from the aforementioned methods in that we introduce the concept of grouping, achieving excellent performance. As shown in Table 2, our method achieves the highest accuracy and AUC on both the TCGA-LUNG and CPTAC datasets. On the HCH-LUNG dataset, although our method's AUC is slightly lower than DGRMIL's 99.65 (a difference of only 0.18), it still demonstrates robust performance. These experimental results strongly confirm the effectiveness and practicality of the pathological feature stratification proposed in this paper.

[0156] Experiments have shown that this fusion strategy can fully combine the accurate discrimination information of salient groups with the supplementary information of potential groups, and has higher classification accuracy and stability compared with single instance-level prediction or group-level prediction.

[0157] This invention effectively solves many problems existing in weakly supervised pathological image classification methods by organically combining a teacher-student network architecture with a pathological feature hierarchical strategy. It improves the model's feature utilization efficiency, noise resistance, instance-level classification ability, and generalization performance, providing a more reliable and efficient technical solution for computer-aided pathological diagnosis.

[0158] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A pathological image classification method based on teacher-student networks and feature hierarchies, characterized in that... This includes the following steps: Step S1. Data Preprocessing: Process the input whole-slice pathological images The tissue region is extracted and segmented into multiple image patches of equal size to obtain an instance set. ; Step S2. Build and train the teacher model (TeacherModel); Step S21. Model Construction: Construct the model consisting of the feature extraction module ( ), feature embedding module ( Gated attention module ( ), Feature aggregation module ( The teacher model consists of a classifier and a package-level classifier; Step S22. Feature Extraction (FETM): Extract image features using a pre-trained deep convolutional neural network or feature extraction model, and map the image features into a first-dimensional feature vector through an adaptive average pooling layer (AdaptiveAvgPool2d). ; Step S23. Feature Embedding (FEBM): The feature vector is input into the feature embedding module, and sequentially passes through a one-dimensional batch normalization (BatchNorm1d), a fully connected layer (Linear), a ReLU activation function, and an L2 normalization layer to compress the feature dimension to 512 dimensions, thus obtaining the embedded features. ; Step S24. Attention Scoring: Input the embedded features into the gating attention module Attention scores for each instance are calculated using fully connected layers and Sigmoid / Tanh activation functions. ; Step S25. Feature Aggregation and Classification: The attention score of a bag is... and embedded features enter Obtain packet-level feature representation ,Will Input classifier Conduct training; Step S26. Model Training: Repeat the above steps on the entire training set until the model converges or reaches the preset stopping condition, and save the best performing model as the teacher model. Step S3. Pathological Feature Stratification: Step S31. Feature Extraction: Extract image features using a pre-trained deep convolutional neural network or feature extraction model; map the image features into a first-dimensional feature vector using an adaptive average pooling layer (AdaptiveAvgPool2d). ; Step S32. Feature Embedding: Embed all instance features in a WSI Input the feature embedding module of the teacher model trained in step S2 To obtain embedded features ; Step S33. Scoring and Ranking: All embedded features in a WSI are scored and ranked. Input the gated attention module of the teacher model trained in step S2 Obtain the attention score for each instance. Attention scores are calculated for all instances within each bag (WSI) of the training set and sorted from highest to lowest. Step S34. Dynamic Grouping: Set the hyperparameter ratio (Based on a preset ratio threshold), the instances within the package are divided into a significant group (SG), a latent group (LG), and a trivial group based on the sorting results. Step S35. Iterative processing: Divide all instances within each WSI in the training set into three groups: saliency group, latent group, and trivial group; Step S4. Build and train the student model (StudentModel): The student model employs a two-layer collaborative training strategy, consisting of "instance-level training" and "group-level training". Student model construction: The student model consists of a feature embedding module ( Gated attention module ( ), Feature aggregation module ( ), instance-level classifier Group-level classifier composition; feature Embedding: embedding salient group instance features from a WSI and instance features of potential groups Input student model The saliency group embedding features were obtained respectively. and trivial group feature embedding ; Step S5: Inference and Prediction: The trained model is used to achieve accurate classification of whole-slice pathological images (WSI). Standardization and two-dimensional prediction fusion are used to ensure the reliability of the classification results. The specific optimized steps are as follows: Step S51. Data preprocessing: Use the OTSU thresholding method to remove the background and retain the tissue area; cut into non-overlapping instances of a preset uniform size; complete pixel normalization; Step S52. Teacher Model Feature Filtering Output: Input the preprocessed instance into the teacher model, and extract the feature vectors and compress them into embedded features in sequence; Calculate instance attention scores, filter salient and latent group features according to preset hyperparameter ratios, and remove trivial groups; Step S53. Student model group-level prediction S_G: The latent group features are optimized by embedding to generate group-level features, which are input into the group-level classifier and output as group-level prediction results S_G; Step S54. Student model instance-level prediction S_I: Input salient group instances into the instance-level classifier to obtain the prediction result of a single instance; calculate the mean of all instance prediction results and output the instance-level prediction result S_I to reduce the random error of a single instance; Step S55. Fusion Prediction (S_G+I): Using an equal-weighted averaging strategy, the group-level prediction result S_G and the instance-level prediction result S_I are averaged to obtain the final WSI classification probability distribution. The category with the highest probability is the final classification result.

2. The pathological image classification method based on teacher-student network and feature hierarchical structure according to claim 1, characterized in that, The data preprocessing in step S1 includes image segmentation and slicing, obtaining whole-slice pathological images (WSI), removing the background using the OTSU thresholding method, and segmenting the region containing tissue into non-overlapping small patches (i.e., instances), with the size set to a preset uniform size.

3. The pathological image classification method based on teacher-student network and feature hierarchical structure according to claim 1, characterized in that, The method for dividing the three groups in step S34 further includes: Significant Group (SG): The top group with the highest scores Instances, containing the most discriminative pathological patterns, yield the following instance feature set: ; Latent Group (LG): Top scorers in the middle Instances contain information that is not typical but has value for data mining; the resulting set of instance features is... ; Trivial Group: The group with the lowest remaining score. Instances (usually background or normal tissue) are discarded during subsequent training of student models.

4. The pathological image classification method based on teacher-student network and feature hierarchical structure according to claim 1, characterized in that, Step S4, instance-level training (for the salient group), includes the following steps: Step S401. Pseudo-tag assignment: Assign the package (WSI) tags Assign a significant group to each instance within this package ; Step S402. Merging: Merge all salient group instances from different packages in a training batch to form a large instance library (LG). Step S403. Shuffling: Randomly shuffle the merged instance library to generate "unordered salient groups (DLG)"; Step S404. Classification Training: Input the instances from the DLG into the instance-level classifier. Calculate instance-level loss.

5. The pathological image classification method based on teacher-student network and feature hierarchical structure according to claim 1, characterized in that, Step S4, group-level training (for potential groups), includes the following steps: Step S411. Group Feature Aggregation: Instances within potential groups Using the attention module of the student model ( ) and aggregation module ( Generate group-level features ; Step S412. Assigning pseudo-tags: Assigning tags to the package Assign this group-level feature; Step S413. Classification Training: Input group-level features into the group-level classifier Calculate group-level loss; Step S414. Joint Loss Calculation: Use MarginLoss to optimize both instance-level classification and group-level classification tasks simultaneously. The total loss is the sum of the two parts.

6. A pathological image classification system based on teacher-student networks and feature hierarchies, employing the pathological image classification method based on teacher-student networks and feature hierarchies as described in any one of claims 1-5, characterized in that, include: Data preprocessing module: used for WSI background removal, tissue region segmentation, and generation of preset uniform size instances; Feature extraction module: Based on the pre-trained ResNet50 model (truncated to Stage 3), extracts 1024-dimensional feature vectors of instances; Teacher model training module: includes feature extraction module ( ), feature embedding module ( Gated attention module ( ), Feature aggregation module ( A package-level classifier is used to generate attention scores and complete the training of the teacher model; The pathological feature stratification module is used to receive the attention score and embedding features output by the teacher model, divide them into significant, latent, and trivial groups according to the proportion, and remove the trivial group. Student model training module: includes feature embedding module ( Gated attention module ( ), Feature aggregation module ( ), instance-level classifier Group-level classifier Two-layer collaborative optimization is achieved through instance-level training and group-level training of "merging and shuffling"; Fusion Inference Module: Used to calculate the instance-level prediction mean and group-level prediction results, and output the final WSI classification result after fusion.

7. The pathological image weakly supervised processing system according to claim 6, characterized in that, The feature extraction module uses an adaptive average pooling layer. Map instance features to feature vectors of fixed dimensions.

8. The pathological image weakly supervised processing system according to claim 6, characterized in that, The total loss of the student model training module is the sum of the instance-level loss and the group-level loss, and is jointly optimized using MarginLoss.

Images