A pathological whole slide image classification method

Abstract

The application discloses a pathological whole slice image classification method, relates to the technical field of medical image processing, and aims to solve the problems of high labeling cost, low long sequence modeling efficiency, poor semantic heterogeneity adaptation and insufficient lesion focus in WSI weak supervision classification in the prior art, S1, pathological whole slice image training samples are acquired and preprocessed to obtain an initial instance feature sequence; S2, the initial instance feature sequence is input into an SR-Mamba long sequence modeling module to output a fusion feature sequence; S3, the fusion feature sequence is input into a pseudo packet dynamic updating module to generate an enhanced data set; S4, the results of steps S2 and S3 are input into a cross-guided mask attention module to output a final packet-level feature; S5, WSI classification and model parameter optimization are completed based on the packet-level feature, and finally, pathological whole slice image classification results are obtained. Through integration of SR-Mamba long sequence modeling, pseudo packet dynamic updating and cross-guided mask attention modeling, the accuracy and efficiency of WSI weak supervision classification are improved.

Description

Technical Field

[0001] This invention relates to the field of medical image processing technology, specifically to a method for classifying whole pathological slide images. Background Technology

[0002] Whole-slice images (WSIs), with their ultra-high resolution (a single WSI often contains billions of pixels) and complete tissue microstructure information, have become a core carrier for tumor diagnosis, classification, and prognostic assessment. However, the inherent characteristics of WSIs bring the following problems to the application of deep learning models: significant bottlenecks in data scale and storage / computation, high annotation costs, significant problems of sparse lesion distribution and class imbalance, and complex semantic heterogeneity. To address the problem of insufficient annotation, multi-instance learning (MIL), as a weakly supervised paradigm, has been widely used in WSI analysis. In breast cancer metastasis detection and lung cancer classification tasks, the MIL framework has certain advantages, but existing MIL methods still have three major bottlenecks.

[0003] (1) Balancing efficiency and performance in long sequence modeling: A single WSI typically contains tens of thousands of patches, forming an extremely long sequence of instances. The traditional Transformer architecture models global dependencies through self-attention, but the computational complexity increases quadratically with the sequence length. When processing a WSI with 8,000 patches, the computational load reaches approximately 64 million interactions, leading to a significant increase in training time and a tendency to overfit due to excessive parameters. Although some studies have attempted to reduce complexity through sparse attention and local attention, this results in the loss of global contextual information and an inability to capture weakly correlated lesions across regions (such as scattered micrometastases). Structured state-space models (SSMs) such as S4 achieve linear complexity, but are essentially linear time-invariant systems that cannot dynamically adjust information propagation based on the input content, thus limiting their adaptability to semantic differences in pathological images.

[0004] (2) Static package construction cannot adapt to dynamic semantic evolution: Existing MIL methods mostly cut WSI into fixed-structure packages in the preprocessing stage, and the instance grouping within the package remains unchanged during training. Static packages are difficult to capture this evolution, resulting in insufficient modeling of instance semantic associations and limiting the model's generalization ability.

[0005] (3) Insufficient ability to focus on key lesions under weak supervision: MIL relies only on package-level labels, making it difficult for the model to accurately locate sparse lesion instances and susceptible to background noise interference. Existing methods such as ABMIL assign weights to instances through an attention mechanism, but attention weights are easily affected by redundant background patches, resulting in the dilution of key lesion weights; the CLAM series introduces class-related attention, but relies on predefined class structures and cannot adapt to the semantic differences of different WSIs; although DSMIL improves recall through dual branches, its ability to integrate global semantics is insufficient, and there are still missed detection problems when the lesion region is extremely sparse. In addition, most MIL methods ignore the spatial dependencies between instances—the distribution of lesions in pathological tissues has spatial correlation, and ignoring this correlation will cause the model to fail to capture the global lesion distribution pattern, further reducing classification accuracy.

[0006] In summary, existing technologies (such as CN116152566A) face problems such as high annotation costs, low efficiency in long sequence modeling, poor adaptation to semantic heterogeneity, and insufficient lesion focus in WSI weakly supervised classification. Summary of the Invention

[0007] This invention addresses the problems of high annotation costs, low efficiency of long sequence modeling, poor semantic heterogeneity adaptation, and insufficient lesion focusing in existing technologies for weakly supervised WSI classification. It proposes a pathological whole-slice image classification method that improves the accuracy and efficiency of weakly supervised WSI classification by integrating SR-Mamba long sequence modeling, pseudo-packet dynamic updating, and cross-guided masked attention (CGMA) modeling.

[0008] To achieve the above objectives, the present invention adopts the following technical solution: a method for classifying pathological whole-section images, comprising the following steps: S1, Obtain training samples of whole pathological slide images and perform preprocessing to obtain the initial instance feature sequence; S2. Input the initial instance feature sequence into the SR-Mamba long sequence modeling module and output the fused feature sequence; S3. Input the fused feature sequence into the pseudo-packet dynamic update module to generate the enhanced dataset; S4. Input the results of steps S2 and S3 into the cross-guided mask attention module and output the final packet-level features. S5. Based on package-level features, complete WSI classification and optimize model parameters to finally obtain the classification results of pathological whole slide images.

[0009] This technical solution proposes a pathological whole-slice image classification method. By integrating the SR-Mamba module for long sequence modeling, the pseudo-packet dynamic update mechanism, and the global vector-guided cross-mask attention module, it improves the accuracy and efficiency of WSI weakly supervised classification, meets the dual requirements of accuracy and efficiency in clinical pathological diagnosis, and promotes the practical application of digital pathology in clinical diagnosis.

[0010] Furthermore, step S1 includes: S11: Collect whole pathological slide images and their corresponding package-level labels, and cut the images into non-overlapping image blocks of the same size. S12, based on the information entropy of each non-overlapping image block, remove non-overlapping image blocks that are to be classified as background regions; S13, extract the instance-level feature vector of each valid non-overlapping image patch according to the feature extractor, and perform linear projection processing to form the initial instance feature sequence.

[0011] The technical solution in step S1 can transform the original WSI into a low-dimensional feature sequence suitable for model processing, while removing invalid background information.

[0012] Furthermore, step S2 also includes: constructing an SR-Mamba long sequence modeling module, which includes an original branch unit for modeling the sequential dependencies of feature sequences, a rearranged branch unit for enhancing the correlation modeling of weakly correlated instances across regions, and a gated fusion unit for adaptively fusing the output features of the original branch and the rearranged branch.

[0013] In this technical solution, the input of the original order branch unit is the initial instance feature sequence, and the output is the original order feature representation; the input of the rearrange branch unit is the initial instance feature sequence, and the output is the rearranged version after restoring the original order; and the output of the gated fusion unit is the fused feature sequence.

[0014] Furthermore, step S2 includes: S21, Project, activate and model the initial instance feature sequence, and output the original sequence feature representation; S22, the initial instance feature sequence is input into the rearranged branch unit for rearrangement, and the rearranged feature representation is output; S23, input the original feature representation and the rearranged feature representation into the gated fusion unit, fuse the two branch features and output the fused feature sequence through residual connection.

[0015] In this technical solution, the initial instance feature sequence is input into the original order branch unit, and the original order feature representation is output. The original order feature representation and the rearranged feature representation are input into the gated fusion unit, and the two branch features are fused through an adaptive gating mechanism, and the fused feature sequence is output through residual connection.

[0016] Furthermore, step S3 also includes: constructing a pseudo-packet dynamic update module, which includes a prototype clustering unit for initially dividing phenotypic clusters, a phenotypic fine-tuning unit for optimizing phenotypic cluster centers, a hierarchical sampling unit for generating semantically consistent pseudo-packets, a pseudo-packet mixup unit for generating augmented datasets, and a periodic update unit for dynamically updating the pseudo-packet structure.

[0017] In this technical solution, the pseudo-packet dynamic update module is only enabled during the model training phase and disabled during the testing phase to ensure inference consistency; the input of the prototype clustering unit is the fused feature sequence output from step S2, and the output is the preliminary phenotypic cluster; the input of the phenotypic fine-tuning unit is the preliminary phenotypic cluster, and the output is the optimized phenotypic cluster; the input of the hierarchical sampling unit is the optimized phenotypic cluster, and the output is the initial pseudo-packet set; the input of the pseudo-packet mixup is the initial pseudo-packet set corresponding to the two WSIs, and the output is the mixed pseudo-packet and the corresponding label; the input of the periodic update unit is the training round signal, and the output is the updated pseudo-packet set.

[0018] Furthermore, step S3 includes: S31, divide the fused feature sequence into several semantically related phenotypic clusters; S32 uses K-means clustering to iteratively optimize the center and instance assignment of phenotypic clusters and outputs the optimized phenotypic clusters; S33, randomly divide each phenotypic cluster and sample it in equal amounts to obtain the initial pseudo-packet set; S34, through mask vector filtering and cross-sample combination, mixes pseudo-packets of different WSIs to generate augmented datasets; S35, repeat S31-S34, dynamically update the pseudo-packet structure to output the pseudo-packet set.

[0019] Furthermore, step S4 includes: constructing a cross-guided mask attention module; the cross-guided mask attention module includes an instance mask multi-head attention unit for modeling local instance interactions, a global vector mask multi-head attention unit for modeling global semantic information, a mask cross attention unit for filtering local features guided by global semantics, and a nonlinear mapping unit for optimizing feature representation.

[0020] In this technical solution, the input of the instance mask multi-head attention unit is the feature output in steps S2 and S3, and the output is the local interaction feature; the input of the global vector mask multi-head attention unit is the learnable global vector, and the output is the global interaction feature; the input of the mask cross attention unit is the global interaction feature and the local interaction feature, and the output is the cross-guided feature; the input of the nonlinear mapping unit is the cross-guided feature, and the output is the final packet-level feature.

[0021] Furthermore, step S4 includes: S41, based on the results of steps S2 and S3, apply masked multi-head self-attention and output local interaction features; S42 introduces a learnable global vector, which is then processed by masked multi-head attention to output global interaction features. S43, the global vector is the query, the instance feature is the key value, and the global semantically guided feature representation is calculated through masked cross-attention; S44, the cross-attention output is processed by the feedforward network and masked multi-head attention to output the final packet-level features.

[0022] Furthermore, step S5 also includes: constructing a classification prediction and optimization module; the classification prediction and optimization module includes a classifier unit and a loss optimization unit, the classifier unit is able to output WSI bag-level classification labels, and the loss optimization unit is applicable to end-to-end training of the HyG-MIL framework.

[0023] In this technical solution, the input of the classifier unit is the final bag-level feature output in step S4, and the output is the predicted probability and classification label; the input of the loss optimization unit is the predicted probability, the true label and the instance feature, and the output is the optimized model parameters.

[0024] Furthermore, step S5 includes: S51, the final bag-level features are input into the classifier unit, the prediction probability is calculated by the Softmax function, and the bag-level classification label of WSI is determined based on the prediction probability; S52, the model framework is trained using a joint loss function, which includes classification cross-entropy loss, triplet loss, and diversity loss.

[0025] The present invention can bring about the following technical effects: This invention provides an efficient solution for weakly supervised classification of whole pathological slide images by innovatively combining Mamba long sequence modeling and cross-attention guidance mechanism, and introducing pseudo-packet augmented data and its dynamic update mechanism. It can be widely used in clinical pathology diagnosis assistance and tumor screening scenarios, and has important practical application value. Attached Figure Description

[0026] Figure 1 This is a schematic diagram of the HyG-MIL framework of the present invention.

[0027] Figure 2 This is a schematic diagram of the Mamba long sequence modeling module of the present invention.

[0028] Figure 3 This is a schematic diagram of the sequence reorder of the present invention. Detailed Implementation

[0029] Example 1

[0030] To address the problems of high annotation costs, low efficiency in long sequence modeling, poor semantic heterogeneity adaptation, and insufficient lesion focusing in existing technologies for weakly supervised WSI classification, this embodiment proposes a pathological whole-slice image classification method, referencing... Figure 1 , Figure 2 as well as Figure 3 It mainly includes the following steps.

[0031] Step S1: Obtain training samples of whole pathological slide images and perform preprocessing to obtain initial instance feature sequences.

[0032] Step S1 mainly involves transforming the original WSI into a low-dimensional feature sequence suitable for model processing, while removing invalid background information. Specifically, it includes the following sub-steps.

[0033] Step S11: Collect the whole pathological slide images and their corresponding package-level tags, and cut the images into several non-overlapping image blocks of the same size.

[0034] In this embodiment, whole pathological slice images (WSI) and corresponding package-level labels (positive / negative or tumor subtype) are acquired. The input whole pathological slice images are then cut into non-overlapping image patches of the same size, such as 224×224 pixels.

[0035] Step S12: Based on the information entropy of each non-overlapping image block, remove the non-overlapping image blocks that are to be classified as background regions.

[0036] In this embodiment, step S12 uses information entropy to filter and remove background regions. Specifically, the information entropy of each non-overlapping image patch is calculated. If the information entropy is less than 5, the patch is discarded, and valid instances containing histological information are retained.

[0037] Step S13: Extract instance-level feature vectors for each valid non-overlapping image patch using the feature extractor, and perform linear projection processing to form an initial instance feature sequence.

[0038] In this embodiment, a pre-trained ResNet-50 network is selected as the feature extractor to extract the instance-level feature vector of each valid patch. The extracted instance-level feature vectors are then processed by linear projection to form an initial instance feature sequence X, which consists of L feature vectors (L is the number of valid patches), and each feature vector has a dimension of D.

[0039] Step S2: Input the initial instance feature sequence into the constructed SR-Mamba long sequence modeling module to output the fused feature sequence.

[0040] Before step S2, there is also a step of constructing the SR-Mamba long sequence modeling module. The constructed SR-Mamba long sequence modeling module mainly includes the original order branching unit, the rearranged branching unit, and the gated fusion unit.

[0041] The original branch unit is used to model the sequential dependency of the feature sequence. Its input is the initial instance feature sequence and its output is the original-order feature representation. The rearrangement branch unit is used to enhance the correlation modeling of weakly related instances across regions. Its input is the initial instance feature sequence and its output is the rearranged version after restoring the original order. The gated fusion unit is used to adaptively fuse the output features of the original branch and the rearranged branch. Its output is the fused feature sequence.

[0042] After completing the construction of the SR-Mamba long sequence modeling module, the process of step S2 is carried out, which specifically includes the following sub-steps.

[0043] Step S21: Project, activate, and model the initial instance feature sequence to output the original sequence feature representation.

[0044] For step S21, the initial instance feature sequence obtained in step S1 is input into the original order branch unit mentioned above. Specifically, after linear projection, one-dimensional convolution and SiLU activation are performed on the instance feature sequence X, the state space SSM model is input to model the order dependency relationship, and the original order feature representation Y' is output.

[0045] Step S22: The initial instance feature sequence is input into the rearranged branch unit for rearrangement, and the rearranged feature representation is output.

[0046] For step S22, the initial instance feature sequence obtained in step S1 is input into the rearrangement branch unit. Specifically, the feature sequence length L is reshaped into a two-dimensional representation of N×R (R is the segment length, N is the number of segments), and the columns are rearranged to make weakly correlated instances across regions more proximate in the sequence, resulting in a new sequence Xr. The input is processed using the same procedure as the original branch unit (linear projection, one-dimensional convolution, SiLU activation, SSM modeling), and the rearranged feature representation Yr is output. Yr is then reversed to restore the original order, resulting in Yr'.

[0047] Step S23: Input the original feature representation and the rearranged feature representation into the gated fusion unit, fuse the two branch features, and output the fused feature sequence through residual connection.

[0048] For step S23, the original feature representation Y and the rearranged feature representation Yr' are fed into the input gated fusion unit. Specifically, based on the standardized feature X' of feature X, a gate coefficient S is generated through linear projection and SiLU activation; S is applied to Y' and Yr' respectively through element-wise multiplication to obtain the enhanced feature sequence representations X" and Xr", which are then added together; finally, after linear transformation, a residual connection is performed with the original feature sequence X to output the fused feature. .

[0049] Step S3: Input the fused feature sequence into the constructed pseudo-packet dynamic update module to generate the enhanced dataset.

[0050] Before step S3, there is also a process of constructing a pseudo-packet dynamic update module, which includes a prototype clustering unit, a phenotypic fine-tuning unit, a hierarchical sampling unit, a pseudo-packet mixup unit, and a periodic update unit.

[0051] The pseudo-packet dynamic update module is only enabled during the model training phase and disabled during the testing phase to ensure inference consistency.

[0052] The prototype clustering unit is responsible for initially dividing phenotypic clusters. Its input is the fused feature sequence output from step S2, and its output is the initial phenotypic clusters. The phenotypic fine-tuning unit is responsible for optimizing the phenotypic cluster centers. Its input is the initial phenotypic clusters, and its output is the optimized phenotypic clusters. The hierarchical sampling unit is responsible for generating semantically consistent pseudo-packets. Its input is the optimized phenotypic clusters, and its output is the initial set of pseudo-packets. The pseudo-packet mixing unit is responsible for generating augmented datasets. Its input is the initial set of pseudo-packets corresponding to the two WSIs, and its output is the mixed pseudo-packets and their corresponding labels. The periodic update unit is responsible for dynamically updating the pseudo-packet structure. Its input is the training round signal, and its output is the updated set of pseudo-packets.

[0053] After completing the construction of the pseudo-package dynamic update module, the process of step S3 is carried out, which specifically includes the following sub-steps.

[0054] Step S31: Divide the fused feature sequence into several semantically related phenotypic clusters.

[0055] For step S31, the fused feature sequence output from step S2 is input into the prototype clustering unit. Specifically, the initial prototype vector of the instance features is calculated by averaging the feature sequence. Based on the cosine similarity between the instance and the prototype, the instance is initially divided into l phenotypic clusters (l=6-8, specifically set according to the WSI organizational complexity) to ensure semantic relevance within the cluster.

[0056] Step S32: Use K-means clustering to iteratively optimize the center and instance assignment of phenotypic clusters, and output the optimized phenotypic clusters.

[0057] For step S32, the initial phenotypic clusters are input into the phenotypic fine-tuning unit. Specifically, the K-means clustering method is used to iteratively update the phenotypic cluster centers. Instances are then reassigned to the nearest clusters based on the updated cluster centers, and the optimized phenotypic clusters are output.

[0058] Step S33: Randomly divide each phenotypic cluster and sample it in equal amounts to obtain the initial pseudo-packet set.

[0059] For step S33, the optimized phenotypic clusters are input into the hierarchical sampling unit. Specifically, each phenotypic cluster is randomly divided into n subsets, and one subset instance is randomly selected from each phenotypic cluster to form a pseudo-packet, resulting in a total of n pseudo-packets.

[0060] Step S34: By filtering through mask vectors and combining across samples, pseudo-packets of different WSIs are mixed to generate an augmented dataset.

[0061] For step S34, it sets the pseudo-packet sets corresponding to the two WSIs (A and B). and The input is fed into the pseudo-packet Mixup unit; specifically, an n-dimensional mask vector is defined. (Elements can be 1 or 0), through the mask vector Filter out pseudo-packets of A ( The position where the element is 1 is retained. The pseudo-packet at the corresponding location; Remove positions where the element is 0. (corresponding to the pseudo-packet); and simultaneously via 1- Filtering pseudo-packets of B (1- The position where the element is 1 is retained. The pseudo-packet at the corresponding position; 1- Remove positions where the element is 0. (corresponding to pseudo-packages), and finally the filtered ones Counterfeit packages and Pseudo-packet combinations form hybrid packets. ; after the above filtering Pseudo-packets directly constitute hybrid packets. And provide the corresponding package-level tags. The new fake package label is calculated by weighting the original WSI label (e.g., if fake packages A account for 2 / 3 and fake packages B account for 1 / 3, then the label is 2 / 3 × label A + 1 / 3 × label B).

[0062] Step S35: Repeat S31-S34 to dynamically update the pseudo-packet structure to output the pseudo-packet set.

[0063] For step S35, the pseudo-packet structure is dynamically updated through the periodic update unit. Specifically, every E (E=20) training rounds, the periodic update signal is triggered, and steps S31-S34 are repeated to regenerate the pseudo-packet set and augmented dataset to avoid feature solidification caused by static pseudo-packets. This allows the pseudo-packets to adapt to the dynamic evolution of feature distribution during training, balancing feature distribution adaptability and training stability.

[0064] Step S4: Input the results of steps S2 and S3 into the constructed cross-guided mask attention module and output the final packet-level features.

[0065] Before step S4, there is a process of constructing a cross-guided mask attention module, which includes an instance mask multi-head attention unit, a global vector mask multi-head attention unit, a mask cross attention unit, and a non-linear mapping unit.

[0066] The instance mask multi-head attention unit models local instance interactions, taking the features output from steps S2 and S3 as input and outputting local interaction features. The global vector mask multi-head attention unit models global semantic information, taking a learnable global vector as input and outputting global interaction features. The mask cross-attention unit performs global semantic-guided local feature filtering, taking global and local interaction features as input and outputting cross-guided features. The nonlinear mapping unit optimizes feature representation, taking cross-guided features as input and outputting final packet-level features.

[0067] After completing the construction of the cross-guided mask attention module, the process of step S4 is carried out, which mainly includes the following sub-steps.

[0068] Step S41: Based on the results of steps S2 and S3, apply masked multi-head self-attention and output local interaction features.

[0069] For step S41, in the instance mask multi-head attention unit, the fused feature sequence output from step S2 and the mixed pseudo-packet feature output from step S3 (denoted as...) are combined. Input, and obtain the query through linear transformation. ,key ,value When calculating the self-similarity of queries and keys, a mask matrix is ​​applied to suppress interactions between irrelevant instances. Attention weights are then calculated and applied to... Above, after masked multi-head attention, local interaction features are output. .

[0070] Step S42: Introduce a learnable global vector, and output global interaction features through masked multi-head attention.

[0071] For step S42, m learnable global vectors are introduced into the global vector mask multi-head attention unit. Similar to step S41, the query is obtained through a linear transformation. ,key ,value Apply a mask matrix Suppress interactions between irrelevant vectors, compute attention weights and apply them to... Above, after masked multi-head attention, the global interaction features are output. .

[0072] Step S43: The global vector is the query, and the instance features are the key values. The global semantically guided feature representation is calculated through masked cross-attention.

[0073] For step S43, in the masked cross-attention unit, the global vector multi-head attention output result is... After passing through the feedforward network FFN, the sequence features are used as the query. After passing through the feedforward network FFN, the key-value pairs are used to compute the globally semantically guided feature representation CrossAttn through masked cross-attention.

[0074] Step S44: After the cross-attention output is processed by the feedforward network and masked multi-head attention, the final packet-level features are output.

[0075] For step S44, in the nonlinear mapping unit, the CrossAttn output is processed by the feedforward network (FFN) and masked multi-head attention, and then the final packet-level feature Z is output through the linear layer.

[0076] Step S5: Based on package-level features, complete WSI classification and optimize model parameters to finally obtain the classification results of the whole pathological slide images.

[0077] Before step S5, there is a process of constructing a classification prediction and optimization module; the classification prediction and optimization module includes a classifier unit and a loss optimization unit.

[0078] The classifier unit outputs the bag-level classification label of WSI. Its input is the final bag-level feature output in step S4, and its output is the predicted probability and the classification label. The loss optimization unit trains the HyG-MIL (HybridGuided Multi-Instance Learning) framework end-to-end. Its input is the predicted probability, the true label and the instance feature, and its output is the optimized model parameters.

[0079] After completing the process of constructing the classification prediction and optimization module, proceed to step S5, which includes the following sub-steps.

[0080] Step S51: Input the final bag-level features into the classifier unit, calculate the prediction probability using the Softmax function, and determine the bag-level classification label of WSI based on the prediction probability.

[0081] The final bag-level feature Z output from step S4 is input into the classifier unit. Specifically, it is mapped to the label category dimension through a fully connected layer, and the prediction probability is calculated using the softmax function. ,according to Determine the package-level classification label (positive / negative or tumor subtype) of WSI.

[0082] Step S52: The model framework is trained using a joint loss function, which includes classification cross-entropy loss, triplet loss, and diversity loss.

[0083] The entire framework is trained through a loss optimization unit. Specifically, the parameters of the entire HyG-MIL framework are optimized through backpropagation based on the joint loss function. The joint loss function consists of a weighted sum of three parts: classification cross-entropy loss, triplet loss, and diversity loss.

[0084] The specific calculations for each loss function are as follows: (1) Classification cross-entropy loss: For each WSI sample, calculate the product of the true label and the logarithm of the predicted probability, then take the negative value of the product result for all samples and sum them up to finally obtain the classification cross-entropy loss value. In binary classification tasks, the true label is marked as 1 for positive labels and 0 for negative labels; in multi-class tasks (such as tumor subtype classification), the label is represented by one-hot encoding.

[0085] (2) Triplet Loss: For each global vector, calculate its cosine distance (denoted as positive distance) to the corresponding positive packet center vector and its cosine distance (denoted as negative distance) to the corresponding negative packet center vector. Subtract the negative distance from the positive distance and add a fixed boundary parameter (valued at 0.2). If the result is positive, it is directly used as the loss contribution of the global vector; if it is negative, it is set to 0. Finally, sum the loss contributions of all global vectors to obtain the Triplet loss value. This loss term, through the constraint of "bringing the positive distance closer and widening the negative distance," makes similar features more clustered and dissimilar features more dispersed.

[0086] (3) Diversity Loss: All learnable global vectors are combined into a global vector matrix. First, the product of this matrix and its transpose is calculated. Then, a very small numerical stability constant (with a value of 1e-6) is added to the product of the identity matrix (to avoid matrix singularity). Finally, the determinant of the final matrix is ​​calculated, and the determinant value is the diversity loss value. This loss encourages the global vectors to be orthogonal to enhance representativeness.

[0087] This invention provides an efficient solution for weakly supervised classification of whole pathological slide images by innovatively combining Mamba long sequence modeling and cross-attention guidance mechanism, and introducing pseudo-packet augmented data and its dynamic update mechanism. It can be widely used in clinical pathology diagnosis assistance and tumor screening scenarios, and has important practical application value. Example 2 Based on Example 1, this example provides further supplementary explanations, please refer to... Figure 1 The HyG-MIL (Hybrid Guided Multi-Instance Learning) framework in Example 1 mainly includes five stages: preprocessing and feature extraction, SR-Mamba long sequence modeling, pseudo-packet dynamic updating, cross-guided mask attention (CGMA) modeling, classification prediction and optimization. Each stage works together to solve the core bottleneck of existing methods.

[0088] The SR-Mamba module can efficiently capture long-range dependencies of ultra-long sequences while ensuring modeling efficiency, based on the linear complexity of state-space models and combined with sequence rearrangement strategies.

[0089] The pseudo-packet dynamic update module can adapt to the semantic heterogeneity and dynamic evolution characteristics of pathological images by periodically reconstructing pseudo-packets and enhancing mixup data. The cross-guided mask attention module can enhance the focus on key lesion areas and suppress background noise interference through global vector guidance and mask attention.

[0090] For preprocessing and feature extraction, it transforms the original WSI into a low-dimensional feature sequence suitable for model processing, while removing invalid background information.

[0091] During the WSI cutting process, the original WSI is cut into non-overlapping patches of 224×224 pixels, using a magnification of 20×, which preserves sufficient tissue cell details while avoiding a surge in computational load due to excessive magnification.

[0092] During the background removal process, the information entropy of each patch is calculated, and blank or noisy patches with an entropy value less than 5 are discarded, retaining only valid instances containing histological information to reduce invalid calculations.

[0093] During feature extraction, a ResNet-50 network pre-trained on ImageNet is used to extract a 2048-dimensional feature vector for each valid patch. After linear projection, an instance feature sequence X is formed, which consists of L feature vectors (L is the number of valid patches), and each feature vector has a dimension of D. For SR-Mamba long sequence modeling, please refer to step S2 in Example 1. For the dynamic update of pseudo-packets, please refer to step S3 in Example 1. The core difference between the dynamic update module and existing technologies lies in the fact that existing methods such as PseMix use a static pseudo-packet construction strategy, while this invention introduces a periodic update mechanism to dynamically adapt to the evolution of feature distribution, avoiding the feature solidification problem caused by static pseudo-packets. Ablation experiments verify that this module improves the classification accuracy by 0.77% / 0.39% compared to the baseline model. For cross-guided masked attention (CGMA) modeling, please refer to step S4 in Example 1. The core difference between the cross-guided masked attention module and existing technologies is that DGR-MIL only uses multi-head cross-attention, while this invention introduces a masking mechanism combined with multi-head attention. Through a triple mask design (instance interaction mask, global vector interaction mask, and cross-attention mask), it precisely controls the interaction range and strengthens the guidance of global semantics on local features, solving the problems of inaccurate lesion focusing and susceptibility to background interference in existing methods. Ablation experiments verify that this module achieves optimal performance when working in conjunction with other modules. For classification prediction and optimization, please refer to step S5 in Example 1.

[0094] The above scheme will be further illustrated with specific experiments.

[0095] 1. Experimental environment: Hardware: NVIDIA RTX 3090 GPU (24GB VRAM); Software: Python 3.10, PyTorch 2.0.1, CUDA 11.3.

[0096] 2. Experimental Dataset: (1) CAMELYON16 dataset: contains 400 breast cancer lymph node WSIs (270 training, 130 testing), with positive regions accounting for about 10%, and an average of 8000 20×patches per WSI; (2) TCGA-LUNG dataset: contains 1054 lung cancer WSIs (840 training images and 210 test images), with a positive region rate of over 80%, and an average of 5000 20×patch per WSI.

[0097] 3. Experimental Results: (1) The comparison results between this method and the existing mainstream MIL method are as follows (Table 1, Table 2): Table 1. Comparative experimental results on the CAMELYON16 dataset. ; Table 2 Comparative experimental results on the TCGA-LUNG dataset ; CAMELYON16 dataset: The ACC was 0.9419±0.013, F1 score was 0.9402±0.020, AUC was 0.9828±0.026, and training time was 153 minutes. Compared with DGR-MIL (average ACC = 0.9302), the performance was improved by 1.9%, and compared with MambaMIL (average ACC = 0.9296), the performance was improved by 1.2%, and the training efficiency was improved by 28% compared with MambaMIL. TCGA-LUNG dataset: The ACC was 0.9535±0.013, F1 score was 0.9535±0.020, AUC was 0.9814±0.026, and training time was 151 minutes. Compared with DGR-MIL (average ACC equals 0.9380), the performance was improved by 1.5%, compared with MambaMIL (average ACC equals 0.9339), the performance was improved by 1.9%, and the training efficiency was improved by 16.6% compared with DGR-MIL.

[0098] (2) Ablation experiment verification: Verify the effectiveness of each module on the two datasets (Table 3). Table 3 Performance verification of each module on the CAMELYON16 and TCGA-LUNG datasets. .

[0099] CAMELYON16 dataset: Baseline model DGR-MIL (no SR-Mamba, no pseudo-packet dynamic updates, no CGMA): The average ACC is 0.9225; Baseline + SR-Mamba: Mean ACC equals 0.9380 (+1.55%); Baseline + pseudo-packet dynamic update: average ACC equals 0.9302 (+0.77%); HyG-MIL (SR-Mamba + Pseudo-packet dynamic updates + CGMA): Average ACC equals 0.9419 (+1.94%). TCGA-LUNG dataset: Baseline model DGR-MIL (no SR-Mamba, no pseudo-packet dynamic updates, no CGMA): The average ACC is 0.9380; Baseline + SR-Mamba: Mean ACC equals 0.9424 (+0.44%); Baseline + pseudo-packet dynamic update: Average ACC equals 0.9419 (+0.39%); HyG-MIL (SR-Mamba + Pseudo-packet dynamic updates + CGMA): Average ACC equals 0.9535 (+1.5%).

[0100] The ablation experiment results show that all three core modules can improve model performance and produce a significant synergistic effect when combined. This proves that the module combination of the present invention is not a simple superposition, but forms a technical closed loop of "efficient modeling - dynamic adaptation - precise focusing", which synergistically solves multiple bottlenecks of existing methods.

[0101] The above technical solutions and experiments can achieve the following technical effects: 1. Efficiency Advantage: The SR-Mamba module, based on the linear complexity of SSM (O(L), where L is the sequence length), solves the bottleneck of quadratic complexity of Transformer, reducing training time by 53% compared to Trans-MIL (324 minutes); 2. Accuracy advantage: The pseudo-packet is dynamically updated to adapt to semantic heterogeneity, and CGMA enhances lesion focusing. On both types of datasets, it outperforms existing methods in ACC, F1, and AUC. 3. Robustness advantage: Through the combination of multiple modeling methods and joint loss constraints, the model performs stably in both sparse (CAMELYON16) and dense (TCGA-LUNG) positive regions, and has strong generalization ability.

[0102] 4. Practical advantages: Only WSI-level labels are required, without the need for instance-level fine annotation, reducing data annotation costs; the inference speed is fast during the testing phase, which can meet the efficiency requirements of clinical diagnosis. Example 3

[0103] This embodiment provides a specific implementation method for detecting lymph node metastasis in breast cancer; the detection of lymph node metastasis in breast cancer (CAMELYON16 dataset) includes the following process: 1. Data preparation: Download the CAMELYON16 dataset, obtain 270 training images and 130 test images according to the official partitioning scheme, and divide the training set into a sub-training set and a validation set in a 9:1 ratio for model training; 2. Preprocessing: The WSI with a magnification of 20× is cut into non-overlapping patches of 224×224 pixels; the information entropy of each patch is calculated, and valid patches with an entropy value of not less than 5 are retained; 2048-dimensional features of each valid patch are extracted by a pre-trained ResNet-50 network, and mapped to 512 dimensions by linear projection to form an instance feature sequence. 3. Pseudo-packet update configuration: The number of phenotypic clusters is 6, and the number of pseudo-packets is 6. During training, prototype clustering, phenotypic fine-tuning, hierarchical sampling and pseudo-packet mixup operations are re-executed every 20 rounds to update the pseudo-packet structure. 4. Model Training: Learning Rate AdamW optimizer, trained for 100 epochs, with triplet loss weights in the joint loss. The diversity loss weight is 0.1; 5. Prediction and Evaluation: The test set outputs packet-level classification results, and the ACC, F1, and AUC evaluation indicators are calculated. The average ACC of the test set classification is 0.9419, F1 is 0.9402, and AUC is 0.9828, which meets the sensitivity and specificity requirements for clinical detection of lymph node metastasis in breast cancer. Example 4

[0104] This embodiment provides a specific implementation method for lung cancer subtype classification; lung cancer subtype classification (TCGA-LUNG dataset) includes the following process: 1. Data preparation: The TCGA-LUNG dataset is divided into training / validation / test sets according to the proportion of patients (65% / 10% / 25%), including two categories: LUAD (lung adenocarcinoma) and LUSC (lung squamous cell carcinoma). 2. Preprocessing: Same as in Example 3, each WSI retains an average of 5000 valid patches; 3. Model Training: Learning Rate The remaining parameters are the same as in Example 3;

[0105] 4. Prediction and Evaluation: The test set outputs packet-level classification results, and calculates the evaluation indicators ACC, F1, and AUC. The average classification ACC of the test set is 0.9535, F1 is 0.9535, and AUC is 0.9814, achieving accurate differentiation of lung cancer subtypes.

Claims

1. A method for classifying whole-section pathological images, characterized in that, Includes the following steps: S1, Obtain training samples of whole pathological slide images and perform preprocessing to obtain the initial instance feature sequence; S2, input the initial instance feature sequence into the SR-Mamba long sequence modeling module, and output the fused feature sequence; S3, input the fused feature sequence into the pseudo-packet dynamic update module to generate the enhanced dataset; S4. Input the results of steps S2 and S3 into the cross-guided mask attention module and output the final packet-level features. S5, based on package-level features, completes WSI classification and model parameter optimization, and finally obtains the classification results of pathological whole slide images.

2. The method for classifying pathological whole-section images according to claim 1, characterized in that, Step S1 includes: S11: Collect whole pathological slide images and their corresponding package-level labels, and cut the images into non-overlapping image blocks of the same size. S12, based on the information entropy of each non-overlapping image block, remove non-overlapping image blocks that are to be classified as background regions; S13, extract the instance-level feature vector of each valid non-overlapping image patch according to the feature extractor, and perform linear projection processing to form the initial instance feature sequence.

3. A method for classifying pathological whole-section images according to claim 1 or 2, characterized in that, Step S2 further includes: constructing an SR-Mamba long sequence modeling module, which includes an original branch unit for modeling the sequential dependencies of feature sequences, a rearranged branch unit for enhancing the correlation modeling of weakly correlated instances across regions, and a gated fusion unit for adaptively fusing the output features of the original branch and the rearranged branch.

4. The method for classifying pathological whole-section images according to claim 3, characterized in that, Step S2 includes: S21, Project, activate and model the initial instance feature sequence, and output the original sequence feature representation; S22, the initial instance feature sequence is input into the rearranged branch unit for rearrangement, and the rearranged feature representation is output; S23, input the original feature representation and the rearranged feature representation into the gated fusion unit, fuse the two branch features and output the fused feature sequence through residual connection.

5. A method for classifying pathological whole-section images according to claim 1 or 2, characterized in that, Step S3 further includes: constructing a pseudo-packet dynamic update module, which includes a prototype clustering unit for initially dividing phenotypic clusters, a phenotypic fine-tuning unit for optimizing phenotypic cluster centers, a hierarchical sampling unit for generating semantically consistent pseudo-packets, a pseudo-packet mixup unit for generating augmented datasets, and a periodic update unit for dynamically updating the pseudo-packet structure.

6. The method for classifying pathological whole-section images according to claim 5, characterized in that, Step S3 includes: S31, divide the fused feature sequence into several semantically related phenotypic clusters; S32 uses K-means clustering to iteratively optimize the center and instance assignment of phenotypic clusters and outputs the optimized phenotypic clusters; S33, randomly divide each phenotypic cluster and sample it in equal amounts to obtain the initial pseudo-packet set; S34, through mask vector filtering and cross-sample combination, mixes pseudo-packets of different WSIs to generate augmented datasets; S35, repeat S31-S34, dynamically update the pseudo-packet structure to output the pseudo-packet set.

7. A method for classifying pathological whole-section images according to claim 1 or 2, characterized in that, Step S4 includes: constructing a cross-guided mask attention module; the cross-guided mask attention module includes an instance mask multi-head attention unit for modeling local instance interactions, a global vector mask multi-head attention unit for modeling global semantic information, a mask cross attention unit for filtering local features guided by global semantics, and a nonlinear mapping unit for optimizing feature representation.

8. A method for classifying pathological whole-section images according to claim 7, characterized in that, Step S4 includes: S41, based on the results of steps S2 and S3, apply masked multi-head self-attention and output local interaction features; S42 introduces a learnable global vector, which is then processed by masked multi-head attention to output global interaction features. S43, the global vector is the query, the instance feature is the key value, and the global semantically guided feature representation is calculated through masked cross-attention; S44, the cross-attention output is processed by the feedforward network and masked multi-head attention to output the final packet-level features.

9. A method for classifying pathological whole-section images according to claim 1 or 2, characterized in that, Step S5 further includes: constructing a classification prediction and optimization module; the classification prediction and optimization module includes a classifier unit and a loss optimization unit, the classifier unit can output WSI bag-level classification labels, and the loss optimization unit is applicable to end-to-end training of the HyG-MIL framework.

10. A method for classifying pathological whole-section images according to claim 9, characterized in that, Step S5 includes: S51, the final bag-level features are input into the classifier unit, the prediction probability is calculated by the Softmax function, and the bag-level classification label of WSI is determined based on the prediction probability; S52, the model framework is trained using a joint loss function, which includes classification cross-entropy loss, triplet loss, and diversity loss.

Images