Pathological image classification system and method based on Mama and hybrid expert model

Abstract

The invention discloses a pathological image classification system and method based on Mama and a hybrid expert model, the pathological image classification system proposes a challenging task for WSI classification by combining the advantages of the Mama and the hybrid expert model, and develops a novel Mama-based hierarchical network called Mama-WSI. In order to promote feature representation while ensuring efficiency, the Mama-WSI introduces bidirectional Mama blocks in a pseudo packet and across the pseudo packet to explore correlation between instances in the pseudo packet and global pseudo packet representation. Meanwhile, a dynamic memory library is provided to store historical information related to label prediction, and a cross-pseudo packet fusion module is designed to fuse effective information from a dynamically updated dictionary module so as to enhance current pseudo packet representation and global modeling. Furthermore, the expert hybrid module is capable of adaptively coordinating global and local packet representations to achieve robust packet label prediction.

Description

Technical Field

[0001] The present invention belongs to the technical field of pathological images, and relates to a pathological image classification method, and specifically to a pathological image classification system and method based on Mamba and mixed expert models (Whole Slide Image Classification Meets Mamba and Mixture of Experts, referred to as Mamba-WSI). Background Art

[0002] Histopathology slides contain rich biological structural information, which is crucial for the accuracy of cancer diagnosis. However, diagnosing tissue sections using manual optical microscopy is a time-consuming and tedious task for pathologists. With the advancement of digital technology, tissue sections can be scanned as whole slide images (WSI), which provides new possibilities for traditional pathological diagnosis. In particular, deep learning methods have shown advantages over manual feature extraction methods in parsing tissue and cell structure information in WSI. Nevertheless, WSI has gigapixel-level high resolution and complex microstructure, making it extremely time-consuming and challenging to collect pixel-level annotations.

[0003] To address these challenges, the Multi-Instance Learning (MIL) method has attracted widespread attention because it only requires package-level labels as weak constraints. This method regards each WSI as a "package", cuts the WSI into a series of instances, and forms a package-level feature representation for classifier training by mining the correlation between local and global instances. Although the MIL paradigm combined with Convolutional Neural Networks (CNN) has made significant progress in performance, it still has limitations in learning the global perspective and capturing long-range dependencies. Different from CNN-based methods, some studies have successfully captured the spatial and morphological information of large-scale instances by introducing the self-attention mechanism. In particular, Transformer shows advantages in enhancing global context communication, but its computational cost is often high when applied to high-resolution WSI. For example, a WSI with a resolution of 106,496×51,968 pixels is tiled into 55,852 instances of size 256×256 at 20x magnification, each of which needs to be encoded as a 1024-dimensional feature vector, which makes the computational cost of the popular Transformer-based TransMIL model as high as 152.03 GFLOPs.

[0004] In order to deal with the quadratic complexity of Transformer when processing WSI, researchers have proposed a variety of methods to reduce the amount of computation. A common strategy is to apply a random mask to the input instance features, that is, randomly discard some input features to shorten the length of long sequences. However, this method inevitably leads to partial information loss, which may affect the spatial pattern capture of the tumor area. Another method is to reduce the original complexity from O(N) to g by grouping N instances into g pseudo-packets. 2 ) is reduced to Where N is the number of instances. Despite the improvements of these methods, the quadratic complexity problem of Transformers has not been fundamentally solved, which still limits its application in high-resolution WSI.

[0005] In order to solve the computational bottleneck problem of Transformer when modeling long sequences, Structured State Space Sequence (S4) has attracted widespread attention due to its excellent performance under linear complexity. However, since the Mamba model was originally designed for unidirectional one-dimensional continuous data, it shows limitations in processing spatial hierarchical structures and local context analysis. To overcome this weakness, Mamba-based variants have achieved excellent results by designing multiple scanning modes to balance local and global representations. For example, some researchers combine Graph Neural Networks (GNNs) with Mamba to capture local and global spatial relationships between instances. However, trying to pack all instance sequences into GNNs or Mamba models often incurs learning and computational burdens, making it difficult for the model to efficiently process large-scale WSI data. Summary of the invention

[0006] In order to solve the problem that existing methods produce unsatisfactory results and unbearable computational overhead due to the local connectivity in CNN and the quadratic complexity of Transformer, the present invention provides a pathological image classification system and method based on Mamba and Mixed Expert Model (Mixed Expert Model, MoE for short). The present invention proposes to combine the advantages of Mamba and expert mixed models for the challenging task of WSI classification, and develops a novel Mamba-based hierarchical network, called Mamba-WSI. In order to promote feature representation while ensuring efficiency, Mamba-WSI introduces bidirectional Mamba blocks within and across pseudo-packages to explore the correlation between instances within pseudo-packages and global pseudo-package representation. At the same time, the present invention proposes a dynamic memory library to store historical information related to label prediction, and designs a cross-pseudo-package fusion module to fuse effective information from the dynamically updated dictionary module to enhance the current pseudo-package representation and global modeling. In addition, an expert mixed module can adaptively coordinate global and local package representations to achieve robust package label prediction.

[0007] The objective of the present invention is achieved through the following technical solutions:

[0008] A pathological image classification system based on Mamba and hybrid expert model includes a full-slice tissue pathological image extraction instance module, an instance feature extraction module, a cross pseudo-bag fusion module (Cross Pseudo-bag Fusion Module, CPFM), a local and global bag representation module (Global and Local Bag Representation Module, GLBRM), and an expert mixture module (Experts Mixture Module, EMM), wherein:

[0009] The full-slice histopathology image extraction instance module is responsible for histopathology image instance extraction. The specific steps of histopathology image instance extraction are as follows:

[0010] (1) Given a full-slice dataset of pathological images It contains N WSIX i and its corresponding label Y i ;

[0011] (2) The whole-slice image WSI is read into the computer at a downsampled resolution and converted from RGB to HSV color space;

[0012] (3) The non-background tissue area of ​​WSI is calculated based on the image saturation and channel threshold after median blur, and the morphological closing operation is combined to fill small gaps and holes as the final tissue area;

[0013] (4) According to the calculated tissue area, for each WSI X i Crop B at specified magnification i Image patch instances Get instance collection and store, where H and W represent the height and width of an instance, b∈[1,B i ] represents the b-th cropped instance;

[0014] The instance feature extraction module is responsible for encoding the instance features of the pathological image. The specific steps of encoding the instance features are as follows:

[0015] (1) Use a deep neural network pre-trained on ImageNet or a large pathology image dataset as an encoder to transform each instance x i,b Embedded into a D-dimensional feature vector In this case, we get a set of feature vectors

[0016] (2) To facilitate the exploration of spatial and structural relationships between instances, we define a feature vector set Introducing learnable positional encodings

[0017] (3) In order to promote the intrinsic correlation of instance features in Mamba-WSI learning and improve feature expression, the larger packet is randomly sampled into T smaller pseudo packets. where M is the number of instances in the pseudo-bag, Represents the pseudo-packet instance features generated by sampling at time t;

[0018] The cross-pseudo-packet fusion module CPFM includes a bidirectional Mamba block (Bidirectional Mamba Block, referred to as BiMambaB), a cross-attention module (Cross-Attention Module, referred to as CAM), and a dynamic memory bank (Dynamic Memory Bank, referred to as DMB). At time t, a set of instance features are sampled Capture using BiMambaB The semantic and spatial features between the instances in the t-th time are aggregated into a pseudo-bag representation at the current time t. middle; Integrate historical knowledge from dynamic memory bank DMB using criss-cross attention module CPFM Enter an enhanced pseudo-package representation Among them, Represents DMB at time t-1, enhanced pseudo-packet representation Passed to a Multi-Layer Perceptron (MLP) to infer the enhanced pseudo-packet representation The corresponding pseudo label probability Also use package tag Y i Act as pseudo-label to supervise The relevant process is expressed as:

[0019]

[0020]

[0021]

[0022] The dynamic memory library DMB stores K discriminative pseudo-packet feature expressions To facilitate the expression of pseudo-packages at the current time t Enter an enhanced pseudo-package representation in, represents the K pseudo-packet features retained, is the corresponding category probability, According to the category confidence Ranking from high to low; for DMB at time t-1, and As input, by comparing Compared with the one stored before time t-1 Rearrange Introduction As a category-related hint and a Softmax function, the stored feature representation Transformed into refined feature representation Then, Quantize to the optimal feature representation Generate weights Then the weight do The coefficient of Perform an addition operation to aggregate the scattered discriminant information into a most informative feature representation set The above operation is expressed as:

[0023]

[0024]

[0025] Then, the splicing representation The cross-attention operation is used to represent the pseudo bag at the current time t. Provide more tag-related features;

[0026] The local and global packet representation module GLBRM includes two parallel sub-branches: a bidirectional Mamba block (Bidirectional Mamba Block, BiMambaB for short) and a convolution and maximum pooling block (ConvBlock for short). BiMambaB is used to refine the global packet representation between pseudo packets. ConvBlock is used to extract the salient features in the pseudo-bag, called local bag representation. The above process is encapsulated as:

[0027]

[0028]

[0029] Predict the global bag representation through the linear layer and Softmax function respectively The corresponding category probability Local package representation The corresponding category probability and pass and Refined into a more fine-grained package representation The process is expressed as:

[0030]

[0031] The expert hybrid module EMM includes two expert modules and a routing network. Expert module 1 is composed of 1×1 convolution layer, Relu and maximum pooling, and expert module 2 is composed of 3×3 convolution layer, Relu and maximum pooling. The weights of expert module 1 and expert module 2 are generated by the routing network (Routing) composed of linear layer and Softmax activation function. and Used to aggregate the output results of expert module 1 and expert module 2 and The weighted features are then input into a multi-layer perceptron for final packet label prediction. The process is described as follows:

[0032]

[0033] A pathological image classification method based on Mamba and mixed expert model includes the following steps:

[0034] Step 1: Given a full-slice dataset of pathological images It contains N WSI Xi and its corresponding label Y i , X i is regarded as the i-th “packet”; X i Cut out B i Instances;

[0035] Step 2: For each instance x i,b Embedded into a D-dimensional feature vector In this case, we get a set of feature vectors

[0036] Step 3: To facilitate the exploration of spatial and structural relationships between instances, we create a feature vector set Introducing learnable positional encodings

[0037] Step 4: Mamba-WSI randomly samples the larger packet into T smaller pseudo packets

[0038] Step 5: Capture using BiMambaB The semantic and spatial structural features between the instances in the current pseudo-package are aggregated into the feature expression of the current pseudo-package

[0039] Step 6: To enhance The discrimination information in Use CPFM to integrate historical features related to predicted labels in DMB and use MLP to predict labels

[0040] Step 7: Introduce package tag Y i Act as pseudo-label to supervise In order to Extract more feature representations related to labels;

[0041] Step 8: Use BiMambaB module to extract pseudo-packet feature expression The global package representation in ConvBlock preserves pseudo-packet feature representation The local package representation of

[0042] Step 9: and Score by corresponding category and Refined into more fine-grained package representations And use EMM to achieve more robust packet label prediction

[0043] Compared with the prior art, the present invention has the following advantages:

[0044] 1. This paper proposes a Mamba-WSI framework for hierarchical fusion and representation learning that combines group sampling with Mamba. This framework combines Mamba with MoE to efficiently process WSI tasks, aiming to address the challenge of modeling long-distance instance dependencies in WSI tasks and improve classification performance. Unlike traditional self-attention methods that explore the global relationship of all pixels or packages, this Mamba-based hierarchical fusion scheme decomposes the learning task into two subtasks: pseudo-package internal fusion and cross-pseudo-package external representation learning.

[0045] 2. In the task of internal fusion of pseudo-packets, the present invention designs a bidirectional Mamba block to capture the instance correlation within the pseudo-packet, thereby generating a richer representation of the current pseudo-packet. In order to solve the sparsity and redundancy problems of pseudo-packet representation and enhance the discriminative power of the current pseudo-packet, the present invention introduces DMB to store historical information of category dependency, and enhances the pseudo-packet representation by integrating information-rich features into the representation of the current pseudo-packet through CPFM to provide more refined label dependency features.

[0046] 3. In the pseudo-packet representation learning task, the present invention combines the representations of all pseudo-packets and extracts the local and global package features between pseudo-packets. Specifically, the present invention designs a BiMambaB module to integrate all pseudo-packet representations, extract the global package representation between pseudo-packets, and designs a convolution and maximum pooling block ConvBlock to retain the significant information in the pseudo-packet as a local package representation, avoiding mutual interference and homogenization problems between pseudo-packet representations. By coordinating the advantages of global package representation and local package representation of BiMambaB and ConvBlock, the present invention achieves a more fine-grained package-level representation, greatly improving the classification performance of the WSI task.

[0047] 4. In order to further enhance the applicability and robustness of the model, the present invention introduces EMM to alleviate the deviation in packet representation and thus improve the accuracy of packet label prediction.

[0048] 5. Experimental results show that Mamba-WSI surpasses the most recent state-of-the-art ACMIL method on the ESCA dataset by 3.8% and 4.1% in accuracy and AUC respectively. These results show that Mamba-WSI has obvious advantages in performance and efficiency and can better cope with the challenges in WSI tasks. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 This is a flow chart of pathological image classification based on Mamba and hybrid expert model;

[0050] Figure 2 It is a bidirectional Mamba module;

[0051] Figure 3 It is a convolution module;

[0052] Figure 4 It is an expert hybrid module. DETAILED DESCRIPTION

[0053] The technical solution of the present invention is further described below in conjunction with the accompanying drawings, but is not limited thereto. Any modification or equivalent replacement of the technical solution of the present invention without departing from the spirit and scope of the technical solution of the present invention should be included in the protection scope of the present invention.

[0054] The present invention provides a pathological image classification system based on Mamba and hybrid expert model, such as Figure 1 As shown, it includes a full-slice tissue pathology image extraction instance module, an instance feature extraction module, a CPFM module, a GLBRM module, and an EMM module, wherein:

[0055] 1. Full-slice tissue pathology image extraction example module:

[0056] The full-slice histopathology image extraction instance module is responsible for histopathology image instance extraction. The specific steps of histopathology image instance extraction are as follows:

[0057] (1) Given a full-slice dataset of pathological images It contains N WSIX i and its corresponding label Y i , X i Considered as the i-th "package";

[0058] (2) The whole-slice image WSI is read into the computer at a downsampled resolution and converted from RGB to HSV color space;

[0059] (3) The non-background tissue area of ​​WSI is calculated based on the image saturation and channel threshold after median blur, and the morphological closing operation is combined to fill small gaps and holes as the final tissue area;

[0060] (4) According to the calculated tissue area, for each WSI X i Crop B at specified magnification i Image patch instances Get instance collection and store, where H and W represent the height and width of an instance, b∈[1,B i ] represents the b-th cropped instance;

[0061] 2. Instance Feature Extraction Module

[0062] The instance feature extraction module is responsible for encoding the instance features of the pathological image. The specific steps of encoding the instance features are as follows:

[0063] (1) Use a deep neural network pre-trained on ImageNet or a large pathology image dataset as an encoder to transform each instance x i,b Embedded into a D-dimensional feature vector In this case, we get a set of feature vectors

[0064] (2) To facilitate the exploration of spatial and structural relationships between instances, we define a feature vector set Introducing learnable positional encodings

[0065] (3) In order to promote the intrinsic correlation of instance features in Mamba-WSI learning and improve feature expression, the larger packet is randomly sampled into T smaller pseudo packets. where M is the number of instances in the pseudo-bag, Represents the pseudo packet instance features generated by sampling at time t.

[0066] 3. CPFM module

[0067] The purpose of the CPFM is to effectively and efficiently model instance relationships to promote pseudo-package instance fusion and package feature representation, including: BiMambaB, CAM, DMB. The specific operation steps are:

[0068] A set of instance features sampled at time t Operation as an example, first design a Figure 2 The BiMambaB module shown in the figure consists of a two-layer bidirectional Mamba structure and a linear attention module (LAM) to fully explore the complex relationships between instances in the pseudo-bag and aggregate them into the pseudo-bag representation at the current time t. middle.

[0069] In order to deal with the current pseudo package representation The sparsity of disease features in Equipped with a cross-attention module CAM to integrate historical knowledge from the dynamic memory bank DMB Enter an enhanced pseudo-package representation Among them, Represents DMB at time t-1.

[0070] Finally, the enhanced pseudo-package representation It is passed to a Multi-Layer Perceptron (MLP) constructed by a linear layer and a Softmax activation function to infer the enhanced pseudo-bag representation The corresponding pseudo label probability The relevant process is expressed as:

[0071]

[0072]

[0073] The Cross-Attention operation is decomposed into:

[0074]

[0075] Q, K, and V are the query, key, and value in the cross-attention mechanism respectively. Represents DMB at time t-1. Represents a pseudo-packet feature expression at time t, Represents the instance feature set formed by sampling M instance features at time t.

[0076] DMB aims to preserve K discriminative pseudo-bag feature expressions To facilitate the expression of pseudo-packages at the current time t Enter an enhanced pseudo-package representation in, represents the K pseudo-packet features retained, is the corresponding category probability. According to the category confidence Ranking from high to low. For DMB at time t-1, and As input, by comparing Compared with the one stored before time t-1 To rearrange Although increasing K may be beneficial, the redundancy and computational burden will reduce the model performance. As a category-related hint and a Softmax function, the stored feature representation Transformed into refined feature representation Then, Quantize to the optimal feature representation Generate weights Then the weight do The coefficient of Perform an addition operation to aggregate the scattered discriminant information into a most informative feature representation set The above operation is expressed as:

[0077]

[0078]

[0079] Then, the splicing representation Through the cross-attention operation Provides more tag-related features.

[0080] 4. GLBRM module

[0081] The purpose of the CCRM module is to alleviate the pseudo-packet representation bias and promote a more comprehensive packet feature expression. It includes two parallel sub-branches: Bidirectional Mamba Blocks (BiMambaB) and Convolutional and MaxPooling Block (ConvBlock). The structure of ConvBlock is as follows: Figure 3 BiMambaB is used to refine the global package representation between pseudo packages. In order to avoid the homogenization of pseudo-packet representation, ConvBlock is used to retain and integrate the optimal features in the pseudo-packet, which is called local pack representation.

[0082] The above process is encapsulated as:

[0083]

[0084]

[0085] In order to more accurately represent the package and improve the accuracy of package label prediction, the present invention predicts the global package representation through the linear layer and the Softmax function respectively. The corresponding category probability Local package representation The corresponding category probability and pass and Refined into a more fine-grained package representation The process is expressed as:

[0086]

[0087] 5. EMM module

[0088] like Figure 4As shown, the expert hybrid module EMM includes two expert modules and a routing network. Expert module 1 is composed of 1×1 convolution layer, Relu and maximum pooling, and expert module 2 is composed of 3×3 convolution layer, Relu and maximum pooling. The weights of expert module 1 and expert module 2 are generated by the routing network (Routing) composed of linear layer and Softmax activation function. and Used to aggregate the output results of expert module 1 and expert module 2 and The weighted features are then input into a multi-layer perceptron for final packet label prediction. The process is described as follows:

[0089]

[0090]

[0091] 6. Loss Function

[0092] In order to achieve efficient global instance modeling and high-quality feature representation to better infer packet labels, the present invention explores composite cross entropy loss functions, which involve: The commonality bag of the commonality and feature representation module represents the corresponding label probability loss The feature bag represents the corresponding label probability loss Label probability loss corresponding to the expert mixture module The formulas for these loss functions are as follows:

[0093]

[0094]

[0095] The final total loss is defined as:

[0096]

[0097] Among them, λ is used to adjust the pseudo-label probability loss In total loss The proportion of Y is adjusted from 0 to 0.2 by integrating the duration and cosine curve. i is the true label corresponding to the WSI, T is the number of pseudo packets generated, and N is the number of WSIs contained in the dataset.

[0098] The present invention also provides a pathological image classification method based on Mamba and a hybrid expert model, the method comprising the following steps:

[0099] Step 1: Given a full-slice dataset of pathological images It contains N WSI X i and its corresponding label Y i , X i is regarded as the i-th “packet”; X i Cut out B i instances.

[0100] Step 2: For each instance x i,b Embedded into a D-dimensional feature vector In this case, we get a set of feature vectors

[0101] Step 3: To facilitate the exploration of spatial and structural relationships between instances, Introducing learnable positional encodings

[0102] Step 4: Mamba-WSI randomly samples the larger packet into T smaller pseudo packets in Represents the pseudo packet instance features sampled at time t M is the number of instances in the pseudo-bag.

[0103] Step 5: Capture using BiMambaB The semantic and spatial structural features between the instances in the current pseudo-package are aggregated into the feature expression of the current pseudo-package

[0104] Step 6: To enhance The discrimination information in Use CPFM to integrate historical features related to predicted labels in DMB and use MLP to predict labels

[0105] Step 7: Introduce package tag Y i Act as pseudo-label to supervise In order to Extract more label-related feature representations.

[0106] Step 8: To facilitate global package representation, extract pseudo-packages using the BiMambaB module Inter-feature expression, summarized as a global package representation ConvBlock preserves pseudo packets The feature expression between the two is summarized as a local package representation

[0107] Step 9: and By the corresponding category probability and Refined into more fine-grained package representations And use EMM to achieve more robust packet label prediction

[0108] In order to verify the performance of the Mamba-WSI of the present invention, extensive experimental comparisons were conducted on the ESCA dataset with a variety of typical WSI analysis MIL methods. These methods include: MeanPooling, ABMIL, SetTransformer, DeepAttnMIL, TransMIL, MuRCL, MHIM, SSMMIL, MaxPooling, RNNMIL, DSMIL, CLAM, DTFD, DGMIL, IAT, BCL, CIMIL, SSMMIL and MambaMIL. The evaluation indicators include area under the curve (AUC), accuracy, precision and recall, where the threshold is set to 0.5. The Mamba-WSI model was trained under 200 iterations, using the AdaMax optimizer with an initial learning rate of 0.0004. The quantitative results are shown in Table 1. The Mamba-WSI of the present invention has achieved competitive performance in all evaluation indicators. In particular, it surpasses the most advanced ACMIL method in accuracy and AUC by 3.8% and 4.1% respectively.

[0109] Table 1: Quantitative comparison of the proposed Mamba-WSI results with representative multi-instance learning methods on the ESCA dataset

[0110]

[0111]

Claims

1. A pathological image classification system based on Mamba and hybrid expert model, characterized by The pathological image classification system includes a full-slice tissue pathological image extraction instance module, an instance feature extraction module, a cross-pseudo-packet fusion module CPFM, a local and global packet representation module GLBRM, and an expert mixture module EMM, wherein: The full-slice histopathology image extraction instance module is responsible for histopathology image instance extraction; The instance feature extraction module is responsible for encoding instance features of tissue pathology images; The cross-pseudo-packet fusion module CPFM includes a bidirectional Mamba module BiMambaB, a cross attention module CAM, and a dynamic memory bank DMB. At time t, a set of instance features are sampled. Capture using BiMambaB The semantic and spatial features between the instances in the t-th time are aggregated into a pseudo-bag representation at the current time t. middle; Integrating historical knowledge from DMB using CPFM Enter an enhanced pseudo-package representation Among them, Represents DMB at time t-1, enhanced pseudo-packet representation is passed into a multi-layer perceptron (MLP) to infer the enhanced pseudo-bag representation The corresponding pseudo label probability Also use package tag Y i Act as pseudo-label to supervise The relevant process is expressed as: The dynamic memory library DMB stores K discriminative pseudo-packet feature expressions To facilitate the expression of pseudo-packages at the current time t Enter an enhanced pseudo-package representation in, represents the K pseudo-packet features retained, is the corresponding category probability, According to the category confidence Ranking from high to low; for DMB at time t-1, and As input, by comparing Compared with the one stored before time t-1 Rearrange Introduction As a category-related hint and a Softmax function, the stored feature representation Transformed into refined feature representation Then, Quantize to the optimal feature representation Generate weights Then the weight do The coefficient of Perform an addition operation to aggregate the scattered discriminant information into a most informative feature representation set The above operation is expressed as: Then, the splicing representation The cross-attention operation is used to represent the pseudo bag at the current time t. Provide more tag-related features; The local and global bag representation module GLBRM includes two parallel sub-branches: a bidirectional Mamba module BiMambaB and a convolution and maximum pooling module ConvBlock. BiMambaB is used to refine the global bag representation between pseudo-bags. ConvBlock is used to extract the salient features in the pseudo-bag, called local bag representation. The above process is encapsulated as: Predict the global bag representation through the linear layer and Softmax function respectively The corresponding category probability Local Package Representation The corresponding category probability and pass and Refined into a more fine-grained package representation The process is expressed as: The expert hybrid module EMM includes two expert modules and a routing network. The expert module 1 is composed of a 1×1 convolution layer, Relu and maximum pooling, and the expert module 2 is composed of a 3×3 convolution layer, Relu and maximum pooling. The weights of the expert module 1 and the expert module 2 are generated by the routing network composed of a linear layer and a Softmax activation function. and Used to aggregate the output results of expert module 1 and expert module 2 and The weighted features are then input into a multi-layer perceptron for final packet label prediction. The process is described as follows:

2. The pathological image classification system based on Mamba and hybrid expert model according to claim 1 is characterized in that The specific steps of extracting the tissue pathology image instance are as follows: (1) Given a full-slice dataset of pathological images It contains N WSIX i and its corresponding label Y i ; (2) The whole-slice image WSI is read into the computer at a downsampled resolution and converted from RGB to HSV color space; (3) The non-background tissue area of ​​WSI is calculated based on the image saturation and channel threshold after median blur, and the morphological closing operation is combined to fill small gaps and holes as the final tissue area; (4) According to the calculated tissue area, for each WSI X i Crop B at specified magnification i Image patch instances Get instance collection and store, where H and W represent the height and width of an instance, b∈[1,B i ] represents the b-th cropped instance.

3. The pathological image classification system based on Mamba and hybrid expert model according to claim 1 is characterized in that The specific steps of the instance feature encoding are as follows: (1) Use a deep neural network pre-trained on ImageNet or a large pathology image dataset as an encoder to transform each instance x i,b Embedded into a D-dimensional feature vector In this case, we get a set of feature vectors (2) To facilitate the exploration of spatial and structural relationships between instances, we define a feature vector set Introducing learnable positional encodings (3) In order to promote the intrinsic correlation of instance features in Mamba-WSI learning and improve feature expression, the larger packet is randomly sampled into T smaller pseudo packets. where M is the number of instances in the pseudo-bag, Represents the pseudo packet instance features generated by sampling at time t.

4. A method for pathological image classification based on Mamba and hybrid expert model using the pathological image classification system according to any one of claims 1 to 3, characterized in that The method comprises the following steps: Step 1: Given a full-slice dataset of pathological images It contains N WSI X i and its corresponding label Y i , X i is regarded as the ith "packet"; X i Cut out B i Instances; Step 2: For each instance x i,b Embedded into a D-dimensional feature vector In this case, we get a set of feature vectors Step 3: To facilitate the exploration of spatial and structural relationships between instances, we create a feature vector set Introducing learnable positional encodings Step 4: Mamba-WSI randomly samples the larger packet into T smaller pseudo packets Step 5: Capture using BiMambaB The semantic and spatial structural features between the instances in the current pseudo-package are aggregated into the feature expression of the current pseudo-package Step 6: To enhance The discrimination information in Use CPFM to integrate historical features related to predicted labels in DMB and use MLP to predict labels Step 7: Introduce package tag Y i Act as pseudo-label to supervise In order to Extract more feature representations related to labels; Step 8: Use BiMambaB module to extract pseudo-packet feature expression The global package representation in ConvBlock preserves pseudo-packet feature representation The local package representation of Step 9: and Score by corresponding category and Refined into more fine-grained package representations And use EMM to achieve more robust packet label prediction

Images