A pathological image calculation method based on hierarchical adaptive low-rank approximate attention

Abstract

The application provides a pathological image calculation method based on layered adaptive low-rank approximate attention, comprising the following steps: step 1, feature extraction and multi-instance learning modeling; step 2, constructing a staining-invariant adaptive regularization module; step 3, designing a wavelet frequency domain and spatial domain dual-flow complementary pathological feature adaptive re-calibration module; step 4, realizing a region-level adaptive attention mechanism of pathological perception; step 5, realizing a global-level attention mechanism; step 6, fusing bidirectional spatial and channel interaction attention; step 7, a multi-level adaptive optimization strategy; and step 8, feature aggregation and classification prediction. The method guarantees global optimality, landmark point selection has mathematical optimal representativeness guarantee, is more reliable compared with a heuristic method, and optimally considers the mutual position relationship between landmark points, so that the selection of redundant landmark points which are too concentrated in a feature space is avoided.

Description

Technical Field

[0001] This invention belongs to the field of computer vision and medical image processing technology, specifically relating to a pathological image calculation method based on hierarchical adaptive low-rank approximate attention. Background Technology

[0002] In computational pathology, accurate analysis of panoramic pathological slide images (WSIs) is crucial for key tasks such as tumor diagnosis, subtype classification, and prognostic prediction. WSIs are obtained by digitizing tissue slides using high-resolution scanning equipment, typically containing billions of pixels of information. Due to the ultra-high resolution and scarcity of fine-grained annotations in WSIs, Multi-Instance Learning (MIL) has become a mainstream technical framework. MIL treats WSIs as bags and image patches as instances, achieving slide-level prediction through weakly supervised learning. This modeling approach aligns closely with clinical practice, as pathologists' diagnostic reports are usually based on entire slides rather than single fields of view. The core assumption of MIL is that if a bag is positive, there is at least one positive instance in the bag; if a bag is negative, all instances in the bag are negative.

[0003] Traditional MIL (Multi-Layered Clustering) methods generally employ a three-stage process: feature extraction, aggregation, and prediction. In the feature extraction stage, a pre-trained deep neural network is used to extract fixed-dimensional feature vectors for each image patch. In the aggregation stage, various strategies are used to integrate instance-level features into a bag-level representation. In the prediction stage, a classifier is used for final diagnosis. This field has evolved from early simple aggregation methods such as ABMIL to complex methods such as TransMIL. ABMIL proposes an instance-weighted aggregation strategy based on an attention mechanism, but assumes that instances are independent of each other. CLAM introduces clustering constraints to model the grouping patterns of instances. TransMIL significantly improves modeling capabilities by introducing a Transformer self-attention mechanism, but introduces secondary computational complexity.

[0004] Current methods face three key challenges in WSI analysis of large-scale panoramic pathological slide images. The first challenge is the insufficient feature adaptability. Existing methods generally rely on feature extractors pre-trained on natural image datasets such as ImageNet. While these pre-trained models can extract general visual features, they lack adaptability to specific pathological tasks. Natural images and pathological images differ significantly in visual characteristics; pathological images contain a large number of specific histological structures and cell morphology features. Even when using models pre-trained on pathological image datasets, static feature extractors struggle to adequately adapt to specific diagnostic needs due to the semantic gap between the pre-training task and the downstream diagnostic task. Research shows that even with high-quality pre-trained features, online feature learning can still bring significant performance improvements.

[0005] The second challenge is the excessively high computational complexity. Panoramic pathological slide images (WSIs), after image segmentation, typically contain thousands or even tens of thousands of image patch instances. If the traditional Transformer self-attention mechanism is used to model the relationships between instances, the computational complexity is O(I²), where I represents the number of instances. This quadratic complexity leads to unacceptable computational overhead and memory consumption when processing large-scale panoramic pathological slide images (WSIs) containing thousands of image patches. For large-scale panoramic pathological slide images (WSIs) containing thousands of image patches, this quadratic complexity becomes a major bottleneck in practical applications.

[0006] The third challenge is the lack of spatial and channel modeling. While linear attention methods from the field of natural language processing can effectively reduce computational complexity, these methods are designed for text sequences with natural sequential relationships and are difficult to directly adapt to pathological images with two-dimensional spatial distribution characteristics. In panoramic pathological slice images (WSI), the spatial location of image patches contains important pathological significance, and adjacent tissues often have similar morphological features. In addition, the channel dimension of features also contains rich semantic information. Different feature channels may encode different types of pathological information such as nuclear features, cytoplasmic features, and tissue structure features. However, existing methods still lack sufficient research on the integration of spatial mechanisms and channel attention, and usually adopt a simple concatenation approach, failing to achieve deep interaction between space and channels.

[0007] While the Nyström attention mechanism can reduce computational complexity from O(n²) to O(n) using low-rank matrix factorization, directly applying this efficient attention method to medical image analysis still faces key challenges. One is the landmark selection problem. Existing Nyström methods generally employ uniform random sampling to select landmarks, with some improvements using heuristic sampling based on importance scores. However, these methods have shortcomings. First, they lack mathematical guarantees of the overall representativeness of the landmark set; independent selection of each landmark does not consider redundancy; and the sampling strategy is disconnected from the geometric structure of the feature distribution, potentially leading to insufficient coverage of critical tumor regions and reduced model performance. Furthermore, heuristic learning methods lack theoretical optimality support. Second is the spatial adaptability problem. Lexical sequences in natural language processing have natural temporal relationships, while image patches in panoramic pathological slice images (WSI) exhibit a two-dimensional spatial distribution. Simply arranging image patches into a one-dimensional sequence disrupts the original spatial topology. Finally, there is the issue of spatial and channel synergy. Existing methods use a simple concatenation approach for spatial and channel attention, failing to fully leverage their synergistic enhancement effects. Therefore, it is of great significance to develop an efficient MIL method that can maintain linear computational complexity, achieve adaptive landmark selection for pathological perception, and effectively integrate multi-dimensional spatial and channel information. Summary of the Invention

[0008] Purpose of the Invention: This invention aims to address the core technical problems in existing computational pathology image analysis methods, such as insufficient feature adaptability, excessive computational complexity, lack of pathological perception in landmark selection, and missing spatial and modeling features. It provides an efficient panoramic pathological slide image WSI analysis method to meet the needs of cancer diagnosis, subtype classification, and prognostic prediction. Specifically, this invention provides a pathological image computation method based on hierarchical adaptive low-rank approximate attention. It can be applied to clinical decision support scenarios such as tumor diagnosis, cancer subtype classification, and patient prognostic prediction.

[0009] The method includes the following steps:

[0010] Step 1, Feature Extraction and Multi-Instance Learning Modeling: The input panoramic pathological slide image WholeSlideImage (WSI) is divided into image blocks to generate an instance set. A pre-trained feature extractor is used to extract features from each image block to obtain a fixed feature representation. The panoramic pathological slide image WSI analysis task is modeled as a multi-instance learning problem.

[0011] Step 2, Construct a staining-invariant adaptive regularization module: To address the color distribution variation problem caused by differences in hematoxylin and eosin (H&E) staining batches and scanning equipment, a staining-invariant adaptive regularization module is designed. This module generates multi-view instance features through learnable color perturbation and calculates the entropy-regularized Wasserstein distance between the empirical distributions of features before and after perturbation as an auxiliary loss for color consistency, which is then added to the training.

[0012] Step 3: To address the problem that the pre-trained feature extractor is insufficient in capturing high-frequency texture details of pathological images, a pathological feature adaptive recalibration module with complementary wavelet frequency and spatial domain streams is designed. The pathological feature adaptive recalibration module processes the spatial semantic stream and the frequency texture stream in parallel, calculates the entropy regularized Wasserstein distance between empirical distributions as a distribution alignment auxiliary loss, and finally outputs the task adaptive feature Z.

[0013] Step 4, implementation of the regional adaptive Nyström attention mechanism for pathological perception: The landmark selection problem is modeled as the Wasserstein centroid problem in optimal transport theory. By constructing the transport cost matrix for pathological perception, entropy regularization optimization and Sinkhorn iterative solution, the optimal set of landmarks that can represent the feature distribution of all instances with the minimum transport cost is found from a geometric perspective. End-to-end training is achieved through a differentiable soft aggregation mechanism.

[0014] Step 5, global Nyström attention mechanism implementation: After regional processing, a global Nyström attention mechanism is constructed to model cross-regional semantic relationships and long-range dependencies. The adaptive landmark selection strategy of pathological perception is also adopted, and global modeling is achieved through regional feature aggregation, cross-regional attention calculation and feature reprojection.

[0015] Step 6, Two-way spatial and channel interaction attention fusion: Design an iterative two-way interaction mechanism. The channel weight modulates the calculation of the query matrix Query and key matrix Key of spatial attention so that the model focuses on the diagnostic feature channels. The spatial weight guides the weighted pooling of channel attention so that high attention regions contribute more to the calculation of channel weights. And an interaction convergence regularization loss is introduced to ensure that the two-way interaction is fast and stable.

[0016] Step 7, Multi-level adaptive optimization strategy: Employ progressive landmark reduction, zero-fill alignment, and learnable temperature adjustment to improve the model's practicality and robustness;

[0017] Step 8, Feature Aggregation and Classification Prediction: The optimized instance features are integrated into a package-level representation through an aggregation function, a classifier is used for final prediction, and a multi-task loss function is used for end-to-end training.

[0018] Step 1 includes:

[0019] The input panoramic pathological slide image (WSI) undergoes preprocessing and image patch segmentation. Preprocessing includes background segmentation, tissue region detection, and color normalization. At a fixed magnification, the panoramic pathological slide image (WSI) is segmented into regular rectangular image patches, each covering a tissue region of a certain size. After preprocessing and segmentation, an instance set is obtained. Where I represents the total number of instances, This represents the i-th instance;

[0020] A pre-trained deep neural network feature extractor is used to extract features from each image patch. After each image patch passes through the feature extractor, a fixed-dimensional feature vector is obtained. The feature vectors of all instances are used to form the initial feature matrix. ;in Represents the space of real numbers; For feature dimensions;

[0021] The task of panoramic pathological slide image WSI analysis is modeled as a standard multi-instance learning problem, and the training dataset is defined as follows: , This represents the set of instances corresponding to the WSI of the i-th panoramic pathological slide image. Indicates the first Image patch instances, i-th packet-level label The label constraint for the i-th panoramic pathological slide image (WSI) is defined as the positive caching condition. If and only if there exists at least one instance j such that the instance's true label is true. Negative package conditions If and only if for all instances j, .

[0022] Step 2 includes: color-invariant adaptive regularization module on the initial feature matrix Global average pooling is used to obtain global feature statistics. Global feature statistics Input perturbation prediction network Generate HSV color space perturbation matrices for hue, saturation, and lightness. Among them, the perturbation prediction network For lightweight multilayer perceptron (MLP), 3rd order identity matrix Ensure that the initial perturbation is close to zero; for the first One original image block The conversion from the RGB color space to the HSV color space (hue, saturation, and lightness) yields... Applying a perturbation yields Then convert back to the RGB color space to obtain the i-th color-enhanced image patch. ; will the first One original image block With the i-th color-enhanced image patch The pre-trained feature extractor, with the freeze parameters input separately, yields the original feature matrix. With the perturbation characteristic matrix The feature extractor parameters are frozen but retained in the computation graph to support gradient-directed perturbation prediction networks. Return;

[0023] Additional calculation of the original feature matrix With the perturbation characteristic matrix Entropy-regularized 2-Wasserstein distance between empirical distributions as an auxiliary loss for color consistency The color entropy regularization coefficient To control the intensity of entropy regularization, and using the color loss balance coefficient as a hyperparameter. Add the total loss function; this module is only enabled during the training phase.

[0024] Step 3 includes: designing a wavelet frequency domain and spatial domain dual-stream complementary recalibration module before the Nyström attention layer. The recalibration module processes the spatial domain semantic stream and the frequency domain texture stream in parallel. The spatial domain semantic stream preserves the semantic structure and organization distribution of the original features, and obtains the spatial features through lightweight transformation processing. ,in For learnable weight matrix, These are learnable bias parameters; ReLU represents the linear rectified function; LayerNorm represents layer normalization;

[0025] Frequency domain texture flow first uses a zero-padding strategy to transform a one-dimensional sequence Reconstructed into a regular two-dimensional feature map , ,in Two-dimensional layout height, The width of the two-dimensional layout is represented; then, a two-level wavelet packet decomposition is performed independently on each channel to obtain a multi-resolution sub-band coefficient tree. Subband selection masks are then generated using a lightweight multilayer perceptron (MLP). , The coefficients are obtained by adaptively enhancing the high-frequency subbands to determine the subband number. ,in Activated for Sigmoid. This represents element-wise multiplication;

[0026] Finally, the enhanced coefficients were analyzed. Perform inverse wavelet packet transform, extract the real part and flatten it back to a one-dimensional sequence to obtain the frequency domain enhanced features. ;

[0027] Introducing learnable scalars during two-stream fusion The original features H are preserved through residual connections, and task-adaptive features are calculated. Simultaneously, the entropy-regularized 2-Wasserstein distance between the empirical distributions of the spatial and frequency domain flows is additionally calculated as an auxiliary loss for distribution alignment. ,in Represents the entropy regularization coefficient. These are preset hyperparameters greater than 0. And expressed as coefficients. Add to the total loss function.

[0028] Step 4 includes: designing a region-level Nyström attention strategy: first, performing spatial awareness preprocessing on feature sequences containing I instances. The layout is transformed into a regular two-dimensional space through adaptive fill operation, and the fill length is calculated according to the formula. , ,in and These represent the height and width of the padded 2D layout, respectively. A zero-padding strategy is used, which involves adding Δ zero vectors to the end of the feature sequence so that the total length of the feature sequence is exactly equal to... This allows it to be reshaped into a complete square layout; then, region division is performed, and the filled feature sequence is reshaped into a two-dimensional spatial layout. Then, it is uniformly divided into L×L non-overlapping rectangular regions, where L is the number of regions; the region division process is represented by the region division function. The i-th region Include Instances of spatially adjacent image patches For the first The number of instances in each region is full. The region division follows the principle of spatial proximity, with each region containing spatially adjacent image blocks. Then, for each region, an adaptive Nyström attention mechanism based on optimal transmission theory is independently applied to learn local features. The landmark selection problem is modeled as the Wasserstein centroid problem in optimal transmission theory, and the optimal set of landmarks that can represent the feature distribution of all instances with the minimum transmission cost is found from a geometric perspective.

[0029] Instance feature distribution modeling: [This refers to the distribution of features within a region.] Individual Instance Features Modeled as an empirical distribution in feature space ,in Indicates that it is located in the feature Dirac measure at the location, This represents the uniform empirical distribution of features across all instances. Indicates the first Individual instance characteristics;

[0030] Definition of representative distribution of landmarks: Let the points to be selected be... The set of landmark features is as follows , Corresponding discrete distribution Defined as ,in and , This represents the weight allocation of the j-th landmark. For the position located at the Features of candidate landmarks Dirac measure at the location; Indicates the first to be selected Features of each landmark;

[0031] Constructing the Wasserstein barycenter objective function: The landmark selection problem is transformed into the problem of finding the optimal Wasserstein barycenter that represents the original feature distribution. ,in The 2-Wasserstein distance is defined as follows: ; The marginal distributions are respectively and The set of all joint distributions, where γ is the transmission plan matrix;

[0032] Design the transmission cost matrix for pathological perception. Instead of the standard Euclidean distance, the transmission cost matrix simultaneously considers three factors: feature space distance, pathological importance, and spatial continuity.

[0033] ,

[0034] in The first term represents the pathological perception transmission cost between the i-th instance feature and the j-th candidate landmark feature. For instance features Features of candidate landmarks The squared Euclidean distance in the feature space; the second term As a modulating factor of pathological importance, exp is the natural exponential function. For the first The pathological importance score of each instance. The first term is the adjustment coefficient; the third term is the adaptive spatial continuity constraint factor, where... For example With candidate landmarks The Euclidean distance in the WSI two-dimensional space of the original panoramic pathological slide image, where β is the spatial penalty coefficient and the penalty function is... Defined as:

[0035] ,

[0036] in An adaptive threshold;

[0037] Pathological Importance Score Through a lightweight pathological importance assessment network calculate:

[0038] ,

[0039] in It is the first The feature vector of each instance and It is a learnable weight matrix. It's the dimensionality reduction ratio. and It is a bias term. It is the sigmoid activation function;

[0040] Entropy regularization and Sinkhorn iterative solution: Introducing an entropy regularization term transforms the original problem into: ,

[0041] in For entropy regularization of optimal transmission loss, Ent For the entropy of the transmission plan, For the element in the i-th row and j-th column of the transmission plan matrix γ, The regularization coefficient is used.

[0042] The Sinkhorn-Knopp iterative algorithm is used for efficient solution: First, the kernel matrix is ​​calculated. ,initialization That is, initializing the column scaling vector initial values. Equal to the number of landmark points and the vector of all 1 dimensions Iterative updates , Where t is the current iteration step number, It is the column scaling vector of the t-th iteration. It is the row scaling vector for the (t+1)th iteration. The source distribution weights are equal to the total number of instances. The reciprocal of the number of instances multiplied by a vector of all 1s This indicates that all instances have uniform weights; That is, the target distribution weight vector Equal to the landmark weight vector The optimal transmission plan is obtained after convergence. ,in This is the converged row scaling vector. The converged column scaling vector. This is a matrix diagonalization operation;

[0043] Differentiable landmark location learning: The features of the j-th landmark... The position parameterization is a soft aggregation of the original instance features: ,in The weights for assigning to the j-th landmark for the i-th instance are calculated using a learnable assignment network combined with the Gumbel-Softmax sampling mechanism. The value of the landmark traversal index k ranges from 1 to the total number of landmarks m; A parameterized assignment scoring function; and The noise variable is independently and identically distributed from the standard Gumbel distribution; τ is the temperature parameter; based on the optimal transmission plan. This further constrains the consistency between the assigned weight π and the transmission plan. ;

[0044] Optimal transmission regularization loss: The optimal transmission target is used as the regularization loss during training.

[0045] + ,

[0046] in To achieve optimal transmission loss using entropy regularization, To allocate consistency loss, To allocate entropy loss, This is the balance coefficient;

[0047] Landmark feature matrix based on optimal transmission selection The low-rank approximation Nyström attention is calculated within the region, and the output features after processing the l-th region are... ,in Representation layer normalization; This represents a low-rank approximate attention function;

[0048] Spatial attention is calculated by concatenating the outputs of each region according to their original spatial positions using the concat operation. The complete feature sequence after reconstructing the region :

[0049] ,

[0050] Then remove the previously added padding to ensure the output dimension matches the input; then concatenate the regions to form the complete feature sequence. A bidirectional spatial and channel interactive attention function is applied to perform regional-level spatial and channel co-optimization, resulting in regional-level optimization features. ,in These are features optimized for regional-level bidirectional spatial and channel interaction attention. This represents the bidirectional spatial and channel interaction attention function.

[0051] Step 5 includes: introducing a global Nyström attention mechanism and regional output. Enter the second layer for global-level processing:

[0052] ,

[0053] ,

[0054] in This represents the bidirectional spatial and channel interaction attention function. This represents the globally Nyström optimal transport attention function. The number of global landmarks is used; firstly, K representative features are extracted from each region through a learnable soft aggregation mechanism. Then, global Nyström attention is applied to the regional representative features to capture the semantic relationships between different regions. The global Nyström attention selects... The system uses several landmarks to approximate the complete cross-regional attention matrix. That is, after global attention processing, residual connections are performed, followed by further channel dimension optimization through channel attention.

[0055] Finally, classification prediction is achieved through the MultipleInstanceLearning (MIL) aggregation module: ,

[0056] in This is the prediction result at the package level.

[0057] Step 6 includes: adopting an iterative bidirectional interactive framework, where spatial attention and channel attention alternate and modulate each other, with the number of iterations set to T, and initializing the spatial weights. and channel weight In the t-th iteration: first, spatial attention calculation for channel modulation is performed, followed by spatially guided channel attention update, and then a bidirectional spatial and channel interactive attention function is executed. The input features are uniformly denoted as H. When called at the region level, H corresponds to the complete feature sequence after concatenating the regions. When called at the global level, H corresponds to the feature after global Nyström attention processing. The specific process includes:

[0058] Spatial attention through channel modulation: Introducing channel weights to modulate the query and key.

[0059] ,

[0060] ,

[0061] ,

[0062] in This is the channel weight vector obtained from the previous iteration, where Q is the query matrix, K is the key matrix, and V is the value matrix. These represent the query transformation weight matrix, the key transformation weight matrix, and the value transformation weight matrix, respectively.

[0063] To maintain linear computational complexity, spatial attention is calculated using the aforementioned low-rank approximation mechanism based on optimal transport landmarks:

[0064] ,

[0065] in, For spatial attention output, This represents the optimal transmission landmark matrix for the corresponding multiplexing layer. Low-rank approximate attention computation is performed; then spatial weights are extracted from the attention output. Specifically, this involves summing and normalizing each row of the attention matrix to obtain the spatial importance score for each instance.

[0066] Spatially Guided Channel Attention: Introducing Spatially Weighted Weighted Pooling

[0067] ,

[0068] ,

[0069] in It is a spatially weighted average pooling feature. It is a spatially weighted max pooling feature; It is the spatial attention weight of the i-th instance in the current iteration;

[0070] The two pooling results are input into a shared multilayer perceptron network, subjected to nonlinear transformation, and then summed. The updated channel weights are obtained after sigmoid activation.

[0071] ,

[0072] in The channel weight vector is updated for the (t+1)th iteration;

[0073] Interactive convergence regularization loss: Introducing interactive convergence regularization loss:

[0074] ,

[0075] in To achieve interactive convergence of regularization loss, Let be the spatial attention weights for the t-th iteration;

[0076] Final Feature Fusion: Spatial Attention Weights after T iterations and channel attention weights To achieve a mutually consistent stable state, the final characteristics are obtained through dual modulation:

[0077] ,

[0078] in This is a broadcast expansion operation used to copy and expand the low-dimensional weight vector along the corresponding dimension to match the size of the input feature matrix.

[0079] Step 7 includes employing three multi-level adaptive optimization techniques:

[0080] The first approach is a progressive landmark reduction strategy, which uses different numbers of landmarks at different depths of the network. Specifically, the hierarchical Nyström attention of this invention includes two levels. The first level is regional Nyström attention, which is a shallower layer and uses more landmarks. The first level fully models fine-grained feature relationships within the region; the second level is global Nyström attention, which is a deeper level. Because the semantics are more abstract and compact after the features are processed at the region level, fewer landmarks are used. This can effectively approximate the global attention matrix across regions. The depth of the layers is determined by the processing order of the data in the network, that is, the layers processed first are shallow and the layers processed later are deep. The number of landmarks gradually decreases as the layers deepen.

[0081] The second type is the zero-padding alignment strategy. When reshaping a one-dimensional feature sequence into a two-dimensional spatial layout for region division, if the number of instances is not a perfect square, padding is required to ensure that the region can be evenly divided into regular regions. The zero-padding strategy is adopted, that is, adding all-zero vectors to the end of the feature sequence until the sequence length meets the requirement of a perfect square.

[0082] The third approach is a learnable temperature regulation strategy, which introduces a learnable temperature parameter into the Gumbel-Softmax sampling process to dynamically adjust the sharpness of the sampling distribution.

[0083] Step 8 includes:

[0084] Use a lightweight attention network to optimize instance features. As input, the output is the attention score for each instance. The attention score is then normalized using softmax to obtain the importance weight for each instance, ensuring that the sum of all weights is 1. This results in the final bag-level feature representation. The result is obtained through weighted summation: After obtaining the package-level feature representation, a linear classifier is used for the final diagnostic prediction. The classifier adopts a single-layer linear mapping structure, which directly maps the aggregated package-level features to the output space of each category.

[0085] The model training employs a multi-task joint loss function, and the total loss function... for:

[0086] ,

[0087] in It is a distribution alignment auxiliary loss. It is the interactive convergence loss. It is the balance coefficient.

[0088] The present invention also provides an electronic device, including a processor and a memory, the memory storing program code that, when executed by the processor, causes the processor to perform the steps of the method.

[0089] Beneficial Effects: This invention achieves task-adaptive feature representation learning. By introducing an online feature re-embedding module, it overcomes the limitations of traditional MIL methods that rely on fixed pre-trained features, enabling the model to dynamically adjust feature representations according to the needs of specific diagnostic tasks, effectively bridging the semantic gap between general pre-trained features and specific pathological targets. This invention achieves landmark selection with mathematical optimality guarantees. Unlike traditional Nyström methods that use uniform random sampling or heuristic importance sampling, it innovatively models the landmark selection problem as the Wasserstein centroid problem in optimal transport theory, seeking the optimal set of landmarks that can represent the feature distribution of all instances with minimum transport cost from a geometric perspective. This method guarantees global optimality, and landmark selection has a mathematically optimal representativeness guarantee, making it more reliable than heuristic methods. Furthermore, optimal transport naturally considers the relative positions of landmarks, avoiding the selection of overly clustered redundant landmarks in the feature space. Finally, through the design of a transport cost matrix for pathological perception, diagnostic value information is integrated into the geometric framework, achieving a unity of pathological semantics and mathematical optimality. Furthermore, by introducing a spatial continuity constraint term into the transmission cost matrix, this invention addresses the problem of existing optimal transmission methods that only consider feature space distance while neglecting the spatial distribution characteristics of pathological images. This ensures that landmarks uniformly cover different pathological regions in the WSI space of the panoramic pathological slide image, avoiding diagnostic blind spots caused by landmark clustering in the feature space. This invention achieves efficient linear computational complexity by innovatively designing a hierarchical Nyström attention mechanism, successfully reducing the quadratic computational complexity of traditional Transformer self-attention. Reduced to linear complexity Where m is the number of landmarks and This invention significantly improves the computational efficiency of processing large-scale panoramic pathological slide images (WSI). It achieves spatially aware pathological feature modeling. Regional-level Nyström attention fully utilizes the inherent spatial locality of pathological images, while global-level Nyström attention further models cross-regional semantic relationships and long-range dependencies, realizing hierarchical feature learning from local details to global structure. This invention achieves bidirectional deep interactive learning between space and channels. Unlike traditional methods that use simple concatenation, it innovatively designs an iterative bidirectional interactive mechanism. Channel weights modulate spatial attention, focusing the model on diagnostically relevant channels, while spatial weights guide channel attention, making high-attention regions contribute more. This breaks the limitation of unidirectional information flow, achieving deep coupling optimization of spatial and channel dimensions, significantly enhancing the discriminative power of feature representation. This invention achieves end-to-end differentiable training. Through Sinkhorn iteration and soft aggregation mechanisms, the entire landmark selection process is fully differentiable, supporting end-to-end joint optimization without the need for additional pre-training or staged training. This invention achieves multi-level adaptive optimization, comprehensively employing various optimization strategies such as progressive landmark reduction, zero-fill alignment, and learnable temperature adjustment, thereby improving the model's flexibility and robustness at different levels. This invention has broad clinical application scenarios; the proposed method is a general feature learning and aggregation framework, independent of specific task types or dataset features, thus enabling flexible application to various computational pathology tasks, including cancer diagnosis, subtype classification, and prognostic prediction. This invention exhibits good scalability and practicality; its lightweight architecture and linear computational complexity allow it to handle larger-scale panoramic pathological slide image (WSI) data, meeting the needs of practical clinical applications. Furthermore, through a wavelet frequency and spatial domain dual-stream complementary recalibration module and Wasserstein distribution alignment loss, it significantly compensates for the pre-trained model's insufficient capture of high-frequency texture details in pathology, and directly couples frequency domain enhancement to the optimal transport landmark selection mechanism, improving the geometric discriminative power and landmark representativeness accuracy in the feature space. Meanwhile, the transmission cost matrix was further optimized by using an adaptive spatial continuity threshold (based on thumbnail tissue density prediction) to ensure uniform coverage of landmarks in pathological regions of different densities, avoiding diagnostic blind spots and achieving more robust pathological perception modeling. Furthermore, the robustness of the model to batch variations in hematoxylin and eosin staining (H&E) was further improved through a staining-invariant adaptive regularization module and color consistency Wasserstein loss, achieving significant improvements in generalization performance on multi-center, multi-scanning device datasets. Simultaneously, the optimal transmission computation mechanism was reused to improve overall efficiency. Attached Figure Description

[0090] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments, and the advantages of the present invention in the above and / or other aspects will become clearer.

[0091] Figure 1a This is a flowchart of the method of the present invention.

[0092] Figure 1b Low-rank approximate attention architecture based on adaptive landmark selection.

[0093] Figure 1c This is a diagram of the bidirectional spatial and channel interaction attention architecture.

[0094] Figure 1d This is a diagram illustrating the Chinese meanings of the symbols in the image.

[0095] Figure 2a This is a comparison chart of experimental results on the CAMELYON-16 dataset.

[0096] Figure 2b This is a comparison chart of experimental results on the TCGA-BRCA dataset.

[0097] Figure 2c Comparison of experimental results on the TCGA-NSCLC dataset.

[0098] Figure 3 Visualize the experimental results. Detailed Implementation

[0099] like Figure 1a As shown, this embodiment of the invention provides a pathological image calculation method based on hierarchical adaptive low-rank approximate attention, including the following steps:

[0100] Step 1, Feature Extraction and Multi-Instance Learning Modeling: The input panoramic pathological slide image WSI is divided into image blocks to generate an instance set. A pre-trained feature extractor is used to extract features from each image block to obtain a fixed feature representation. The panoramic pathological slide image WSI analysis task is modeled as a multi-instance learning problem.

[0101] Step 2: Construction of the Stain Invariant Adaptive Regularization Module. To address the color distribution variation problem caused by differences in hematoxylin and eosin staining batches and scanning equipment, this invention designs a stain invariant adaptive regularization module. This module predicts a learnable HSV color space perturbation matrix from global feature statistics using a lightweight multilayer perceptron. It applies perturbation to the original image patch in the HSV color space to generate a color-enhanced image patch. The original image patch and the color-enhanced image patch are respectively input into a pre-trained feature extractor with frozen parameters for online feature extraction, obtaining the original features and perturbation features. The entropy-regularized Wasserstein distance between their empirical distributions is calculated and added to the training as a color consistency auxiliary loss to ensure the model's robustness to color variations. Simultaneously, it reuses the subsequent optimal transmission computation module to improve efficiency. This module is only enabled during the training phase and does not add additional computational overhead during the inference phase. The original feature matrix extracted in Step 1 is directly used as the input for subsequent wavelet frequency domain and spatial domain dual-stream recalibration. By introducing this color consistency auxiliary loss during the training phase, the entire model improves its robustness to color variations and pathological discrimination ability during the optimization process.

[0102] Step 3: Construct an adaptive recalibration module for pathological features based on complementary wavelet frequency and spatial domain streams. Addressing the issue of insufficient high-frequency texture details capture in pre-trained feature extractors (such as ResNet-50 or PLIP) in pathological images, this invention designs a complementary recalibration module based on complementary wavelet frequency and spatial domain streams. This module processes the spatial semantic stream (preserving macroscopic tissue structure and semantic distribution) and the frequency texture stream in parallel (using a multi-resolution sub-band gating mechanism through wavelet packet decomposition, adaptively enhancing high-frequency detail sub-bands with learnable masks, and then reconstructing them inversely). In addition to direct fusion, the two streams also calculate the entropy-regularized Wasserstein distance between empirical distributions as an auxiliary loss for distribution alignment, ensuring geometric consistency and improving feature space discriminative power. This provides a better transmission cost basis and representative accuracy for the optimal transmission landmark selection in Step 4. The final output is the task-adaptive feature Z, which preserves the original semantics while enhancing the ability to express microscopic pathological textures.

[0103] Step 4: Implementation of the regional adaptive Nyström attention mechanism for pathological perception. Considering the local clustering and spatial correlation of pathological features in panoramic pathological slice images (WSI), a regional Nyström attention strategy is designed. The landmark selection problem is innovatively modeled as the Wasserstein centroid problem in optimal transport theory. By constructing the transport cost matrix for pathological perception, entropy regularization optimization, and Sinkhorn iterative solution, the optimal set of landmarks that can represent the feature distribution of all instances with the minimum transport cost is found from a geometric perspective. End-to-end training is achieved through a differentiable soft aggregation mechanism.

[0104] Step 5: Implementation of global Nyström attention mechanism. After regional processing, global Nyström attention is constructed to model cross-regional semantic relationships and long-range dependencies. The adaptive landmark selection strategy of pathological perception is also adopted. Global modeling is achieved through regional feature aggregation, cross-regional attention calculation and feature reprojection.

[0105] Step 6: Integrating bidirectional spatial and channel interaction attention, an innovative iterative bidirectional interaction mechanism is designed. Channel weight modulates the Query and Key calculations of spatial attention, enabling the model to focus on diagnostic-related feature channels. Spatial weight guides the weighted pooling of channel attention, making high-attention regions contribute more to the channel weight calculation. An interaction convergence regularization loss is introduced to ensure fast and stable bidirectional interaction.

[0106] Step 7: Multi-level adaptive optimization strategy. Multi-level adaptive optimization techniques such as progressive landmark reduction, zero-fill alignment and learnable temperature adjustment are used to further improve the practicality and robustness of the model.

[0107] Step 8, Feature Aggregation and Classification Prediction: The optimized instance features are integrated into a package-level representation through an aggregation function, a classifier is used for final prediction, and a multi-task loss function is used for end-to-end training.

[0108] In step 1, feature extraction and multi-instance learning modeling include:

[0109] The input panoramic pathological slide images (WSI) undergo preprocessing and image patch segmentation. Preprocessing includes background segmentation, tissue region detection, and color normalization to remove blank areas on the slides and standardize color differences between different scanning devices. Image patch segmentation is performed at a fixed magnification, dividing the panoramic pathological slide image (WSI) into regular rectangular image patches, each 256×256 pixels in size. At 20x magnification, the corresponding actual tissue region is approximately 128×128 micrometers. After preprocessing and segmentation, an instance set is obtained. Where I represents the total number of instances. A pre-trained deep neural network feature extractor is used to extract features from each image patch. The feature extractor can be a ResNet-50 model pre-trained on the large-scale natural image dataset ImageNet, or a model specifically pre-trained on pathological image datasets, such as PLIP. PLIP is a vision and language pre-trained model based on a contrastive learning framework. By learning the correspondence between pathological images and text descriptions, feature representations more suitable for pathological analysis tasks can be extracted. After each image patch passes through the feature extractor, a fixed-dimensional feature vector is obtained. The feature vectors of all instances form the initial feature matrix. The panoramic pathology slide image WSI analysis task is modeled as a standard multi-instance learning problem, and the training dataset is defined as follows: , This represents the set of instances corresponding to the WSI of the i-th panoramic pathological slide image, containing Image patch instance, package level label This represents the diagnostic result of the panoramic pathological slide image WSI. The label constraint for multi-instance learning is defined as the positive caching condition. If and only if there exists at least one instance j such that the true label of that instance exists. Negative package conditions If and only if for all instances j, .

[0110] Step 2 includes: Let the initial instance feature matrix be... To improve the robustness of the model to H&E mutations caused by hematoxylin and eosin staining, this invention designs a staining-invariant adaptive regularization module. First, global average pooling is performed on the initial feature matrix H to obtain global feature statistics. Input h into a perturbation prediction network consisting of a lightweight multilayer perceptron (MLP). Generate HSV color space perturbation matrix The 3rd order identity matrix Ensure that the initial perturbation is close to zero during the training phase.

[0111] Next, regarding the first One original image block The result of converting from RGB color space to HSV color space Applying a perturbation yields Then convert back to the RGB color space to obtain the i-th color-enhanced image patch. All original image patches With corresponding color-enhanced image patches Input the pre-trained feature extractor with frozen parameters to obtain the original feature matrix H and the perturbed feature matrix. .

[0112] Additional calculation of the entropy-regularized 2-Wasserstein distance between the empirical distributions H and H' (Reuse the Sinkhorn iteration from step 4 to solve,) (where entropy regularization coefficient is the coefficient), this loss corresponds to Figure 1a L_color in, and with coefficients A total loss function is incorporated. This loss forces the model to maintain a consistent distribution of color perturbations during optimization, improving geometric stability and providing a more robust feature foundation for subsequent wavelet frequency and spatial dual-stream recalibration and optimal transport landmark selection. The original feature matrix H will be directly input into the module of subsequent step 3 for processing.

[0113] Step 3 includes:

[0114] Let the initial instance feature matrix be... Where I is the number of instances and D is the feature dimension. To compensate for the insufficient capture of high-frequency texture details in pathological tissues by the pre-trained model, this invention designs a wavelet frequency domain and spatial domain dual-stream complementary recalibration module before the Nyström attention layer. This module processes the spatial domain semantic stream and the frequency domain texture stream in parallel. The spatial domain semantic stream, which preserves the semantic structure and tissue distribution of the original features, is obtained through lightweight transformation processing: ,

[0115] in , These are learnable parameters. Frequency domain texture flow first utilizes a zero-padding strategy to transform the one-dimensional sequence... Reconstructed into a regular two-dimensional feature map ( Then, a two-level wavelet packet decomposition is performed independently on each channel (using the wavelet basis 'db4' commonly used in pathology) to obtain a multi-resolution subband coefficient tree. Subband selection masks are then generated using a lightweight MLP (predicting from global average pooling features). ( (where the number of sub-bands is used), adaptive enhancement is performed on the high-frequency sub-bands to obtain... ,in For Sigmoid activation, finally adjust the enhanced coefficients. Perform inverse wavelet packet transform, extract the real part and flatten it back to a one-dimensional sequence to obtain the frequency domain enhanced features. Introducing a learnable scalar during two-stream fusion (Initialized to 0.1), calculate (Including the original) (The residual connections are used to ensure gradient stability). Simultaneously, to ensure that frequency domain enhancement directly serves subsequent optimal transmission landmark selection, the entropy-regularized 2-Wasserstein distance between the empirical distributions of the spatial and frequency domain flows is additionally calculated.

[0116] ,

[0117] Figure 1a L_align in this context represents the distribution alignment auxiliary loss. Solve using Sinkhorn iteration. Let be the entropy regularization coefficient, and let be the coefficient By incorporating a total loss function, the frequency domain texture enhancement is forced to maintain geometric consistency with the spatial domain semantic distribution, while simultaneously improving the distributional diversity of the feature space. This makes the transmission cost matrix in step 4 more reflective of pathological texture discrimination information, thereby improving the representativeness of landmarks and the accuracy of attention. After processing by this module, the output features are... It not only adapts to downstream tasks but also significantly improves the expression of high-frequency pathological details, providing higher-quality input for subsequent stratified Nyström attention.

[0118] Step 4 includes: Considering the local clustering and spatial correlation commonly exhibited by pathological features in panoramic pathological slide images (WSI), this invention designs a region-level Nyström attention strategy. First, spatially aware preprocessing is performed on the feature sequence containing I instances. It is transformed into a regular two-dimensional spatial layout through an adaptive fill operation, and the fill length is calculated according to the formula. , ,in and These represent the height and width of the padded 2D layout, respectively. A zero-padding strategy is used, which involves adding Δ zero vectors to the end of the feature sequence so that the total length of the feature sequence is exactly equal to... This allows it to be reshaped into a complete square layout. Then, region division is performed, and the filled feature sequence is reshaped into a two-dimensional spatial layout. Then, it is uniformly divided into L×L non-overlapping rectangular regions. The region division process is represented as follows:

[0119] ,

[0120] Each region Include Spatially adjacent image blocks, satisfying The region division follows the principle of spatial proximity, with each region containing spatially adjacent image patches. This division strategy is theoretically based on the local consistency assumption in pathology, which states that spatially close regions in tissue sections often have similar histological features and pathological manifestations. Then, an adaptive Nyström attention mechanism based on optimal transport theory is independently applied to each divided region for local feature learning. Unlike the traditional Nyström method, which uses uniform random sampling to select landmarks, this invention innovatively models the landmark selection problem as the Wasserstein centroid problem in optimal transport theory, geometrically seeking the optimal set of landmarks that can represent the feature distribution of all instances with minimal transport cost.

[0121] Instance feature distribution modeling: [This refers to the distribution of features within a region.] Individual Instance Features Modeled as an empirical distribution in feature space ,in Indicates that it is located in the feature Dirac measure at the location, This represents the uniform empirical distribution of features across all instances.

[0122] Definition of representative distribution of landmarks: Let the points to be selected be... The characteristics of each landmark are Its corresponding discrete distribution is defined as ,in and , which represents the weight allocation of each landmark.

[0123] Wasserstein barycenter objective function construction: The landmark selection problem is transformed into the problem of finding the optimal Wasserstein barycenter that represents the original feature distribution. ,in The 2-Wasserstein distance is defined as follows: . The marginal distributions are respectively and The set of all joint distributions, γ, is called the transport plan matrix. The physical meaning of this objective function is to find a set of landmarks that minimizes the total cost required to "transfer" the original feature distribution to the landmark distribution, thereby ensuring that the set of landmarks has mathematically optimal representativeness.

[0124] Design of the transmission cost matrix for pathological perception: To incorporate prior knowledge of pathology, this invention designs a transmission cost matrix for pathological perception. Instead of the standard Euclidean distance, this transmission cost matrix simultaneously considers three factors: feature space distance, pathological importance, and spatial continuity.

[0125] ,

[0126] The first item For instance features Features of candidate landmarks The squared Euclidean distance in the feature space; the second term As a modulatory factor of pathological importance, For the pathological importance score of the i-th instance, To adjust the coefficients and reduce the transmission cost of instances with high pathological importance (such as tumor areas), landmarks are more likely to be selected near key diagnostic areas.

[0127] The third term is the adaptive spatial continuity constraint factor, in which For example With candidate landmarks The penalty function is defined as the Euclidean distance in the two-dimensional space of the original panoramic pathological slide image WSI, and is as follows:

[0128] ,

[0129] in To achieve an adaptive threshold, a lightweight convolutional neural network (CNN) predicts the global tissue density map from low-magnification thumbnail images of panoramic pathological slides (WSI). The threshold is then calculated based on the average tissue density of the currently processed region (high-density tumor areas have larger thresholds, while low-density sparse areas have smaller thresholds). This adaptive design ensures that landmarks achieve spatially uniform coverage in WSI images of panoramic pathological slides with different pathological density distributions, avoiding the problems of insufficient representativeness in sparse tumor areas or excessive clustering of landmarks in dense areas caused by a fixed threshold. This further improves the representativeness of the landmark set for the overall instance feature distribution and the accuracy of attention modeling. Pathological Importance Scoring Through a lightweight pathological importance assessment network calculate:

[0130] ,

[0131] in It is the first The feature vector of each instance , It is a learnable weight matrix. This is the dimensionality reduction ratio, usually set to 4. and It is a bias term. The sigmoid activation function ensures the output is within the range (0,1). The pathological significance of this design lies in the reduced transmission cost for instances of high pathological importance (such as tumor regions), leading to a greater tendency for landmarks to be selected near diagnostically critical areas. Furthermore, the spatial continuity constraint ensures that landmarks are uniformly distributed in the WSI space of the panoramic pathological slide image, fully covering different pathological regions. Without spatial constraints, the optimal transmission solution might lead to all landmarks clustering in a localized region of the feature space, ignoring other diagnostically valuable pathological regions. Through this design, the present invention achieves a triple unity of feature space optimality, pathological semantic importance, and uniform spatial coverage.

[0132] Entropy regularization and Sinkhorn iteration: Directly solving the optimal transport problem has high computational complexity. This invention introduces an entropy regularization term, transforming the original problem into: ,

[0133] in For the entropy of the transmission plan, Here, represents the regularization coefficient. The introduction of the entropy regularization term transforms the optimization problem into a strongly convex problem with a unique solution that can be solved using an efficient iterative algorithm. The Sinkhorn-Knopp iterative algorithm is employed for efficient solution. First, the kernel matrix G is calculated. ,initialization Iterative updates , ,in For the source distribution weights, The target distribution weights are defined. The Sinkhorn iteration has a linear convergence rate, typically converging to sufficient accuracy within 10-20 iterations. After convergence, the optimal transport plan is obtained. .

[0134] Differentiable Landmark Location Learning: To achieve end-to-end training, this invention parameterizes landmark locations as a soft aggregation of original instance features. ,in The weights for assigning to the j-th landmark for the i-th instance are calculated using a learnable assignment network combined with the Gumbel-Softmax sampling mechanism. . A lightweight MLP is used to implement the parameterized assignment scoring function; and Let be the noise variable sampled independently and identically distributed from the standard Gumbel distribution; τ is the temperature parameter, which is gradually decreased during training to make the assignment approximate discrete selection. Based on the optimal transmission plan. This further constrains the consistency between the assigned weight π and the transmission plan. This constraint ensures that the landmarks obtained by soft aggregation are consistent with the optimal transport solution.

[0135] Optimal transmission regularization loss: The optimal transmission target is used as the regularization loss during training.

[0136] + ,

[0137] in To achieve optimal transmission loss using entropy regularization, To allocate consistency losses;

[0138] To distribute entropy loss, diversity in allocation is encouraged to avoid degradation. The balance coefficient. Landmark feature matrix based on optimal transmission selection. Nyström attention within the computation region ,in Representation layer normalization and residual connections ensure stable feature propagation. Through an optimal transmission mechanism, the computational complexity of a single region is reduced from... Reduce to Furthermore, because landmarks possess mathematically optimal representativeness, the accuracy of attention modeling is improved. For example... Figure 1b As shown, both the regional-level attention mechanism and the global-level attention mechanism follow the low-rank approximation computation framework of the Nyström method, achieving efficient attention computation by performing a low-rank approximation on the attention matrix. The input feature matrix X is linearly mapped to generate a query matrix Q, a key matrix K, and a value matrix V. Q and K obtain representative landmarks through adaptive landmark selection and construct the landmark query matrix. and landmark key matrix After obtaining the landmarks, attention calculation is performed according to the Nyström low-rank approximation calculation process, where pINV represents the pseudo-inverse operation, O represents the output feature matrix, n represents the number of instances, m represents the number of landmarks, and d represents the input feature dimension. This represents the feature dimensions of the query and the key. The feature dimension representing the value.

[0139] After all regions have been processed, the outputs from each region are spliced ​​together according to their original spatial locations to reconstruct a complete feature sequence: , Figure 1a In this context, H1, H2, H3, and H4 represent multiple sets of regional feature representations obtained after regional-level processing, and Concat represents concatenating these regional feature representations.

[0140] The previously added padding is then removed to ensure the output dimension matches the input. This results in a complete feature sequence after region concatenation. A bidirectional spatial and channel interactive attention function is applied to perform regional-level spatial and channel co-optimization, resulting in regional-level optimization features. ,in These are features optimized for regional-level bidirectional spatial and channel interaction attention. This represents the bidirectional spatial and channel interaction attention function. This dense interaction within the region enables the model to identify and enhance local pathological patterns with diagnostic value, laying the foundation for subsequent global semantic modeling.

[0141] The implementation of the global Nyström attention mechanism in step 5 includes:

[0142] While region-level processing can effectively capture local pathological patterns and reduce computational complexity, many key diagnostic clues and prognostic information rely on cross-regional semantic relationships and global structural organization. To capture these complex cross-regional semantic relationships, this invention introduces a global-level Nyström attention mechanism after region-level processing. Region-level output. Enter the second layer for global-level processing:

[0143] ,

[0144] ,

[0145] Global Nyström attention processes complete feature sequences directly without region partitioning. It establishes long-range dependencies between any two instances through global landmark sampling, capturing cross-regional pathological association patterns and global tissue structure information. Specifically, it first extracts K representative features from each region using a learnable soft aggregation mechanism, and then applies global Nyström attention to these representative features to capture semantic relationships between different regions. Global Nyström attention selects… A total of 10 landmarks are used to approximate the complete cross-regional attention matrix, where The computational complexity is significantly reduced compared to the total number of region-representing features. After global attention processing, residual connections are also performed, followed by further channel dimension optimization through channel attention. Through a hierarchical design of two layers of Nyström attention—region-level and global-level—the overall computational complexity is reduced from that of traditional self-attention. Reduce to Where m is the number of landmarks and This two-stage architecture achieves an effective balance between local pathological pattern learning and global structural understanding, leveraging the inherent spatial locality of medical images while retaining the ability to globally model complex semantic patterns. Finally, classification prediction is completed through the MIL aggregation module. .

[0146] Step 6, the bidirectional spatial and channel interaction attention fusion, includes:

[0147] like Figure 1c As shown, this invention designs a bidirectional spatial and channel interactive attention mechanism. Although Nyström attention effectively reduces computational complexity in the spatial dimension, diagnostic information in pathological images is distributed not only in the spatial dimension but also in the channel dimension of features. Existing methods use a simple concatenation of spatial and channel attention, failing to fully leverage their synergistic enhancement effect. Therefore, this invention designs an innovative bidirectional interactive spatial and channel attention mechanism, enabling the two attention types to mutually enhance and synergistically optimize each other. This invention employs an iterative bidirectional interactive framework, where spatial attention and channel attention alternate and modulate each other. Let the number of iterations be T, and initialize the spatial weights... (Uniform weight) and channel weight (Uniform weights). In the t-th iteration: first, spatial attention calculation for channel modulation is performed, followed by spatially guided channel attention update. The bidirectional spatial and channel interactive attention function... The input features are uniformly denoted as H. When called at the region level, H corresponds to the complete feature sequence after concatenating the regions. When called at the global level, H corresponds to the feature after global Nyström attention processing. The specific process is as follows:

[0148] Spatial attention with channel modulation: Traditional spatial attention treats all channels equally, while this invention introduces channel weights to modulate the query and key.

[0149] ,

[0150] ,

[0151] ,

[0152] in This is the channel weight vector obtained from the previous iteration, where ⊙ represents element-wise multiplication and is extended to all instances via a broadcast mechanism. By modulating Q and K with the channel weights, the model automatically focuses on diagnostically relevant feature channels when calculating spatial relationships, suppressing interference from background noise channels. To maintain linear computational complexity, spatial attention is calculated using the aforementioned low-rank approximation mechanism based on optimal transport landmarks.

[0153] ,

[0154] Spatial weights are then extracted from the attention output. Specifically, this involves summing and normalizing each row of the attention matrix to obtain the spatial importance score for each instance.

[0155] Spatially Guided Channel Attention: Traditional channel attention compresses the spatial dimension through global pooling, losing spatial distribution information. This invention introduces spatially weighted weighted pooling:

[0156] ,

[0157] ,

[0158] in This represents the spatial attention weight (scalar) of the i-th instance in the current iteration. Unlike traditional global pooling, this weighted pooling allows high-attention regions (such as tumor cell clusters) to contribute more to the channel weight calculation, thus learning a more pathologically meaningful channel representation. The two pooling results are input into a shared multilayer perceptron network, subjected to nonlinear transformations, and then summed. After sigmoid activation, the updated channel weights are obtained.

[0159] .

[0160] Interactive convergence regularization loss: To ensure that bidirectional interactions converge quickly to a stable state, an interactive convergence regularization loss is introduced.

[0161] ,

[0162] This loss encourages attention weights to stabilize quickly during iteration, avoiding oscillations.

[0163] Final Feature Fusion: Spatial Attention Weights after T iterations and channel attention weights To achieve a mutually consistent stable state, the final characteristics are obtained through dual modulation:

[0164] ,

[0165] This includes residual connections to maintain gradient flow. This bidirectional interaction mechanism breaks the unidirectional information flow limitation of traditional serial designs, achieving deep coupling optimization of spatial and channel dimensions, and significantly enhancing the discriminative power of feature representations. For example... Figure 1d As shown, Figure 1b and Figure 1c The main symbols in the Chinese text have the following meanings: Figure 1d As shown, Figure 1d In this context, Softmax represents the normalization function.

[0166] Step 7 includes: To further improve the practicality and robustness of the model, this invention employs three multi-level adaptive optimization techniques. The first is a progressive landmark reduction strategy. In different layers of a deep neural network, the abstraction level and semantic richness of features vary significantly. This invention uses a strategy of using different numbers of landmarks at different network depths. The number of landmarks used in the first layer, i.e., the region Nyström attention, is... To fully model the fine-grained feature relationships within the region, the number of landmarks used in the second-level, global Nyström attention is [number missing]. The first approach is to adapt to the abstraction and compactness of features after region-level processing. The second approach is a zero-padding alignment strategy. When reshaping a one-dimensional feature sequence into a two-dimensional spatial layout for region partitioning, if the number of instances is not a perfect square, padding is required to ensure uniform division into regular regions. This invention adopts a zero-padding strategy, which adds all-zero vectors to the end of the feature sequence until the sequence length meets the requirement of a perfect square. The advantage of zero-padding is that it is simple, efficient, and does not introduce any noise information. The third approach is a learnable temperature adjustment strategy. A learnable temperature parameter is introduced during the Gumbel-Softmax sampling process to dynamically adjust the sharpness of the sampling distribution. The temperature parameter, as a trainable parameter of the network, is learned end-to-end through backpropagation along with other parameters, enabling the model to automatically learn the optimal attention sharpness and sampling concentration based on the training data.

[0167] Step 8 includes: After processing by the online feature re-embedding module, the features of each instance are optimized in a task-adaptive manner. Next, these optimized instance-level features need to be aggregated into a bag-level representation to support WSI-level diagnostic prediction of panoramic pathological slide images. This invention employs an attention-weighted aggregation strategy, calculating the importance weight of each instance through an attention mechanism, and then performing a weighted summation of all instance features. Specifically, a lightweight attention network is used to aggregate the optimized instance features... As input, the output is the attention score for each instance. The attention scores are then normalized using softmax to obtain the importance weights for each instance, ensuring that the sum of all weights is 1. The final bag-level feature representation is calculated through a weighted summation: After obtaining the bag-level feature representation, a linear classifier is used for final diagnostic prediction. The classifier adopts a single-layer linear mapping structure, directly mapping the aggregated bag-level features to the output space of each category. The number of neurons in the output layer is determined according to the number of categories in the task.

[0168] The model is trained using a multi-task joint loss function, and the total loss function is:

[0169] ,

[0170] in It is the cross-entropy classification loss. It is the optimal transmission regularization loss. It is the interactive convergence loss. It is the balance coefficient. It is the Wasserstein loss for frequency-space distribution alignment introduced in step 3. ), It is the balance coefficient. The training process uses the backpropagation algorithm to simultaneously optimize all parameters of the wavelet frequency and spatial dual-stream recalibration module, the pathological importance assessment network, the attention aggregation mechanism, and the classifier, achieving end-to-end joint learning.

[0171] In one specific embodiment of the present invention, the method includes:

[0172] Step 1: Data Preparation and Feature Extraction. The model was trained and validated using publicly available pathological image datasets. These datasets included TCGA-NSCLC (TCGA Non-Small Cell Lung Cancer Dataset), with 854 panoramic pathological slide images in WSI; TCGA-BRCA (TCGA Invasive Breast Cancer Dataset), with 977 panoramic pathological slide images in WSI; TCGA-BLCA (TCGA Bladder Urothelial Carcinoma Dataset), with 377 panoramic pathological slide images in WSI; and CAMELYON-16 (Breast Cancer Lymph Node Metastasis Detection Dataset). A standardized preprocessing procedure was performed on each panoramic pathological slide image (WSI), including tissue region segmentation and color normalization. Tissue region segmentation used deep learning methods to identify and remove background areas from the slide, retaining only areas containing valid tissue. Color normalization employed a standardization method to eliminate color differences between different scanning devices and staining batches, improving the model's generalization ability. The preprocessed panoramic pathological slide images (WSI) were then divided into 256×256 pixel image blocks at a fixed magnification. Image patch features are extracted using a pre-trained deep learning model, with ResNet-50 and PLIP being selected as general-purpose models. The feature vectors of all instances are organized into a feature matrix H, which serves as input to the subsequent online feature learning module.

[0173] Step 2, configuration of the color-invariant adaptive regularization module. Using the initial instance feature matrix H obtained in Step 1 and the corresponding original image patch... To improve the model's robustness to batch variations in hematoxylin and eosin staining (H&E) and differences in scanning equipment, a staining-invariant adaptive regularization module is constructed as input. Specifically, the initial feature matrix H is first subjected to global average pooling to obtain global feature statistics. and will Input is a perturbation prediction network composed of lightweight multilayer sensing mechanisms. Generate HSV color space perturbation matrix Then, for the first... One original image block Converting from RGB color space to HSV color space yields... Applying a perturbation yields Then convert it back to the RGB color space to obtain color-enhanced image patches. The original image patch and its corresponding color-enhanced image patch are input into a pre-trained feature extractor with frozen parameters, respectively, to obtain the original feature matrix H and the perturbed feature matrix H′. The entropy-regularized 2-Wasserstein distance between the empirical distributions of H and H′ is then calculated. As an auxiliary loss for color consistency, this loss corresponds to the one in Figure 1a. and with coefficient Add to the total loss function. This module is only enabled during the training phase and does not add additional computational overhead during the inference phase. Perturbation Prediction Network It is implemented using a lightweight multilayer perceptron, with color perturbation performed in the HSV space, and the entropy regularization coefficient... and loss balance coefficient Configure based on the performance of the training set.

[0174] Step 3: Configuration of the wavelet frequency and spatial dual-stream recalibration module. This invention inserts a wavelet frequency and spatial dual-stream recalibration module before the Nyström attention layer. It uses the 'db4' wavelet basis for two-level wavelet packet decomposition, and a lightweight MLP predicts subband selection masks from global average pooling features. The learnable scalar α and the frequency-space distribution alignment loss coefficients are automatically optimized through training.

[0175] Step 4: Divide the features evenly into 8×8 non-overlapping rectangular regions. For each region, first model the instance features within the region as an empirical distribution. .

[0176] Constructing the pathological perception transmission cost matrix: Calculating the importance score for each instance using a pathological importance assessment network. The evaluation network employs a two-layer MLP structure: the first layer reduces the D-dimensional features to D / 4 dimensions and applies ReLU activation; the second layer maps the features to 1 dimension and applies Sigmoid activation. The transmission cost matrix is ​​calculated as follows:

[0177] ,

[0178] Where λ=0.5 is the pathological perception regulation coefficient. The spatial penalty coefficient, Set to half the diagonal length of the current processing region. Set the entropy regularization coefficient ε = 0.1, and the maximum number of iterations. Initialize the number of landmarks. The optimal transmission plan γ* is calculated using Sinkhorn iteration, and the convergence criterion is... The assignment weights π are calculated based on a learnable assignment network. This assignment network uses a single-layer MLP to map D-dimensional features to... The dimension is normalized using Softmax. Temperature τ is set to 1.0 initially during training and linearly annealed to 0.1 as training progresses. Landmark features are calculated using soft aggregation. The standard Nyström attention computation is then applied. The main process is that the input H (dimension I×D) is transformed linearly to generate three branches Q, K, and V, where Q and K are approximated by downsampling of landmarks selected by optimal transmission.

[0179] Step 5, Bidirectional Spatial and Channel Interactive Attention. After concatenating all the region features processed by Nyström, apply the bidirectional spatial and channel interactive attention module. Set the iteration count T=3. In each iteration: first, modulate the calculation of Q and K using the current channel weight C; then calculate the spatial attention and extract the spatial weight S; next, use the spatial weight to guide the weighted pooling of the channel attention to calculate the updated channel weights. Specifically, the weighted average pooling is... Weighted max pooling The two pooling results are summed after being processed through a shared MLP (dimensionality reduction ratio r=4), and then activated by a Sigmoid function to obtain new channel weights. After T iterations, the features are doubly modulated using the final spatial and channel weights.

[0180] Step 6, Global Nyström Attention. The module then uses global Nyström attention to model cross-regional semantic relationships and long-range dependencies, also employing an adaptive landmark selection strategy for pathological perception. Each layer contains a global landmark. Following the global Nyström attention layer, a bidirectional spatial and channel interaction attention module is applied to optimize the channel dimension. This tightly coupled design ensures that the semantic information learned by spatial attention can guide channel selection and recalibration in a timely manner, avoiding information loss or dilution during multi-layer propagation. Finally, the MIL model is used for downstream tasks such as cancer diagnosis, subtype classification, and survival analysis.

[0181] Step 7, Training Configuration and Loss Function. The total loss function is:

[0182] ,

[0183] in It is the cross-entropy classification loss. It is the optimal transmission regularization loss. It is the interactive convergence loss. It is the balance coefficient. It is the Wasserstein loss for frequency-space distribution alignment introduced in step 3. ), These are the balance coefficients. The training process uses the backpropagation algorithm to simultaneously optimize all parameters of the wavelet frequency and spatial dual-stream recalibration module, the pathological importance assessment network, the attention aggregation mechanism, and the classifier, achieving end-to-end joint learning. The balance coefficients are automatically optimized during training. The AdamW optimizer is used with a learning rate of 1e-4 and a weight decay of 1e-5. Training lasts for 200 epochs with a batch size of 1 (processing one panoramic pathological slice image WSI each time).

[0184] Step 8: Evaluate the performance of the trained model on an independent test set. The test set includes samples that the model has never seen during training to ensure the objectivity and reliability of the evaluation results. AUC is an important metric for evaluating performance. Figure 2a , Figure 2b and Figure 2c As shown. To comprehensively evaluate the advantages of the method of this invention, comparative experiments were conducted with several existing MIL methods. The comparative methods include ABMIL, CLAM, DSMIL, TransMIL, DTFD-MIL, IBMIL, and RRT-MIL. NAHA (Ours) indicates the method proposed in this invention. Figures 2a to 2c PLIP and ResNet-50 in the table represent experimental results using different pre-trained feature extractors. All comparison methods were trained and tested on the same dataset, with the same feature extractor and the same data partitioning settings to ensure fairness. Experimental results show that our proposed method achieves competitive classification performance while maintaining linear computational complexity, outperforming existing methods on certain datasets and metrics. During training, this method uses 5-fold cross-validation for all methods except CAMELYON-16, which uses 3-fold cross-validation.

[0185] Step 9: To improve the model's interpretability, this method employs a heatmap visualization method to analyze the model's attention distribution, demonstrating its decision-making process in panoramic pathological slide image WSI analysis. On the TCGA-NSCLC and CAMELYON-16 datasets, the attention heatmap shows that the model successfully focuses on tumor cell aggregation areas, confirming the effectiveness of the local consistency hypothesis. Specifically, by visualizing the importance scores output by the pathological importance assessment network, it can be observed that high-score areas highly overlap with the tumor areas labeled by pathologists, proving the pathological rationale for the adaptive landmark selection strategy. The model's predicted attention scores are converted into percentages, and the normalized scores are mapped to the corresponding spatial locations of the input panoramic pathological slide image WSI to generate a heatmap. Furthermore, Figure 3 The top k most relevant local regions in the panoramic pathological slide image WSI are shown, highlighting the feature regions that the model identifies as most relevant to the tumor.

[0186] This invention provides a pathological image calculation method based on hierarchical adaptive low-rank approximate attention. Many methods and approaches exist for implementing this technical solution; the above description is merely a preferred embodiment of the invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of this invention, and these improvements and modifications should also be considered within the scope of protection of this invention. All components not explicitly stated in this embodiment can be implemented using existing technologies.

Claims

1. A pathological image calculation method based on hierarchical adaptive low-rank approximate attention, characterized in that, Includes the following steps: Step 1, Feature Extraction and Multi-Instance Learning Modeling: For the input panoramic pathological slide image WSI, the image is divided into blocks to generate an instance set. A pre-trained feature extractor is used to extract features from each image block to obtain a fixed feature representation. The panoramic pathological slide image WSI analysis task is modeled as a multi-instance learning problem. Step 2, constructing a staining-invariant adaptive regularization module: To address the color distribution variation problem caused by differences in hematoxylin and eosin staining (H&E) batches and scanning equipment, a staining-invariant adaptive regularization module is designed. This module generates multi-view instance features through learnable color perturbation and calculates the entropy-regularized Wasserstein distance between the empirical distributions of features before and after perturbation as an auxiliary loss for color consistency, which is then added to the training. Step 3: To address the problem that the pre-trained feature extractor is insufficient in capturing high-frequency texture details of pathological images, a pathological feature adaptive recalibration module with complementary wavelet frequency and spatial domain streams is designed. The pathological feature adaptive recalibration module processes the spatial semantic stream and the frequency texture stream in parallel, calculates the entropy regularized Wasserstein distance between empirical distributions as a distribution alignment auxiliary loss, and finally outputs the task adaptive feature Z. Step 4, implementation of the regional adaptive Nyström attention mechanism for pathological perception: The landmark selection problem is modeled as the Wasserstein centroid problem in optimal transport theory. By constructing the transport cost matrix for pathological perception, entropy regularization optimization and Sinkhorn iterative solution, the optimal set of landmarks that can represent the feature distribution of all instances with the minimum transport cost is found from a geometric perspective. End-to-end training is achieved through a differentiable soft aggregation mechanism. Step 5, global Nyström attention mechanism implementation: After regional processing, a global Nyström attention mechanism is constructed to model cross-regional semantic relationships and long-range dependencies. The adaptive landmark selection strategy of pathological perception is also adopted, and global modeling is achieved through regional feature aggregation, cross-regional attention calculation and feature reprojection. Step 6, Two-way spatial and channel interaction attention fusion: Design an iterative two-way interaction mechanism. The channel weight modulates the calculation of the query matrix Query and key matrix Key of spatial attention so that the model focuses on the diagnostic feature channels. The spatial weight guides the weighted pooling of channel attention so that high attention regions contribute more to the calculation of channel weights. And an interaction convergence regularization loss is introduced to ensure that the two-way interaction is fast and stable. Step 7, Multi-level adaptive optimization strategy: Employ progressive landmark reduction, zero-fill alignment, and learnable temperature adjustment to improve the model's practicality and robustness; Step 8, Feature Aggregation and Classification Prediction: The optimized instance features are integrated into a package-level representation through an aggregation function, a classifier is used for final prediction, and a multi-task loss function is used for end-to-end training.

2. The method according to claim 1, characterized in that, Step 1 includes: preprocessing and segmenting the input panoramic pathological slide image (WSI). Preprocessing includes background segmentation, tissue region detection, and color normalization. At a fixed magnification, the panoramic pathological slide image (WSI) is segmented into regular rectangular image blocks, each covering a tissue region of a certain size. After preprocessing and segmentation, an instance set is obtained. Where I represents the total number of instances, This represents the i-th instance; A pre-trained deep neural network feature extractor is used to extract features from each image patch. After each image patch passes through the feature extractor, a fixed-dimensional feature vector is obtained. The feature vectors of all instances are used to form the initial feature matrix. ;in Represents the space of real numbers; For feature dimensions; The task of panoramic pathological slide image WSI analysis is modeled as a standard multi-instance learning problem, and the training dataset is defined as follows: , This represents the set of instances corresponding to the WSI of the i-th panoramic pathological slide image. Indicates the first Image patch instances, i-th packet-level label The label constraint for the i-th panoramic pathological slide image (WSI) is defined as the positive caching condition. If and only if there exists at least one instance j such that the instance's true label is true. Negative package conditions If and only if for all instances j, .

3. The method according to claim 2, characterized in that, Step 2 includes: color-invariant adaptive regularization module on the initial feature matrix Global average pooling is used to obtain global feature statistics. Global feature statistics Input perturbation prediction network Generate HSV color space perturbation matrices for hue, saturation, and lightness. Among them, the perturbation prediction network For lightweight multilayer perceptron (MLP), 3rd order identity matrix Ensure that the initial perturbation is close to zero; for the first One original image block The conversion from the RGB color space to the HSV color space (hue, saturation, and lightness) yields... Applying a perturbation yields Then convert back to the RGB color space to obtain the i-th color-enhanced image patch. ; will the first One original image block With the A color-enhanced image block The pre-trained feature extractor, with the freeze parameters input separately, yields the original feature matrix. With the perturbation characteristic matrix The feature extractor parameters are frozen but retained in the computation graph to support gradient-directed perturbation prediction networks. Return; Additional calculation of the original feature matrix With the perturbation characteristic matrix Entropy-regularized 2-Wasserstein distance between empirical distributions as an auxiliary loss for color consistency The color entropy regularization coefficient To control the intensity of entropy regularization, and using the color loss balance coefficient as a hyperparameter. Add the total loss function; this module is only enabled during the training phase.

4. The method according to claim 3, characterized in that, Step 3 includes: designing a wavelet frequency domain and spatial domain dual-stream complementary recalibration module before the Nyström attention layer. The recalibration module processes the spatial domain semantic stream and the frequency domain texture stream in parallel. The spatial domain semantic stream preserves the semantic structure and organization distribution of the original features, and obtains the spatial features through lightweight transformation processing. ,in For learnable weight matrix, These are learnable bias parameters; ReLU represents the linear rectified function; LayerNorm represents layer normalization; Frequency domain texture flow first uses a zero-padding strategy to transform a one-dimensional sequence Reconstructed into a regular two-dimensional feature map , ,in Two-dimensional layout height, The width of the two-dimensional layout is represented; then, a two-level wavelet packet decomposition is performed independently on each channel to obtain a multi-resolution sub-band coefficient tree. Subband selection masks are then generated using a lightweight multilayer perceptron (MLP). , The coefficients are obtained by adaptively enhancing the high-frequency subbands to determine the subband number. ,in Activated for Sigmoid. This represents element-wise multiplication; Finally, the enhanced coefficients were analyzed. Perform inverse wavelet packet transform, extract the real part and flatten it back to a one-dimensional sequence to obtain the frequency domain enhanced features. ; Introducing learnable scalars during two-stream fusion The original features H are preserved through residual connections, and task-adaptive features are calculated. Simultaneously, the entropy-regularized 2-Wasserstein distance between the empirical distributions of the spatial and frequency domain flows is additionally calculated as an auxiliary loss for distribution alignment. ,in Represents the entropy regularization coefficient. For preset hyperparameters greater than 0, and expressed as coefficients Add to the total loss function.

5. The method according to claim 4, characterized in that, Step 4 includes: designing a region-level Nyström attention strategy: first, performing spatial awareness preprocessing on feature sequences containing I instances. The layout is transformed into a regular two-dimensional space through adaptive fill operation, and the fill length is calculated according to the formula. , ,in and These represent the height and width of the padded 2D layout, respectively. A zero-padding strategy is used, which involves adding Δ zero vectors to the end of the feature sequence so that the total length of the feature sequence is exactly equal to... This allows it to be reshaped into a complete square layout; then, region division is performed, and the filled feature sequence is reshaped into a two-dimensional spatial layout. Then, it is uniformly divided into L×L non-overlapping rectangular regions, where L is the number of regions; the region division process is represented by the region division function. The i-th region Include Instances of spatially adjacent image patches For the first The number of instances in each region is full. The region division follows the principle of spatial proximity, with each region containing spatially adjacent image blocks. Then, for each region, an adaptive Nyström attention mechanism based on optimal transmission theory is independently applied to learn local features. The landmark selection problem is modeled as the Wasserstein centroid problem in optimal transmission theory, and the optimal set of landmarks that can represent the feature distribution of all instances with the minimum transmission cost is found from a geometric perspective. Instance feature distribution modeling: [This refers to the distribution of features within a region.] Individual Instance Features Modeled as an empirical distribution in feature space ,in Indicates that it is located in the feature Dirac measure at the location, This represents the uniform empirical distribution of features across all instances. Indicates the first Individual instance characteristics; Definition of representative distribution of landmarks: Let the points to be selected be... The set of landmark features is as follows , Corresponding discrete distribution Defined as ,in and , This represents the weight allocation of the j-th landmark. For the position located at the Features of candidate landmarks Dirac measure at the location; Indicates the first to be selected Features of each landmark; Constructing the Wasserstein barycenter objective function: The landmark selection problem is transformed into the problem of finding the optimal Wasserstein barycenter that represents the original feature distribution. ,in The 2-Wasserstein distance is defined as follows: ; The marginal distributions are respectively and The set of all joint distributions, where γ is the transmission plan matrix; Design the transmission cost matrix for pathological perception. Instead of the standard Euclidean distance, the transmission cost matrix simultaneously considers three factors: feature space distance, pathological importance, and spatial continuity. ， in The first term represents the pathological perception transmission cost between the i-th instance feature and the j-th candidate landmark feature. For instance features Features of candidate landmarks The squared Euclidean distance in the feature space; the second term As a modulating factor of pathological importance, exp is the natural exponential function. For the pathological importance score of the i-th instance, The first term is the adjustment coefficient; the third term is the adaptive spatial continuity constraint factor, where... For example With candidate landmarks The Euclidean distance in the WSI two-dimensional space of the original panoramic pathological slide image, where β is the spatial penalty coefficient and the penalty function is... Defined as: ， in An adaptive threshold; Pathological Importance Score Through a lightweight pathological importance assessment network calculate: ， in It is the first The feature vector of each instance and It is a learnable weight matrix. It's the dimensionality reduction ratio. and It is a bias term. It is the sigmoid activation function; Entropy regularization and Sinkhorn iterative solution: Introducing an entropy regularization term transforms the original problem into: ， in For entropy regularization of optimal transmission loss, Ent For the entropy of the transmission plan, For the element in the i-th row and j-th column of the transmission plan matrix γ, The regularization coefficient is used. The Sinkhorn-Knopp iterative algorithm is used for efficient solution: First, the kernel matrix is ​​calculated. ,initialization That is, initializing the column scaling vector initial values. Equal to the number of landmark points and the vector of all 1 dimensions Iterative updates , Where t is the current iteration step number, It is the column scaling vector of the t-th iteration. It is the row scaling vector for the (t+1)th iteration. The source distribution weights are equal to the total number of instances. The reciprocal of the number of instances multiplied by a vector of all 1s This indicates that all instances have uniform weights; That is, the target distribution weight vector Equal to the landmark weight vector The optimal transmission plan is obtained after convergence. ,in This is the converged row scaling vector. The converged column scaling vector. This is a matrix diagonalization operation; Differentiable landmark location learning: The features of the j-th landmark... The position parameterization is a soft aggregation of the original instance features: ,in The weights for assigning to the j-th landmark for the i-th instance are calculated using a learnable assignment network combined with the Gumbel-Softmax sampling mechanism. The value of the landmark traversal index k ranges from 1 to the total number of landmarks m; A parameterized assignment scoring function; and The noise variable is independently and identically distributed from the standard Gumbel distribution; τ is the temperature parameter; based on the optimal transmission plan. This further constrains the consistency between the assigned weight π and the transmission plan. ; Optimal transmission regularization loss: The optimal transmission target is used as the regularization loss during training. + ， in To achieve optimal transmission loss using entropy regularization, To allocate consistency loss, To allocate entropy loss, This is the balance coefficient; Landmark feature matrix based on optimal transmission selection The low-rank approximation Nyström attention is calculated within the region, and the output features after processing the l-th region are... ,in Representation layer normalization; This represents a low-rank approximate attention function; Spatial attention is calculated by concatenating the outputs of each region according to their original spatial positions using the concat operation. The complete feature sequence after reconstructing the region : ， Then remove the previously added padding to ensure the output dimension matches the input; then concatenate the regions to form the complete feature sequence. A bidirectional spatial and channel interactive attention function is applied to perform regional-level spatial and channel co-optimization, resulting in regional-level optimization features. ,in These are features optimized for regional-level bidirectional spatial and channel interaction attention. This represents the bidirectional spatial and channel interaction attention function.

6. The method according to claim 5, characterized in that, Step 5 includes: introducing a global Nyström attention mechanism and regional output. Enter the second layer for global-level processing: ， ， in This represents the bidirectional spatial and channel interaction attention function. This represents the globally Nyström optimal transport attention function. The number of global landmarks is used; firstly, K representative features are extracted from each region through a learnable soft aggregation mechanism. Then, global Nyström attention is applied to the regional representative features to capture the semantic relationships between different regions. The global Nyström attention selects... The system uses several landmarks to approximate the complete cross-regional attention matrix. That is, after global attention processing, residual connections are performed, followed by further channel dimension optimization through channel attention. Finally, classification prediction is achieved through the MultipleInstanceLearning (MIL) aggregation module: , in This is the prediction result at the package level.

7. The method according to claim 6, characterized in that, Step 6 includes: adopting an iterative bidirectional interactive framework, where spatial attention and channel attention alternate and modulate each other, with the number of iterations set to T, and initializing the spatial weights. and channel weight In the t-th iteration: first, spatial attention calculation for channel modulation is performed, followed by spatially guided channel attention update, and then a bidirectional spatial and channel interactive attention function is executed. The input features are uniformly denoted as H. When called at the region level, H corresponds to the complete feature sequence after concatenating the regions. When called at the global level, H corresponds to the feature after global Nyström attention processing. The specific process includes: Spatial attention through channel modulation: Introducing channel weights to modulate the query and key. ， ， ， in This is the channel weight vector obtained from the previous iteration, where Q is the query matrix, K is the key matrix, and V is the value matrix. These represent the query transformation weight matrix, the key transformation weight matrix, and the value transformation weight matrix, respectively. Spatial attention is calculated using the aforementioned low-rank approximation mechanism based on optimal transport landmarks: ， in, For spatial attention output, This represents the optimal transmission landmark matrix for the corresponding multiplexing layer. Low-rank approximate attention computation is performed; then spatial weights are extracted from the attention output. Specifically, this involves summing and normalizing each row of the attention matrix to obtain the spatial importance score for each instance. Spatially Guided Channel Attention: Introducing Spatially Weighted Weighted Pooling ， ， in It is a spatially weighted average pooling feature. It is a spatially weighted max pooling feature; It is the spatial attention weight of the i-th instance in the current iteration; The two pooling results are input into a shared multilayer perceptron network, subjected to nonlinear transformation, and then summed. The updated channel weights are obtained after sigmoid activation. ， in The channel weight vector is updated for the (t+1)th iteration; Interactive convergence regularization loss: Introducing interactive convergence regularization loss: ， in To achieve interactive convergence of regularization loss, Let be the spatial attention weights for the t-th iteration; Final Feature Fusion: Spatial Attention Weights after T iterations and channel attention weights To achieve a mutually consistent stable state, the final characteristics are obtained through dual modulation: ， in This is a broadcast expansion operation used to copy and expand the low-dimensional weight vector along the corresponding dimension to match the size of the input feature matrix.

8. The method according to claim 7, characterized in that, Step 7 includes employing three multi-level adaptive optimization techniques: The first strategy is a progressive reduction of landmarks, which uses different numbers of landmarks at different depths of the network. More landmarks are used in shallower layers to fully model fine-grained feature relationships, while fewer landmarks are used in deeper layers to adapt to the abstraction and compactness of features. The second type is the zero-padding alignment strategy. When reshaping a one-dimensional feature sequence into a two-dimensional spatial layout for region division, if the number of instances is not a perfect square, padding is required to ensure that the region can be evenly divided into regular regions. The zero-padding strategy is adopted, that is, adding all-zero vectors to the end of the feature sequence until the sequence length meets the requirement of a perfect square. The third approach is a learnable temperature regulation strategy, which introduces a learnable temperature parameter into the Gumbel-Softmax sampling process to dynamically adjust the sharpness of the sampling distribution.

9. The method according to claim 8, characterized in that, Step 8 includes: Use a lightweight attention network to optimize instance features. As input, the output is the attention score for each instance. The attention score is then normalized using softmax to obtain the importance weight for each instance, ensuring that the sum of all weights is 1. This results in the final bag-level feature representation. The result is obtained through weighted summation: After obtaining the package-level feature representation, a linear classifier is used for the final diagnostic prediction. The classifier adopts a single-layer linear mapping structure, which directly maps the aggregated package-level features to the output space of each category. The model training employs a multi-task joint loss function, and the total loss function... for: ， in It is a distribution alignment auxiliary loss. It is the interactive convergence loss. It is the balance coefficient.

10. An electronic device, characterized in that, It includes a processor and a memory, the memory storing program code that, when executed by the processor, causes the processor to perform the steps of the method as described in any one of claims 1 to 9.

Images