Full-slice image classification processing method and device based on topological latent space reconstruction
By employing a topological latent space reconstruction method, and utilizing techniques such as pre-trained feature extraction, saliency scoring, and topological reconstruction, the problems of noise interference and local convolution limitations in whole-slice image classification are solved. This achieves efficient global spatial dependency capture and feature interaction, thereby improving the accuracy and robustness of pathological image classification.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- HUNAN HUISEN BIOTECHNOLOGY CO LTD
- Filing Date
- 2026-03-14
- Publication Date
- 2026-05-19
AI Technical Summary
Existing whole-slice image classification methods struggle to effectively capture long-range dependencies and histological pattern co-occurrence relationships. They are also susceptible to distributional noise interference and limited receptive fields in local convolutions, resulting in low signal-to-noise ratios that fail to meet the needs of pathological analysis.
By employing pre-trained feature extraction, saliency scoring, topology reconstruction, bidirectional Mamba encoding and multi-scale feature fusion, dense interaction aggregation of fully self-attention modules, and standard cross-entropy loss function, a high signal-to-noise ratio feature grid is constructed to capture global spatial dependencies and co-occurrence relationships.
Physical removal of redundant background noise improves the model's signal-to-noise ratio, overcomes the limitations of local convolution, enables global spatial dependency capture and many-to-many interaction between features, enhances the ability to represent pathological features, and improves robustness and computational efficiency.
Smart Images
Figure CN121837803B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the fields of digital pathological image processing and computer vision technology, specifically to a method and apparatus for classifying whole-slice images based on topological latent space reconstruction. Background Technology
[0002] Digital pathology is a crucial tool in modern oncology for pathological analysis, prognostic assessment, and treatment planning. Whole-slice images, as the core data carrier in this field, can fully present the morphological characteristics of pathological tissues, providing comprehensive visual evidence for pathological analysis. Due to the massive data volume of whole-slice images, traditional fully supervised learning methods are difficult to apply directly, and pixel-by-pixel fine annotation requires significant time and manpower. Therefore, weakly supervised multi-instance learning has become the mainstream research paradigm in whole-slice image analysis. In this paradigm, the whole-slice image is considered as a feature package containing multiple patches, and the model is trained solely based on slice-level labels. Existing methods based on weakly supervised multi-instance learning have made some progress in feature aggregation through attention mechanisms, providing a feasible technical path for whole-slice image classification.
[0003] Existing weakly supervised multi-instance learning methods still face numerous unresolved issues when processing whole-slice images, failing to meet the demands of practical pathological analysis. Histopathological images inherently exhibit strong heterogeneity, and variations in staining protocols, tissue preparation, and scanning equipment can easily lead to distribution shifts within membranes and between slices. This distribution noise can mislead attention mechanisms, causing them to focus on artifacts rather than true lesion areas. While some methods propose distribution recalibration strategies to correct feature distribution shifts, these strategies still retain the entire feature sequence, and massive amounts of non-diagnostic background noise dilute key pathological signals, limiting the model's signal-to-noise ratio. Furthermore, their spatial encoding relies on local convolution, limiting the receptive field and failing to capture long-range dependencies within the tumor microenvironment. Static prototype queries in the aggregation phase create an interaction bottleneck, hindering effective inference of co-occurrence relationships between different histological patterns, ultimately impacting the actual performance of whole-slice image classification.
[0004] Therefore, there is an urgent need for a whole-slice image classification method based on topological latent space reconstruction to solve the problems of existing technologies, such as difficulty in capturing long-range dependencies and co-occurrence relationships of histological patterns. Summary of the Invention
[0005] To address these issues, the present invention provides a whole-slice image classification processing method and apparatus based on topological latent space reconstruction, which solves the problems of existing whole-slice image classification methods being affected by distributed noise, limited receptive fields of local convolution, and interactive bottlenecks in feature aggregation, making it difficult to capture long-range dependencies and co-occurrence relationships of histological patterns.
[0006] To achieve the above objectives, the present invention provides the following technical solution: a whole-slice image classification processing method based on topological latent space reconstruction, comprising:
[0007] The unordered feature package is obtained by extracting features from the whole slice image to be classified using a pre-trained feature extractor.
[0008] The saliency score of each patch feature in the unordered feature package is obtained by calculating the saliency score; based on the saliency score, the highest-scoring feature is selected using a deterministic Top-K sampling strategy. K From the features of each map patch, a subset of key features is obtained;
[0009] By performing topological reconstruction on the key feature subset, the disordered key feature subset is mapped into a regular feature grid; if the number of tiles in the key feature subset is insufficient... K Then, the zero-filling strategy is used to complete the potential spatial topology reconstruction;
[0010] The feature mesh is input into a bidirectional Mamba encoder embedded in the U-Net architecture; the isotropic global spatial dependency of the feature mesh is captured by bidirectional Mamba encoding and multi-scale feature fusion, and the encoded mesh features are output.
[0011] By flattening the encoded grid features and attaching a learnable classification token to the flattened feature sequence, the input sequence of the fully self-attention module is obtained; the input sequence is then fed into the fully self-attention module for dense interactive aggregation to obtain slice-level feature representations.
[0012] Based on the standard cross-entropy loss function, the slice-level feature representation is trained end-to-end to complete the classification inference of the whole slice image and output the classification result of the whole slice image.
[0013] As a preferred embodiment of the whole-slice image classification processing method based on topological latent space reconstruction, the formula for calculating the saliency score of each patch feature in the unordered feature bag is as follows:
[0014] ;
[0015] In the formula, For the first i The saliency score of each patch feature; For the unordered feature bag, the first i Each patch feature; , These are learnable parameters; , This is the bias term for a lightweight scoring network.
[0016] As a preferred approach for whole-slice image classification based on topological latent space reconstruction, the deterministic Top-K sampling strategy is used to select the highest-scoring image. K During the feature extraction process for each patch, the deterministic Top-K sampling strategy selects the patch using an index set. K Each map feature; the expression for the index set is:
[0017] ;
[0018] In the formula, For index set; A subset of the tile feature indexes for an unordered feature package; N This represents the total number of patch features in the unordered feature bag.
[0019] As a preferred embodiment of the whole-slice image classification processing method based on topological latent space reconstruction, the steps for obtaining the encoded grid features are as follows:
[0020] The feature grid is serialized, and standard Mamba coding blocks with shared weights are applied to the original sequence and the spatially flipped sequence respectively to obtain the forward feature sequence and the backward feature sequence.
[0021] The forward feature sequence and the backward feature sequence are concatenated and then input into a linear layer for feature fusion to obtain preliminary encoded features.
[0022] By using the U-Net architecture's stepwise max pooling and bilinear interpolation, multi-scale feature fusion is performed on the initial encoded features to output the encoded grid features.
[0023] As a preferred approach for whole-slice image classification based on topological latent space reconstruction, the attention calculation formula is as follows during dense interaction aggregation using a fully self-attention module:
[0024] ;
[0025] ;
[0026] ;
[0027] ;
[0028] In the formula, For query matrix; The key matrix; It is a value matrix; The input sequence for the fully self-attention module; , , It is a learnable linear projection matrix; The feature dimension of the key vector; This is the attention mask matrix.
[0029] As a preferred embodiment of the full-slice image classification processing method based on topological latent space reconstruction, during the end-to-end training of the slice-level feature representation based on the standard cross-entropy loss function, the expression of the standard cross-entropy loss function is as follows:
[0030] ;
[0031] In the formula, This is the loss value; This refers to the training batch size; The total number of categories for the classification task; For the first m The sample at the th c One-hot encoding of the true label on the class; For the model to predict the first m The sample belongs to the first c The probability value of the class.
[0032] The present invention also provides a whole-slice image classification processing apparatus based on topological latent space reconstruction, which employs the above-mentioned whole-slice image classification processing method based on topological latent space reconstruction, including:
[0033] The unordered feature bag acquisition unit is used to extract features from the full slice image to be classified using a pre-trained feature extractor to obtain an unordered feature bag.
[0034] The key feature subset acquisition unit is used to obtain the saliency score of each patch feature by calculating the saliency score of each patch feature in the unordered feature package; based on the saliency score, the unit selects the highest-scoring feature using a deterministic Top-K sampling strategy. K From the features of each map patch, a subset of key features is obtained;
[0035] A key feature subset topology reconstruction unit is used to map the disordered key feature subset into a regular feature mesh by performing topology reconstruction on the key feature subset; if the number of tiles in the key feature subset is insufficient... K Then, the zero-filling strategy is used to complete the potential spatial topology reconstruction;
[0036] The encoded mesh feature output unit is used to input the feature mesh into a bidirectional Mamba encoder embedded in the U-Net architecture; by combining bidirectional Mamba encoding with multi-scale feature fusion, the isotropic global spatial dependency of the feature mesh is captured, and the encoded mesh features are output.
[0037] The slice-level feature representation acquisition unit is used to flatten the encoded grid features and attach a learnable classification token to the flattened feature sequence to obtain the input sequence of the fully self-attention module; the input sequence is then input into the fully self-attention module for dense interactive aggregation to obtain the slice-level feature representation.
[0038] The slice-level feature representation training unit is used to perform end-to-end training on the slice-level feature representation based on the standard cross-entropy loss function, complete the classification inference of the whole slice image, and output the classification result of the whole slice image.
[0039] As a preferred embodiment of the whole-slice image classification processing device based on topological latent space reconstruction, in the key feature subset acquisition unit, during the calculation of the saliency score for each patch feature in the unordered feature package, the formula for calculating the saliency score is as follows:
[0040] ;
[0041] In the formula, For the first i The saliency score of each patch feature; For the unordered feature bag, the first i Each patch feature; , These are learnable parameters; , This is the bias term for a lightweight scoring network.
[0042] As a preferred embodiment of a full-slice image classification processing device based on topological latent space reconstruction, the key feature subset acquisition unit selects the highest-scoring feature subset using the deterministic Top-K sampling strategy. K During the feature extraction process for each patch, the deterministic Top-K sampling strategy selects the patch using an index set. K Each map feature; the expression for the index set is:
[0043] ;
[0044] In the formula, For index set; A subset of the tile feature indexes for an unordered feature package; N This represents the total number of patch features in the unordered feature bag.
[0045] As a preferred embodiment of a full-slice image classification processing device based on topological latent space reconstruction, the step of obtaining the encoded grid features in the encoded grid feature output unit is as follows:
[0046] The feature grid is serialized, and standard Mamba coding blocks with shared weights are applied to the original sequence and the spatially flipped sequence respectively to obtain the forward feature sequence and the backward feature sequence.
[0047] The forward feature sequence and the backward feature sequence are concatenated and then input into a linear layer for feature fusion to obtain preliminary encoded features.
[0048] By using the U-Net architecture's stepwise max pooling and bilinear interpolation, multi-scale feature fusion is performed on the initial encoded features to output the encoded grid features.
[0049] As a preferred embodiment of a full-slice image classification processing device based on topological latent space reconstruction, in the slice-level feature representation acquisition unit, during the dense interactive aggregation process through the fully self-attention module, the attention calculation formula is as follows:
[0050] ;
[0051] ;
[0052] ;
[0053] ;
[0054] In the formula, For query matrix; The key matrix; It is a value matrix; The input sequence for the fully self-attention module; , , It is a learnable linear projection matrix; The feature dimension of the key vector; This is the attention mask matrix.
[0055] As a preferred embodiment of the full-slice image classification processing device based on topological latent space reconstruction, in the slice-level feature representation training unit, during the end-to-end training of the slice-level feature representation based on the standard cross-entropy loss function, the expression of the standard cross-entropy loss function is:
[0056] ;
[0057] In the formula, This is the loss value; This refers to the training batch size; The total number of categories for the classification task; For the first m The sample at the th cOne-hot encoding of the true label on the class; For the model to predict the first m The sample belongs to the first c The probability value of the class.
[0058] The present invention has the following advantages: First, it physically removes a large amount of redundant background noise, constructs a high signal-to-noise ratio feature grid, significantly reduces the interference of non-diagnostic information on the model, and strengthens the expression of pathological core signals.
[0059] Second, it breaks through the field-of-view limitations of local convolution, achieves isotropic global spatial dependency capture of whole-slice images, and fully explores long-range correlation information in the tumor microenvironment.
[0060] Third, it enables dense interaction aggregation of many-to-many features, which can effectively infer the co-occurrence relationship of different histological patterns and improve the model's ability to represent complex pathological features.
[0061] Fourth, the deterministic Top-K selection strategy makes the model inference results more stable, exhibits good robustness on different datasets, and has a wider range of applications.
[0062] Fifth, the model does not rely on complex auxiliary loss functions; it can achieve stable end-to-end training using only standard cross-entropy loss, simplifying the training process while ensuring training effectiveness.
[0063] Sixth, the unordered multi-instance learning problem is reconstructed into a structured visual task, which is adapted to mature computer vision operators, thereby improving the overall computational efficiency and adaptability of the model. Attached Figure Description
[0064] To more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings in the following description are merely exemplary, and those skilled in the art can derive other embodiments based on the provided drawings without creative effort.
[0065] The structures, proportions, sizes, etc. illustrated in this specification are only for the purpose of assisting those skilled in the art in understanding and reading the content disclosed herein, and are not intended to limit the conditions under which the present invention can be implemented. Therefore, they have no substantial technical significance. Any modifications to the structure, changes in the proportions, or adjustments to the size, without affecting the effects and objectives that the present invention can produce, should still fall within the scope of the technical content disclosed in the present invention.
[0066] Figure 1 This is a flowchart illustrating the whole-slice image classification processing method based on topological latent space reconstruction provided in Embodiment 1 of the present invention.
[0067] Figure 2 This is a schematic diagram of the specific process architecture of the full-slice image classification processing method based on topological latent space reconstruction provided in Embodiment 1 of the present invention.
[0068] Figure 3 This is a schematic diagram of the architecture of the whole-slice image classification processing device based on topological latent space reconstruction provided in Embodiment 2 of the present invention. Detailed Implementation
[0069] The following specific embodiments illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention. Example 1
[0070] See Figure 1 and Figure 2 Embodiment 1 of the present invention provides a whole-slice image classification processing method based on topological latent space reconstruction, including the following steps:
[0071] S1. Use a pre-trained feature extractor to extract features from the full slice image to be classified, and obtain an unordered feature package;
[0072] S2. A saliency score is obtained by calculating the saliency score of each patch feature in the unordered feature package; based on the saliency score, the highest-scoring feature is selected using a deterministic Top-K sampling strategy. K From the features of each map patch, a subset of key features is obtained;
[0073] S3. By performing topological reconstruction on the key feature subset, the disordered key feature subset is mapped into a regular feature grid; if the number of tiles in the key feature subset is insufficient... K Then, the zero-filling strategy is used to complete the potential spatial topology reconstruction;
[0074] S4. Input the feature mesh into a bidirectional Mamba encoder embedded in the U-Net architecture; capture the isotropic global spatial dependency of the feature mesh by bidirectional Mamba encoding and multi-scale feature fusion, and output the encoded mesh features;
[0075] S5. Flatten the encoded grid features and attach a learnable classification token to the flattened feature sequence to obtain the input sequence of the fully self-attention module; input the input sequence into the fully self-attention module for dense interactive aggregation to obtain slice-level feature representation;
[0076] S6. Based on the standard cross-entropy loss function, perform end-to-end training on the slice-level feature representation to complete the classification inference of the whole slice image and output the classification result of the whole slice image.
[0077] In this embodiment, in step S1, a pre-trained feature extractor is used to extract features from the full slice image to be classified, thereby obtaining an unordered feature package.
[0078] Specifically, the whole-slice image to be classified pathologically is first divided into several image patches according to a preset size. Then, all image patches are input into a pre-trained feature extractor. The feature extractor performs deep feature mining and mapping on each image patch, outputting a high-dimensional feature vector for each patch. The feature vectors of all patches together constitute an unordered feature package, which is a set of features without spatial arrangement rules, denoted as . ,in The number of tiles. This serves as the feature dimension, providing the original feature foundation for subsequent feature selection and reconstruction.
[0079] In this embodiment, in step S2, a saliency score is obtained by calculating the saliency score of each feature in the unordered feature package; based on the saliency score, the feature with the highest score is selected using a deterministic Top-K sampling strategy. K From the features of each image patch, a subset of key features is obtained.
[0080] Specifically, firstly, a lightweight scoring network is used to calculate the saliency score of each feature vector in the unordered feature package. Through a combination of linear transformation and activation function, the saliency score corresponding to each feature is obtained, which is used to quantify the pathological discrimination value of the feature. Then, based on the calculated saliency scores, a deterministic Top-K sampling strategy is executed, abandoning the traditional probabilistic sampling method and directly selecting the top-ranked features. K The map features are extracted by indexing and then integrated into a subset of key features. This process enables the initial removal of non-diagnostic background features, reducing noise interference for subsequent topology reconstruction.
[0081] The formula for calculating the significance score is as follows:
[0082] ;
[0083] In the formula, For the first i The saliency score of each patch feature; For the unordered feature bag, the first i Each patch feature; , These are learnable parameters; , This is the bias term for a lightweight scoring network.
[0084] In this embodiment, the highest-scoring sample is selected using the deterministic Top-K sampling strategy. K During the feature extraction process for each patch, the deterministic Top-K sampling strategy selects the patch using an index set. K Each map feature; the expression for the index set is:
[0085] ;
[0086] In the formula, For index set; A subset of the tile feature indexes for an unordered feature package; N This represents the total number of patch features in the unordered feature bag.
[0087] In this embodiment, in step S3, the disordered key feature subset is mapped to a regular feature grid by performing topological reconstruction on the key feature subset; if the number of tiles in the key feature subset is insufficient... K Then, a zero-filling strategy is used to complete the potential spatial topology reconstruction.
[0088] Specifically, firstly, based on the preset grid capacity K, the one-dimensional arrangement of key feature subsets is mapped to a two-dimensional regular feature grid, the dimension of which is... H × W × D ,in This transforms a loosely ordered set of features into a compact, regular grid structure; if the number of tiles in the key feature subset after Top-K filtering does not reach the preset threshold... K The value is filled with a zero-filling strategy to fill in the missing positions of the feature grid, ensuring the regularity of the feature grid structure and completing the topological reconstruction of the entire latent space. This transforms the original multi-instance learning problem into a high signal-to-noise ratio structured computer vision task.
[0089] In this embodiment, in step S4, the feature mesh is input into a bidirectional Mamba encoder embedded in the U-Net architecture; through bidirectional Mamba encoding and multi-scale feature fusion, the isotropic global spatial dependency of the feature mesh is captured, and the encoded mesh features are output.
[0090] Specifically, the reconstructed regular feature grid is first input into a bidirectional Mamba encoder, which embeds a bidirectional Mamba module into the classic U-Net architecture, combining long sequence modeling and multi-scale feature processing capabilities. The specific processing steps are as follows:
[0091] T1. Serialize the feature grid and apply standard Mamba coding blocks with shared weights to the original sequence and the spatially flipped sequence respectively to obtain the forward feature sequence and the backward feature sequence.
[0092] Specifically, the core of the Mamba model lies in its ability to discretize the state equations of a continuous system, thereby achieving linear complexity. For processing long sequences, the discretized state update equation is defined as:
[0093] ;
[0094] ;
[0095] In the formula, This represents the current time step (sequence position). For a moment Input features; This is the hidden state of the system, responsible for remembering historical information; For output features; and To pass through time scale parameters For continuous parameter state evolution matrix and input projection matrix The parameter matrix obtained after zero-order preserved discretization; To output the projection matrix;
[0096] However, standard Mamba is causal, meaning the current state depends only on historical information, which contradicts the isotropic nature of image data. To overcome this directional limitation and achieve panoramic modeling, this invention proposes a bidirectional Mamba module (Bi-SSM). The feature mesh is serialized, and SSM is applied to both the original sequence and the spatially flipped sequence:
[0097] ;
[0098] ;
[0099] In the formula, It is a forward feature sequence; It is a backward feature sequence; The input is a sequence of reconstructed feature grids; Standard Mamba encoded blocks with shared weights; This is a spatial flipping operation along the sequence dimension;
[0100] T2. After concatenating the forward feature sequence and the backward feature sequence, input them into a linear layer for feature fusion to obtain preliminary encoded features;
[0101] T3. Through stepwise max pooling and bilinear interpolation of the U-Net architecture, multi-scale feature fusion is performed on the preliminary encoded features to output the encoded grid features.
[0102] Specifically, the encoded grid features are:
[0103] ;
[0104] In the formula, This refers to the encoded grid features.
[0105] In this embodiment, in step S5, the encoded grid features are flattened and a learnable classification token is attached to the flattened feature sequence to obtain the input sequence of the fully self-attention module; the input sequence is input to the fully self-attention module for dense interactive aggregation to obtain slice-level feature representation.
[0106] Specifically, in obtaining mesh features that include global context Furthermore, existing prototype query methods often suffer from information bottlenecks due to their "one-to-many" compression mechanism, making it difficult to handle highly heterogeneous tumor patterns. Therefore, this invention employs a self-attention mechanism in the aggregation stage to achieve dense interaction.
[0107] Flatten the grid features and append a learnable classification token (CLS Token). The input sequence is denoted as... The self-attention mechanism achieves co-occurrence inference by calculating the correlation matrix among all key instances:
[0108] The formula for calculating attention is:
[0109] ;
[0110] ;
[0111] ;
[0112] ;
[0113] In the formula, For query matrix; The key matrix; It is a value matrix; The input sequence for the fully self-attention module; , , It is a learnable linear projection matrix; The feature dimension of the key vector; This is the attention mask matrix.
[0114] In this embodiment, in step S6, the slice-level feature representation is trained end-to-end based on the standard cross-entropy loss function to complete the classification inference of the whole slice image and output the classification result of the whole slice image.
[0115] Specifically, thanks to the high signal-to-noise ratio input from Top-K reconstruction and the strong context modeling capability of Bi-Mamba, TopoMIL can achieve stable convergence without relying on complex auxiliary loss functions. It directly uses standard cross-entropy loss for end-to-end training.
[0116] ;
[0117] In the formula, This is the loss value; This refers to the training batch size; The total number of categories for the classification task; For the first m The sample at the th c One-hot encoding of the true label on the class; For the model to predict the first m The sample belongs to the first c The probability value of the class.
[0118] In one possible implementation, a verification example is provided as follows:
[0119] The method of this invention is defined as TopoMIL. To verify the superiority of TopoMIL, this invention is compared with the current mainstream Multiple Instance Learning (MIL) methods (DSMIL, RankMix), including the state-of-the-art Transformer-based method (TransMIL, FR-MIL). Table 1 shows the quantitative evaluation results of each model on the test set.
[0120] Table 1. Comparison results of different models on the whole slide image dataset of colorectal cancer pathology:
[0121] .
[0122] As shown in Table 1, TopoMIL achieved best or near-best performance across all evaluation metrics. In particular, compared to the baseline model FR-MIL, TopoMIL improved the area under the ROC curve (AUC) by 16.08% and the accuracy (ACC) by 16.29%. This directly verifies the core assumption of this invention: that physically removing background noise through "topology reconstruction" is more effective than FR-MIL's "feature distribution calibration." While FR-MIL suppressed some noise, the low signal-to-noise ratio caused by the full Bag input still limited the classifier's discrimination boundary. TopoMIL, by constructing a pure feature grid, fundamentally solves this problem.
[0123] To further verify the effectiveness and robustness of the TopoMIL architecture in general multi-instance learning tasks, this embodiment extends the experiments to five widely used classic MIL benchmark datasets. The detailed results are shown in Table 2.
[0124] Table 2. Comparison results of different models on multi-instance learning benchmark datasets:
[0125] .
[0126] Experimental data show that TopoMIL not only achieves state-of-the-art classification accuracy on challenging tasks such as MUSK2 and ELEPHANT, but more importantly, its standard deviation in multiple independent repeated experiments is significantly lower than that of the baseline model. This phenomenon profoundly reveals the difference in stability between different aggregation paradigms: traditional methods are constrained by the probability uncertainty of soft attention mechanisms or random sampling, leading to drastic fluctuations in inference results; while thanks to the deterministic Top-K selection strategy proposed in this invention, TopoMIL can consistently and stably lock the most discriminative key instances in different rounds of experiments, thus achieving excellent robustness while maintaining high accuracy, proving that the architecture has general noise-resistant inference capabilities.
[0127] The application scenarios of this invention are as follows: In the scenario of tumor pathology diagnosis, this invention can efficiently classify whole-slice images, assist pathologists in quickly identifying lesion areas, and improve the overall efficiency of pathological analysis.
[0128] In the context of cancer prognostic assessment, this invention can perform feature analysis and classification on whole-section images of tumor tissue, providing a reliable image analysis basis for clinical development of subsequent treatment plans.
[0129] In digital pathology big data analysis scenarios, this invention can automatically classify and process large batches of whole-slice images, enabling rapid screening and classification of pathological images and reducing the cost of manual analysis.
[0130] In pathology teaching and research scenarios, this invention can accurately classify and organize various types of whole pathological slide images, providing data support for the selection of pathology-related teaching cases and the mining of research data.
[0131] In remote pathological diagnosis scenarios, this invention can automatically classify and analyze whole-slice images transmitted from different locations, providing standardized image analysis results for remote pathological diagnosis and assisting in the diagnostic work.
[0132] In the context of pathological image annotation assistance, this invention can first classify and filter whole-slice images to locate regions with diagnostic value, thereby reducing the amount of ineffective work for subsequent manual fine annotation.
[0133] It should be noted that the method of this disclosure embodiment can be executed by a single device, such as a computer or server. The method of this embodiment can also be applied to a distributed scenario, where multiple devices cooperate to complete the task. In such a distributed scenario, one of these devices may execute only one or more steps of the method of this disclosure embodiment, and the multiple devices will interact with each other to complete the method described.
[0134] It should be noted that the above description describes some embodiments of this disclosure. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in a different order than that shown in the above embodiments and still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require a specific or sequential order to achieve the desired result. In some embodiments, multitasking and parallel processing are also possible or may be advantageous. Example 2
[0135] See Figure 3 Embodiment 2 of the present invention also provides a whole-slice image classification processing device based on topological latent space reconstruction, comprising:
[0136] The unordered feature bag acquisition unit 001 is used to extract features from the full slice image to be classified using a pre-trained feature extractor to obtain an unordered feature bag.
[0137] The key feature subset acquisition unit 002 is used to obtain the saliency score of each patch feature by calculating the saliency score of each patch feature in the unordered feature package; based on the saliency score, the highest-scoring feature is selected using a deterministic Top-K sampling strategy. K From the features of each map patch, a subset of key features is obtained;
[0138] The key feature subset topology reconstruction unit 003 is used to map the disordered key feature subset into a regular feature mesh by performing topology reconstruction on the key feature subset; if the number of tiles in the key feature subset is insufficient... K Then, the zero-filling strategy is used to complete the potential spatial topology reconstruction;
[0139] The encoded mesh feature output unit 004 is used to input the feature mesh into a bidirectional Mamba encoder embedded in the U-Net architecture; by bidirectional Mamba encoding and multi-scale feature fusion, the isotropic global spatial dependency of the feature mesh is captured, and the encoded mesh features are output.
[0140] The slice-level feature representation acquisition unit 005 is used to flatten the encoded grid features and attach a learnable classification token to the flattened feature sequence to obtain the input sequence of the fully self-attention module; and input the input sequence into the fully self-attention module for dense interactive aggregation to obtain the slice-level feature representation.
[0141] The slice-level feature representation training unit 006 is used to perform end-to-end training on the slice-level feature representation based on the standard cross-entropy loss function, complete the classification inference of the whole slice image, and output the classification result of the whole slice image.
[0142] In this embodiment, in the key feature subset acquisition unit 002, during the saliency score calculation for each patch feature in the unordered feature package, the formula for calculating the saliency score is as follows:
[0143] ;
[0144] In the formula, For the first i The saliency score of each patch feature; For the unordered feature bag, the first i Each patch feature; , These are learnable parameters; , This is the bias term for a lightweight scoring network.
[0145] In this embodiment, the key feature subset acquisition unit 002 selects the highest-scoring feature subset using the deterministic Top-K sampling strategy. K During the feature extraction process for each patch, the deterministic Top-K sampling strategy selects the patch using an index set. K Each map feature; the expression for the index set is:
[0146] ;
[0147] In the formula, For index set; A subset of the tile feature indexes for an unordered feature package; N This represents the total number of patch features in the unordered feature bag.
[0148] In this embodiment, the step of obtaining the encoded grid features in the encoded grid feature output unit 004 is as follows:
[0149] The feature grid is serialized, and standard Mamba coding blocks with shared weights are applied to the original sequence and the spatially flipped sequence respectively to obtain the forward feature sequence and the backward feature sequence.
[0150] The forward feature sequence and the backward feature sequence are concatenated and then input into a linear layer for feature fusion to obtain preliminary encoded features.
[0151] By using the U-Net architecture's stepwise max pooling and bilinear interpolation, multi-scale feature fusion is performed on the initial encoded features to output the encoded grid features.
[0152] In this embodiment, in the slice-level feature representation acquisition unit 005, during the dense interaction aggregation process through the fully self-attention module, the attention calculation formula is as follows:
[0153] ;
[0154] ;
[0155] ;
[0156] ;
[0157] In the formula, For query matrix; The key matrix; It is a value matrix; The input sequence for the fully self-attention module; , , It is a learnable linear projection matrix; The feature dimension of the key vector; This is the attention mask matrix.
[0158] In this embodiment, in the slice-level feature representation training unit 006, during the end-to-end training of the slice-level feature representation based on the standard cross-entropy loss function, the expression of the standard cross-entropy loss function is:
[0159] ;
[0160] In the formula, This is the loss value; This refers to the training batch size; The total number of categories for the classification task; For the first m The sample at the th c One-hot encoding of the true label on the class; For the model to predict the first m The sample belongs to the first c The probability value of the class.
[0161] It should be noted that the information interaction and execution process between the modules of the above system are based on the same concept as the method embodiment in Embodiment 1 of this application, and the resulting technical effects are the same as those in the method embodiment of this application. For details, please refer to the description in the method embodiment shown above in this application, and it will not be repeated here. Example 3
[0162] Embodiment 3 of the present invention provides a non-transitory computer-readable storage medium storing program code for a full-slice image classification processing method based on topological latent space reconstruction. The program code includes instructions for executing the full-slice image classification processing method based on topological latent space reconstruction of Embodiment 1 or any possible implementation thereof.
[0163] Computer-readable storage media can be any available medium that a computer can access, or a data storage device such as a server or data center that integrates one or more available media. The available medium can be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., DVDs), or semiconductor media (e.g., solid-state drives (SSDs)). Example 4
[0164] Embodiment 4 of the present invention provides an electronic device, including: a memory and a processor; the processor and the memory communicate with each other through a bus; the memory stores program instructions that can be executed by the processor, and the processor can execute the full-slice image classification processing method based on topological latent space reconstruction according to Embodiment 1 or any possible implementation thereof by calling the program instructions.
[0165] Specifically, a processor can be implemented in hardware or software. When implemented in hardware, the processor can be a logic circuit, an integrated circuit, etc. When implemented in software, the processor can be a general-purpose processor that reads software code stored in memory. This memory can be integrated into the processor or located outside the processor and exist independently.
[0166] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of the present invention are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable system. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means.
[0167] It is obvious to those skilled in the art that the modules or steps of the present invention described above can be implemented using general-purpose computing systems. They can be centralized on a single computing system or distributed across a network of multiple computing systems. Optionally, they can be implemented using program code executable by a computing system, thereby storing them in a storage system for execution by the computing system. In some cases, the steps shown or described can be performed in a different order than those presented herein, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. Thus, the present invention is not limited to any particular combination of hardware and software.
[0168] Although the present invention has been described in detail above with general descriptions and specific embodiments, modifications or improvements can be made to it, which will be obvious to those skilled in the art. Therefore, all such modifications or improvements made without departing from the spirit of the present invention fall within the scope of protection claimed by the present invention.
Claims
1. A whole-slice image classification processing method based on topological latent space reconstruction, characterized in that, include: The unordered feature package is obtained by extracting features from the whole slice image to be classified using a pre-trained feature extractor. The whole slice image to be classified for pathology is divided into several image patches according to a preset size, and then all image patches are input into a pre-trained feature extractor. The saliency score of each patch feature in the unordered feature package is obtained by calculating the saliency score; based on the saliency score, the highest-scoring feature is selected using a deterministic Top-K sampling strategy. K From the features of each map patch, a subset of key features is obtained; By performing topological reconstruction on the key feature subset, the disordered key feature subset is mapped into a regular feature grid; if the number of tiles in the key feature subset is insufficient... K Then, the zero-filling strategy is used to complete the potential spatial topology reconstruction; The feature mesh is input into a bidirectional Mamba encoder embedded in the U-Net architecture; the isotropic global spatial dependency of the feature mesh is captured by bidirectional Mamba encoding and multi-scale feature fusion, and the encoded mesh features are output. By flattening the encoded grid features and attaching a learnable classification token to the flattened feature sequence, the input sequence of the fully self-attention module is obtained; the input sequence is then fed into the fully self-attention module for dense interactive aggregation to obtain slice-level feature representations. Based on the standard cross-entropy loss function, the slice-level feature representation is trained end-to-end to complete the classification inference of the whole slice image and output the classification result of the whole slice image. The steps for obtaining the encoded grid features are as follows: The feature grid is serialized, and standard Mamba coding blocks with shared weights are applied to the original sequence and the spatially flipped sequence respectively to obtain the forward feature sequence and the backward feature sequence. The forward feature sequence and the backward feature sequence are concatenated and then input into a linear layer for feature fusion to obtain preliminary encoded features. By using the U-Net architecture's stepwise max pooling and bilinear interpolation, multi-scale feature fusion is performed on the initial encoded features to output the encoded grid features.
2. The whole-slice image classification processing method based on topological latent space reconstruction according to claim 1, characterized in that, In the process of calculating the saliency score for each tile feature in the unordered feature package, the formula for calculating the saliency score is as follows: ； In the formula, For the first i The saliency score of each patch feature; For the unordered feature bag, the first i Each patch feature; , These are learnable parameters; , This is the bias term for a lightweight scoring network.
3. The whole-slice image classification processing method based on topological latent space reconstruction according to claim 2, characterized in that, The highest-scoring sample is selected using the deterministic Top-K sampling strategy. K During the feature extraction process for each patch, the deterministic Top-K sampling strategy selects the patch using an index set. K Each map feature; the expression for the index set is: ； In the formula, For index set; A subset of the tile feature indexes for an unordered feature package; N This represents the total number of patch features in the unordered feature bag.
4. The whole-slice image classification processing method based on topological latent space reconstruction according to claim 3, characterized in that, During the dense interaction aggregation process using the fully self-attention module, the attention calculation formula is as follows: ； ； ； ； In the formula, For query matrix; The key matrix; It is a value matrix; The input sequence for the fully self-attention module; , , It is a learnable linear projection matrix; The feature dimension of the key vector; This is the attention mask matrix.
5. The whole-slice image classification processing method based on topological latent space reconstruction according to claim 4, characterized in that, In the process of end-to-end training of the slice-level feature representation based on the standard cross-entropy loss function, the expression of the standard cross-entropy loss function is: ； In the formula, This is the loss value; This refers to the training batch size; The total number of categories for the classification task; For the first m The sample at the th c One-hot encoding of the true label on the class; For the model to predict the first m The sample belongs to the first c The probability value of the class.
6. A whole-slice image classification processing apparatus based on topological latent space reconstruction, employing the whole-slice image classification processing method based on topological latent space reconstruction as described in any one of claims 1-5, characterized in that, include: The unordered feature bag acquisition unit is used to extract features from the full slice image to be classified using a pre-trained feature extractor to obtain an unordered feature bag. The key feature subset acquisition unit is used to obtain the saliency score of each patch feature by calculating the saliency score of each patch feature in the unordered feature package; based on the saliency score, the unit selects the highest-scoring feature using a deterministic Top-K sampling strategy. K From the features of each map patch, a subset of key features is obtained; A key feature subset topology reconstruction unit is used to map the disordered key feature subset into a regular feature mesh by performing topology reconstruction on the key feature subset; if the number of tiles in the key feature subset is insufficient... K Then, the zero-filling strategy is used to complete the potential spatial topology reconstruction; The encoded mesh feature output unit is used to input the feature mesh into a bidirectional Mamba encoder embedded in the U-Net architecture; by combining bidirectional Mamba encoding with multi-scale feature fusion, the isotropic global spatial dependency of the feature mesh is captured, and the encoded mesh features are output. The slice-level feature representation acquisition unit is used to flatten the encoded grid features and attach a learnable classification token to the flattened feature sequence to obtain the input sequence of the fully self-attention module; the input sequence is then input into the fully self-attention module for dense interactive aggregation to obtain the slice-level feature representation. The slice-level feature representation training unit is used to perform end-to-end training on the slice-level feature representation based on the standard cross-entropy loss function, complete the classification inference of the whole slice image, and output the classification result of the whole slice image.
7. The whole-slice image classification processing device based on topological latent space reconstruction according to claim 6, characterized in that, In the key feature subset acquisition unit, during the saliency score calculation for each tile feature in the unordered feature package, the formula for calculating the saliency score is as follows: ； In the formula, For the first i The saliency score of each patch feature; For the unordered feature bag, the first i Each patch feature; , These are learnable parameters; , This is the bias term for a lightweight scoring network.
8. The whole-slice image classification processing device based on topological latent space reconstruction according to claim 7, characterized in that, In the key feature subset acquisition unit, the highest-scoring feature is selected using the deterministic Top-K sampling strategy. K During the feature extraction process for each patch, the deterministic Top-K sampling strategy selects the patch using an index set. K Each map feature; the expression for the index set is: ； In the formula, For index set; A subset of the tile feature indexes for an unordered feature package; N This represents the total number of patch features in the unordered feature bag.
9. The whole-slice image classification processing device based on topological latent space reconstruction according to claim 8, characterized in that, In the encoded grid feature output unit, the steps for obtaining the encoded grid features are as follows: The feature grid is serialized, and standard Mamba coding blocks with shared weights are applied to the original sequence and the spatially flipped sequence respectively to obtain the forward feature sequence and the backward feature sequence. The forward feature sequence and the backward feature sequence are concatenated and then input into a linear layer for feature fusion to obtain preliminary encoded features. By using the U-Net architecture's stepwise max pooling and bilinear interpolation, multi-scale feature fusion is performed on the initial encoded features to output the encoded grid features; In the slice-level feature representation acquisition unit, during the dense interaction aggregation process through the fully self-attention module, the attention calculation formula is as follows: ； ； ； ； In the formula, For query matrix; The key matrix; It is a value matrix; The input sequence for the fully self-attention module; , , It is a learnable linear projection matrix; The feature dimension of the key vector; For attention mask matrix; In the slice-level feature representation training unit, during the end-to-end training of the slice-level feature representation based on the standard cross-entropy loss function, the expression of the standard cross-entropy loss function is: ； In the formula, This is the loss value; This refers to the training batch size; The total number of categories for the classification task; For the first m The sample at the th c One-hot encoding of the true label on the class; For the model to predict the first m The sample belongs to the first c The probability value of the class.