Pathological image memory efficient high-resolution feature backtracking method based on gated token pruning

Abstract

This application discloses a method for efficient and high-resolution feature backtracking of pathological images based on gated token pruning, belonging to the field of image recognition technology. The method includes: downsampling the pathological image; inputting the downsampling result into an image encoder; extracting a global feature sequence and the spatial location corresponding to each global feature; calculating the saliency score of the global features in the global feature sequence; selecting a preset number of feature subsets from the global feature sequence based on the saliency score; extracting local image patches corresponding to the feature subsets in the pathological image according to their spatial location; encoding the local image patches; mapping the encoding result to the embedding space corresponding to the global feature sequence to obtain local features; weighted fusing the local features with the corresponding global features to obtain fused features; and determining the pathological classification result based on the fused features.

Description

Technical Field

[0001] This application relates to the field of image recognition technology, and in particular to a method for efficient and high-resolution feature backtracking of pathological image memory based on gated token pruning. Background Technology

[0002] In related technologies, the identification methods for pathological full-view slice images usually first perform global downsampling on the original high-resolution image, extract global features at low resolution for preliminary analysis, and then select candidate regions from the global features through saliency heatmaps or attention mechanisms. Then, the original high-resolution image blocks corresponding to the candidate regions are encoded, and the high-resolution local features and low-resolution global features are spliced ​​and fused to complete the classification.

[0003] However, the process of selecting candidate regions from global features typically relies on post-training threshold truncation or soft filtering based on attention scores. In these cases, the number of selected regions cannot be constrained to a fixed value during model forward propagation, making the number of high-resolution image patches recalled in a single inference uncontrollable. In this situation, there is a lack of structural coupling between computational budget and diagnostic accuracy, making stable operation difficult to guarantee in real-world deployment environments with limited computing power.

[0004] The above content is only used to help understand the technical solution of this application and does not represent an admission that the above content is prior art. Summary of the Invention

[0005] The main purpose of this application is to provide a method for efficient and high-resolution feature backtracking of pathological image memory based on gated token pruning, which aims to solve the technical problem of unstable recognition effect of pathological full-view slice images.

[0006] To achieve the above objectives, this application provides a method for efficient and high-resolution feature backtracking of pathological image memory based on gated token pruning. The method includes the following steps: The pathological image is downsampled, and the downsampling result is input into the image encoder to extract the global feature sequence and the spatial location corresponding to each global feature; Calculate the saliency score of the global features in the global feature sequence, and select a preset number of feature subsets from the global feature sequence based on the saliency score; Based on the spatial location, extract local image patches corresponding to the feature subset from the pathological image; The local image patch is encoded, and the encoding result is mapped to the embedding space corresponding to the global feature sequence to obtain local features; The local features are weighted and fused with the corresponding global features to obtain fused features, and the pathological classification result is determined based on the fused features.

[0007] In one embodiment, the step of calculating the saliency score of the global features in the global feature sequence and selecting a predetermined number of feature subsets from the global feature sequence based on the saliency score includes: Based on the correlation between the semantic information of each global feature and the pathological diagnosis task, the diagnostic relevance score of each global feature is calculated. Based on the spatial adjacency relationship between each global feature and its neighboring global features, the spatial aggregation degree score of each global feature is calculated. The significance score is obtained by weighted summation of the diagnostic relevance score and the spatial aggregation score. Based on the saliency score, a preset number of feature subsets are selected from the global feature sequence.

[0008] In one embodiment, the step of weighted summing of the diagnostic relevance score and the spatial aggregation score to obtain the significance score includes: The diagnostic relevance score is multiplied by a first preset weight to obtain a first weighted score; Multiply the spatial aggregation degree score by the second preset weight to obtain the second weighted score; The significance score is obtained by adding the first weighted score and the second weighted score.

[0009] In one embodiment, the step of selecting a preset number of feature subsets from the global feature sequence based on the saliency score includes: By using a differentiable sparse selection layer, and based on Guboul soft maximum sampling, the selection probability corresponding to each global feature is generated; A hard selection mask is generated based on the selection probability, and the number of global features selected in the hard selection mask is equal to the preset number. The global features selected by the hard selection mask are determined as the feature subset.

[0010] In one embodiment, the step of encoding the local image patch and mapping the encoding result to the embedding space corresponding to the global feature sequence to obtain local features includes: Perform multi-layer residual convolution operations on each of the local image blocks to obtain convolutional features; The convolutional features are subjected to global mean pooling to obtain pooled features; By linearly projecting the pooling features, the pooling features are mapped to the embedding space corresponding to the global feature sequence, thus obtaining the local features.

[0011] In one embodiment, the step of weightedly fusing the local features with the corresponding global features to obtain fused features, and determining the pathological classification result based on the fused features, includes: The fusion features are pooled and aggregated to obtain the global representation vector corresponding to the pathological image; The global representation vector is input into a multi-label classification head, which then outputs predicted probabilities for each of the preset pathological categories. The predicted probability corresponding to each of the pathological categories is determined as the pathological classification result.

[0012] In one embodiment, after the step of determining the predicted probability corresponding to each of the pathological categories as the pathological classification result, the method further includes: Based on the class activation mapping heatmap, the coverage ratio of the feature subset to the lesion area is calculated to obtain the coverage loss; Based on the semantic consistency between the global features and the corresponding local features, calculate the cross-scale loss; Calculate the topology preservation loss based on the number of spatially connected components in the feature subset; The gating consistency loss is calculated based on the offset of the weights corresponding to the local features during the weighted fusion process and the variance of the fused features. The classification loss is calculated based on the pathological classification results and the true labels. The classification loss, the coverage loss, the cross-scale loss, the topology preservation loss and the gating consistency loss are weighted and summed to obtain the total loss. The network parameters are then optimized using the total loss.

[0013] In one embodiment, after the steps of calculating the classification loss based on the pathological classification results and the true labels, weighting and summing the classification loss, the coverage loss, the cross-scale loss, the topology preservation loss, and the gating consistency loss to obtain the total loss, and optimizing the network parameters with the total loss, the method further includes: Configure the selection method of the feature subset to a deterministic selection mode; The significance scores of each global feature are sorted in descending order of numerical value; Select the aforementioned preset number of global features from the sorting results to form the feature subset.

[0014] One or more technical solutions proposed in this application have at least the following technical effects: This application, after downsampling the pathological image and extracting the global feature sequence, selects a preset number of feature subsets from the global feature sequence based on the saliency score of each global feature. Local image patches corresponding to the feature subsets are extracted from the pathological image according to their spatial location and encoded. The encoded results are mapped to the embedding space corresponding to the global feature sequence to obtain local features. These local features are then weighted and fused with the corresponding global features to determine the pathological classification result. This ensures that the number of high-resolution local image patches recalled in each inference is constrained to the preset number, no longer fluctuating with changes in the input image. This predetermines the computation time and memory overhead of a single inference, enabling stable operation in clinical deployment environments with limited computing resources. Simultaneously, the selection of local image patches is still based on the saliency score, ensuring that the recalled image patches are concentrated in regions that substantially contribute to pathological classification, achieving controllable inference computation without sacrificing diagnostic accuracy. Attached Figure Description

[0015] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0016] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 This is a flowchart illustrating the first embodiment of the method for efficient and high-resolution feature backtracking of pathological image memory based on gated token pruning in this application. Figure 2 This is a schematic diagram of the pathological image feature recognition process involved in the first embodiment of the method for efficient and high-resolution feature backtracking of pathological image memory based on gated token pruning in this application. Figure 3 This is a schematic diagram of the PATH-Net five-module cascaded structure involved in the first embodiment of the method for efficient and high-resolution feature backtracking of pathological image memory based on gated token pruning in this application; Figure 4 This is a flowchart illustrating the second embodiment of the method for efficient and high-resolution feature backtracking of pathological image memory based on gated token pruning in this application. Figure 5 This is a flowchart illustrating the third embodiment of the method for efficient and high-resolution feature backtracking of pathological image memory based on gated token pruning in this application. Figure 6This is a schematic diagram of the hardware operating environment of the pathological image display memory high-efficiency and high-resolution feature backtracking device based on gated token pruning, which is involved in the embodiments of this application.

[0018] The realization of the purpose, functional features and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0019] It should be understood that the specific embodiments described herein are merely illustrative of this application and are not intended to limit this application.

[0020] To better understand the above technical solutions, the following will provide a detailed explanation of the technical solutions in conjunction with the accompanying drawings and specific implementation methods.

[0021] The main solution of this application embodiment is as follows: downsampling the pathological image, inputting the downsampling result into the image encoder, extracting the global feature sequence and the spatial location corresponding to each global feature, calculating the saliency score of the global features in the global feature sequence, selecting a preset number of feature subsets from the global feature sequence based on the saliency score, extracting local image patches corresponding to the feature subsets in the pathological image according to the spatial location, encoding the local image patches, mapping the encoding result to the embedding space corresponding to the global feature sequence to obtain local features, weighted fusing the local features with the corresponding global features to obtain fused features, and determining the pathological classification result based on the fused features.

[0022] Current methods for identifying pathological full-view slice images typically involve first downsampling the original high-resolution image globally to extract global features at low resolution for preliminary analysis. Candidate regions are then selected from these global features using saliency heatmaps or attention mechanisms. The corresponding original high-resolution image patches are then encoded, and the high-resolution local features are concatenated with the low-resolution global features to complete the classification. However, the process of selecting candidate regions from global features usually relies on post-training threshold truncation or soft selection based on attention scores. The number of selected regions cannot be constrained to a fixed value during model forward propagation, making the number of high-resolution image patches recalled in a single inference uncontrollable. In this situation, there is a lack of structural coupling between computational budget and diagnostic accuracy, making stable operation difficult to guarantee in real-world deployment environments with limited computing power.

[0023] This application, after downsampling pathological images and extracting global feature sequences, selects a predetermined number of feature subsets from the global feature sequences based on the saliency scores of each global feature. Local image patches corresponding to these feature subsets are extracted from the pathological images according to their spatial location and encoded. The encoded results are mapped to the embedding space corresponding to the global feature sequences to obtain local features. These local features are then weighted and fused with the corresponding global features to determine the pathological classification result. This ensures that the number of high-resolution local image patches recalled in each inference is constrained to the predetermined number, no longer fluctuating with changes in the input image. This predetermines the computation time and memory overhead of a single inference, enabling stable operation in clinical deployment environments with limited computing resources. Simultaneously, the selection of local image patches is still based on saliency scores, ensuring that the recalled image patches are concentrated in regions that substantially contribute to pathological classification, achieving controllable inference computation without sacrificing diagnostic accuracy.

[0024] To better understand the above technical solutions, exemplary embodiments of this application will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of this application are shown in the drawings, it should be understood that this application can be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided to enable a more thorough understanding of this application and to fully convey the scope of this application to those skilled in the art.

[0025] It should be noted that the executing entity in this embodiment can be a pathological image recognition system, or a computing service device with data processing, network communication, and program execution functions, such as a tablet computer, personal computer, or mobile phone, or an electronic device capable of achieving the above functions, or a device for efficient and high-resolution feature backtracking of pathological image memory based on gated token pruning, etc. This embodiment does not specifically limit it. The following uses a pathological image recognition system as an example to describe this embodiment and the following embodiments.

[0026] Based on this, this application provides a method for efficient and high-resolution feature backtracking of pathological image memory based on gated token pruning, referring to... Figure 1 , Figure 1 This is a flowchart illustrating the first embodiment of the method for efficient and high-resolution feature backtracking of pathological image memory based on gated token pruning in this application.

[0027] In this embodiment, the method for efficient and high-resolution feature backtracking of pathological images based on gated token pruning includes steps S10 to S50: Step S10: Downsample the pathological image, input the downsampling result into the image encoder, and extract the global feature sequence and the spatial location corresponding to each global feature; In this embodiment, the pathological image refers to an ultra-high resolution full-view slice image acquired through a digital pathology scanning device. Since full-view slice images typically have billions of pixels, processing them directly at their original resolution would consume extremely high computational resources. Therefore, the pathological image recognition system first performs a downsampling operation on the original pathological image, scaling it to a lower preset resolution. The downsampled image retains the overall tissue structure and regional distribution information, but the data volume is significantly reduced. The image encoder is a pre-trained neural network model that can transform the visual information in the input image into a sequence of multiple feature vectors, each corresponding to a spatial region in the downsampled image. The specific structure of the image encoder can be a visual transformer model, which captures the long-distance dependencies between regions in the image through a multi-head self-attention mechanism. Each output feature vector carries the tissue morphology semantic information of the corresponding region. While extracting features, the pathological image recognition system also records the coordinate information of each feature vector's position in the downsampled image, and establishes a correlation between the global features and the corresponding region in the original high-resolution image based on this coordinate information.

[0028] Specifically, the pathological image recognition system acquires a full-view pathological slide image file input by the user, scales the image to a preset target size using a bilinear interpolation algorithm, and generates a low-resolution thumbnail. The system divides this thumbnail into multiple fixed-size image blocks. Each image block is converted into a vector through linear projection, and then added to its corresponding positional code to form an input sequence. The input sequence is fed into an image encoder composed of multiple stacked transformer layers for layer-by-layer processing. Each transformer layer sequentially performs multi-head self-attention calculation and feedforward neural network transformation, ultimately outputting a global feature sequence of the same length as the input sequence. The dimension of each global feature is the embedding dimension of the image encoder. The system also maintains a list of spatial coordinates corresponding one-to-one with the global feature sequence, with each coordinate recording the spatial position of the corresponding image block in the thumbnail using row and column indices.

[0029] Optionally, the image encoder can employ a visual transformer model pre-trained on a general image dataset and fine-tuned on pathological image data. The pre-trained model already possesses basic visual feature extraction capabilities, while the fine-tuning process adapts the model to the texture and color features of pathological tissue images, resulting in a global feature sequence with stronger pathological semantic discrimination capabilities.

[0030] Optionally, the downsampling operation can employ an adaptive downsampling strategy. The system dynamically determines the downsampling factor based on the resolution of the original pathological image and the available memory capacity of the target processing device, ensuring that the downsampled image retains sufficient tissue structure information while meeting the storage constraints of the processing device.

[0031] It should be noted that after the image encoder extracts the global feature sequence, each global feature in this sequence becomes the basis for saliency scoring and feature subset selection in subsequent steps. The global feature sequence corresponds to each spatial region of the downsampled image; therefore, each global feature actually carries the global contextual semantics of the corresponding region in the pathological image. Since the downsampling operation irreversibly loses high-resolution detail information, the global feature sequence only provides an overview of the tissue structure of the pathological image. Detailed pathological features such as cell nuclear morphology and mitotic figures need to be retrieved from the original high-resolution image as needed.

[0032] Step S20: Calculate the significance score of the global features in the global feature sequence, and select a preset number of feature subsets from the global feature sequence based on the significance score; In this embodiment, the saliency score is used to measure the importance of each global feature in the pathological diagnosis task. For a global feature, a higher saliency score indicates that the corresponding image region contains richer information related to pathological diagnosis, and should be prioritized for recall. The calculation method of the saliency score is not limited to a single dimension. In this embodiment, each global feature is evaluated from two dimensions—semantic relevance and spatial aggregation—through parallel diagnostic scoring branches and spatial topology scoring branches, respectively. The preset number is a pre-defined fixed integer value, which physically represents the maximum number of high-resolution image patches allowed to be recalled in each inference process. This value is determined by the computing power budget of the deployment environment. The system filters each feature in the global feature sequence based on the saliency score, selecting exactly the preset number of global features to form a feature subset. This ensures that the computing power consumption does not exceed the preset upper limit, and that the selected feature subset has a high contribution to the pathological classification task.

[0033] Specifically, the diagnostic scoring branch is a scorer module composed of a fully connected neural network. This module receives a single global feature vector as input and outputs a scalar value representing the relevance score of that feature to the pathological diagnosis task. The parameters of this scorer module are updated synchronously with the main network during model training, ensuring that the scoring rules learned by the scorer are consistent with the optimization objective of the downstream classification task. The spatial topology scoring branch receives a single global feature vector and its spatial coordinates as input. By considering the feature similarity between each global feature and other global features in its spatial neighborhood, it outputs the spatial aggregation score of that global feature in the current tissue space. Global features located inside lesion regions and surrounded by similar pathological features have higher spatial aggregation scores, while global features located at discrete points in isolated background regions have lower spatial aggregation scores. The system performs a weighted summation of the diagnostic relevance score and the spatial aggregation score of each global feature to obtain the significance score of each global feature. The weighting coefficients are adaptively adjusted during training according to the optimization objective. The system sorts each global feature in descending order according to the significance score and selects a predetermined number of top-ranked global features from the global feature sequence to form a feature subset based on the sorting result.

[0034] As an optional implementation, the diagnostic scoring branch consists of two cascaded fully connected neural networks. The first fully connected layer maps the input global feature vector to a low-dimensional hidden representation and activates it through a modified linear unit. The second fully connected layer maps the hidden representation to a scalar output value. The system applies a sigmoid activation function to this scalar value to normalize it to the 0-1 range, obtaining a diagnostic relevance score. The weight parameters of the diagnostic scoring branch share the backpropagation link with the backbone network and are updated synchronously during model training. Since the gradient of the diagnostic scoring branch originates from the classification loss of the downstream classification task, the scoring criteria learned by this branch after training convergence are inherently consistent with the decision boundary of the classification head, meaning it can accurately distinguish image regions that substantially contribute to the final classification decision. Simultaneously, the spatial topology scoring branch constructs a spatial adjacency graph based on the spatial coordinates of each global feature, establishing adjacency relationships between each global feature and its neighboring global features within a preset neighborhood range. For example, an eight-neighborhood definition is used to make each global feature adjacent to its neighboring features in the up, down, left, right, and four diagonal directions.

[0035] For each global feature, the spatial topology scoring branch calculates the cosine similarity of the feature vectors between the global feature and all neighboring global features, and averages all similarity values. This average is used as the spatial aggregation score of the global feature. Global features located within lesion regions and surrounded by similar pathological features have higher similarity to neighboring features and thus receive higher spatial aggregation scores, while discrete points located in isolated background regions have lower similarity to neighboring features and thus receive lower spatial aggregation scores.

[0036] The system obtains pre-configured first and second preset weights. It multiplies the diagnostic relevance score by the first preset weight to obtain a first weighted score, and multiplies the spatial aggregation score by the second preset weight to obtain a second weighted score. The two weighted scores are then summed to obtain the significance score for each global feature. The ratio of the first and second preset weights determines the relative contribution of the semantic and spatial dimensions to the overall score. For example, when the first preset weight is 0.6 and the second preset weight is 0.4, semantic relevance dominates while spatial aggregation serves as a supplementary constraint.

[0037] As an alternative implementation, the system inputs the calculated saliency scores of each global feature into a differentiable sparse selection layer. The differentiable sparse selection layer first applies independent random noise sampled from a Guboll distribution to each saliency score. This noise introduces an exploratory element to the selection process in the early stages of training, ensuring that each global feature has a chance to be selected, thus preventing the model from prematurely locking into a suboptimal selection strategy before the parameters are fully trained. The scores after applying the noise perturbation are transformed into a continuous selection probability distribution using a soft-maximum function. Each global feature corresponds to a selection probability between 0 and 1, and the sum of all selection probabilities is 1. During the forward propagation phase of training, the system sorts the selection probabilities and generates a hard selection mask through hard thresholding and binarization. The hard selection mask is a binary vector of the same length as the global feature sequence. Positions with a value of 1 indicate that the corresponding global feature is selected, and positions with a value of 0 indicate that it is not selected. The system adjusts the binarization threshold so that the number of values ​​of 1 in the mask is exactly equal to a preset number, thereby strictly enforcing the hard constraint of computational power during forward propagation.

[0038] During backpropagation in the training phase, hard thresholding binarization, acting as a step function, has a gradient that is zero or nonexistent, thus preventing direct gradient propagation. The system employs a pass-through estimator strategy to address this issue. In forward propagation, the discrete output of the hard selection mask is maintained, while in backpropagation, the gradient at the hard selection mask is directly copied and propagated to the output of the soft maximum function. This bypasses the non-differentiable binarization operation, allowing the gradient to continue propagating upstream to the parameters of the diagnostic scoring branch and the spatial topology scoring branch. Therefore, the feature subset selection module, while satisfying a fixed number of hard constraints during forward inference, ensures that the entire selection chain is differentiable during training, and the selection strategy can co-evolve end-to-end with the goal of optimizing classification accuracy.

[0039] Optionally, during the inference phase, the differentiable sparse selection layer can be configured to switch to a deterministic ranking selection mode, that is, no longer applying Guboll noise, directly sorting in descending order of significance scores and selecting a preset number of global features that rank highly, thus ensuring the determinism and reproducibility of the inference results.

[0040] Step S30: Based on spatial location, extract local image patches corresponding to the feature subset from the pathological image; In this embodiment, spatial location refers to the coordinate information of the region in the downsampled image recorded for each global feature during the downsampling and encoding extraction steps. Since the feature subset corresponds to a specific global feature in the global feature sequence, each selected global feature is associated with a spatial location coordinate.

[0041] Specifically, the pathological image recognition system queries the list of spatial coordinates recorded in the aforementioned downsampling and encoding extraction steps based on the index of the selected global feature in the global feature sequence to obtain the row and column coordinates of each selected global feature. The system then maps these row and column coordinates back to the corresponding regions in the original high-resolution pathological image based on the downsampling magnification. Based on the mapped pixel coordinates and a preset cropping window size, it crops out various local image blocks from the original pathological image. Each local image block is a sub-image directly extracted from the original high-resolution image, retaining all pixel details at its original resolution, including fine-grained pathological features visible under high magnification, such as cell nuclear morphology, nuclear chromatin texture, nuclear membrane irregularities, and mitotic structures.

[0042] As an optional implementation, the size of the cropping window can be adaptively adjusted based on the downsampling factor and the spatial span of the selected global feature in the global feature sequence. For feature subsets with a large spatial span, the cropping window size can be appropriately increased to cover a wider tissue region, avoiding the loss of tissue context information caused by a fixed window.

[0043] As an alternative implementation, the system can perform preprocessing operations on local image blocks while cropping them, including color normalization and background removal. Color normalization eliminates staining differences between different batches of pathological images, and background removal filters out irrelevant blank areas, improving the efficiency of subsequent encoding steps.

[0044] Step S40: Encode the local image patch and map the encoding result to the embedding space corresponding to the global feature sequence to obtain the local features; In this embodiment, encoding local image patches refers to converting high-resolution local image patches into feature vectors using a dedicated image patch encoding network. Since the local image patches originate from the original high-resolution image, whose resolution is much higher than the downsampled image, directly fusing their features with the global features extracted from the downsampled image in the aforementioned downsampling and encoding extraction steps would face the dual obstacles of feature dimension mismatch and semantic space inconsistency. Therefore, after encoding, this step further maps the encoding result to the embedding space of the global feature sequence through a projection operation, ensuring that the mapped local features and global features reside in the same semantic space and have the same vector dimension, providing mathematical feasibility for the weighted fusion operation in subsequent steps.

[0045] Specifically, the pathological image recognition system feeds the cropped local image patches obtained in the aforementioned image patch extraction step into the image patch alignment network for processing. The image patch alignment network consists of multiple stacked residual convolutional layers. Each residual convolutional layer includes convolution operations, batch normalization, and a non-linear activation function. Residual connections add the features before and after convolution, effectively mitigating the gradient vanishing problem in deep networks. The convolutional layers extract visual features at different levels of abstraction from the local image patches layer by layer. Lower-level convolutions capture detailed information such as edges and textures, while higher-level convolutions capture semantic features such as cell nucleus morphology and tissue structure. The system performs global mean pooling on the convolutional feature map output by the image patch alignment network, compressing the two-dimensional feature map into a one-dimensional feature vector in spatial dimension. This pooled feature vector carries the overall visual semantics of the local image patch. The system maps the pooled feature vector to an embedding space with the same dimension as the global feature sequence in the aforementioned downsampling and encoding extraction steps through a linear projection layer, obtaining the final local features.

[0046] As an optional implementation, the image patch alignment network can share some low-level convolutional weights with the image encoder in the downsampling step. The image encoder, during pre-training, has learned general visual filters on large-scale natural or pathological image data, including the ability to extract low-level visual features such as edges, textures, and color patches. These low-level filters exhibit generalization characteristics across resolutions and scales. When initializing the image patch alignment network, the pathological image recognition system directly copies the first few layers of convolutional weights from the image encoder to the corresponding convolutional layers of the image patch alignment network. During subsequent training, these shared convolutional layers are fine-tuned with a small learning rate to retain the general visual knowledge in the pre-trained filters while adapting to the extraction requirements of high-resolution pathological details. Higher-level convolutions in the image patch alignment network, located after the shared layers, are randomly initialized and trained from scratch, specifically learning abstract semantic features related to high-resolution pathological image patches. This parameter sharing and hierarchical training strategy significantly reduces the number of parameters in the image patch alignment network, significantly improves training convergence speed, and avoids the slow convergence and accuracy jitter problems caused by random gradient fluctuations when training low-level filters from scratch.

[0047] As an alternative implementation, the linear projection layer can be replaced with a multilayer perceptron projection structure. Conventional linear projection maps pooled feature vectors to the embedding space using only a single-layer weight matrix multiplication; this mapping transformation is purely linear and lacks the ability to compensate for information compression biases caused by the pooling operation. The multilayer perceptron projection structure inserts at least one nonlinear hidden layer between the pooled feature vectors and the output local features. This hidden layer transforms the pooled feature vectors to a higher-dimensional feature space and introduces nonlinear transformation capabilities through a modified linear unit activation function, before mapping them to the embedding space through a second-layer weight matrix. The nonlinear transformation of the multilayer perceptron can compensate for information loss during the pooling process, particularly by implicitly reconstructing the spatial distribution information within regions lost during global mean pooling when compressing spatial information. This allows for a more refined alignment between the projected local features and the corresponding global features in the coordinate dimensions of the embedding space.

[0048] For example, the image patch alignment network contains three residual convolutional layers, each with a kernel size of 3x3, and output channels of 256, 512, and 1024 respectively. The pooled feature vector obtained after global mean pooling has a dimension of 1024. The linear projection layer maps the 1024-dimensional pooled feature vector to a 768-dimensional embedding space, consistent with the global feature dimension.

[0049] It should be noted that the design of the image patch alignment network balances the requirements of high-resolution detail extraction and embedding space alignment. Multi-layer residual convolution ensures the full extraction of rich pathological details from local image patches, providing a parameter-efficient and training-stable dimensional alignment scheme for the combination of global mean pooling and linear projection. Each component of the mapped local features is consistent with its corresponding component of the global features in terms of the coordinate axes in the embedding space, allowing for direct vector addition without introducing semantic misalignment.

[0050] Step S50: Weighted fusion of local features and corresponding global features to obtain fused features, and determination of pathological classification results based on fused features.

[0051] In this embodiment, weighted fusion refers to multiplying local features by a fusion weight and then adding them to the corresponding global features. The fusion weight controls the contribution ratio of local features, i.e., high-resolution detail information, in the fusion result. In this embodiment, the fusion weight gradually increases from zero with each training epoch during model training. In the early stages of training, the fusion weight is close to zero, and the fusion features are mainly dominated by global features. The image patch alignment network and the classification head rely on stable global semantic signals to learn basic feature representations. As training progresses, the fusion weight gradually increases, and high-resolution local detail information is gradually introduced. The classification head learns and integrates fine-grained pathological features based on the existing global semantic framework. After the fusion features are generated, the system performs pooling aggregation on each fusion feature to obtain the global representation vector corresponding to the pathological image. This global representation vector is input into the classification head to output the predicted probability of each pathological category, and the predicted probability is determined as the pathological classification result.

[0052] Specifically, the pathological image recognition system maintains a fusion weight value that monotonically increases with the current training epoch. In each training iteration, the system calculates the product of each local feature and the current fusion weight to obtain a weighted local feature. This weighted local feature is then added element-wise with the corresponding global feature and the corresponding positional encoding. The sum is then normalized to obtain the fusion feature. The system performs attention pooling on each fusion feature, calculates the attention weight of each fusion feature relative to the learnable query vector, and sums the fusion features according to their attention weights to obtain the global representation vector. The system inputs the global representation vector into a classification head composed of a fully connected layer and a multi-label activation function. The classification head outputs the same number of predicted probability values ​​as the preset number of pathological categories, where each predicted probability value represents the confidence that the pathological image belongs to the corresponding pathological category.

[0053] As an optional implementation, the fusion weights can employ a phased incremental strategy combined with a training stability diagnostic mechanism. In this embodiment, the pathological image recognition system divides the training process into multiple stages, such as a warm-up stage, a progressive fusion stage, and a fine-tuning stage, totaling three consecutive stages. In the warm-up stage, the fusion weights are fixed at zero. At this time, the fusion features are entirely dominated by global features. The image patch alignment network learns basic feature extraction capabilities without local feature interference, and the classification head quickly converges to basic classification accuracy based on stable global semantic signals. In the progressive fusion stage, the fusion weights are gradually increased from zero to a preset upper limit value with a preset step size. Each sub-stage continues for a preset number of training rounds. At the end of each sub-stage, the system records various evaluation indicators on the validation set, including classification accuracy and loss value. By monitoring the changes in indicators between adjacent sub-stages, the system determines whether training oscillations occur during the fusion process. If a decrease in classification accuracy exceeding a preset threshold is detected compared to the previous sub-stage, the system automatically pauses the increase of the fusion weights and adds additional training rounds to the current weight value to stabilize the fusion process. During the fine-tuning phase, the fusion weights are kept at the preset upper limit, and the system performs fine-tuning of the entire network parameters under the condition of balanced fusion of global and local features.

[0054] As an alternative implementation, the pooling aggregation operation can employ a multi-head attention pooling structure. In this embodiment, the pathological image recognition system sets up multiple parallel attention pooling heads, each maintaining its own independent learnable query vector. During training, each query vector is guided by gradient signals to focus on different semantic aspects of information in the fused feature sequence. For example, one attention head learns to focus on fused features related to global tissue structure features, another attention head learns to focus on fused features related to high-resolution cellular detail features, and a third attention head learns to focus on fused features related to cross-regional contrast features. Each attention head independently calculates the attention score of the query vector and each fused feature in the fused feature sequence, and then sums them according to the weighted scores to obtain the aggregation vector corresponding to each head. The system concatenates the aggregation vectors output by all attention heads in the dimensional direction into a longer global representation vector. The length of this global representation vector is equal to the dimension of a single aggregation vector multiplied by the number of attention heads.

[0055] In one implementation, such as Figure 2 As shown, Figure 2This is a schematic diagram of the pathological image feature recognition process in this embodiment. As shown in the figure, the system as a whole adopts a cascaded processing flow of "low-resolution global perception, dual-branch gating scoring, budget hard constraint selection, high-resolution local recall, alignment and progressive fusion, and multi-label classification". First, the input high-resolution full-view pathological image is downsampled to generate a 224×224 low-resolution thumbnail to reduce the computational burden of subsequent global modeling. Then, the global token sequence and corresponding tissue context features are extracted from this thumbnail. Based on this, the diagnostic relevance score and spatial topology score of each token are calculated by a dual-branch pathological saliency scorer and fused into a saliency score. Next, exactly K tokens are selected by a budget hard constraint differentiable token selection layer to obtain the corresponding spatial coordinates. Based on the coordinates, the corresponding high-resolution pathological blocks are accurately cropped and recalled from the original high-resolution image. The local features are then projected into an embedding space consistent with the global tokens by an image block alignment network. The global contextual features and local high-definition fine-grained features are stably fused through a gated progressive cross-scale fusion layer to generate a unified feature representation that can be directly used for model training or clinical inference, and then input into the classification head to complete pathological multi-label recognition.

[0056] Specifically, the overall architecture of the ATH-Net system includes modules A through E in the figure, and its input is ultra-high resolution full-view pathological images. PATH-Net comprises the following five interdependent dedicated modules. (See reference...) Figure 3 , Figure 3 This is a schematic diagram of the cascaded five-module PATH-Net structure involved in this embodiment. Module A is a low-resolution global encoder. Downsampling generates low-resolution thumbnails Input the visual Transformer backbone network and extract... Global feature sequence of each spatial token: .

[0057] The spatial coordinate mapping of each token on the full-view slice image is preserved synchronously. Module B is a two-branch pathological significance scorer. Diagnostic relevance scores are output in parallel for each token. Spatial topological fractions The fusion score is: .

[0058] In the formula , These can be learned weights or preset constants; Characterizing the first The degree of correlation between each token region and the pathological classification label; Characterizing the first The degree of association between each token and the continuity of its neighborhood organization. This dual-branch design differs from a general single scoring head, providing a structural anchor for subsequent topology loss and coverage loss. Module C is a budget-constrained differentiable token selection layer. A fixed recall number is determined given the hardware computing power budget. ( ), receive fusion score The output is exactly Hard-choice mask for each token: .

[0059] The implementation can employ a differentiable Top-K (first K) selection layer or a Gumbel-Softmax sparse selection layer. During the training phase, a pass-through estimator is used to make the selection operation differentiable, and the forward propagation strictly outputs the results. Token index and spatial coordinates The computational budget is enforced at the structural layer, rather than relying on external scaling regularization for indirect approximation. Module D is a high-resolution local recall and image patch alignment network. Based on coordinates... From the original image medium crop A pathological image block Alignment features are obtained by PAN encoding of each image block. ,satisfy: .

[0060] Module E is a gated progressive cross-scale fusion layer and a classification head. For each recalled location... The fusion features are: .

[0061] In the formula Encodes spatial location. To follow the training rounds Increasing learnable gating coefficients. The fused features, after pooling, are input into a multi-label classification head, which outputs the predicted class probability. .

[0062] Furthermore, by integrating the main task supervision with four types of PATH-Net-specific constraint objectives, the total loss function expression is as follows: .

[0063] In the formula, Ltotal is the total loss function of the model, which is the only core supervision signal for network backpropagation and parameter iterative updates. Lcls is the cross-entropy classification loss, used to ensure the core accuracy of multi-label recognition of pathological images. coverThis represents the loss of lesion coverage integrity, used to constrain the coverage integrity of the lesion area by the recalled token. xscale This is a cross-scale semantic consistency loss used to constrain the semantic alignment of high- and low-resolution features at the same location. topo A sparse selection loss is used to maintain the topology, preventing misselection of tokens and ensuring spatial continuity. L gate λ1, λ2, λ3, and λ4 are gated asymptotic consistency loss used to stabilize the cross-scale fusion process and preserve pre-trained knowledge; λ1, λ2, λ3, and λ4 are non-negative adjustable hyperparameters used to adjust the weight strength of each specific constraint in the overall optimization objective.

[0064] In scenarios with limited hardware resources, K can be reduced and λ1 increased to strengthen the lesion coverage constraint under limited computing power. When model training oscillates or region selection becomes fixed, λ3 can be increased to improve spatial topology exploration capabilities. When fine-tuning the pre-trained model, λ4 can be increased to extend the gating process. In conventional training scenarios, small-weight, weak regularization constraints are used to prioritize the accuracy of pathological identification.

[0065] Batch multi-label cross-entropy loss is used as the main supervisory loss. This accurately measures the deviation between the model's predictions and the actual pathological labels, taking into account the characteristics of multi-label and multi-classification tasks involving pathological slides. The formula is defined as follows: .

[0066] In the formula, B represents the number of samples in a single training batch, C represents the total number of categories in the pathological identification task, and y b，c This represents the true label of class c corresponding to the b-th sample. b，c This represents the predicted probability of the b-th sample corresponding to the c-th class, as output by the model's classification head.

[0067] This loss function serves as the core supervisory signal for the entire PATH-Net, driving end-to-end parameter iteration updates for the visual Transformer backbone network, bi-branch saliency scorer, budget hard constraint selection layer, image patch alignment network, and gated feature fusion module. It minimizes the deviation between the predicted distribution and the true label distribution, fundamentally ensuring the core accuracy of pathological slide typing, grading, and lesion identification, and adapting to the industry characteristics of imbalanced categories in pathological datasets.

[0068] In this embodiment, after downsampling the pathological image and extracting the global feature sequence, a preset number of feature subsets are selected from the global feature sequence based on the saliency score of each global feature. Local image patches corresponding to the feature subsets are extracted from the pathological image according to their spatial location and encoded. The encoded results are mapped to the embedding space corresponding to the global feature sequence to obtain local features. These local features are then weighted and fused with the corresponding global features to determine the pathological classification result. This ensures that the number of high-resolution local image patches recalled in each inference is constrained to the preset number, no longer fluctuating with changes in the input image. This predetermines the computation time and memory overhead of a single inference, enabling stable operation in clinical deployment environments with limited computing resources. Simultaneously, the selection of local image patches is still based on the saliency score, ensuring that the recalled image patches are concentrated in areas that substantially contribute to pathological classification, achieving controllability of inference computation without sacrificing diagnostic accuracy.

[0069] Based on the same inventive concept, this application also provides a second embodiment, referring to... Figure 4 , Figure 4 This is a flowchart illustrating the second embodiment of the method for efficient and high-resolution feature backtracking of pathological image memory based on gated token pruning in this application.

[0070] In this embodiment, the method for efficient and high-resolution feature backtracking of pathological images based on gated token pruning includes steps S61-S65: Step S61: Based on the class activation mapping heatmap, calculate the coverage ratio of the feature subset to the lesion area to obtain the coverage loss; Step S62: Calculate the cross-scale loss based on the semantic consistency between global features and corresponding local features; Step S63: Calculate the topology preservation loss based on the number of spatially connected components in the feature subset; Step S64: Calculate the gating consistency loss based on the offset of the weights corresponding to the local features during the weighted fusion process and the variance of the fused features; Step S65: Calculate the classification loss based on the pathological classification results and the true labels. The classification loss, coverage loss, cross-scale loss, topology preservation loss and gating consistency loss are weighted and summed to obtain the total loss. The network parameters are then optimized using the total loss.

[0071] In this embodiment, the pathological image recognition system constructs a multi-objective joint training system composed of five types of loss functions. Each of the five loss types corresponds to a different functional module of the network, forming a structure-bound supervised closed loop. The classification loss corresponds to the multi-label classification head, measuring the difference between the classification prediction result and the true label, driving the classification head to output accurate pathological category prediction probabilities. The coverage loss corresponds to the feature subset selection process, using a class activation map heatmap as a spatial reference for the lesion region, measuring the completeness of the selected feature subset's spatial coverage of the lesion region, driving the selection of feature subsets to cover the lesion region as much as possible. The cross-scale loss corresponds to the image patch alignment network, measuring the semantic consistency between global features and corresponding local features, driving aligned local features to maintain a similar orientation to global features in the semantic space. The topology preservation loss corresponds to the feature subset selection process, measuring the spatial connectivity of feature subsets, driving the selected feature subsets to form spatially continuous regions rather than discrete points. The gating consistency loss corresponds to the weighted fusion process, measuring the smoothness of the fusion weight change trajectory and the stability of the fusion feature numerical distribution, driving the smooth progress of the fusion process.

[0072] Specifically, the pathological image recognition system acquires the true labels of pathological images under preset pathological categories. It then compares the predicted probabilities of each pathological category output by the multi-label classification head with the true labels category-by-category, calculating the multi-label cross-entropy classification loss. The system utilizes class activation mapping technology for forward inference on the pathological images, generating a heatmap of lesion regions corresponding to the spatial dimension of the global feature sequence. The value at each spatial location in the lesion region heatmap represents the activation intensity of that location for the classification decision. The system calculates the sum of the heatmap values ​​of the selected global features in the feature subset at their corresponding locations on the lesion region heatmap, divides this sum by the sum of the heatmap values ​​of all global features at their corresponding locations on the lesion region heatmap, and obtains the coverage ratio of the feature subset to the lesion region. Subtracting this coverage ratio from 1 yields the coverage loss. Minimizing the coverage loss means that the feature subset covers as many high-activation locations of the lesion region as possible in the heatmap.

[0073] Furthermore, the system calculates cosine similarity between each selected global feature and its corresponding local feature. Cosine similarity measures the directional consistency of two vectors in the embedding space. For feature pairs with cosine similarity below a preset similarity threshold, the difference between the cosine similarity and the threshold is calculated as a penalty value. The mean of the penalty values ​​for all feature pairs is the cross-scale loss. The cross-scale loss drives the local features output by the image patch alignment network to semantically converge towards the corresponding global features. The system constructs a binary space mask corresponding to the feature subset, where selected positions are set to 1 and unselected positions are set to 0. The system performs connected component analysis on this binary space mask to obtain the number of connected components. Dividing the number of connected components by a preset number yields the topology-preserving loss. Minimizing the topology-preserving loss drives the feature subset to aggregate into continuous connected regions in space as much as possible. During training, the system tracks the changes in the fusion weights corresponding to the local features involved in weighted fusion as the training rounds progress. It calculates the deviation between this curve and the preset target weight curve, and simultaneously calculates the statistical variance of each fusion feature to measure the stability of the numerical distribution of the fusion features. The system then performs a weighted summation of the deviation and variance to obtain the gating consistency loss. The system weights and sums the five types of losses according to their respective preset weight coefficients to obtain the total loss. Based on the total loss, it sequentially calculates the gradient of each network module parameter relative to the total loss using the backpropagation algorithm, and updates the network parameters using the gradient descent optimizer.

[0074] This embodiment establishes a structured supervision closed loop by binding five types of loss functions to each network module in a one-to-one correspondence. Classification loss ensures the accuracy of the final diagnosis; coverage loss and topology preservation loss constrain the completeness of lesion coverage and the continuity of spatial distribution of feature subsets from the spatial dimension, respectively; cross-scale loss constrains the semantic alignment between global and local features; and gating consistency loss constrains the numerical stability and smoothness of weight changes in the fusion process.

[0075] Since the system described in Embodiment 2 of this application is a system used to implement the method of Embodiment 1 of this application, those skilled in the art can understand the specific structure and variations of the system based on the method described in Embodiment 1 of this application, and therefore will not be described again here. All systems used in the method of Embodiment 1 of this application fall within the scope of protection of this application.

[0076] Based on the same inventive concept, this application also provides a third embodiment, referring to... Figure 5 , Figure 5 This is a flowchart illustrating the third embodiment of the method for efficient and high-resolution feature backtracking of pathological image memory based on gated token pruning in this application.

[0077] In this embodiment, the method for efficient and high-resolution feature backtracking of pathological images based on gated token pruning includes steps S71-S73: Step S71: Configure the feature subset selection method to deterministic selection mode; Step S72: Sort the significance scores of each global feature in descending order of numerical value; Step S73: Select the first preset number of global features from the sorting results to form a feature subset.

[0078] In this embodiment, after training is completed, the system switches the configuration for the feature subset selection method during the inference deployment phase. During training, feature subset selection uses Gubohr soft-maximum sampling to generate selection probabilities, which are then binarized into a hard selection mask. This introduces randomness to promote extensive exploration of different feature combinations in the global feature sequence. During inference, the system aims to produce stable and reproducible results for the same input pathological image in each inference iteration. Therefore, it switches to a deterministic selection mode, selecting features based on numerical ranking of saliency scores, without involving random sampling.

[0079] Specifically, the pathological image recognition system disables the calculation of coverage loss, cross-scale loss, topology preservation loss, and gating consistency loss during the inference phase. Only forward propagation is performed during inference, without backward gradient propagation, and the network parameters remain in their fixed state after training. The system switches the feature subset selection method from the differentiable sparse selection layer sampling mode configured during training to a deterministic descending selection mode. In deterministic selection mode, the system numerically sorts the saliency scores of each global feature in descending order and directly selects a preset number of top-ranked global features as the feature subset. This selection process is purely based on numerical comparison and ranking, consistently producing the same feature subset for the same input pathological image in any number of inferences, ensuring the determinism and reproducibility of the diagnostic results. The deterministic selection mode eliminates the need for Guboll noise sampling and hard thresholding binarization, making the inference computation path more streamlined compared to the training phase.

[0080] This embodiment eliminates the uncertainty of results caused by random factors in the inference process by switching the selection method from random sampling in the training phase to deterministic descending order selection during the inference phase. The calculation of the four types of regularization loss specific to training is disabled during the inference phase, and the computational path of the selection operation is simplified, so that the inference computation flow only retains the forward computation that directly affects the final classification result, reducing computational overhead and inference latency in the deployment environment.

[0081] This application provides a device for efficient and high-resolution feature backtracking of pathological image memory based on gated token pruning. The device includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the method for efficient and high-resolution feature backtracking of pathological image memory based on gated token pruning in the above embodiment 1.

[0082] The following is for reference. Figure 6 This document illustrates a structural schematic diagram suitable for implementing a gated token pruning-based high-efficiency, high-resolution feature backtracking device for pathological image memory. The gated token pruning-based high-efficiency, high-resolution feature backtracking device for pathological image memory in this embodiment can include, but is not limited to, mobile terminals such as mobile phones, laptops, digital radio receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Description), PMPs (Portable Media Players), and in-vehicle terminals (e.g., in-vehicle navigation terminals), as well as fixed terminals such as digital TVs and desktop computers. Figure 6 The illustrated device for efficient and high-resolution feature backtracking of pathological image memory based on gated token pruning is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments in this application.

[0083] like Figure 6As shown, the efficient high-resolution feature tracing device for pathological image display memory based on gated token pruning may include a processing unit 1001 (e.g., a core processor, graphics processor, etc.), which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM) 1002 or the program loaded from the storage device 1003 into the random access memory (RAM) 1004. The random access memory 1004 also stores various programs and data required for the operation of the efficient high-resolution feature tracing device for pathological image display memory based on gated token pruning. The processing unit 1001, the read-only memory 1002, and the random access memory 1004 are interconnected via a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Typically, the following systems can be connected to I / O interface 1006: input devices 1007 including, for example, touchscreens, touchpads, keyboards, mice, image sensors, microphones, accelerometers, gyroscopes, etc.; output devices 1008 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 1003 including, for example, magnetic tapes, hard disks, etc.; and communication devices 1009. Communication device 1009 allows the gated token-based pathology image memory high-efficiency high-resolution feature tracing device to communicate wirelessly or wiredly with other devices to exchange data. Although the figure shows a gated token-based pathology image memory high-efficiency high-resolution feature tracing device with various systems, it should be understood that it is not required to implement or possess all the systems shown. More or fewer systems can be implemented alternatively.

[0084] Specifically, according to the embodiments disclosed in this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments disclosed in this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device, or installed from storage device 1003, or installed from read-only memory 1002. When the computer program is executed by processing device 1001, it performs the functions defined in the methods of the embodiments disclosed in this application.

[0085] The device for efficient and high-resolution feature backtracking of pathological images based on gated token pruning provided in this application, employing the gated token pruning method for efficient and high-resolution feature backtracking of pathological images based on gated token pruning in the above embodiments, can solve the technical problem of unstable recognition effect of pathological full-view slice images. Compared with the prior art, the beneficial effects of the device for efficient and high-resolution feature backtracking of pathological images based on gated token pruning provided in this application are the same as the beneficial effects of the gated token pruning method for efficient and high-resolution feature backtracking of pathological images based on gated token pruning provided in the above embodiments, and other technical features in this device for efficient and high-resolution feature backtracking of pathological images based on gated token pruning are the same as those disclosed in the previous embodiment method, and will not be repeated here.

[0086] It should be understood that the various parts disclosed in this application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any suitable manner in one or more embodiments or examples.

[0087] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0088] This application provides a computer-readable storage medium having computer-readable program instructions (i.e., a computer program) stored thereon, which are used to execute the efficient and high-resolution feature backtracking method for pathological image memory based on gated token pruning in the above embodiments.

[0089] The computer-readable storage medium provided in this application may be, for example, a USB flash drive, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, system, or device. The program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, radio frequency (RF), etc., or any suitable combination thereof.

[0090] The aforementioned computer-readable storage medium may be included in a device for efficient and high-resolution feature backtracking of pathological image display memory based on gated token pruning; or it may exist independently and not be assembled into a device for efficient and high-resolution feature backtracking of pathological image display memory based on gated token pruning.

[0091] The aforementioned computer-readable storage medium carries one or more programs. When these programs are executed by the gated token pruning-based high-efficiency, high-resolution feature backtracking device for pathological image memory, the device performs the following actions: downsampling of the pathological image; inputting the downsampling result into an image encoder; extracting a global feature sequence and the spatial location corresponding to each global feature; calculating the saliency score of the global features in the global feature sequence; selecting a preset number of feature subsets from the global feature sequence based on the saliency score; extracting local image patches corresponding to the feature subsets in the pathological image according to their spatial location; encoding the local image patches; mapping the encoding result to the embedding space corresponding to the global feature sequence to obtain local features; weighted fusion of the local features and the corresponding global features to obtain fused features; and determining the pathological classification result based on the fused features.

[0092] Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0093] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation that may be implemented in systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0094] The modules described in the embodiments of this application can be implemented in software or hardware. The names of the modules do not necessarily limit the functionality of the unit itself.

[0095] The readable storage medium provided in this application is a computer-readable storage medium that stores computer-readable program instructions (i.e., a computer program) for executing the above-described method for efficient and high-resolution feature backtracking of pathological image memory based on gated token pruning. This method can solve the technical problem of unstable recognition performance of pathological full-view slice images. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in this application are the same as those of the efficient and high-resolution feature backtracking method for pathological image memory based on gated token pruning provided in the above embodiments, and will not be repeated here.

[0096] The above description is only a part of the embodiments of this application and does not limit the patent scope of this application. All equivalent structural transformations made under the technical concept of this application and using the contents of the specification and drawings of this application, or direct / indirect applications in other related technical fields, are included in the patent protection scope of this application.

Claims

1. A method for efficient and high-resolution feature backtracking of pathological image memory based on gated token pruning, characterized in that, The method includes the following steps: The pathological image is downsampled, and the downsampling result is input into the image encoder to extract the global feature sequence and the spatial location corresponding to each global feature; Calculate the saliency score of the global features in the global feature sequence, and select a preset number of feature subsets from the global feature sequence based on the saliency score; Based on the spatial location, extract local image patches corresponding to the feature subset from the pathological image; The local image patch is encoded, and the encoding result is mapped to the embedding space corresponding to the global feature sequence to obtain local features; The local features are weighted and fused with the corresponding global features to obtain fused features, and the pathological classification result is determined based on the fused features.

2. The method for efficient and high-resolution feature backtracking of pathological images based on gated token pruning as described in claim 1, characterized in that, The step of calculating the saliency score of the global features in the global feature sequence and selecting a predetermined number of feature subsets from the global feature sequence based on the saliency score includes: Based on the correlation between the semantic information of each global feature and the pathological diagnosis task, the diagnostic relevance score of each global feature is calculated. Based on the spatial adjacency relationship between each global feature and its neighboring global features, the spatial aggregation degree score of each global feature is calculated. The significance score is obtained by weighted summation of the diagnostic relevance score and the spatial aggregation score. Based on the saliency score, a preset number of feature subsets are selected from the global feature sequence.

3. The method for efficient and high-resolution feature backtracking of pathological images based on gated token pruning as described in claim 2, characterized in that, The step of weighted summing of the diagnostic relevance score and the spatial aggregation score to obtain the significance score includes: The diagnostic relevance score is multiplied by a first preset weight to obtain a first weighted score; Multiply the spatial aggregation degree score by the second preset weight to obtain the second weighted score; The significance score is obtained by adding the first weighted score and the second weighted score.

4. The method for efficient and high-resolution feature backtracking of pathological images based on gated token pruning as described in claim 2, characterized in that, The step of selecting a preset number of feature subsets from the global feature sequence based on the saliency score includes: By using a differentiable sparse selection layer, and based on Guboul soft maximum sampling, the selection probability corresponding to each global feature is generated; A hard selection mask is generated based on the selection probability, and the number of global features selected in the hard selection mask is equal to the preset number. The global features selected by the hard selection mask are determined as the feature subset.

5. The method for efficient and high-resolution feature backtracking of pathological images based on gated token pruning as described in claim 1, characterized in that, The step of encoding the local image patch and mapping the encoding result to the embedding space corresponding to the global feature sequence to obtain local features includes: Perform multi-layer residual convolution operations on each of the local image blocks to obtain convolutional features; The convolutional features are subjected to global mean pooling to obtain pooled features; By linearly projecting the pooling features, the pooling features are mapped to the embedding space corresponding to the global feature sequence, thus obtaining the local features.

6. The method for efficient and high-resolution feature backtracking of pathological image memory based on gated token pruning as described in claim 1, characterized in that, The step of weightedly fusing the local features with the corresponding global features to obtain fused features, and determining the pathological classification result based on the fused features, includes: The fusion features are pooled and aggregated to obtain the global representation vector corresponding to the pathological image; The global representation vector is input into a multi-label classification head, which then outputs predicted probabilities for each of the preset pathological categories. The predicted probability corresponding to each of the pathological categories is determined as the pathological classification result.

7. The method for efficient and high-resolution feature backtracking of pathological images based on gated token pruning as described in claim 6, characterized in that, After the step of determining the predicted probability corresponding to each of the pathological categories as the pathological classification result, the method further includes: Based on the class activation mapping heatmap, the coverage ratio of the feature subset to the lesion area is calculated to obtain the coverage loss; Based on the semantic consistency between the global features and the corresponding local features, calculate the cross-scale loss; Calculate the topology preservation loss based on the number of spatially connected components in the feature subset; The gating consistency loss is calculated based on the offset of the weights corresponding to the local features during the weighted fusion process and the variance of the fused features. The classification loss is calculated based on the pathological classification results and the true labels. The classification loss, the coverage loss, the cross-scale loss, the topology preservation loss and the gating consistency loss are weighted and summed to obtain the total loss. The network parameters are then optimized using the total loss.

8. The method for efficient and high-resolution feature backtracking of pathological images based on gated token pruning as described in claim 7, characterized in that, After the steps of calculating the classification loss based on the pathological classification results and the true labels, and then weighting and summing the classification loss, the coverage loss, the cross-scale loss, the topology preservation loss, and the gating consistency loss to obtain the total loss, and optimizing the network parameters with the total loss, the method further includes: Configure the selection method of the feature subset to a deterministic selection mode; The significance scores of each global feature are sorted in descending order of numerical value; Select the aforementioned preset number of global features from the sorting results to form the feature subset.

Images