A multi-view histopathological image classification method and system
By employing dual-path parallel feature extraction and dynamic field-of-view decision-making with a sparse decision head, combined with temperature annealing training and multi-loss function optimization, the problems of rigid multi-scale fusion and neglect of pathological type differences in existing technologies are solved, thereby improving the accuracy and interpretability of histopathological image classification.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- HANGZHOU YICE TECH CO LTD
- Filing Date
- 2026-02-24
- Publication Date
- 2026-06-05
AI Technical Summary
Existing technologies cannot adaptively adjust the multi-scale feature fusion weights in histopathological image classification, ignore differences in pathological types, and lack end-to-end optimization, resulting in insufficient classification accuracy and uninterpretable decisions.
We employ a dual-path parallel feature extraction approach, dynamically adjust vision decision values through a multi-view feature fusion module and a sparse decision head, and combine temperature annealing training and multi-loss function optimization to achieve end-to-end vision decision and classification result optimization.
It significantly improves classification accuracy, especially for malignant invasive lesions, with an improvement of 5-8%. The accuracy of small field category remains the same or increases slightly by 1-2%, and the overall accuracy is improved by about 5%. It also has strong generalization ability and interpretability.
Smart Images
Figure CN122156737A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of medical image processing, and in particular to a multi-view histopathological image classification method and system. Background Technology
[0002] In the field of histopathological image classification, existing technologies are mainly divided into two categories: single-scale methods and multi-scale fusion methods. The technical characteristic of single-scale methods is that they use only image patches of a single field of view size (usually a fixed physical distance) as input, extracting features and performing classification through convolutional neural networks (such as ResNet and DenseNet). A typical example is the standard ResNet classifier: it directly uses images with field of view patches resized to 360×360 pixels as input to pre-train a CNN for classification. Another example is the Inception-v3 architecture: it uses a fixed-size input and captures local features through multi-scale convolutional kernels. The technical limitations of this method are that it cannot adapt to the differentiated field of view requirements of different pathological structures, especially categories that require observation of surrounding environmental information.
[0003] Existing multi-scale methods are mainly divided into two types: intra-field-of-view multi-scale methods and heterogeneous field-of-view multi-scale methods. Intra-field-of-view multi-scale methods are characterized by using image patches of different sizes as input within the same field of view (same physical distance). Typical implementations include pyramid pooling, which extracts feature pyramids of different scales from the same field-of-view image and fuses them through pooling layers. Another approach is multi-cropping strategies, which extract cropped images of different sizes from the same field-of-view region, input them separately into the network, and then average the prediction results. The limitation of these methods is that they cannot obtain true multi-scale contextual information; they only show size variations under different magnifications. Heterogeneous field-of-view multi-scale methods are characterized by extracting image features across different field-of-view ranges. Typical examples include: two-stage cascaded methods, which first perform preliminary classification on the smaller field of view and then extract features from the larger field of view for uncertain samples for reclassification; parallel dual-stream network methods, which simultaneously process images of the smaller field of view (details) and the larger field of view (context), fusing them through feature concatenation or weighted averaging; and spatial pyramid pooling, which extracts features from different fields of view and constructs a multi-resolution feature pyramid.
[0004] Fixed fusion weights: Most methods use fixed weights to fuse multi-scale features, which cannot be adaptively adjusted according to image content; Lack of category awareness: The differences in field of view requirements for different pathological types are not considered, and a uniform fusion strategy is adopted; Manual rule decision-making: Two-stage methods require setting manual thresholds to determine whether a large field of view is needed, and lack end-to-end optimization.
[0005] The closest technology to this patent is the "parallel dual-stream heterogeneous field-of-view multi-scale fusion method". The technical features of this method include: simultaneously inputting two images with different field-of-view sizes (such as physical distances of 100 micrometers and 200 micrometers), corresponding to small and large field-of-views at the same resolution, respectively; extracting features using two independent CNN backbone networks; fusing multi-scale features through simple feature concatenation, weighted summation, or attention mechanisms; and using the fused features for classification prediction.
[0006] Existing fusion strategies are rigid, with fixed-weight fusion: simple stitching or average weighting, which cannot adaptively adjust the fusion weights according to the image content. Even simple stacking adaptive fusion may not be effective; it ignores feature complementarity: it does not fully explore the cross-enhancement potential between features of different field sizes.
[0007] Category requirements are ignored; existing technologies do not consider the essential differences in field of view requirements for different pathological types. Malignant invasive lesions require a large field of view to observe the surrounding infiltration and microenvironment. Focal benign lesions can be clearly identified with a small field of view, while a large field of view may introduce noise or supplement environmental information to make the prediction more reliable. Existing methods use the same multi-field strategy for all categories and do not achieve differentiated processing.
[0008] The end-to-end optimization is insufficient. The two-stage method requires manual setting of thresholds and rules, and cannot be jointly optimized; the decision is not differentiable: it cannot incorporate the decision of "whether a large field of view is needed" into end-to-end training; the loss design is simple: it only uses classification loss and does not design a special loss function for the characteristics of multiple fields of view. Summary of the Invention
[0009] This invention addresses the shortcomings of existing technologies, such as the use of fixed weights to fuse multi-scale features, the inability to adaptively adjust based on image content, the lack of category awareness (failing to consider the differences in field of view requirements for different pathological types and adopting a uniform fusion strategy), and the need for manual rule-based decision-making (two-stage methods require setting manual thresholds to determine whether a large field of view is needed, lacking end-to-end optimization). It provides a multi-field histopathological image classification method and system.
[0010] To solve the above-mentioned technical problems, the present invention provides the following technical solution: A multi-view histopathological image classification method, characterized in that the method includes: Step 1: Obtain large-field and small-field images of the tissue pathology to be classified; Step 2: Use dual-path parallel feature extraction. Input the large field-of-view image and the small field-of-view image respectively, extract the initial features corresponding to the large field-of-view image and the small field-of-view image, and obtain the large field-of-view feature vector and the small field-of-view feature vector. Step 3: Obtaining the fused feature vector L_fused. The large-field feature vector and the small-field feature vector are fused using the multi-field feature fusion module MVFM to obtain the fused feature vector L_fused. Step 4: Output the vision decision value α. The fused feature vector L_fused is simultaneously input into the sparse decision head SDH and the classification head. The sparse decision head outputs the vision decision value α, which is used to represent the information of a large field of view. Step 5: Decision judgment of the field of view decision value α. If the decision value α is greater than the preset threshold, a large field of view is extremely needed to provide environmental information; otherwise, a small field of view is sufficient, and the large field of view is used to assist the surrounding environment so that the structure of the small field of view is clear. Step 6: Obtain the final classification result logits_final. Input the fused feature vector L_fused, the classification result logits, and the field decision value α. Dynamically adjust the contribution weights of the large field features and small field features to the classification result based on the field decision value α, and output the final classification result logits_final.
[0011] Preferably, the large-field-of-view feature vector and the small-field-of-view feature vector are fused using the Multi-Field-of-View Feature Fusion (MVFM) module to obtain the fused feature vector L_fused, which includes: Channel-dimensional concatenation involves concatenating the large-field-of-view feature vector and the small-field-of-view feature vector along the channel dimension to obtain the concatenated feature vector. The gated attention weighting process involves inputting the concatenated feature vector into the gated attention submodule. Two attention weight vectors are generated through a fully connected layer and a sigmoid activation function, corresponding to the large field-of-view feature gate weight g_l and the small field-of-view feature gate weight g_s. The large field-of-view feature gate weight g_l and the small field-of-view feature gate weight g_s are then multiplied element-wise with the original large field-of-view feature vector and the small field-of-view feature vector, respectively, to obtain the weighted large field-of-view feature vector f_l' and the weighted small field-of-view feature vector f_s'. Cross-feature interaction: The weighted large-field feature vector f_l' and the weighted small-field feature vector f_s' are bidirectionally mapped through a transformation network to obtain the feature-enhanced large-field feature vector f_cross_l and the feature-enhanced small-field feature vector f_cross_s. Finally, the feature vectors f_cross_l and f_cross_s after feature enhancement are concatenated again and then passed through a fully connected enhancement network to output the fused feature vector L_fused.
[0012] Preferably, the fused feature vector L_fused is simultaneously input into the sparse decision head SDH and the classification head, and the sparse decision head outputs the field-of-view decision value α to characterize the large field-of-view information, including: The fused feature vector L_fused is input into the main decision network structure, and the main decision network structure outputs two-dimensional logits probabilities. Temperature annealing training is performed using two-dimensional logits probabilities, and the randomness of the decision is controlled by a dynamic temperature parameter τ related to the number of training rounds.
[0013] Preferably, temperature annealing training is performed using two-dimensional logits probabilities, and the decision randomness is controlled by a dynamic temperature parameter τ related to the number of training rounds, including: During the training phase, Gumbel-Softmax sampling is used to make soft decisions based on temperature τ(t) and output the field decision value α with large field of view information; during the inference phase, low-temperature hard decision-making is adopted and the field decision value α with large field of view information is output.
[0014] As a preferred method, the final classification result logits_final is obtained by taking the fused feature vector L_fused, the classification result logits, and the field of view decision value α as inputs. The contribution weights of the large and small field of view features to the classification result are dynamically adjusted based on the field of view decision value α. The final classification result logits_final is then output, including: The field-of-view perception refinement involves concatenating the fused feature vector L_fused with the field-of-view decision value α of the large field-of-view information and inputting it into the field-of-view perception network to obtain a classification correction term based on the field-of-view decision. The classification correction terms are fused with the original classification results to obtain the final classification result logits_final.
[0015] Preferably, after acquiring the large-field and small-field images of the histopathological tissue to be classified, the following steps are also included: The classification loss is calculated by inputting the final classification result logits_final and the true class label into the classification loss module, and then using the cross-entropy loss function to calculate the cross loss, which is used to characterize the degree of difference between the classification result and the true label. The sparsity loss calculation involves inputting the vision decision value α into the sparsity loss module, calculating the uniformity loss of α using the entropy loss function, and calculating the deviation loss of α from the preset benchmark value using the deviation loss function. The uniformity loss and the deviation loss are then weighted and summed according to preset weights to obtain the sparsity loss. The category-aware loss is calculated by inputting the final classification result logits_final and the true class label into the category-aware loss module, and then calculating the category-conditional loss through the category-conditional loss function to improve the model's classification accuracy for a few classes. The total loss is calculated by inputting the cross loss, sparsity loss, and class conditional loss, and then weighting and summing the three losses according to the preset loss weight coefficients to obtain the total loss of the model training. Backpropagation updates the parameters. Based on the total loss, the gradient of the parameters of each layer of the model is calculated using the gradient descent algorithm, and the model parameters are adjusted in the opposite direction of the gradient. Steps 2 to 5 are executed iteratively until the total loss of the model converges to the preset threshold, thus completing the training of the histopathological image classification model.
[0016] To address the aforementioned technical problems, the present invention also provides a multi-view histopathological image classification system, which is used to implement the aforementioned multi-view histopathological image classification method.
[0017] This invention, by adopting the above technical solutions, has significant technical effects: This invention performs fusion at a deep feature level rather than a shallow image level, reducing redundant calculations. This significantly improves classification accuracy.
[0018] This invention requires a significant reduction in the accuracy of large-field categories and an improvement of 5-8% in the accuracy of proliferative polyps; it also requires maintaining or slightly increasing the accuracy of small-field categories by 1-2%; and an overall improvement in classification accuracy of approximately 5%. The gating attention mechanism of this invention enables the model to dynamically adjust the fusion weights based on the content.
[0019] This invention features completely differentiable features from feature extraction, fusion, decision-making to post-processing. Joint optimization: Classification loss, sparsity loss, and category-aware loss are jointly optimized. Temperature annealing strategy: Smoothly connects training and inference, avoiding optimization difficulties caused by the discreteness of decisions. Decision interpretability: The α value output by the sparse decision head provides an interpretable basis for why a large field of view is needed, and to what extent. Strong generalization ability: Validated on multiple independent datasets, including different coloring methods and scanner models.
[0020] The optimized model of this invention will help in the execution of auxiliary diagnostic tasks, reduce labor costs, greatly improve work efficiency, reduce the possibility of misjudgment and missed detection, and ensure the accuracy of diagnosis. Attached Figure Description
[0021] Figure 1 This is a flowchart of the forward propagation process of the complete model of this invention.
[0022] Figure 2 This is a flowchart of the multi-view feature fusion module of the present invention.
[0023] Figure 3 This is a flowchart of the sparse decision head of the present invention.
[0024] Figure 4 This is a flowchart of the post-processing refining module of the present invention.
[0025] Figure 5 This is a flowchart of the loss function calculation process of this invention. Detailed Implementation
[0026] The present invention will now be described in further detail with reference to the accompanying drawings and embodiments.
[0027] Example
[0028] A multi-view histopathological image classification method, characterized in that the method includes: Step 1: Obtain large-field and small-field images of the tissue pathology to be classified; Step 2: Use dual-path parallel feature extraction. Input the large field-of-view image and the small field-of-view image respectively, extract the initial features corresponding to the large field-of-view image and the small field-of-view image, and obtain the large field-of-view feature vector and the small field-of-view feature vector. Step 3: Obtaining the fused feature vector L_fused. The large-field feature vector and the small-field feature vector are fused using the multi-field feature fusion module MVFM to obtain the fused feature vector L_fused. Step 4: Output the vision decision value α. The fused feature vector L_fused is simultaneously input into the sparse decision head SDH and the classification head. The sparse decision head outputs the vision decision value α, which is used to represent the information of a large field of view. Step 5: Decision judgment of the field of view decision value α. If the decision value α is greater than the preset threshold, a large field of view is extremely needed to provide environmental information; otherwise, a small field of view is sufficient, and the large field of view is used to assist the surrounding environment so that the structure of the small field of view is clear. Step 6: Obtain the final classification result logits_final. Input the fused feature vector L_fused, the classification result logits, and the field decision value α. Dynamically adjust the contribution weights of the large field features and small field features to the classification result based on the field decision value α, and output the final classification result logits_final.
[0029] In this embodiment, the physical size of the patch in the input full-slice feature extraction module is set to x micrometers. In the experiment, the physical size of the patch with the corresponding large field of view is 4x micrometers, thereby cutting out all patch images of the slide. Figure 1 The model takes an arbitrary full-slice image as input, along with the effective patch image size and the augmented image. Through inference by the proposed model, it obtains the classification probability value of the corresponding small field-of-view patch and the field-of-view decision α value, where B is the number of patches input at one time and num_classes is the number of classes. The model simultaneously receives paired small field-of-view images (B×3×360×360) and large field-of-view images (B×3×1080×1080) as input.
[0030] Two independent backbone networks (which can both be ResNet50 or different) process the small and large field-of-view images respectively. The small field-of-view network focuses on extracting local detail features, while the large field-of-view network captures global contextual information. Both the small and large field-of-view images output small and large field-of-view spatial feature maps, respectively. These feature maps are then converted into small and large field-of-view spatial feature vectors by global average pooling. Each small and large field-of-view spatial feature vector is a B×2048-dimensional feature vector.
[0031] Multi-view feature fusion: The feature vectors of the small-view image space and the large-view image space are fed into the multi-view feature fusion module MVFM, and are deeply integrated through an adaptive mechanism. The fused features are B×4096 dimensions.
[0032] Parallel decision-making and classification: When the fused features are fed into two branches simultaneously, the classification head generates a preliminary class prediction, and the sparse decision head generates the vision selection probability α.
[0033] The preliminary classification results, combined with fused features and the view decision α value, are optimized through a post-processing refinement module. The final output is the refined classification result B×num_classes and the view decision explanation B×1.
[0034] Data preprocessing mainly includes multi-view pairing pruning and data augmentation. A precise spatial correspondence multi-view pruning strategy is employed.
[0035] Under a standard diagnostic field of view (x micrometers), a sliding window with a fixed x-step size traverses the entire slice image to extract small field-of-view image patches. For each small field-of-view image patch, the system uses its lower right corner pixel as a spatial anchor point, and uses this as the center to crop a large field-of-view image patch within a larger field of view (4x micrometers). The system ensures that the small field-of-view image is precisely located in the upper left center region of the large field-of-view image, such that for any x value, the physical distance of half the large field-of-view image minus the physical distance of the overlapping small field-of-view image is an integer multiple of x, conforming to a symmetrical spatial correspondence. For boundary cases, the system automatically adjusts the cropping position to ensure that the large field-of-view image is always located within the tissue region. Finally, a set of strictly paired image pairs is generated, with each sample containing views of the same pathological region at the same magnification in two different fields of view.
[0036] A synchronous and independent data augmentation strategy was adopted to first adjust the image size to a fixed size for both small and large field-of-view images. The small field-of-view image was resized to 360×360 pixels, and the large field-of-view image was resized to 1080×1080 pixels. Then, HSV color space transformation and Gamma correction were applied with a certain probability. Finally, horizontal and vertical flipping and matrix regularization were performed with a certain probability.
[0037] Multi-view feature fusion module (MVFM) Figure 2 The specific steps include: Gated attention weighting: First, the 2048-dimensional small-field feature f_s and the 2048-dimensional large-field feature f_l are concatenated to form a 4096-dimensional joint representation. A two-layer fully connected network (fc layer, 4096-dimensional → 2048-dimensional → 2-dimensional) with Softmax activation is used to generate the small-field attention weight g_s and the large-field attention weight g_l; and g_s + g_l = 1. Dynamic calculation is performed based on the specific content of each sample, achieving adaptive feature importance allocation to obtain the weighted large-field weight f_l' and the weighted small-field weight f_s'. f_s'= f_s× g_s, f_l'= f_l×g_l; Where f_l is the large field of view weight, f_s is the small field of view weight; g_l is the large field of view attention weight, g_s is the small field of view attention weight; f_l' is the weighted large field of view weight, and f_s' is the weighted small field of view weight. Cross-feature interaction: The weighted large field-of-view weights f_l' and the weighted small field-of-view weights f_s' are bidirectionally mapped through two independent transformation networks; the small field-of-view features are transformed into the large field-of-view feature space through a linear layer from 2048 dimensional to 2048 dimensional; T_l→s(f_l'), and the large field-of-view features are transformed into the small field-of-view feature space T_s→l(f_s'); cross-enhancement is performed to obtain the small field-of-view feature enhancement f_cross_s and the large field-of-view feature enhancement f_cross_s. f_cross_s = f_s' + T_l→s(f_l'); f_cross_s = f_l' + T_s→l(f_s'); Small-field features provide context awareness, while large-field features incorporate local details.
[0038] The small-field feature enhancement f_cross_s and large-field feature enhancement f_cross_s after fusion and enhancement cross-enhancement are concatenated again and passed through a fully connected enhancement network to output a 4096-dimensional fused feature vector.
[0039] The sparse decision head module implements the model's key intelligent decision-making capabilities. The main decision network structure consists of a 4096-dimensional fusion feature layer that first passes through a three-layer fully connected network (dimensions 512, 256, and 2 respectively), incorporating BatchNorm, ReLU, and Dropout operations, ultimately outputting two-dimensional logits probabilities. Temperature annealing training strategy: This is the core innovation of this system's training. During training, a dynamic temperature parameter τ, related to the number of training epochs, controls the randomness of the decisions. τ(t) = max(τ_final,τ_init×γ^t); Where τ_init=1.0, τ_final=0.1, γ=0.95, and t is the number of training rounds. The initial high temperature (τ=1.0) promotes exploration, while the later low temperature (τ=0.1) promotes model convergence.
[0040] During the training phase, Gumbel-Softmax sampling is used to make soft decisions based on temperature τ(t) (the calculation formula is torch.nn.functional.gumbel_softmax(logits probability, tau=τ(t), hard=False)), and the output is a continuous probability α∈[0,1], which maintains differentiability.
[0041] The inference phase uses low-temperature (τ=0.1) hard decision-making (hard=True) and outputs a binary choice α∈{0,1}.
[0042] The α value represents the probability threshold for selecting a large field of view. When α > 0.5, the model determines that the sample desperately needs contextual assistance from a large field of view for classification. When α ≤ 0.5, the model determines that the information from a small field of view is basically sufficient, and a small amount of assistance is needed from a large field of view. The vision-aware refinement process concatenates the fused features with the α value (4096+1 dimensions) and inputs it into a specialized vision-aware network, such as... Figure 4 As shown in the mid-field perception refinement network, classification correction terms based on field-of-view decisions are generated. This allows the model to further adjust the classification boundary according to its own judgment of field-of-view requirements.
[0043] The output of the residual fusion refinement network is fused into the original logits with a small weight (0.1), ultimately outputting the refined classification prediction. This forms a complete feedback loop of "classification → decision → refinement"; it is based on a vision-aware adaptive mechanism; and the model can be stably enhanced: the small-weight residual connections ensure the stability of the refinement process and avoid destroying existing knowledge.
[0044] The input data for the loss function consists of refined logits, the true label y_true, and the view decision α, all of which are vectors of dimension num_classes*1. The classification loss uses the standard cross-entropy loss to ensure basic classification accuracy: L_cls = CE(logits, y_true). The sparsity loss includes entropy loss and bias loss. Entropy loss encourages clear decisions, while the bias term pushes decisions away from ambiguous intermediate values (0.5). The combined effect is to encourage the model to make clear large / small view choices, avoiding ambiguity. The entropy loss is calculated as follows: Hα= -α·logα- (1-α)·log(1-α)]; The mean value is obtained after calculation. The deviation loss is obtained by calculating the mean value according to Dα = |α - 0.5|. The sparsity loss formula is: L_sparse = mean(Hα) + mean(Dα); For the class-aware loss function L_class_aware, for categories labeled "requiring a large field of view" (such as atrophy), the loss function encourages α to approach 1; for other categories, it encourages α to approach 0. This design allows the model to learn prior knowledge about the field of view requirements for different pathological types. Based on human experience, we set C_large as the set of categories requiring a large field of view. When a target in a certain field of view belongs to the large field of view category set, the loss for the field of view is (1-α)²; otherwise, the loss is α².
[0045] The total loss is: L_total = L_cls + λ1·L_sparse + λ2·L_class_aware; The weights are adaptively adjusted, for example: λ1=0.5, λ2=0.3. The synergistic effect of the three losses ensures the model's balance among multiple objectives: the classification loss guarantees basic performance, the sparsity loss improves computational efficiency, and the class-aware loss enhances the ability to identify difficult samples.
Claims
1. A multi-view histopathological image classification method, characterized in that, The methods include: Step 1: Obtain large-field and small-field images of the tissue pathology to be classified; Step 2: Use dual-path parallel feature extraction. Input the large field-of-view image and the small field-of-view image respectively, extract the initial features corresponding to the large field-of-view image and the small field-of-view image, and obtain the large field-of-view feature vector and the small field-of-view feature vector. Step 3: Obtaining the fused feature vector L_fused. The large-field feature vector and the small-field feature vector are fused using the multi-field feature fusion module MVFM to obtain the fused feature vector L_fused. Step 4: Output the vision decision value α. The fused feature vector L_fused is simultaneously input into the sparse decision head SDH and the classification head. The sparse decision head outputs the vision decision value α, which is used to represent the information of a large field of view. Step 5: Decision judgment of the field of view decision value α. If the decision value α is greater than the preset threshold, a large field of view is extremely needed to provide environmental information. Otherwise, the narrow field of view is sufficient, and the wide field of view is used to assist in understanding the surrounding environment, so that the structure of the narrow field of view is clear; Step 6: Obtain the final classification result logits_final. Input the fused feature vector L_fused, the classification result logits, and the field decision value α. Dynamically adjust the contribution weights of the large field features and small field features to the classification result based on the field decision value α, and output the final classification result logits_final.
2. The multi-view histopathological image classification method according to claim 1, characterized in that, The large-field-of-view feature vector and the small-field-of-view feature vector are fused using the Multi-Field-of-View Feature Fusion (MVFM) module to obtain the fused feature vector L_fused, which includes: Channel-dimensional concatenation involves concatenating the large-field-of-view feature vector and the small-field-of-view feature vector along the channel dimension to obtain the concatenated feature vector. The gated attention weighting process involves inputting the concatenated feature vector into the gated attention submodule. Two attention weight vectors are generated through a fully connected layer and a sigmoid activation function, corresponding to the large field-of-view feature gate weight g_l and the small field-of-view feature gate weight g_s. The large field-of-view feature gate weight g_l and the small field-of-view feature gate weight g_s are then multiplied element-wise with the original large field-of-view feature vector and the small field-of-view feature vector, respectively, to obtain the weighted large field-of-view feature vector f_l' and the weighted small field-of-view feature vector f_s'. Cross-feature interaction: The weighted large-field feature vector f_l' and the weighted small-field feature vector f_s' are bidirectionally mapped through a transformation network to obtain the feature-enhanced large-field feature vector f_cross_l and the feature-enhanced small-field feature vector f_cross_s. Finally, the feature vectors f_cross_l and f_cross_s after feature enhancement are concatenated again and then passed through a fully connected enhancement network to output the fused feature vector L_fused.
3. The multi-view histopathological image classification method according to claim 1, characterized in that, The fused feature vector L_fused is simultaneously input into the sparse decision head SDH and the classification head, and the sparse decision head outputs the field-of-view decision value α, which represents the information of a large field of view, including: The fused feature vector L_fused is input into the main decision network structure, and the main decision network structure outputs two-dimensional logits probabilities. Temperature annealing training is performed using two-dimensional logits probabilities, and the randomness of the decision is controlled by a dynamic temperature parameter τ related to the number of training rounds.
4. The multi-view histopathological image classification method according to claim 3, characterized in that, Temperature annealing training is performed using two-dimensional logits probabilities, with the decision randomness controlled by a dynamic temperature parameter τ related to the number of training epochs, including: During the training phase, Gumbel-Softmax sampling is used to make soft decisions based on temperature τ(t) and output the field decision value α with large field of view information; during the inference phase, low-temperature hard decision-making is adopted and the field decision value α with large field of view information is output.
5. The multi-view histopathological image classification method according to claim 1, characterized in that, The final classification result `logits_final` is obtained by taking the fused feature vector `L_fused`, the classification result `logits`, and the field-of-view decision value `α` as input. The contribution weights of the large and small field-of-view features to the classification result are dynamically adjusted based on the field-of-view decision value `α`. The final classification result `logits_final` is output, including: The field-of-view perception refinement involves concatenating the fused feature vector L_fused with the field-of-view decision value α of the large field-of-view information and inputting it into the field-of-view perception network to obtain a classification correction term based on the field-of-view decision. The classification correction terms are fused with the original classification results to obtain the final classification result logits_final.
6. The multi-view histopathological image classification method according to claim 1, characterized in that, After acquiring large-field and small-field images of the tissue pathology to be classified, the process also includes: The classification loss is calculated by inputting the final classification result logits_final and the true class label into the classification loss module, and then using the cross-entropy loss function to calculate the cross loss, which is used to characterize the degree of difference between the classification result and the true label. The sparsity loss calculation involves inputting the vision decision value α into the sparsity loss module, calculating the uniformity loss of α using the entropy loss function, and calculating the deviation loss of α from the preset benchmark value using the deviation loss function. The uniformity loss and the deviation loss are then weighted and summed according to preset weights to obtain the sparsity loss. The category-aware loss is calculated by inputting the final classification result logits_final and the true class label into the category-aware loss module, and then calculating the category-conditional loss through the category-conditional loss function to improve the model's classification accuracy for a few classes. The total loss is calculated by inputting the cross loss, sparsity loss, and class conditional loss, and then weighting and summing the three losses according to the preset loss weight coefficients to obtain the total loss of the model training. Backpropagation updates the parameters. Based on the total loss, the gradient of the parameters of each layer of the model is calculated using the gradient descent algorithm, and the model parameters are adjusted in the opposite direction of the gradient. Steps 2 to 5 are executed iteratively until the total loss of the model converges to the preset threshold, thus completing the training of the histopathological image classification model.
7. A multi-view histopathological image classification system, characterized in that, This method is used to implement a multi-view histopathological image classification method as described in any one of claims 1-6.