A method for classifying cross-lineage small blue round cell tumors, electronic device, storage medium and program product

Abstract

The application discloses a cross-lineage small blue round cell tumor classification method, which comprises the following steps: inputting a hematoxylin-eosin staining whole section image into a cross-lineage small blue round cell tumor classification model to obtain cross-lineage classification of the small blue round cell tumor of the whole section image; and the construction method of the model comprises the following steps: acquiring hematoxylin-eosin staining whole section image samples, and establishing a two-level label system of lineage categories and subtype categories; carrying out tissue segmentation on the whole section image, extracting tissue regions and cutting the tissue regions into image blocks; using a pathological basic model to extract features of the image blocks to obtain image block embedding vectors; constructing a section representation generation network based on the image block embedding vectors to generate section representations; constructing a classifier, wherein the classifier performs joint prediction of lineage classification and subtype classification according to the section representations; and training the classification model by using a coarse-fine granularity joint supervision strategy.

Description

Technical Field

[0001] This disclosure belongs to the field of tumor diagnostic technology, and specifically relates to a method for classifying cross-lineage small blue round cell tumors, electronic devices, storage media, and program products. Background Technology

[0002] Small blue round cell tumors (SRBCTs) are a group of tumors that appear very similar on routine hematoxylin and eosin (H&E) stained sections, but actually originate from different biological lineages (mesenchymal, neuroectodermal, hematolymphatic, epithelial, and melanocyte lineages), requiring different follow-up work and treatments. Current research has primarily focused on single tumor families or limited subsets of sarcomas, failing to address the cross-lineage differential diagnostic challenges encountered in initial histopathological evaluation. Currently, the diagnosis of these tumors typically relies on expensive immunohistochemistry and molecular testing, resulting in high diagnostic costs, long processing times, and the depletion of precious tissue samples. Summary of the Invention

[0003] The purpose of this disclosure is to provide a hierarchical deep learning framework based on a fundamental model to achieve cross-lineage classification of SRBCTs, addressing the lack of cross-lineage differential diagnosis in existing technologies. This provides pathologists with rapid and accurate preliminary classification, thereby optimizing the clinical diagnostic process, reducing unnecessary immunohistochemical and molecular testing, and preserving valuable tissue samples. This disclosure is intended for the field of pathological diagnosis, aiming to solve the cross-lineage classification problem of "small blue round cell tumors" (SRBCTs) by using artificial intelligence technology to achieve rapid and accurate tumor lineage identification, providing auxiliary decision support for clinical diagnosis.

[0004] One aspect of this disclosure is a method for classifying small blue round cell tumors across lineages. This method involves inputting a hematoxylin-eosin stained whole-section image into a classification model for small blue round cell tumors across lineages to obtain the cross-lineage classification of the small blue round cell tumors in the whole-section image. The method for constructing this model includes: Obtain whole-section image samples stained with hematoxylin and eosin, and establish a two-level labeling system of lineage and subtype categories; The full-slice image is segmented into tissue regions, which are then divided into image blocks. The image patch is used to extract features using a pathological model to obtain the image patch embedding vector; A slice representation generation network is constructed based on the image patch embedding vector to generate slice representations; A classifier is constructed, which performs joint prediction of lineage classification and subtype classification based on the slice representation; The classification model is trained using a joint supervision strategy of coarse and fine granularity.

[0005] Furthermore, the step of constructing a slice representation generation network based on the image patch embedding vector and generating slice representations includes: After linear projection and normalization, the image patch embedding vector is used to generate initial slice labels through an attention pooler. The initial slice markers and image block marker sequences are input into a hierarchical transformer, and the slice representation is output after multi-head self-attention processing.

[0006] The training using a joint coarse- and fine-grained supervision strategy includes: The sum of the phylogenetic cross-entropy loss and the subtype cross-entropy loss is used as the total loss function for optimization.

[0007] The pathological model is VIRCHOW2.

[0008] The method for constructing the model also includes: A heatmap generation module is constructed to map image block-level attention weights or contribution scores back to slice coordinates, thereby generating an attention heatmap.

[0009] The step of constructing a slice representation generation network based on the image patch embedding vector includes: The image block embedding vectors are grouped and pooled according to the full slice two-dimensional spatial coordinates to form a coarse-grained region representation. The coarse-grained region representation is input into a lightweight multi-head self-attention module to generate a suspicion score for each coarse-grained region.

[0010] The generated slice representation also includes: Based on the suspicion score, a differentiable Top-K gating mechanism is used to filter out the coarse-grained region with the highest suspicion score; The image patch embedding vectors within the range covered by the selected coarse-grained regions are extracted to form a local image patch label sequence.

[0011] The remaining unfiltered coarse-grained regions are aggregated into a background context vector with low weight.

[0012] The suspicion scores of the selected coarse-grained regions are weighted and aggregated, and then a global coarse-grained context label is generated by linear projection. The local image patch label sequence is input into an attention pooler to generate local slice labels, and the global coarse-grained context label is fused with the local slice labels to form fused slice labels.

[0013] The fused slice markers and the local image patch marker sequence are input together into a hierarchical transformer, and the slice representation is output after multi-head self-attention processing. The fusion slice marker, as a special sequence marker, interacts with the local image block marker through a self-attention mechanism to guide the allocation of attention weights to key diagnostic regions.

[0014] The differentiable Top-K gating mechanism uses Gumbel-Softmax relaxation to achieve a differentiable approximation of Top-K selection.

[0015] One aspect of this disclosure is a cross-lineage small blue round cell tumor classification system, comprising: The image acquisition module is used to acquire images of whole sections stained with hematoxylin and eosin; The preprocessing module is used to perform tissue segmentation on the whole slice image, extract tissue regions and cut them into image blocks; The feature extraction module is used to extract features from the image patch using a pathological basic model to obtain the image patch embedding vector; A slice representation generation module is used to generate slice representations based on the image patch embedding vectors; The classification module is used to perform joint prediction of phylogenetic classification and subtype classification on the whole slice image based on the slice representation.

[0016] In one aspect of this disclosure, an electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, the processor running the computer program to implement the aforementioned cross-lineage small blue round cell tumor classification method.

[0017] In one aspect of this disclosure, a storage medium storing a computer program that, when executed by a processor, implements the aforementioned cross-lineage small blue round cell tumor classification method.

[0018] In one aspect of this disclosure, a computer program product includes a computer program that is executed by a processor to implement the cross-lineage small blue round cell tumor classification method. Detailed Implementation

[0019] This disclosure proposes a cross-lineage hierarchical classification model for small round blue cell tumors (SMCs) in whole-section H&E images. Due to the significant cross-lineage complexity in H&E morphological diagnosis of SMCs, tumors from various biological origins can share characteristics such as dense cell growth, sparse cytoplasm, high nucleocytoplasmic ratio, and blue-stained small round cell-like morphology. This necessitates extensive differentiation among neuroendocrine, soft tissue, hematopoietic lymphoid, melanocyte, and undifferentiated epithelial tumors during the initial slide reading stage. Therefore, the design goal of this model is to learn morphological features from H&E whole-section images that support lineage identification and subtype differentiation without using immunohistochemistry or molecular detection inputs, and to use the output as a basis for subsequent auxiliary detection selection in triage. The challenges in constructing this model include: (1) Different lineages of tumors are suitable for small round blue cell-like morphology. Traditional visual morphological features are significantly confused, and discriminative characteristics such as cell nuclear morphology, arrangement, necrosis, interstitial reaction and local structural patterns need to be extracted from H&E images.

[0020] (2) Diagnostic clues are distributed in both local areas at the cellular level and tissue structures at the whole slice level, requiring multi-scale fusion of local high-information tiles and whole slice context.

[0021] (3) Rare subtypes have limited sample size and long-tailed class distribution, so it is necessary to use pathological domain pre-training representation, hierarchical label structure and external validation to reduce the risk of overfitting.

[0022] (4) Since clinical applications emphasize rapid triage and interpretability, the model should output lineage probability, subtype ranking and heat map evidence that can be traced back to slice coordinates.

[0023] The English terms or abbreviations used in this disclosure include: H&E — Hematoxylin-eosin staining; WSI — Whole Slide Image (a high-resolution image obtained by digitizing an entire pathology slide); RNA sequencing; NGS — Next-Generation Sequencing; pixel — pixel; µm / pixel — micrometer per pixel; objective magnification — objective lens magnification; physical resolution; Low-resolution tissue segmentation; Five-fold cross-validation; Information leakage; tile — image tile / image block (a fixed-size small image unit divided from a full slice); patch — an image patch; Patch encoder — an image patch encoder; VIRCHOW2 — Virchow 2: Large-scale pre-trained visual models for pathology; Embedding — embedding vector / feature embedding; Tile embeddings—After inputting each tissue image tile into the pathological basic model, the output is a high-dimensional feature vector. Each tile corresponds to a vector, which is used to represent the morphological information of the local region. feature vector; linear projection — linear projection; latent space — potential space; normalization; pre-trained representation; Sub-visual feature — subtle image features that are difficult to distinguish directly with the naked eye; attention pooler — attention pooling device; Slide token — a slice marker / token (representing the aggregated feature vector of the entire slice); patch-token — an image patch token (representing the feature vector of a single image patch); transformer — Transformer (a neural network architecture based on self-attention mechanism); encoder block — encoder block; hidden dimension; Attention head; head dimension; feed-forward dimension; activation function — the function that activates the user. GELU — Gaussian Error Linear Unit; dropout — random deactivation; Multi-head self-attention; residual connection; Layer normalization; position-wise feed-forward network; Slide representation — a representation of a slice; slide-to-patch — slice to image patch; linear output head — linear output head; coarse-to-fine — from coarse to fine / coarse-to-fine granularity; Joint supervision; CE (Cross-Entropy) — Cross-entropy; λ — Weighting coefficient / loss balance coefficient; macro-F1 — Macro average F1 score (arithmetic mean of F1 scores across all categories). checkpoint — a snapshot of the model (the state of the model saved during training); AdamW — AdamW optimizer; weight decay — the reduction or reduction of weight values. cosine annealing — cosine annealing (learning rate decay strategy); warm-up — the initial stage of training where the learning rate gradually increases; batch size — the size of the batch; epoch — round / cycle (the number of iterations to fully traverse the training data once); slide-level — slice level; accuracy — precision rate; macro-precision — the average macro precision. macro-recall — the average macro recall rate; balanced accuracy — balanced accuracy rate; one-versus-rest — one-to-many / one-to-remain (multi-classification strategy); macro-AUC — the area under the macro-average curve; bootstrap — self-help method / boot-pulling method; Confidence interval (CI) — a confidence interval; MIL (Multiple Instance Learning) — a weakly supervised learning paradigm that treats slices as packets and image patches as instances. TransMIL — TransMIL (a multi-instance learning method based on Transformer); DSMIL — DSMIL (Deep Set Multi-Instance Learning Method); CLAM — CLAM (Clustering Constrained Attention Multi-Instance Learning Method); ABMIL — ABMIL (Attention-Based Multi-Instance Learning Method); UMAP — Uniform Manifold Approximation and Projection (a dimensionality reduction visualization algorithm). manifold — a low-dimensional structure in high-dimensional data; Neighborhood relationship; Heatmap — A visualization of attention weights used to locate diagnostic regions that the model focuses on. backpropagation — the process of propagating backwards. forward propagation; Weakly supervised — using only slice-level labels without requiring image patch-level annotations; throughput — throughput; cache — cache; pre-compute — pre-compute.

[0024] Scratch — Train from scratch; ImageNet — ImageNet (a large-scale natural image classification dataset); ablation — ablation (experiment) (an experimental method of verifying the contribution of a component by removing it); H&E-first — H&E is the top priority.

[0025] Disease subtype abbreviations include: NEC — Neuroendocrine carcinoma, SCLC — Small cell lung cancer, NET — Neuroendocrine tumor, CT — Carcinoid tumor, ONB — Olfactory neuroblastoma, RMS — Rhabdomyosarcoma, ES — Ewing sarcoma, SS — Synovial sarcoma, DSRCT — Fibroproliferative small round cell tumor, CIC-RS — CIC rearranged sarcoma, GT — Hematopoietic lymphoid system-associated subtype, DLBCL — Diffuse large B-cell lymphoma, PC — Plasma cell tumor, MM — Malignant melanoma, UC — Undifferentiated carcinoma.

[0026] According to one or more embodiments, a hierarchical deep learning method based on a base model is used for cross-lineage small blue round cell tumor classification, said model including the following features: (1) The data processing and feature extraction of this model include: The H&E full-slice image is segmented into tissue regions, which are then divided into patches containing the tissue. The pathological model is used to extract features from each patch, resulting in patch-level features.

[0027] (2) Hierarchical Transformer architecture: A hierarchical Transformer was designed to transform slices into patches. Patch-level features were processed by an attention pooler and a hierarchical Transformer to generate a shared slice representation for simultaneous prediction of 5 cell lineages and 13 cell subtypes.

[0028] (3) Adopt a joint supervision mechanism from coarse to fine: A joint coarse-to-fine supervision strategy is adopted, which optimizes 5 lineage classification tasks and 13 subtype classification tasks at the same time, avoiding the limitations of single-level training.

[0029] (4) Initialization of the basic pathological model: Using a pathology-specific pre-trained model (VIRCHOW2) as the initialization of the feature extractor significantly improves the model's ability to identify rare subtypes.

[0030] (5) The model has interpretable heatmap generation capabilities: By mapping patch-level attention or contribution scores back to slice coordinates, an attention heatmap is generated, showing the morphological regions that the model focuses on, making it consistent with the diagnostic observations of pathologists.

[0031] The construction of the model in this disclosure has the following understandings and requirements: (1) The difficulty in diagnosing SRBCTs lies in the morphological similarity of tumors of different lineages, and it is necessary to solve the problem of differential diagnosis across lineages.

[0032] (2) The pathological basic model disclosed herein needs to be able to effectively capture the deep features of pathological images and solve the problem of data scarcity.

[0033] (3) The model’s coarse-to-fine hierarchical design simulates the pathological diagnosis process, first determining the general lineage and then refining the subtypes.

[0034] (4) The model adopts a joint supervision mechanism to ensure that the model retains the genealogy-level structural information while learning subtype features.

[0035] A method for constructing a cross-lineage small blue round cell tumor classification model includes the following steps: S101 collected multicenter SRBCT pathological slides and constructed a cohort containing 663 whole-slice images, covering 5 coarse categories (neuroendocrine tumors, soft tissue tumors, hematologic lymphomas, malignant melanomas, and undifferentiated carcinomas) and 13 fine-grained subtypes.

[0036] S102, perform tissue segmentation and patch extraction on the H&E whole-slice image.

[0037] S103 uses a Wilshaw 2 large-scale pre-trained visual model to extract features for each patch.

[0038] S104 generates a shared slice representation by processing patch features through an attention pooler and a hierarchical Transformer.

[0039] S105 employs a joint coarse-to-fine supervision strategy, simultaneously optimizing 5 lineage classification tasks and 13 subtype classification tasks.

[0040] S106 generates an attention heatmap, showing the morphological regions that the model focuses on.

[0041] Based on this embodiment, this disclosure has been further verified. The verification of the implementation effect includes: In the 13 subtype classifications, the internal validation macro F1=0.724 and the external validation macro F1=0.728.

[0042] In the five genealogical classifications, the internal validation accuracy was 0.926 and the external validation accuracy was 0.910.

[0043] Compared with existing methods, this disclosure improves long-tail category identification capability by more than 6 times.

[0044] According to one or more embodiments, a method for constructing a cross-lineage small blue round cell tumor classification model includes the following steps: S201, Sample Label Collection Hematoxylin-eosin (H&E) whole-slice images (WSI) were obtained, and a two-level labeling system of 5-level lineage categories and 13-level fine-grained subtypes was established. The H&E morphology, immunohistochemistry and necessary molecular detection were used for verification.

[0045] S202, Image Preprocessing and Tissue Segmentation Digital sections were acquired using a scanner with a 40x objective lens, achieving a physical resolution of 0.125 micrometers per pixel. Low-resolution tissue segmentation was first performed to remove the blank background, and then the effective tissue area was divided into 224×224 pixel image tiles.

[0046] S203, Pre-trained Feature Encoding for the Pathology Domain The pathological model VIRCHOW2 is used as the image patch encoder to encode each image patch into a d-dimensional feature vector (tile embedding), thereby decoupling the high-cost image encoding from the downstream classification task. This embedding vector can be pre-computed and cached.

[0047] S204, Full-slice hierarchical aggregation modeling Image patch embedding vectors are linearly projected into a 512-dimensional latent space and normalized. An attention pooler learns normalized weights for each image patch, and the weighted sum generates an initial slide token. This token, along with the image patch token sequence, is input to the slide-to-patch transformer, which includes two encoder blocks, eight attention heads, a hidden dimension of 512, a feedforward dimension of 2048, Gaussian error linear unit (GELU) activation, and a dropout of 0.1. After multi-head self-attention, residual connections, and layer normalization, the output is a slide representation.

[0048] S205, Two-level Joint Supervision Classification The classifier has two linear output heads, predicting 5 lineage labels and 13 subtype labels, respectively. Training employs coarse-grained and fine-grained joint supervision: the total loss is the sum of the lineage cross-entropy loss and the subtype cross-entropy loss, with each weighted at 1.0. The 5-class output is used for clinical lineage triage, and the 13-class output is used for subtype priority ranking, with the macro-average F1 score (macro-F1) of the 13 classes used as the model selection criterion.

[0049] S206, Model Optimization and Validation The AdamW optimizer was used with an initial learning rate of 1×10⁻⁶. -4 Weight decay 1×10 -5 The training employed cosine annealing for learning rate scheduling and five-round linear warm-up, totaling 30 training rounds with a batch size of one slice. Internally, case-level five-fold cross-validation was used to avoid information leakage. Externally, generalization performance was validated in a cohort of 213 independent institutions. Evaluation metrics included slice-level accuracy, macro-precision, macro-recall, macro-average F1 score, and macro-average area under the curve (macro-AUC).

[0050] S207, Interpretable Output and Clinical Positioning A single forward propagation simultaneously outputs the probabilities of 5 lineages, 13 subtypes, and an attention heatmap. The uniform manifold approximation and projection (UMAP) validate the structured distribution of the slice embedding vectors. The heatmap locates the key diagnostic regions of interest to the model, serving as an auxiliary triage basis after the first round of H&E slide review, and helping pathologists select targeted immunohistochemical combinations and molecular detection pathways.

[0051] According to one or more embodiments, the sample inclusion, labeling system and image preprocessing methods in the construction of a cross-lineage small blue round cell tumor classification model are described below.

[0052] The case samples were retrospectively collected from pathology archives of four hospitals, covering the period from February 2012 to August 2022. Case searches used terms such as "small round blue cell tumor" in the initial H&E morphological diagnosis as entry points to cover real-world clinical scenarios requiring cross-lineage differentiation. Candidate cases needed a final integrated pathological diagnosis conforming to the target category, available H&E whole-slide images, and complete clinicopathological information. Case labels were reviewed by pathologists and determined based on a comprehensive assessment of H&E morphology, immunohistochemistry, and necessary molecular testing results; EWSR1, CIC, SS18 rearrangement detection, RNA sequencing, or NGS were included when necessary.

[0053] Exclusion criteria included incomplete clinical information, unclear final diagnosis, repeated sections of the same specimen used for model development, severe defocus scanning, extensive tissue folding, significant compression artifacts, poor staining quality, or lack of sufficient viable tumor tissue. A total of 677 cases were screened, and 663 were retained after quality control. The internal development cohort included 450 cases from XX Hospital, and the external validation cohort included 213 cases from three other institutions. The labeling system included 5 lineage categories and 13 fine-grained subtypes: neuroendocrine tumors, soft tissue tumors, hematopoietic lymphoid system tumors, malignant melanoma, and undifferentiated carcinoma; the fine-grained subtypes included NEC / SCLC, NET / CT, ONB, RMS, ES, SS, DSRCT, CIC-RS, GT, DLBCL, PC, MM, and UC.

[0054] H&E whole sections were scanned using a KF-PRO-400 scanner at 40x objective magnification, with a physical resolution of 0.125 µm / pixel. Digital sections were first segmented at low resolution to remove blank background areas; subsequently, the tissue region was divided into 224 x 224 pixel tiles. Each section was encoded as a d-dimensional feature vector by a pathological baseline model. Internal model development employed case-level five-fold cross-validation, while external validation was performed in an independent institutional cohort, avoiding leakage of information at the section level or within the same case throughout the process.

[0055] The model in this embodiment consists of two parts: a feature extractor based on the pathological model and a WSI (whole slice image) level hierarchical classifier. The VIRCHOW2 patch encoder was ultimately chosen because it achieved optimal performance in internal characterization ablation experiments. For models containing N... NThe image tiles are sliced, and their tile embeddings are first linearly projected onto a 512-dimensional latent space and normalized. Then, an attention pooler learns normalized weights for each tile and generates initial slide tokens through weighted summation. The slide token and patch token sequences are jointly input into a "slide-patch" hierarchical Transformer. This Transformer contains two encoder blocks: a hidden dimension of 512 with eight attention heads (each 64-dimensional); a feed-forward dimension of 2048; a GELU activation function; and a dropout ratio of 0.1. Each encoder block includes a multi-head self-attention mechanism, residual connections, layer normalization, and a location-based feed-forward network. Finally, the updated slide token is read out as the final feature representation of the entire slice. The classifier contains two linear output heads, predicting 5 lineage labels and 13 fine-grained subtype labels, respectively. The training objective is joint supervision of coarse and fine granularity.

[0056] Where λ5=1.0 and λ13=1.0. Five output classes are used for clinical spectrum triage, and 13 output classes are used for subtype priority ranking. The model uses the validation set of 13 classes macro-F1 as the checkpoint selection criterion to ensure optimization for more rigorous, fine-grained tasks with more pronounced long-tail features.

[0057] During training, the model was optimized using AdamW with an initial learning rate of 1 x 10^6. -4 The weight decay is 1 x 10 -5The model employs cosine annealing learning rate scheduling and a 5-epoch linear warm-up. The training batch size is 1 slide, and training lasts for 30 epochs. Internal development uses 450 cases for 5-fold cross-validation; after the final model configuration is determined, it is evaluated in a 213-case independent external validation cohort. Evaluation metrics include slide-level accuracy, macro-precision, macro-recall, balanced accuracy, macro-F1, and one-versus-rest macro-AUC. Confidence intervals are calculated using 1,000 case-level bootstrap resampling.

[0058] Table 1. Performance Results of Main Classifications

[0059] Further characterization, supervision, and aggregation strategy verification.

[0060] Table 2 Ablation Validation of Characterization, Supervision, and Aggregation Strategies

[0061] Characterization ablation results show that pre-training of the pathology domain-based model is the core factor for performance improvement. Compared with Scratch training, VIRCHOW2 significantly improves macro-AUC and macro-F1 on both the 13-class and 5-class tasks. Compared with ImageNet initialization, VIRCHOW2 further enhances fine-grained representation capabilities, indicating that sub-visual features of H&E histological texture and small round blue cell tumors require pathology domain pre-training to be fully captured.

[0062] Coarse-grained and fine-grained joint supervision improves both 5-class and 13-class tasks. The 5-class lineage labels provide clinical diagnostic hierarchy constraints, reducing unstructured confusion in subtype learning; the 13-class subtype labels provide finer-grained morphological differences, inversely enhancing lineage boundaries. Comparison of aggregation strategies shows that the slide-to-patch hierarchical transformer outperforms ABMIL, CLAM, DSMIL, and TransMIL, supporting the necessity of joint modeling of full-slice context and local diagnostic foci.

[0063] Further robustness analysis of rare subtypes was conducted.

[0064] Table 3. Sensitivity analysis of rare subtypes

[0065] Sensitivity analysis of rare subtypes used the established model configuration and original prediction outputs without retraining. After excluding CIC-RS, ONB, and DSRCT in the main analysis, the remaining 410 internal and 199 external subtypes were used to recalculate the indices. The macro-F1 of the internal post-exclusion 10 subtypes was 0.722, similar to the 0.724 result for the complete 13 subtypes; the macro-F1 of the external post-exclusion 10 subtypes was 0.739, also similar to the 0.728 result for the complete 13 subtypes. This analysis indicates that the model conclusions were not dominated by statistical fluctuations in very low-frequency subtypes.

[0066] Further interpretability and diagnostic workflow localization. The model output includes 5 lineage probabilities, 13 subtype probabilities, and an attention heatmap. UMAP projection shows that slide embeddings form an organized structure at both the lineage and subtype levels; UC and MM exhibit relatively compact clustering, while NEN, STT, and LHT show wider manifolds while maintaining subtype neighborhood relationships. This phenomenon is consistent with the coarse-grained structure in SRBCT diagnosis.

[0067] The heatmap shows that the areas of focus of the model are consistent with the areas that pathologists prioritize in diagnosis, including consistent perivascular cell growth in GT, spindle-shaped or herringbone-like areas in SS, rhabdomyoblastic rich areas in RMS, small round cell sheet-like and rosette-like structures in ES, sclerotic intrastromal tumor nests in DSRCT, round to epithelioid cells and prominent nucleoli in CIC-RS, neurofibrillary lobular structures in ONB, organoid neuroendocrine structures in NET / CT, necrotic rich solid nests in NEC / SCLC, mitotically active sheet-like cell areas in DLBCL, plasmacytoid cell rich areas in PC, and mixed areas of small round cells, epithelioid cells or spindle cells in MM.

[0068] Table 4 Results of human-machine image reading

[0069] The clinical application is positioned as an auxiliary triage tool after the initial review of H&E slides. The model does not replace the final pathological diagnosis, but rather provides lineage and subtype prioritization to help pathologists select more targeted immunohistochemical combinations and molecular testing pathways. In scenarios with small specimens, limited tissue volume, insufficient pathology subspecialty resources, or a need to rapidly narrow down the differential diagnosis, this workflow can reduce unnecessary extensive testing and help preserve limited tissue for critical validation.

[0070] The computational implementation of this embodiment is shown in Table 5.

[0071] Table 5 Calculation and Implementation Related Indicators

[0072] This computational flow separates high-cost image encoding from WSI-level classification and supports pre-computation of tile embeddings. The WSI-level model maintains a compact structure and simultaneously generates probability outputs and heatmap evidence for two diagnostic levels through a single forward propagation. Actual runtime depends on WSI size, tissue proportions, number of tiles, hardware platform, and whether feature caching is used; therefore, runtime and throughput tests should be conducted on the target platform before formal deployment. Current results support computational feasibility from four aspects: structural complexity, training configuration, external performance, and baseline comparison.

[0073] This disclosure combines a pathological baseline model, hierarchical WSI aggregation, and coarse- and fine-grained joint supervision to address cross-lineage homomorphic heterogeneity in H&E images of rare small round blue cell tumors. Internal five-fold cross-validation, independent external validation, ablation experiments, rare subtype sensitivity analysis, and slide reading comparisons collectively support the model's stability in lineage triage, subtype prioritization, and interpretable key region localization. This approach can serve as part of an H&E-first auxiliary diagnostic workflow, providing more efficient and targeted decision-making support for subsequent immunohistochemistry and molecular testing.

[0074] The technical effects of the embodiments disclosed herein include: (1) Significantly improves classification accuracy. In the classification task of 13 subtypes, the macro F1 score reached 0.724 in internal validation and 0.728 in external validation.

[0075] (2) Initialization of the pathological basic model improved the average F1 score of long-tailed categories from 0.090 to 0.588, and greatly enhanced the ability to identify rare subtypes (CIC-RS, DSRCT, etc.).

[0076] (3) Rapid pre-screening helps pathologists narrow down the range of differences, prioritize subsequent tests, reduce unnecessary immunohistochemistry and molecular testing, and save costs.

[0077] (4) Attention heatmaps show that the areas the model focuses on are consistent with the pathologist’s diagnostic observations, enhancing clinicians’ confidence in the model’s interpretation.

[0078] (5) The model maintains high accuracy in independent external validation, indicating that it has good generalization ability.

[0079] According to one or more embodiments, this disclosure further reveals that the Heteroeny and Episodes Erythematosus (H&E) diagnostic clues in small round blue cell tumors (SRBCT) often exhibit extremely focal distribution. For example, rhabdomyoblasts in rhabdomyosarcoma may be scattered only in less than 5% of the tumor area, herringbone-like structures in synovial sarcoma may only be seen at the tumor margin, and rosette-like structures in Ewing sarcoma are mostly confined to the periphery of necrosis. In the construction of a cross-lineage small round blue cell tumor classification model, the attention pooler performs a weighted summation of all tissue image blocks in the entire slice to generate an initial slice label, and then inputs this label and the entire image block label sequence into a hierarchical transformer. This globally weighted processing mode has the following technical drawbacks: Firstly, when key diagnostic clues are extremely sparse, their signals are overwhelmed by a large background of homogeneous small round cells, making it difficult to effectively focus attention. Secondly, the computational complexity of the hierarchical transformer increases linearly with the number of image patches due to the full sequence input. For full slices with large tissue areas, the memory usage and inference latency increase significantly. Third, the existing scheme does not explicitly simulate the diagnostic behavior of pathologists who first scan the whole picture at low magnification and then stare at suspicious points at high magnification, and lacks the ability to actively perceive the spatial distribution pattern of diagnostic clues.

[0080] To address this technical problem, this embodiment proposes a method for fusing whole-slice focal diagnostic clues with tissue structure context based on coarse-grained gated sparse screening. This method embeds image blocks from the original file, performs attention pooling, and then inputs them into a hierarchical transformer framework. A pre-emptive coarse-grained gated sparse screening mechanism is added, allowing the hierarchical transformer to process only highly suspicious regions and their neighborhoods. This reduces background noise interference and computational complexity while preserving the ability of the whole-slice tissue structure context to guide local focusing.

[0081] The cross-lineage small blue round cell tumor classification model in this embodiment includes retaining the pathological baseline model VIRCHOW2 as an image patch encoder, and a structure consisting of a linear projection layer, a normalization layer, an attention pooler, and a slide-to-patch hierarchical transformer. Furthermore, it includes the following modules for the data stream: (1) Coarse-grained global perception and suspicion assessment module For the input full slice, all tissue image blocks obtained after tissue segmentation are encoded into image block embedding vectors by VIRCHOW2, then linearly projected into a 512-dimensional latent space and normalized to obtain a fine-grained image block embedding sequence consistent with the original file.

[0082] Building upon this, this embodiment adds a coarse-grained global perception branch. The aforementioned 512-dimensional fine-grained image patch embedding sequence is grouped and pooled according to its two-dimensional spatial coordinates on the whole slice. For example, several adjacent fine-grained image patch embeddings are averaged and pooled to form coarse-grained region representations. Each coarse-grained region representation corresponds to a larger field of view on the original slice, preserving the macroscopic distribution pattern of tissue structure (such as sheet-like, nest-like, sclerotic stroma distribution, etc.).

[0083] The coarse-grained region representation sequence is input into a lightweight, single-layer multi-head self-attention module. This module has the same hidden dimension as the main model and generates a suspicion score for each coarse-grained region, reflecting the probability that the region contains key diagnostic clues. The parameters of this single-layer module are independent of the main layer transformer, avoiding increasing the computational burden on the main model.

[0084] (2) Differentiable sparse gating and local sequence construction module Based on the suspicion score, a differentiable Top-K gating mechanism is used to select several coarse-grained regions with the highest suspicion scores. To ensure gradient propagation during end-to-end training, this embodiment uses Gumbel-Softmax relaxation to achieve a differentiable approximation of Top-K selection: in forward propagation, only the coarse-grained regions with the highest suspicion scores are retained; in backpropagation, the gradient is backpropagated to the suspicion score generation layer through continuous approximations of Gumbel-Softmax. The number of selected coarse-grained regions is dynamically determined based on the model performance on the validation set to ensure a balance between sparsity and information coverage.

[0085] For each selected coarse-grained region, all original fine-grained image patches within its coverage area are extracted and embedded to form a local image patch label sequence. Simultaneously, the remaining unselected coarse-grained regions are aggregated into a background context vector with low weights to avoid completely losing overall background information. The length of the local image patch label sequence is much smaller than the number of original full-image patches, significantly reducing the computational complexity of subsequent layer transformers.

[0086] (3) Global context and local representation fusion module The suspicion scores of the selected coarse-grained regions are weighted and aggregated, and then linearly projected to generate a global coarse-grained context label. This label carries macroscopic distribution information of the entire tissue structure and prior spatial location of highly suspicious regions.

[0087] Simultaneously, the sequence of local image patch labels is input into the attention pooler, which calculates the attention weight of each local image patch and sums them up to generate local slice labels. Since the attention pooler only operates on the sparsely filtered local image patches, its aggregation process avoids noise dilution from a large number of homogeneous background regions, allowing key focal cues to obtain higher representation weights in the slice labels.

[0088] The global coarse-grained context markers are fused with local slice markers (e.g., by splicing or gated weighted addition) to form an initial fused slice marker. This fused marker simultaneously contains representations of the overall slice structural context and local high-information regions.

[0089] (4) Hierarchical Transformer and Classification Output The fused slice markers and local image patch marker sequences are input into the slide-to-patch hierarchical transformer (i.e., the WSI hierarchical transformer). This hierarchical transformer maintains the structure described in the original file: it contains two encoder blocks with a hidden dimension of 512, employs eight attention heads, each with a dimension of 64, a feedforward dimension of 2048, uses Gaussian error linear units as the activation function, and has a random deactivation rate of 0.1. Each encoder block contains a multi-head self-attention network, residual connections, layer normalization, and a location feedforward network.

[0090] In the hierarchical transformer, fused slice markers, as special sequence markers, interact with local image patch markers through a multi-head self-attention mechanism. Specifically, the global context component in the fused slice markers guides the attention weights of local image patch markers towards diagnostically critical regions through self-attention computation; the self-attention between local image patch markers preserves the fine-grained interaction of focal details. This design achieves explicit guidance of local focusing by the global organizational context within the original self-attention framework, without introducing an additional cross-attention mechanism.

[0091] The final slice representation is read from the updated fused slice labels, input to two linear output heads, which predict 5 lineage labels and 13 fine-grained subtype labels, respectively. Training employs joint coarse-grained and fine-grained supervision: the total loss function is the sum of the lineage cross-entropy loss and the subtype cross-entropy loss, with weight coefficients determined based on validation set performance. Furthermore, a sparsity regularization term is added to the total loss to encourage a concentrated distribution of suspicion levels and prevent gating from degenerating into uniform screening.

[0092] The training of the model in this embodiment employs end-to-end joint training. This involves a coarse-grained global perception module, a differentiable sparse gating module, an attention pooler, a hierarchical transformer, and synchronous optimization of the classification head. The optimizer uses AdamW, and the initial learning rate, weight decay, learning rate scheduling, and training epochs are configured consistent with those described in the original documentation. Because sparse gating significantly reduces the input sequence length of the hierarchical transformer, the actual memory usage during training is lower than that of the full-sequence model, allowing for the use of larger model configurations or higher effective batch sizes under the same hardware conditions.

[0093] In the inference phase of this embodiment, for the input full slice, coarse-grained global perception and Top-K sparse screening are first performed to determine the spatial coordinates of high-suspicion regions. Subsequently, only these regions and their neighboring original image patches are subjected to fine-grained encoding and hierarchical transformer processing. Background regions only require coarse-grained perception and do not need to enter fine-grained branches. This mechanism of dynamically tilting computational resources towards high-information regions allows the inference process to focus on key areas for pathological diagnosis.

[0094] This embodiment produces the following technical effects: First, the ability to capture focal clues is enhanced. Coarse-grained global scanning quickly locates suspicious areas, preventing key diagnostic signals from being drowned out by homogeneous background. The attention pooler operates only on sparsely filtered local image patches, giving higher representational weight to sparse diagnostic clues such as rhabdomyoblasts, herringbone-like structures, and rosette-like structures in slice labeling.

[0095] Second, the overall tissue structure context is preserved. Global coarse-grained context labels are fused with slice labels and enter the hierarchical transformer, where they interact with local image patch labels through self-attention. This allows the model to not only capture focal cell details but also understand the macroscopic structural patterns of the cells (such as the boundary between tumor nests and sclerotic stroma, the distribution of necrotic transition zones, etc.), reducing lineage confusion caused by isolated local misjudgments.

[0096] Third, computational complexity is reduced. The sparse gating mechanism reduces the input sequence length of the hierarchical transformer from the original number of full-image patches to the length of a local sequence. Memory usage and computational cost decrease linearly with the sparsity ratio, improving the deployability of the model in clinical scenarios.

[0097] Fourth, interpretability is enhanced. The coarse-grained suspicion distribution map directly reflects the model's diagnostic focus on each region of the entire slide, which is consistent with the pathologist's behavior of first scanning at low magnification and then staring at high magnification; the attention weight of the local sequence accurately locates the focal diagnostic area that the model focuses on. The two layers of interpretable output jointly support the pathologist's clinical decision-making.

[0098] This disclosure can be used as a pathological diagnostic aid system and integrated into a hospital's digital pathology platform. When pathologists review H&E slides of SRBCTs, the system automatically runs a classification model, providing preliminary lineage classification and subtype prediction, as well as heatmaps of morphological regions of interest. Based on this information, doctors can quickly narrow down the range of differences and decide on subsequent immunohistochemical or molecular testing, avoiding unnecessary tests, saving costs, and preserving valuable tissue samples.

[0099] It should be understood that in the embodiments of this disclosure, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / " in this document generally indicates that the preceding and following related objects have an "or" relationship.

[0100] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this disclosure, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this disclosure. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0101] The above description is merely a specific embodiment of this disclosure, but the scope of protection of this disclosure is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this disclosure, and these modifications or substitutions should all be covered within the scope of protection of this disclosure. Therefore, the scope of protection of this disclosure should be determined by the scope of the claims.

Claims

1. A method for classifying small blue round cell tumors across lineages, characterized in that, By inputting the hematoxylin-eosin stained whole-section images into a cross-lineage small blue round cell tumor classification model, the cross-lineage classification of small blue round cell tumors in the whole-section images is obtained. The method for constructing this model includes: Obtain whole-section image samples stained with hematoxylin and eosin, and establish a two-level labeling system of lineage and subtype categories; The full-slice image is segmented into tissue regions, which are then divided into image blocks. The image patch is used to extract features using a pathological model to obtain the image patch embedding vector; A slice representation generation network is constructed based on the image patch embedding vector to generate slice representations; A classifier is constructed, which performs joint prediction of lineage classification and subtype classification based on the slice representation; The classification model is trained using a joint supervision strategy of coarse and fine granularity.

2. The method according to claim 1, characterized in that, The step of constructing a slice representation generation network based on the image patch embedding vector and generating slice representations includes: After linear projection and normalization, the image patch embedding vector is used to generate initial slice labels through an attention pooler. The initial slice markers and image block marker sequences are input into a hierarchical transformer, and the slice representation is output after multi-head self-attention processing.

3. The method according to claim 1 or 2, characterized in that, The training using a joint coarse- and fine-grained supervision strategy includes: The sum of the phylogenetic cross-entropy loss and the subtype cross-entropy loss is used as the total loss function for optimization.

4. The method according to claim 2, characterized in that, The step of constructing a slice representation generation network based on the image patch embedding vector includes: The image block embedding vectors are grouped and pooled according to the full slice two-dimensional spatial coordinates to form a coarse-grained region representation. The coarse-grained region representation is input into a lightweight multi-head self-attention module to generate a suspicion score for each coarse-grained region.

5. The method according to claim 4, characterized in that, The generated slice representation also includes: Based on the suspicion score, a differentiable Top-K gating mechanism is used to filter out the coarse-grained region with the highest suspicion score; Extract the image patch embedding vectors within the range covered by the selected coarse-grained region to form a local image patch label sequence; The remaining unfiltered coarse-grained regions are aggregated into a background context vector with low weight.

6. The method according to claim 5, characterized in that, The generated slice representation also includes: The suspicion scores of the selected coarse-grained regions are weighted and aggregated, and then a global coarse-grained context label is generated by linear projection. The local image patch label sequence is input into an attention pooler to generate local slice labels, and the global coarse-grained context label is fused with the local slice labels to form fused slice labels.

7. The method according to claim 6, characterized in that, The generated slice representation also includes: The fused slice markers and the local image patch marker sequence are input together into a hierarchical transformer, and the slice representation is output after multi-head self-attention processing. The fusion slice marker, as a special sequence marker, interacts with the local image block marker through a self-attention mechanism to guide the allocation of attention weights to key diagnostic regions.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, The processor executes the computer program to implement the method as described in any one of claims 1 to 7.

9. A storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1 to 7.

10. A computer program product, comprising a computer program, characterized in that, The computer program is executed by a processor to implement the method according to any one of claims 1 to 7.

Images