A cancer subtype identification method based on whole-slide histopathology images

Abstract

The application belongs to the technical field of computational pathology, and particularly relates to a cancer subtype identification method and model based on whole slide histopathology images, comprising: cutting the whole slide histopathology image to construct a plurality of image block sets containing tissue regions; for each image block set, extracting a feature vector of each image block in the image block set, and arranging the extracted feature vectors according to the relative positions of the image blocks to obtain a feature image of the image block set; screening out sampling points strongly related to cancer subtype categories on the feature image of the image block set to obtain a fusion feature vector of the image block set; and predicting a cancer subtype category of the whole slide histopathology image based on the fusion feature vectors of the image block sets. The method and model significantly improve the identification accuracy and robustness, reduce the interference of unannotated negative regions, and break through the performance limitation of traditional multiple instance learning methods.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of computational pathology, and particularly relates to a cancer subtype identification method based on whole-slide histopathology images. BACKGROUND

[0002] As one of the important means in medical diagnosis, histopathology is widely used for judgment and typing of major diseases such as cancer through analysis of tissue sections under a microscope. However, due to significant observer differences among pathologists, the traditional manual reading method has certain limitations in accuracy and consistency. In recent years, the rapid development of deep learning technology has promoted the rise of computational pathology, which can quantitatively and qualitatively analyze histology images, realize standardized and automated identification of histopathological features, and help pathologists make more objective, accurate disease diagnosis, prognosis evaluation and prediction analysis, so as to optimize individualized treatment plan and promote the realization of precision medicine.

[0003] Using deep learning models to classify tumor whole-slide images is an important research direction of computational pathology. At present, the application of computational pathology in whole-slide cancer diagnosis has covered multiple cancer types, such as detection and classification of lesion areas of breast cancer, lung cancer subtypes, renal cell carcinoma and neuroblastoma. However, since a whole-slide image usually consists of gigapixels, it cannot be directly input into a standard deep learning model for processing. Existing technologies usually need to divide the whole-slide into thousands of image blocks, which are then input into a neural network model for processing. Traditional fully supervised methods require pathologists to make fine annotations for each image block, which is a huge workload and high cost. If only whole-slide level labels are used to assign uniform labels to all image blocks, a large amount of noise will be introduced, especially in the case where the lesion area accounts for a very small proportion.

[0004] Therefore, multi-instance learning (MIM) has become the mainstream approach for handling weakly supervised problems with only slice-level labels. MIM frameworks solve the binary classification problem of cancer positivity / negative classification by constructing sets of image patches called "bags" and providing positive or negative labels (bags containing lesion patches are labeled positive). With the continuous development of precision medicine, accurate classification of different cancer subtypes has become particularly crucial, as different subtypes typically correspond to different treatment plans and prognoses. Accurate subtype classification can not only assist clinicians in developing personalized treatment plans but also effectively improve patient survival rates and quality of life. Although deep learning-driven MIM frameworks have demonstrated powerful capabilities in cancer positive diagnosis tasks, significant challenges remain in cancer subtype classification tasks. Due to the availability of only full-slice image-level subtype labels, negative samples are severely insufficient or even missing, making it difficult for standard MIM methods to effectively distinguish negative regions from positive sample features of different subtypes. The mixing of negative region features with positive samples introduces significant noise and reduces classification performance. Therefore, there is an urgent need for a deep learning model that can automatically identify feature vectors with high diagnostic value for subtype classification and mitigate the interference of unlabeled negative regions on classification results. Summary of the Invention

[0005] In view of this, the purpose of this application is to provide a cancer subtype identification method based on whole-section histopathological images, specifically targeting the weakly supervised classification problem of cancer subtypes in whole-section pathology images. It is particularly suitable for scenarios lacking negative samples and only having section-level labels, significantly improving classification accuracy and robustness. By adaptively focusing on key features, it reduces interference from unannotated negative regions, overcoming the performance limitations of traditional multi-instance learning methods. This method can be used to assist intelligent clinical pathology diagnosis, contributing to the development of precision medicine, digital pathology analysis, and related AI-assisted diagnosis, and has broad market prospects and significant commercial value.

[0006] This application provides a method for cancer subtype identification based on whole-section histopathological images, including:

[0007] The whole-section histopathological images were segmented to construct multiple sets of image blocks containing tissue regions;

[0008] For each set of image patches, the feature vector of each image patch in the set is extracted, and the extracted feature vectors are arranged according to the relative position of each image patch to obtain the feature image of the set of image patches.

[0009] By selecting sampling points that are strongly correlated with cancer subtypes from the feature images of this image patch set, the fused feature vector of this image patch set is obtained;

[0010] The cancer subtype category of the whole-section histopathological image is predicted based on the fused feature vector of each image patch set.

[0011] Furthermore, the step of segmenting the whole-section histopathological image to construct a set of multiple image blocks containing tissue regions includes:

[0012] The whole-section histopathological image is cut into several image blocks, and multiple image block sets are divided using a 9×9 grid in a left-to-right and top-to-bottom order to preserve the contextual relationship and spatial position information of adjacent image blocks; wherein, the size of the image block is 224×224;

[0013] Calculate the percentage of image patches belonging to tissue regions in each image patch set;

[0014] The set of image blocks that account for at least 30% of the image blocks belonging to the tissue region is retained to obtain multiple sets of image blocks containing the tissue region.

[0015] Furthermore, the step of selecting sampling points strongly correlated with cancer subtypes from the feature images of the image patch set to obtain the fused feature vector of the image patch set includes:

[0016] The number of channels in the feature image of this image patch set is reduced by a 1×1 convolution operation;

[0017] Based on the size of the dimension-reduced feature image and the pre-defined number of sampling points, multiple sampling points and their corresponding coordinates are generated on the dimension-reduced feature image.

[0018] Based on the feature vectors and location embedding features of each sampling point, a progressive update method is used to select sampling points that are strongly correlated with cancer subtypes, and at the same time, a fusion feature vector of the image patch set is generated.

[0019] Furthermore, the feature vectors and position embedding features of each sampling point are obtained in the following manner:

[0020] For each sampling point, the feature vectors of the four integer points closest to that sampling point are weighted and averaged to obtain the feature vector of that sampling point. The coordinates of that sampling point are then subjected to a learnable linear transformation to obtain the position embedding feature of that sampling point.

[0021] Furthermore, based on the feature vectors and location embedding features of each sampling point, a progressive update method is used to select sampling points that are strongly correlated with cancer subtypes, and a fused feature vector of the image patch set is generated, including:

[0022] The feature vectors of each sampling point, the position embedding features, and the fused feature vector output from the previous iteration are summed element by element and then input into the Transformer encoder to obtain the fused feature vector output from the current iteration; wherein, the Transformer encoder includes a multi-head self-attention mechanism and a multilayer perceptron.

[0023] A learnable linear transformation is performed on the fused feature vector output by the current iteration to obtain the sampling point offset, so as to update the coordinates of the sampling points until the iteration number is reached. The fused feature vector output by the last iteration is used as the fused feature vector of the image patch set.

[0024] Furthermore, the prediction of the cancer subtype category of the whole-section histopathological image based on the fused feature vector of each image patch set includes:

[0025] For each set of image patches, based on the fused feature vector of the set of image patches, a multilayer perceptron is used to predict the probability vector of the image patch set belonging to each cancer subtype category.

[0026] The predicted probability vectors of each image patch set belonging to each cancer subtype category are averaged bitwise to obtain the cancer subtype category corresponding to the highest predicted probability.

[0027] This application also provides a cancer subtype identification model based on whole-section histopathological images, the model comprising:

[0028] The image patch set generation module is used to cut whole-slice histopathological images to construct multiple image patch sets containing tissue regions;

[0029] The feature extraction module is used to extract the feature vector of each image patch in each image patch set, and arrange the extracted feature vectors according to the relative position of each image patch to obtain the feature image of the image patch set.

[0030] The feature selection and fusion module is used to filter out sampling points that are strongly correlated with cancer subtypes from the feature images of the image patch set, and obtain the fused feature vector of the image patch set.

[0031] The classification module is used to predict the cancer subtype category of the whole-slice histopathological image based on the fused feature vector of each image patch set.

[0032] The cancer subtype identification method and model based on whole-section histopathological images provided in this application are specifically designed for the weakly supervised classification problem of cancer subtypes in whole-section pathology images. It is particularly suitable for scenarios lacking negative samples and only having section-level labels, significantly improving classification accuracy and robustness. By adaptively focusing on key features, it reduces interference from unannotated negative regions, overcoming the performance limitations of traditional multi-instance learning methods. This method can be used to assist intelligent clinical pathology diagnosis, contributing to the development of precision medicine, digital pathology analysis, and related AI-assisted diagnosis, and has broad market prospects and significant commercial value. Attached Figure Description

[0033] Figure 1 A flowchart of a cancer subtype identification method based on whole-section histopathological images provided in an embodiment of this application is shown;

[0034] Figure 2 This illustration shows a schematic diagram of the structure of a cancer subtype identification model based on whole-section histopathological images provided in an embodiment of this application;

[0035] Figure 3 A schematic diagram of attention visualization of experimental samples provided in an embodiment of this application is shown. Detailed Implementation

[0036] To make the objectives, technical solutions, and advantages of this technical solution clearer, the following detailed description, in conjunction with specific embodiments, further illustrates this technical solution. It should be understood that these descriptions are merely exemplary and not intended to limit the scope of this technical solution.

[0037] Example 1:

[0038] Please see as follows Figure 1 The flowchart shown is for a cancer subtype identification method based on whole-section histopathological images. Figure 1 As shown, the method includes:

[0039] S101. Cut the whole-section histopathological image to construct a set of multiple image blocks containing tissue regions.

[0040] In this step, existing cancer subtype classification methods based on whole-slice histopathological images typically use individual image patches or packets as input. Because the method of cutting image patches fragments the overall structure of the whole-slice image and disrupts the spatial order of the image patches, the model struggles to capture the high-level semantic information of the whole slice. This problem is particularly prominent in cancer subtype multi-classification tasks with only whole-slice level labels, especially when negative samples are lacking. For example, the neighboring region of a tumor image patch of a certain subtype may be a normal tissue image patch, but the model can only uniformly treat all image patches as positive, unable to distinguish the differences between them. This leads to the features of the normal tissue image patch misleading the reasoning process and reducing classification accuracy. Therefore, this application creatively uses the set of image patches as the smallest unit of analysis, preserving the overall structure of the whole slice as much as possible without being limited by computational resources.

[0041] In practical implementation, multiple sets of image patches containing tissue regions can be constructed in the following way:

[0042] Step 1011: Cut the whole-section histopathological image into several image blocks, and use a 9×9 grid to divide multiple image block sets in order from left to right and from top to bottom, so as to preserve the contextual relationship and spatial position information of adjacent image blocks.

[0043] The size of the image block is 224×224.

[0044] Step 1012: Calculate the percentage of image patches belonging to the tissue region in each image patch set.

[0045] Step 1013: Retain the image block set in which at least 30% of the image blocks belong to the tissue region, so as to obtain multiple image block sets containing the tissue region.

[0046] In this step, the image patch set, in addition to containing histological structures, largely retains the morphological structure of the whole slice, thus providing a foundation for capturing the dependencies between image patches, extracting high-level semantic information, and performing feature fusion.

[0047] S102. For each set of image blocks, extract the feature vector of each image block in the set, and arrange the extracted feature vectors according to the relative position of each image block to obtain the feature image of the set of image blocks.

[0048] In this step, each image patch set contains 81 image patches of size 224×224, which can be represented as: X set {x1,x2,x i …,x n}, where x iA feature vector is extracted from each image patch using a feature extractor. These extracted feature vectors are then arranged according to the relative positions of the image patches to obtain the feature image F of the set of image patches. set =Arrange{f(X set )}.

[0049] S103. By selecting sampling points that are strongly correlated with cancer subtypes from the feature images of the image patch set, the fused feature vector of the image patch set is obtained.

[0050] In this step, after obtaining the feature images of each image patch set, a feature selection technique is used to determine which image patches' features are strongly correlated with the cancer subtype category. Since feature selection methods using specific evaluation metrics lack robustness, while data-driven approaches can improve model robustness, this application employs a progressive sampling method. By learning the sampling point offset from the content and location information of the image patches, iterative operations are used to ultimately locate the sampling points strongly correlated with the cancer subtype category, and the features of each sampling point are fused to obtain the fused feature vector of the image patch set.

[0051] In practical implementation, the fused feature vector of this image patch set can be obtained in the following way:

[0052] Step 1031: Reduce the number of channels in the feature image of the image patch set by performing a 1×1 convolution operation.

[0053] In this step, the number of channels in the feature image of the image patch set is reduced from 2048 to 1024 using the following formula:

[0054] F′ set =conv1×1(F set (1)

[0055] Step 1032: Based on the size of the dimension-reduced feature image and the pre-set number of sampling points, generate multiple sampling points and their corresponding coordinates on the dimension-reduced feature image.

[0056] In this step, the initial sampling points of each image patch set are the same, assuming a dimensionality-reduced feature image F′ set ∈R 1024×H×w The total number of upsampled points is (N×N) points s0∈R 2×N×N Then the initial coordinates of the i-th sampling point in [0, N×N-1] are expressed as:

[0057]

[0058] In the formula, Indicates the i-th sampling point, Indicates the vertical grid number of the i-th sampling point. Let H represent the horizontal grid number of the i-th sampling point, H be the height of the reduced feature image, N be the number of sampling points in a single direction, W be the width of the reduced feature image, and τ be the width of the sampled feature image. h =H / N and τ w =W / N represents the sampling step size in the horizontal and vertical directions, respectively.

[0059] Step 1033: Based on the feature vectors and location embedding features of each sampling point, a progressive update method is used to select sampling points that are strongly correlated with cancer subtypes, and at the same time, a fusion feature vector of the image patch set is generated.

[0060] Specifically, the feature vectors and position embedding features of each sampling point are obtained in the following way:

[0061] For each sampling point, the feature vectors of the four integer points closest to that sampling point are weighted and averaged to obtain the feature vector of that sampling point. The coordinates of that sampling point are then subjected to a learnable linear transformation to obtain the position embedding feature of that sampling point.

[0062] Here, since the coordinates of the obtained sampling points are fractions, the coordinates of the sampling points and the feature image F′ set All are differentiable. Assume the coordinates of the sampling point are (a, b), and the eigenvectors of the four integer points closest to the sampling point are F... 01 (a0, b0), F 01 (a0, b1), F 10 (a1, b0), F 11 (a1, b1) ∈ F′ set Using linear interpolation, the feature vector F(a,b) of the sampling points is obtained by weighted averaging using the following formula:

[0063] In the formula, Q0 and Q1 are two intermediate values ​​obtained by linear interpolation in the horizontal direction.

[0064] Since the feature vector of a sampling point is fused with the feature vectors of its four nearest integer points, this application achieves feature fusion synchronously during the updating process of the sampling point through bilinear interpolation.

[0065] Meanwhile, assuming the total number of iterations is I, the dimensionality-reduced feature image F′ set The N×N feature vectors obtained by sampling are F t , t∈{1…,I}; The coordinates s of the sampling points in the tth iteration are... t ∈R2×N×N The position embedding feature S of the sampling point coordinates is obtained through a learnable linear transformation as shown in Equation (8). t ∈R 2014×N×N :

[0066] S t =Ψ t (s t (8)

[0067] Furthermore, in practical implementation, the fused feature vector of this image patch set can be generated in the following ways:

[0068] Step 201: Add the feature vectors of each sampling point, the position embedding features, and the fused feature vector output from the previous iteration element by element, and then input them into the Transformer encoder to obtain the fused feature vector output from the current iteration.

[0069] The Transformer encoder includes a multi-head self-attention mechanism and a multilayer perceptron.

[0070] In this step, the feature vector F of each sampling point is calculated using the following formula. t Location embedding features S t The fused feature vector T output from the previous iteration t-1 Element-wise addition yields the fused feature vector T output by the current iteration. t :

[0071]

[0072] T t =Transformer Encoder(M t (10)

[0073] The Transformer encoder can be represented as:

[0074] M′ t =MSA(LN(M t ))+M t (11)

[0075] T t =MLP(LN(M′) t ))+M′ t (12)

[0076] In the formula, LN represents the layer normalization operation.

[0077] Step 202: Perform a learnable linear transformation on the fusion feature vector output by the current iteration to obtain the sampling point offset, and update the coordinates of the sampling points until the iteration number is reached. Then, use the fusion feature vector output by the last iteration as the fusion feature vector of the image patch set.

[0078] In this step, a learnable linear transformation Φ is used. t ∈R 2×1024 The sampling point offset o is obtained. t And the coordinates s of the sampling point in the next iteration t+1 :

[0079] o t =Φ t (T t (13)

[0080] s t+1 =s t +o t (14)

[0081] Here, based on the features and relative positions of the image patches in the image patch set, the sampling point will be gradually moved to the region of interest. That is, the fused feature vector T, which is strongly correlated with the cancer subtype classification, is output through the feature selection and fusion method in the last iteration. I .

[0082] S104. Based on the fusion feature vector of each image patch set, predict the cancer subtype category of the whole-slice histopathological image.

[0083] In practical implementation, the cancer subtype category of the whole-section histopathological image can be predicted in the following ways:

[0084] Step 1041: For each image patch set, based on the fused feature vector of the image patch set, use a multilayer perceptron to predict the predicted probability vector of the image patch perceptron set belonging to each cancer subtype category.

[0085] In this step, after obtaining the fusion feature vector that is strongly correlated with the cancer subtype classification, this application uses a multilayer perceptron with two hidden layers to classify the fusion feature vector of each image patch set. ReLU is used as the activation function, and in order to reduce the number of model parameters, a parameter random discarding mechanism with a probability of 50% is used in each layer.

[0086] Step 1042: Perform bitwise averaging on the predicted probability vectors of each image patch set belonging to each cancer subtype category to obtain the cancer subtype category corresponding to the highest predicted probability.

[0087] In this step, the predicted probability vector of a whole-section histopathological image is obtained by averaging the predicted probability vectors of the set of image patches it contains. Therefore, the recognition result is expressed as:

[0088]

[0089] In the formula, This represents the predicted cancer subtype category, and argmax(·) is used to find the category with the highest predicted probability.

[0090] Example 2:

[0091] Please see as follows Figure 2 The diagram shows the structure of a cancer subtype identification model based on whole-section histopathological images. The cancer subtype identification model 200 includes:

[0092] The image patch set generation module 201 is used to cut whole-slice histopathological images to construct multiple image patch sets containing tissue regions.

[0093] The image patch set generation module 201 can be pre-trained in the following way:

[0094] Step 301: Collect whole-section histopathological images of different cancer subtypes, and have the pathologist mark the tissue regions; wherein, the tissue regions include: diseased tissue regions and normal tissue regions.

[0095] Step 302: For each whole-section histopathological image, obtain the image block set of the whole-section histopathological image in accordance with the above steps 1011-1013, and mark it as the cancer subtype of the whole-section histopathological image.

[0096] In this step, each image patch set is assigned a category label of the same cancer subtype as the whole-slice histopathological image to which it belongs. This image patch set is then used as the smallest input unit for subsequent modules. This image patch set differs from the concept of a "bag" in traditional multi-instance learning. In traditional multi-instance learning, image patches in a bag may come from slices of multiple subclasses, and the bag label typically depends on whether the bag contains the desired target. In contrast, an image patch set comes only from slices of a specific subclass, although it may contain a large number of normal tissue image patches. This approach is more suitable for multi-class classification problems without negative sample annotations.

[0097] Furthermore, unstandardized histopathological slide datasets often suffer from uneven staining, which negatively impacts model performance by introducing color features into the inference results. Therefore, to alleviate the burden of model training, this application performs Macenko staining normalization during data preprocessing to mitigate the uneven staining caused by differences in staining reagents from different batches, as well as variations in operating techniques and equipment. The specific method is as follows:

[0098] For each whole-section histopathological image, Macenko staining normalization was performed. This was achieved by modeling the spectral characteristics of different staining agents in the tissue sections and recalibrating the color space of the image.

[0099] The feature extraction module 202 is used to extract the feature vector of each image block in each image block set, and arrange the extracted feature vectors according to the relative position of each image block to obtain the feature image of the image block set.

[0100] The feature extraction module 202 can be pre-trained in the following way:

[0101] The feature extraction module 202 is obtained by training a ResNet50 model using whole-section histopathological images. The feature extraction module 202 is used to obtain high-quality histological feature representations from the image patch set.

[0102] The feature selection and fusion module 203 is used to select sampling points that are strongly correlated with cancer subtypes from the feature images of the image patch set to obtain the fused feature vector of the image patch set.

[0103] The classification module 204 is used to predict the cancer subtype category of the whole-slice histopathological image based on the fused feature vector of each image patch set.

[0104] The classification module 204 can be pre-trained in the following way:

[0105] Suppose there are C categories, Γ sets of image patches, and y i (i = 1, ..., Γ) represents the one-hot encoding of the true label of the i-th image patch set, P i This is the model's predicted probability vector for the i-th image patch set. The cross-entropy loss is used to calculate the loss value and for backpropagation to update the model parameters.

[0106] (16)

[0107] In the formula, p i，c y is the probability that the model predicts image patch set i as category c. i，cIt is an indicator function for whether the set of image patches i belongs to category c.

[0108] Example 3:

[0109] To verify the effectiveness of the cancer subtype identification method based on whole-section histopathological images proposed in this application, the method was evaluated on two publicly available sarcoma datasets: TCGA-SARC and SARCOMA-Jilin. In the TCGA-SARC dataset, the eyepiece magnification of the sections is 40x; to preserve tissue structure, images with an objective magnification of 5x were selected for validation. In the SARCOMA-Jilin dataset, the eyepiece magnification of the sections is 20x; to ensure that the downsampling magnification of the samples in this dataset is as close as possible to that of TCGA-SARC, images with an objective magnification of 2x were selected for validation.

[0110] The SARC project, publicly available at TCGA, includes five sarcoma subtypes: synovial sarcoma (20 cases), leiomyosarcoma (155 cases), dedifferentiated liposarcoma (86 cases), undifferentiated sarcoma (107 cases), and fibromyxosarcoma (141 cases), totaling 509 whole-section histopathological images. The internal dataset SARCOA-Jilin contains 30 leiomyosarcomas, 27 synovial sarcomas, 52 undifferentiated sarcomas, and 52 liposarcomas. HE staining was performed in batches, and all samples were collected and labeled by experienced pathologists. The computational pathology research protocol was approved by the ethics committee and by Jilin University and the China-Japan Union Hospital.

[0111] Validation experiments were conducted on an NVIDIA TITAN RTX-24GB GPU equipped with 64GB of RAM, using PyTorch 1.8.0, Python 3.9, and CUDA 11.1. To evaluate the performance of the proposed model, this application reports the following metrics: Accuracy (ACC), Precision (P), Recall (R), F1 score (F1), and Area Under the ROC Curve (AUC). Given that this task is a multi-class classification problem, all performance metrics are calculated and reported using macro-average values. Furthermore, to evaluate the computational efficiency of the model, this application reports the inference time metric.

[0112] Experimental results show that the cancer subtype identification method and model based on whole-section histopathological images proposed in this application achieves a classification accuracy of 88.08% on the TCGA-SARC dataset and 89.36% on the internal dataset, outperforming other multi-instance learning methods. Furthermore, the inference speed is faster than other methods.

[0113] Furthermore, to analyze the ex post-hoc interpretability of the cancer subtype identification method and model based on whole-section histopathological images proposed in this application, this application also visualizes the inference results of an example sample. Please refer to [link to example]. Figure 3 The diagram shows a visualization of the attention process for the experimental samples. Figure 3 As shown, the image patch set within the blue box corresponds to high-probability scores, including diseased tissue regions identified by pathologists. This highlights the ability of image patch set-based classification to distinguish between suspicious and normal tissue regions within a slide. The direction of sampling point movement during inference is marked with blue arrows. Even when the image patch set contains a large number of blank image patches (a common feature of whole slides), sampling points gradually move towards tissue regions. Simultaneously, if the image patch set contains both normal and diseased tissue regions, the model adjusts to focus on the diseased tissue regions. As sampling points move towards points strongly correlated with cancer subtypes, the model assigns different attention scores to these points, enabling it to focus on feature vectors strongly supporting cancer subtype identification while ignoring irrelevant feature vectors. Overall, visualizing regions with high probabilities and high attention scores can provide pathologists with potential diagnostic clues, aiding in the discovery of histological features associated with cancer subtypes.

[0114] In summary, the cancer subtype identification method and model based on whole-section histopathological images proposed in this application have broad applicability and significant commercial potential. Specifically targeting the weakly supervised classification problem of cancer subtypes in whole-section pathology images, it is particularly suitable for scenarios lacking negative samples and possessing only section-level labels. This technology can be used to assist in intelligent clinical pathology diagnosis, contributing to the advancement of precision medicine, digital pathology analysis, and related AI-assisted diagnosis, and possesses broad market prospects and significant commercial value.

[0115] The above content is only a preferred embodiment of the present invention. For those skilled in the art, many changes can be made in the specific implementation and application scope based on the ideas of the present invention. As long as these changes do not depart from the concept of the present invention, they all fall within the protection scope of the present invention.

Claims

1. A method for cancer subtype identification based on whole-section histopathological images, characterized in that, The method includes: The whole-section histopathological images were segmented to construct multiple sets of image blocks containing tissue regions; For each set of image patches, the feature vector of each image patch in the set is extracted, and the extracted feature vectors are arranged according to the relative position of each image patch to obtain the feature image of the set of image patches. By selecting sampling points that are strongly correlated with cancer subtypes from the feature images of this image patch set, the fused feature vector of this image patch set is obtained; The cancer subtype category of the whole-section histopathological image is predicted based on the fused feature vector of each image patch set; The process of segmenting whole-section histopathological images to construct multiple sets of image blocks containing tissue regions includes: The whole-section histopathological image is cut into several image blocks, and multiple image block sets are divided using a 9×9 grid in a left-to-right and top-to-bottom order to preserve the contextual relationship and spatial position information of adjacent image blocks; wherein, the size of the image block is 224×224; Calculate the percentage of image patches belonging to tissue regions in each image patch set; The set of image blocks that belong to the tissue region is retained at least 30% to obtain multiple sets of image blocks containing the tissue region; The step of selecting sampling points strongly correlated with cancer subtypes from the feature images of the image patch set to obtain the fused feature vector of the image patch set includes: The number of channels in the feature image of this image patch set is reduced by a 1×1 convolution operation; Based on the size of the dimension-reduced feature image and the pre-defined number of sampling points, multiple sampling points and their corresponding coordinates are generated on the dimension-reduced feature image. Based on the feature vectors and location embedding features of each sampling point, a progressive update method is used to select sampling points that are strongly correlated with cancer subtypes, and at the same time, a fusion feature vector of the image patch set is generated.

2. The method as described in claim 1, characterized in that, The feature vectors and position embedding features of each sampling point are obtained in the following manner: For each sampling point, the feature vectors of the four integer points closest to that sampling point are weighted and averaged to obtain the feature vector of that sampling point. The coordinates of that sampling point are then subjected to a learnable linear transformation to obtain the position embedding feature of that sampling point.

3. The method as described in claim 1, characterized in that, Based on the feature vectors and location embedding features of each sampling point, a progressive update method is used to select sampling points that are strongly correlated with cancer subtypes, and a fused feature vector for the image patch set is generated, including: The feature vectors of each sampling point, the position embedding features, and the fused feature vector output from the previous iteration are summed element by element and then input into the Transformer encoder to obtain the fused feature vector output from the current iteration; wherein, the Transformer encoder includes a multi-head self-attention mechanism and a multilayer perceptron. A learnable linear transformation is performed on the fused feature vector output by the current iteration to obtain the sampling point offset, so as to update the coordinates of the sampling points until the iteration number is reached. The fused feature vector output by the last iteration is used as the fused feature vector of the image patch set.

4. The method as described in claim 1, characterized in that, The cancer subtype category of the whole-section histopathological image is predicted by the fused feature vector based on each image patch set, including: For each set of image patches, based on the fused feature vector of the set of image patches, a multilayer perceptron is used to predict the probability vector of the image patch perception set belonging to each cancer subtype category. The predicted probability vectors of each image patch set belonging to each cancer subtype category are averaged bitwise to obtain the cancer subtype category corresponding to the highest predicted probability.

5. A cancer subtype identification device based on whole-section histopathological images, characterized in that, The device includes: The image patch set generation module is used to cut whole-slice histopathological images to construct multiple image patch sets containing tissue regions; The feature extraction module is used to extract the feature vector of each image patch in each image patch set, and arrange the extracted feature vectors according to the relative position of each image patch to obtain the feature image of the image patch set. The feature selection and fusion module is used to filter out sampling points that are strongly correlated with cancer subtypes from the feature images of the image patch set, and obtain the fused feature vector of the image patch set. A classification module is used to predict the cancer subtype category of the whole-slice histopathological image based on the fused feature vector of each image patch set; The image patch set generation module, when segmenting a whole-section histopathological image to construct multiple image patch sets containing tissue regions, is specifically used for: The whole-section histopathological image is cut into several image blocks, and multiple image block sets are divided using a 9×9 grid in a left-to-right and top-to-bottom order to preserve the contextual relationship and spatial position information of adjacent image blocks; wherein, the size of the image block is 224×224; Calculate the percentage of image patches belonging to tissue regions in each image patch set; The set of image blocks that belong to the tissue region is retained at least 30% to obtain multiple sets of image blocks containing the tissue region; The feature selection and fusion module, when obtaining the fused feature vector of the image patch set by filtering sampling points strongly correlated with cancer subtype categories from the feature images of the image patch set, is specifically used for: The number of channels in the feature image of this image patch set is reduced by a 1×1 convolution operation; Based on the size of the dimension-reduced feature image and the pre-defined number of sampling points, multiple sampling points and their corresponding coordinates are generated on the dimension-reduced feature image. Based on the feature vectors and location embedding features of each sampling point, a progressive update method is used to select sampling points that are strongly correlated with cancer subtypes, and at the same time, a fusion feature vector of the image patch set is generated.

Images