Transformer-based WSI image classification method, system and related equipment

Abstract

The invention discloses a WSI image classification method, system and related equipment based on Transformer; A WSI image classification method based on Transformer includes: using OTSU algorithm to extract tissue from WSI image; obtaining the minimum rectangular area based on the extracted foreground tissue and performing patch cutting, each patch records the position information; inputting all patches into a pre-trained model and outputting a feature sequence, and forming a data packet with the corresponding cancer label, position information and all feature sequences of the WSI image; inputting the data packet into a position space information encoding module, and performing spatial information extraction and aggregation based on convolution kernels of different sizes; the data processed by the position space information encoding module is input into the Transformer model for category training and prediction. By using the feature sequence as input for prediction and using the position space information encoding module to fully utilize the spatial information of the WSI image, the prediction time is shortened and the prediction accuracy is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of image data processing, and in particular to a Transformer-based WSI image classification method, system and related equipment. Background Art

[0002] In the current field of whole-slice image (WSI) prediction, although deep learning technology, especially convolutional neural network (CNN), has made significant progress, there are still some problems that need to be solved. First, the high resolution and large amount of data of WSI images put tremendous pressure on computing resources, and traditional CNNs may encounter performance bottlenecks when processing such a large amount of data. Secondly, the analysis of WSI images often relies on the subjective judgment of pathologists, which not only increases the uncertainty of diagnosis, but also limits the popularity of automated analysis. In addition, the class imbalance problem in WSI images is also a challenge. Some rare pathological features may be underrepresented in the training data, resulting in limited generalization ability of the model in practical applications.

[0003] The emergence of the Transformer model provides a new approach to solving these problems. Unlike traditional CNNs, the Transformer model can better capture long-distance dependencies in sequence data through the self-attention mechanism, which is crucial for understanding the complex structures and patterns in WSI images; however, directly applying the Transformer model to WSI image analysis also faces challenges because it requires labeling each image block, which is impractical in practice.

[0004] The multiple instance learning (MIL) framework provides an effective way to solve this problem. Under the MIL framework, each WSI image is regarded as a sample bag containing multiple instances (image patches), and the model can learn from the labels of the entire sample bag without the need to annotate each instance separately. This method not only reduces the cost of data annotation, but also enables the model to understand the image content from a more macro perspective, thereby improving the accuracy of prediction.

[0005] For example, the patent document (CN116309333A) performs training by inputting feature sequences, but it performs analysis through each patch and cannot fully consider the spatial location information of the patch and the overall analysis of the WSI image. Summary of the invention

[0006] To solve the above problems, the present invention provides a Transformer-based WSI image classification method, comprising: The OTSU algorithm is used to extract tissue from the WSI image. The OTSU algorithm maximizes the inter-class variance between the foreground and the background to obtain the optimal threshold and thus realize automatic segmentation of the WSI image. Obtaining a minimum rectangular area based on the extracted foreground tissue and performing patch cutting, each patch recording position information; Input all the patches into the pre-trained model and output the feature sequence, and form a data package with the corresponding cancer label, location information and all feature sequences of the WSI image; The data packet is input into the position spatial information encoding module, and spatial information is extracted and aggregated based on convolution kernels of different sizes; The data processed by the position space information encoding module is input into the Transformer model for category training and prediction.

[0007] The WSI image is preliminarily analyzed to obtain the corresponding cancer label; the preliminarily analysis can be obtained by comparing the WSI image with a public data set and / or by a doctor's analysis.

[0008] The method of obtaining the minimum rectangular area based on the extracted foreground tissue and performing patch cutting, and recording the position information of each patch includes the following steps: Binarizing the WSI image based on the optimal threshold; The boundingRect function of the opencv algorithm is used to obtain the minimum external matrix of the foreground tissue; based on the minimum external matrix, the foreground tissue is patch cut, and the image with insufficient length and / or width is filled with zero value. The patch position information is ,in is the i-th patch in the length direction, where is the jth patch in the width direction.

[0009] The corresponding cancer labels and all feature sequences of the WSI images in the data packet are stored in a one-dimensional vector, and the one-dimensional vector is in the form of: cancer label + feature sequence ( x512, where 1 is the cancer label, N is the number of patches, and 512 is the length of the feature sequence), and the position information Stored together to form a data package; all WSI data packages are divided into training set, validation set and test set according to the preset ratio.

[0010] The data packet is input into the position spatial information encoding module, and spatial information extraction and aggregation based on convolution kernels of different sizes include the following steps: After removing the cancer label from the data packets in the training set, the location information is obtained middle Value and value; Value and As the length and width of the space recovery matrix, the length of the feature sequence is taken as the depth of the space recovery matrix. The feature space of x512; if there are missing positions, fill them with zero values; Use convolution kernels of different sizes to extract spatial information of different scales in the feature space, and superimpose the results of each convolution; Use a 1×1 convolution kernel to re-output the superimposed result as X_max×Y_max×512; The coordinates of the location information are sequentially Extract the x512 result and expand it from small to large ×512+1 length sequence, where 1 is the cancer label.

[0011] Among them, the data processed by the position spatial information encoding module is input into the Transformer model for category training and prediction, including: outputting the cancer label of the WSI image; obtaining the best model through training, and the trained model will be used to predict the full digital slice image of the WSI image, and each WSI image corresponds to a heat map to display the model's focus area.

[0012] The present invention also provides a Transformer-based WSI image classification system, which can apply the Transformer-based WSI image classification method as described in any one of the above items, including: Feature extraction module; the feature extraction module is used to extract the foreground tissue of the WSI image and perform patch cutting in the minimum rectangular area; the patch outputs the pre-trained model output feature sequence, and the feature sequence, the corresponding cancer label and location information of the WSI image form a data packet; An information fusion processing module; the information fusion processing module is connected to the feature extraction module to receive the data packet; the information fusion processing module extracts and aggregates spatial information of the data packet through convolution kernels of different sizes and outputs it to the Transformer model for category training and prediction.

[0013] Among them, the information fusion processing module includes a position spatial information encoding module and a Transformer model; the position spatial information encoding module receives a data packet and extracts and aggregates the spatial information of the data packet through convolution kernels of different sizes; the Transformer model receives the data processed by the position spatial information encoding module for category training and prediction.

[0014] The present invention also provides an electronic device, comprising: a memory and a processor coupled to each other, wherein the processor is used to execute program instructions stored in the memory to implement the Transformer-based WSI image classification method as described in any one of the above items.

[0015] The present invention also provides a computer-readable storage medium, wherein the computer-readable storage medium stores program data, and the program data can be executed by a processor to implement the Transformer-based WSI image classification method as described in any one of the above items.

[0016] Beneficial effects of the present invention: Different from the prior art, the present invention discloses a WSI image classification method, system and related equipment based on Transformer; A WSI image classification method based on Transformer includes: using the OTSU algorithm to extract tissue from WSI images; obtaining the minimum rectangular area based on the extracted foreground tissue and performing patch cutting, and recording the position information for each patch; inputting all patches into a pre-trained model and outputting a feature sequence, and forming a data packet with the corresponding cancer label, position information and all feature sequences of the WSI image; the data packet is input into a position space information encoding module, and spatial information is extracted and aggregated based on convolution kernels of different sizes; the data processed by the position space information encoding module is input into the Transformer model for category training and prediction. By using the feature sequence as input for prediction and using the position space information encoding module to fully utilize the spatial information of the WSI image, the prediction time is shortened and the prediction accuracy is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 A flowchart of a Transformer-based WSI image classification method provided by an embodiment of the present invention; Figure 2 for Figure 1 Schematic diagram of the steps of S12; Figure 3 A schematic diagram of the structure of a Transformer-based WSI image classification system provided by an embodiment of the present invention; Figure 4A schematic diagram of the workflow of a Transformer-based WSI image classification system provided in one embodiment of the present invention; Figure 5 is a structural schematic diagram of an embodiment of a computer device of the present invention; Figure 6 It is a schematic diagram of the structure of an embodiment of a computer-readable storage medium of the present invention. DETAILED DESCRIPTION

[0018] The following are specific embodiments of the present invention and the accompanying drawings to further describe the technical solution of the present invention, but the present invention is not limited to these embodiments.

[0019] like Figure 1 As shown, Figure 1 A flowchart of a Transformer-based WSI image classification method provided in an embodiment of the present invention includes the following steps: S11: Use the OTSU algorithm to extract tissue from the WSI image. The OTSU algorithm maximizes the inter-class variance between the foreground and the background to obtain the optimal threshold and then realize automatic segmentation of the WSI image. Specifically, the optimal threshold is the maximum threshold. The maximum threshold is selected as the segmentation threshold to binarize the WSI image. The WSI image is segmented into foreground tissue and background tissue based on the segmentation threshold to realize the extraction of foreground tissue.

[0020] The method used in this embodiment is the maximum inter-class variance method, which is an adaptive threshold determination method. It divides the image into two parts, background and foreground, according to the grayscale characteristics of the image. The larger the inter-class variance between the background and the foreground, the greater the difference between the two parts that constitute the image. When part of the foreground is wrongly divided into the background or part of the background is wrongly divided into the background, the difference between the two parts will become smaller. Therefore, the segmentation that maximizes the inter-class variance means the minimum probability of misclassification.

[0021] In this embodiment, after the length and width of the WSI original image are reduced by 40 times, the OTSU algorithm is used to adaptively find the threshold of each WSI image, and the foreground part of the tissue is extracted by comparing the threshold. In this embodiment, after the length and width of the WSI original image are reduced by 40 times, the foreground tissue is extracted. Different reduction factors can be selected for different WSI images, and no limitation is made here.

[0022] S12: Based on the extracted foreground tissue, the minimum rectangular area is obtained and patch cutting is performed, and each patch records the position information; in this embodiment, the pixel size of the patch is 256×256, and in other embodiments, the pixel size of the patch can be 128×128 or 512×512, without any limitation.

[0023] By extracting the minimum rectangular area and performing patch cutting, the amount of data processed can be significantly reduced and the computational efficiency can be improved; position information is very important for subsequent spatial information encoding and feature fusion, which helps the model better understand the spatial relationship in the image.

[0024] S13: All patches are input into the pre-trained model and the feature sequence is output. The corresponding cancer label, position information and all feature sequences of the WSI image form a data packet. The cancer label indicates whether the WSI image has cancer. The position information indicates the position coordinates of each patch in the original image. The feature sequence indicates a matrix composed of feature vectors generated corresponding to all patches. In this embodiment, the pre-trained model is ResNet50, and one patch generates a 512-dimensional feature vector. The 512-dimensional feature vector of each patch is saved to form a one-dimensional vector N×512, where N is the number of patches.

[0025] S14: The data packet is input into the position spatial information encoding module, which extracts and aggregates spatial information based on convolution kernels of different sizes. By extracting spatial information of different scales through multi-scale convolution, the model can better understand the spatial relationship in the image and improve the accuracy of classification. The feature fusion step stitches together feature maps of different scales, which enhances the expressiveness of the features and helps capture complex structures and patterns in the image. Through position information recovery, the processed feature sequence retains the position information of each patch in the original image, which helps the model better understand the image content.

[0026] S15: The data processed by the position spatial information encoding module is input into the Transformer model for category training and prediction; by adding position encoding, the model can better understand the spatial position of each patch and improve the accuracy of classification; the Transformer model can effectively fuse the features of different patches, capture local and global information in the image, and improve the expressiveness of the model.

[0027] In summary, the present invention provides a WSI image classification method based on Transformer, including: using the OTSU algorithm to extract tissue from WSI images; obtaining the minimum rectangular area based on the extracted foreground tissue and performing patch cutting, and recording the position information of each patch; inputting all patches into a pre-trained model and outputting a feature sequence, and forming a data packet with the corresponding cancer label, position information and all feature sequences of the WSI image; inputting the data packet into a position space information encoding module, and performing spatial information extraction and aggregation based on convolution kernels of different sizes; and inputting the data processed by the position space information encoding module into the Transformer model for category training and prediction. By using the feature sequence as input for prediction and using the position space information encoding module to fully utilize the spatial information of the WSI image, the prediction time is shortened and the prediction accuracy is improved.

[0028] In one embodiment, a preliminary analysis is performed on the WSI image to obtain a corresponding cancer label; the preliminary analysis may be obtained by comparing the WSI image with a public data set and / or by a doctor's analysis; if the cancer label is obtained by comparing the WSI image with the public data, the following steps are included: Collect annotated public WSI datasets, such as TCGA (The Cancer Genome Atlas), CAMELYON16 / 17, etc.; Preprocess the images in the public dataset, including resizing and normalizing, to make them consistent with the WSI image format to be analyzed; Use pre-trained deep learning models (e.g., ResNet50) to extract features from public datasets and WSI images to be analyzed; Compare the features of the WSI image to be analyzed with the features of the annotated images in the public dataset to find the most similar image; use methods such as cosine similarity and Euclidean distance to compare features; the label of the most similar image is used as the preliminary label of the WSI image to be analyzed.

[0029] Cancer labels can be obtained through doctor analysis alone, or preliminary labels can be obtained using public data sets, and then the doctor can judge whether the preliminary labels are correct. Using public data sets and pre-trained models for feature comparison can reduce subjective errors in doctor labeling and improve label consistency.

[0030] In one embodiment, if Figure 2 As shown, Figure 2 for Figure 1 The step diagram of S12 in FIG. 1 includes the following steps: The WSI image is binarized based on the optimal threshold. Using the optimal threshold for binarization can effectively separate the foreground and background and improve the accuracy of segmentation. The binarized image clearly separates the foreground and background, which is convenient for subsequent feature extraction, patch cutting and other processing steps.

[0031] The boundingRect function of the opencv algorithm is used to obtain the minimum bounding rectangle of the foreground tissue; the minimum bounding rectangle can accurately locate the position and range of the foreground tissue. Through the minimum bounding rectangle, the processing area can be limited to a smaller range, reducing the amount of calculation and memory consumption.

[0032] Based on the minimum circumscribed matrix, the foreground tissue is patched, and images with insufficient length and / or width are filled with zero values. The patches with insufficient length and / or width are filled with zero values ​​to ensure that all patches are of the same size, which is convenient for input into the deep learning model for feature extraction. The patch location information is ,in is the i-th patch in the length direction, where is the jth patch in the width direction; each patch and its position information are saved so that its position in the original image can be restored during subsequent processing.

[0033] In one embodiment, the corresponding cancer labels and all feature sequences of the WSI images in the data package are stored in a one-dimensional vector, and the one-dimensional vector format is: cancer label + feature sequence format ( x512, where 1 is the cancer label, N is the number of patches, and 512 is the length of the feature sequence), and location information Stored together to form a data package; all WSI data packages are divided into training set, validation set and test set according to the preset ratio.

[0034] The feature sequence and cancer label in each data packet are separated to form a one-dimensional vector in the form of 1+N×512, where 1 is the cancer label; the one-dimensional vector and the position information are placed separately, and the position information and the one-dimensional vector together form a data packet; the position information is saved separately for easy query and use when needed.

[0035] One WSI corresponds to one data packet, and multiple WSIs correspond to multiple data packets; all WSI data packets are divided into training set, verification set and test set according to a preset ratio (for example, 80% training set, 10% verification set, and 10% test set); for example, the preset ratio is 80% training set, 10% verification set and 10% test set; in other embodiments, the preset ratio can be taken as another value according to the specific situation, and there is no limitation on this.

[0036] In one embodiment, the data packet is input into the position spatial information encoding module, and the spatial information extraction and aggregation based on convolution kernels of different sizes include the following steps: removing the cancer label from the data packet in the training set, and obtaining the position information middle Value and Value; that is, the maximum coordinate value of all patches in the length and width directions.

[0037] Value and The values ​​are used as the length and width of the space recovery matrix, and the length of the feature sequence is used as the depth of the space recovery matrix. x512 feature space; if there are missing positions, fill them with zero values; ensure the integrity and consistency of the feature space to avoid errors caused by missing data.

[0038] Use convolution kernels of different sizes to extract spatial information of different scales from the feature space. For example, use convolution kernels of different sizes (such as 3×3, 5×5, 7×7) to perform convolution operations on the feature space to extract spatial information of different scales; superimpose and concat the results of each convolution to output a new feature space.

[0039] The above results are integrated using a 1×1 convolution kernel to fuse features of different scales while ensuring that the output is x512, ensuring that the depth of the output feature space is still 512, which helps to enhance the expressiveness of the features.

[0040] The coordinates of the location information are sequentially Extract the x512 result and expand it from small to large ×512+1 length sequence, where 1 is the cancer label. The extracted feature sequences are arranged from small to large according to the position information to form a shape of ×512 feature vector, where is the number of patches; add the cancer label after the feature vector to form the final feature sequence, the shape is ×512 + 1.

[0041] In one embodiment, the data processed by the position spatial information encoding module is input into the Transformer model for category training and prediction, including: outputting a cancer label of the WSI image; obtaining the best model through training, and the trained model is used to predict the full digital slice image of the WSI image, and each WSI image corresponds to a heat map to show the model's attention area; the self-attention mechanism of the Transformer model can generate an attention heat map to show the area that the model pays attention to during the classification process, thereby enhancing the interpretability of the model. ×512+1 sequence is input into the Transformer model, and the cancer label is first extracted as the output result. ×512 sequence input model, set parameter training model. In this embodiment, the self-attention module uses the approximate self-attention Nyström Method, the optimization function uses Lookahead, and the parameters are k=5, alpha=1e-4, beta=0.5, epsilon=1e-8. The loss function uses cross entropy loss. The number of iterations epoch is set to 400, and the early stopping technique is set. When the accuracy does not improve for 5 consecutive epochs, the training is stopped. In other embodiments, the values ​​of the parameters can be selected separately, without any limitation.

[0042] The present invention also provides a Transformer-based WSI image classification system, which can apply the Transformer-based WSI image classification method of any of the above embodiments, such as Figure 3 As shown, Figure 3 A schematic diagram of the structure of a Transformer-based WSI image classification system provided in one embodiment of the present invention includes a feature extraction module (SPM) and an information fusion processing module (SETI).

[0043] The feature extraction module is used to extract the foreground tissue of the WSI image and perform patch cutting with the smallest rectangular area; the patch output pre-trained model outputs the feature sequence, and the feature sequence, the corresponding cancer label and location information of the WSI image constitute a data packet; the feature extraction module uses a pre-trained model (such as ResNet50) to extract high-dimensional features, which can capture complex features in the image.

[0044] The information fusion processing module is connected to the feature extraction module to receive data packets; the information fusion processing module extracts and aggregates spatial information of the data packets through convolution kernels of different sizes and outputs it to the Transformer model for category training and prediction; the information fusion processing module extracts spatial information of different scales through multi-scale convolution, which helps the model better understand the spatial relationship in the image.

[0045] Combining the feature extraction module and the information fusion processing module, the performance of the WSI image classification task can be significantly improved, especially when dealing with WSI image imbalanced data and multi-class classification problems.

[0046] In one embodiment, the information fusion processing module includes a position spatial information encoding module and a Transformer model; the position spatial information encoding module receives a data packet and extracts and aggregates spatial information from the data packet through convolution kernels of different sizes; the position spatial information encoding module extracts spatial information of different scales through multi-scale convolution, which helps the model better understand the spatial relationship in the image. The Transformer model receives the data processed by the position spatial information encoding module for category training and prediction. The self-attention mechanism of the Transformer model can generate an attention heat map to show the area that the model pays attention to during the classification process, thereby enhancing the interpretability of the model.

[0047] Attention heatmaps help doctors understand the model’s decision-making process and improve the confidence of diagnosis like Figure 4 As shown, Figure 4 The schematic diagram of the workflow of the Transformer-based WSI image classification system provided in one embodiment of the present invention includes the following steps: S1: Prepare WSI pathology data and corresponding cancer labels; S2: Tissue extraction from WSI images using the OTSU algorithm; S3: Based on the extracted tissue area, the minimum rectangular area is extracted for patch cutting, and the position information of each patch is recorded; S4: Input the patch into the pre-trained ResNet50 model, output 512 feature sequences, and save the labels of each WSI and all sequences into a data package; S5: Input the saved data packet into the spatial information encoding module, extract and aggregate the spatial information through convolution kernels of different sizes, and then output; S6: The data after the position spatial information encoding module is input into the Transformer model for category training and prediction. Finally, an attention heat map is generated based on the self-attention mechanism of the Transformer model.

[0048] For the above embodiment, the present invention provides a computer device, see Figure 5 , Figure 5 1 is a schematic diagram of a computer device according to an embodiment of the present invention. The computer device includes a memory and a processor, wherein the memory and the processor are coupled to each other, the memory stores program data, and the processor is used to execute the program data to implement the steps of any embodiment of the above-mentioned Transformer-based WSI image classification method.

[0049] In this embodiment, the processor may also be referred to as a CPU (Central Processing Unit). The processor may be an integrated circuit chip having signal processing capabilities. The processor may also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0050] The method of the above embodiment can be implemented in the form of a computer program, so the present invention provides a computer readable storage medium, see Figure 6 , Figure 6 1 is a schematic diagram of a computer-readable storage medium according to an embodiment of the present invention. The computer-readable storage medium stores program data that can be run by a processor, and the program data can be executed by the processor to implement the steps of any embodiment of the above-mentioned Transformer-based WSI image classification method.

[0051] The computer-readable storage medium in this embodiment may be a medium that can store program data, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, or may be a server that stores the program data. The server may send the stored program data to other devices for execution, or may execute the stored program data by itself.

[0052] The above description is only an implementation method of the present application, and does not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly used in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A Transformer-based WSI image classification method, characterized in that: include: The OTSU algorithm is used to extract tissue from the WSI image. The OTSU algorithm maximizes the inter-class variance between the foreground and the background to obtain the optimal threshold and thus realize automatic segmentation of the WSI image. Obtaining a minimum rectangular area based on the extracted foreground tissue and performing patch cutting, each patch recording position information; Input all the patches into the pre-trained model and output the feature sequence, and form a data package with the corresponding cancer label, location information and all feature sequences of the WSI image; The data packet is input into the position spatial information encoding module, and spatial information is extracted and aggregated based on convolution kernels of different sizes; The data processed by the position space information encoding module is input into the Transformer model for category training and prediction.

2. The Transformer-based WSI image classification method according to claim 1, characterized in that: A preliminary analysis is performed on the WSI image to obtain a corresponding cancer label; the preliminary analysis can be obtained by comparing the WSI image with a public dataset and / or by a doctor performing analysis.

3. The Transformer-based WSI image classification method according to claim 2, characterized in that: The step of obtaining the minimum rectangular area based on the extracted foreground tissue and performing patch cutting, and recording the position information of each patch, comprises the following steps: Binarizing the WSI image based on the optimal threshold; The boundingRect function of the opencv algorithm is used to obtain the minimum external matrix of the foreground tissue; based on the minimum external matrix, the foreground tissue is patch cut, and the image with insufficient length and / or width is filled with zero value. The patch position information is ,in is the i-th patch in the length direction, where is the jth patch in the width direction.

4. The Transformer-based WSI image classification method according to claim 3, characterized in that: The corresponding cancer labels and all feature sequences of the WSI images in the data packet are stored in a one-dimensional vector, and the one-dimensional vector is in the form of: cancer label + feature sequence ( , where 1 is the cancer label, N is the number of patches, and 512 is the length of the feature sequence), and the position information , stored together to form a data packet; All WSI data packets are divided into training set, validation set and test set according to the preset ratio.

5. The Transformer-based WSI image classification method according to claim 4, characterized in that: The data packet is input into the position spatial information encoding module, and the spatial information extraction and aggregation based on convolution kernels of different sizes include the following steps: removing the cancer label from the data packet in the training set, and obtaining the position information middle Value and value; Value and The values ​​are used as the length and width of the space recovery matrix, and the length of the feature sequence 512 is used as the depth of the space recovery matrix. The feature space of x512; If there are missing positions, fill them with zero values; Use convolution kernels of different sizes to extract spatial information of different scales from the feature space, and superimpose the results of each convolution; use a 1×1 convolution kernel to re-output the superimposed results as ×512 The coordinates of the location information are sequentially Extract the x512 result and expand it from small to large ×512+1 length sequence, where 1 is the cancer label.

6. The Transformer-based WSI image classification method according to claim 5, characterized in that: The data processed by the position spatial information encoding module is input into the Transformer model for category training and prediction, and the steps include: outputting the cancer label of the WSI image; obtaining the best model through training, and the trained model will be used to predict the full digital slice image of the WSI image, and each WSI image corresponds to a heat map to display the model's focus area.

7. A Transformer-based WSI image classification system, which can apply the Transformer-based WSI image classification method according to any one of claims 1 to 6, characterized in that: include: Feature extraction module; The feature extraction module is used to extract the foreground tissue of the WSI image and perform patch cutting in the smallest rectangular area; the patch outputs the feature sequence output by the pre-training model, and the feature sequence, the corresponding cancer label and location information of the WSI image form a data packet; An information fusion processing module; the information fusion processing module is connected to the feature extraction module to receive the data packet; the information fusion processing module extracts and aggregates spatial information of the data packet through convolution kernels of different sizes and outputs it to the Transformer model for category training and prediction.

8. The Transformer-based WSI image classification system according to claim 7, characterized in that: The information fusion processing module includes a position spatial information encoding module and a Transformer model; the position spatial information encoding module receives a data packet and extracts and aggregates spatial information from the data packet through convolution kernels of different sizes; the Transformer model receives the data processed by the position spatial information encoding module for category training and prediction.

9. An electronic device, characterized in that: The electronic device comprises: a memory and a processor coupled to each other, wherein the processor is used to execute program instructions stored in the memory to implement the Transformer-based WSI image classification method according to any one of claims 1 to 6.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores program data, and the program data can be executed by a processor to implement the Transformer-based WSI image classification method according to any one of claims 1 to 6.

Images