Pathological panoramic section image feature extraction method for visual language large model

Abstract

The application discloses a pathological panoramic slice image feature extraction method for a visual language large model, and belongs to the technical field of medical image processing and computer vision. The method comprises the following steps: preprocessing a pathological panoramic slice image to obtain a basic feature map; processing the basic feature map through a feature encoding module to obtain pre-training features, and performing dimension compression on the pre-training features; cooperatively updating the compressed features by combining a local attention mechanism and a global attention mechanism; inputting high-level features into a feature fusion module, fusing the high-level features with text features, and aligning the high-level features and the text features to a same semantic space to generate final visual features for the visual language large model. Through encoding, feature extraction and feature alignment on the input panoramic slice image, the large model can have the ability to process pathological panoramic slice images, the pathological features can be spliced and fused, and the inference ability of the large model can be used to improve the accuracy of computational pathology.

Description

Technical Field

[0001] This invention relates to the fields of medical image processing and computer vision technology, and in particular to a method for feature extraction from panoramic pathological slice images for large-scale visual language models. Background Technology

[0002] With the rapid development of digital pathology, whole-slide images (WSI) have become an important tool for modern pathological diagnosis. These high-resolution digital pathological slides contain rich morphological information, providing an important data foundation for the development of computational pathology. However, traditional computer vision methods face many challenges when processing pathological images, including huge image sizes, complex and diverse features, and high annotation costs. In recent years, methods that extract features from pathological images using pre-trained base models have been widely adopted, enabling rapid and effective feature extraction and subsequent modeling through low-cost training.

[0003] On the other hand, with the rapid development of artificial intelligence technology, Vision-Language Large Models have demonstrated powerful capabilities in multimodal understanding tasks. These models can process both visual and textual information simultaneously, achieving significant results in tasks such as image understanding, visual question answering, and image description generation. However, the application of Vision-Language Large Models in the field of medical pathology faces unique challenges. Existing Vision-Language Large Models often employ natural image processing methods rather than the feature extraction methods used in computational pathology. Due to the massive size of panoramic pathological slide images, these Vision-Language Large Models suffer from high computational resource consumption and slow processing speed, making them almost impractical. Furthermore, Vision-Language Large Models are often trained on natural images, limiting their ability to extract specialized information such as cell morphology, tissue structure, and lesion features from pathological images.

[0004] Therefore, developing a large-scale visual language model suitable for panoramic pathological slide images is of great significance. It can provide pathologists with better decision-making, support and assistance, reduce labor costs and burdens, and improve the accuracy and applicability of downstream tasks. Summary of the Invention

[0005] Purpose of the invention: To address the above problems, the purpose of this invention is to provide a method for feature extraction from pathological panoramic slice images for large-scale visual language models.

[0006] Technical solution: The present invention provides a method for feature extraction from pathological panoramic slide images for large-scale visual language models, comprising the following steps:

[0007] Step 1: Obtain the panoramic pathological slide image to be processed and perform preprocessing. Input the preprocessed panoramic pathological slide image into the pathological basic module to extract the basic feature map.

[0008] Step 2: Process the basic feature map through the pre-trained feature encoding module to extract pre-trained features containing tissue segmentation semantics;

[0009] Step 3: Compress the dimensions of the pre-trained features using convolutional layers to obtain compressed features;

[0010] Step 4: Combine local attention mechanism and global attention mechanism to collaboratively update compressed features, so as to fuse local details and global context information to obtain high-level features;

[0011] Step 5: Input the high-level features into the feature fusion module, fuse them with the text features, and align them to the same semantic space to generate the final visual features for the large visual language model.

[0012] Furthermore, the feature encoding module is a convolutional multilayer perceptron, which includes convolutional kernels with a size of [missing information]. convolutional layers Three identical first linear layers Second linear layer Third linear layer and a SiLU activation function ;

[0013] Using a convolutional multilayer perceptron to process the basic feature map of the input The calculation process is expressed as follows:

[0014] ,

[0015] in, These are pre-trained features.

[0016] Furthermore, step 4 includes:

[0017] Set the width and height dimensions as The compressed feature data is segmented into windows and uniformly cropped into... Square area of ​​size , will the Each region is denoted as and obtain the foreground area ;in, Indicates the height of the panoramic slice image. Indicates the width of the panoramic slice image. Indicates the size of the image patch. Indicates the kernel size;

[0018] For those in Input features of the region First, calculate the local attention. The calculation formula is:

[0019] ,

[0020] in, This represents a multi-head attention calculation method. To be simultaneously in Region and The input set of the region;

[0021] pass After the local attention calculation, local features are obtained. ;

[0022] For local features Recalculate global attention The calculation formula is:

[0023] ,

[0024] in, For being in The input set of the region; This refers to the high-level features obtained.

[0025] Furthermore, the steps of acquiring and preprocessing the panoramic pathological slide images to be processed include:

[0026] First, the red, green, and blue color channels of the pathological panoramic slide image are converted into hue, saturation, and brightness color spaces. Then, threshold segmentation is performed based on the saturation and brightness channels. The threshold segmentation results are then dilated and eroded to remove free and tissue-bound blank areas, distinguishing between the foreground area with tissue and the blank background area. Finally, the entire pathological panoramic slide image is cropped into image blocks of the same size, and image blocks whose background area exceeds a set threshold are discarded.

[0027] Furthermore, step 5 includes:

[0028] High-level features are compressed using convolutional layers to obtain visual features;

[0029] Obtain the foreground region of the tissue segmentation, compare the coordinates of each feature in the original pathological panoramic slice to see if the corresponding region is a tissue part. If the tissue area of ​​the corresponding region is less than the threshold, discard the feature, and rearrange all the remaining features into two-dimensional features to obtain the final visual features.

[0030] Furthermore, the pathology foundation module is a visual transformer pre-trained on a large pathology image dataset.

[0031] Beneficial effects: Compared with the prior art, the significant advantages of this invention are:

[0032] 1. This invention integrates the basic pathology module into the visual language model, making full use of the rich information in panoramic slice images and the performance of the large visual language model to provide precise support for clinical decision-making.

[0033] 2. In view of the large size of panoramic slice images, this invention extracts features with low space occupation and high information content through pathological basic modules and feature compression, respectively, which solves the problem that traditional large vision models cannot handle the huge size of panoramic slice images and realizes efficient and feasible visual feature processing.

[0034] 3. This invention strengthens and updates features by fusing tissue segmentation knowledge with feature encoding module and feature update, and extracts feature distributions with prior knowledge of tissue differences for the foreground and background respectively.

[0035] 4. This invention extracts visual features that can be understood by a large language model through a feature fusion module, and combines the information processing capabilities of the large language model with the rich details of the pathological panoramic slice image to achieve accurate prediction in the field of medical pathology. Attached Figure Description

[0036] Figure 1 This is a schematic diagram of the entire process of a method for feature extraction from panoramic pathological slide images used in a large visual language model.

[0037] Figure 2 This is a flowchart for extracting basic feature maps from panoramic pathological slide images;

[0038] Figure 3 The flowchart shows the pre-training process and the pre-training feature extraction process for the feature encoding module.

[0039] Figure 4 A flowchart for compressing pre-trained features to obtain compressed features;

[0040] Figure 5 A flowchart for obtaining high-level features and visual features. Detailed Implementation

[0041] The embodiments of the present invention will be further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are merely illustrative of the present invention and not intended to limit the scope of the invention. Furthermore, it should be noted that, for ease of description, the accompanying drawings show only the parts relevant to the embodiments of the present invention, and not all structures.

[0042] In the following description, specific details such as target system architecture and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods are omitted so as not to obscure the description of this application with unnecessary detail.

[0043] It should be understood that, when used in this application specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or a collection thereof.

[0044] It should also be understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0045] Furthermore, in the description of this application and the appended claims, the terms "first," "second," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0046] References to "one embodiment" or "some embodiments" in this specification mean that one or more embodiments of this application include the target features, structures, or characteristics described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized.

[0047] The overall flowchart of the pathological panoramic slide image feature extraction method for large-scale visual language models described in this embodiment is as follows: Figure 1 As shown, the method includes the following steps:

[0048] Step 1: Obtain the panoramic pathological slide image to be processed and perform preprocessing. Input the preprocessed panoramic pathological slide image into the pathology basic module to extract the basic feature map.

[0049] Furthermore, the steps of acquiring and preprocessing the panoramic pathological slide images to be processed include:

[0050] First, the red, green, and blue color channels of the pathological panoramic slide image are converted into hue, saturation, and brightness color spaces. Then, threshold segmentation is performed based on the saturation and brightness channels. The threshold segmentation results are then dilated and eroded to remove free and tissue-bound blank areas, distinguishing between the foreground area with tissue and the blank background area. Finally, the entire pathological panoramic slide image is cropped into image blocks of the same size, and image blocks whose background area exceeds a set threshold are discarded.

[0051] Furthermore, the pathology foundation module is a vision transformer (ViT) pre-trained on a large pathology image dataset.

[0052] In one example, the input panoramic slice image is preprocessed for feature extraction using the pathology-based module. The preprocessing includes foreground segmentation based on color space transformation, thresholding, and dilation / erosion morphological operations; fixed-size image patch cropping; and image scaling. Specifically, the RGB color channel image is first converted to the hue, saturation, and brightness HSV color space. Then, thresholding is performed based on the saturation and brightness channels. The thresholded segmentation results are then dilated and eroded to remove small targets and voids, i.e., free and intra-tissue blanks. Finally, the entire panoramic slice image is cropped into image patches of the same size, discarding patches where the background area exceeds a set threshold of 75%. Before preprocessing, forward propagation is performed to extract panoramic slice image features, the process of which is as follows: Figure 2 The forward propagation diagram is shown below. Each panoramic slice image is processed into multiple small image patches for subsequent feature extraction. The small image patches are first cut out from the 20x image. Pixel-sized image blocks, scaled to Pixels. After preprocessing, image patches are acquired, and feature extraction is performed using a pre-trained ViT model. The class tokens in the ViT model are used as the feature results, and each image patch has 1024 dimensions. After feature extraction, the features are stitched together according to the position of each image patch in the panoramic slice image to obtain the basic pathological features. Background features are temporarily filled with values ​​of 0 to facilitate subsequent processing. For dimensions of height... ,width A panoramic slice image with a pixel matrix size of Where 3 represents the values ​​of the RGB color channels (red, green, blue). Each cut image block is 256 in size, so the length and width can be cut into... and The image patches are extracted. The extracted features have a length of 1024 dimensions. The matrix size of the pathological basic features obtained after concatenating the features is [size missing]. .

[0053] In one example, the pre-trained ViT model is fine-tuned. First, a UNI network based on a visual transformer architecture is loaded as initial parameters. Then, to address the domain shift issue inherent in certain tasks, existing techniques such as the DINOv2 self-supervised learning algorithm and the LoRA fine-tuning algorithm are used for fine-tuning training on a proprietary pathology dataset. The hyperparameters used during training maintain the DINOv2 hyperparameter settings, using image sizes of [insert image size here]. Pixel.

[0054] Step 2: Process the basic feature map through the pre-trained feature encoding module to extract pre-trained features containing tissue segmentation semantics.

[0055] Furthermore, the feature encoding module is a convolutional multilayer perceptron, which includes convolutional kernels with a size of [missing information]. convolutional layers Three identical first linear layers Second linear layer Third linear layer and a SiLU activation function ;

[0056] Using a convolutional multilayer perceptron to process the basic feature map of the input The calculation process is expressed as follows:

[0057] ,

[0058] in, These are pre-trained features.

[0059] Combination Figure 3 As shown, step 2 first pre-trains the feature encoding module on manually labeled tissue segmentation, and then applies the trained pre-trained feature encoding module ConvMLP to the forward propagation process of the large model pathological vision module. This module calculates the basic features to obtain the pre-trained features.

[0060] In one example, a custom Convolutional Multilayer Perceptron (ConvMLP) is used as a pre-trained feature encoding module to further extract pre-trained features with downstream task knowledge from the pathological basic features extracted by the pathological basic model. A schematic diagram of the ConvMLP structure is shown below. Figure 3 As shown, the convolutional multilayer perceptron contains convolutional kernels with a size of [missing information]. convolutional layers Three linear layers with identical structures , , and a SiLU activation function Compared to traditional multilayer perceptrons using linear layers, this example uses ConvMLP to capture information from surrounding image patches. Specifically, a 3×3 convolutional layer and SiLU activation function are used as the gate path to adjust feature selection. A 1×1 convolutional layer extracts core features, and a residual connection is used to resample and adjust feature dimensions using a 1×1 convolutional kernel to prevent feature dimension discrepancies. The input to this ConvMLP is the basic feature map obtained in step 1, and the output is pre-trained features with semantic knowledge.

[0061] For this ConvMLP module, a segmentation task is used for pre-training, thereby extracting and learning corresponding pathological tissue semantic knowledge from downstream tasks. The training process uses the D1 dataset, which has been manually annotated with semantic segmentation labels by experts. For the segmentation labels, the same slicing method as for panoramic slice images is used to slice the label mask into blocks. Then, the category of the largest area in each mask block is counted, and the category label of that image block is set to that category. The training input data is the basic pathological features obtained in step 1, and the training labels are the image block categories obtained in the above process. The training network adds a classification head to the convolutional multilayer perceptron to output the predicted category of the corresponding image block. Finally, gradient descent is used to train the ConvMLP module, and the parameters of the convolutional multilayer perceptron part of this classification network are extracted as parameters for the pre-trained feature encoding module. The training parameters use a learning rate of 0.0001, and the existing technique AdamW is selected as the optimizer.

[0062] After pre-training, step 2 is used for forward propagation of the panoramic slice image. The forward propagation process uses the base features obtained in step 1, adding Gaussian noise with a mean of 0 and a variance of 0.0001 to the background region features. This process eliminates the need for feature extraction, which improves the speed of feature computation. Simultaneously, using low-level noise as features enhances the computational stability of the regularization layer, preventing missing and outlier values ​​from arising during variance calculation due to large areas of identical values.

[0063] Step 3: Compress the dimensions of the pre-trained features using convolutional layers to obtain compressed features.

[0064] In step 3, a convolutional layer is used to compress the dimensionality of the pre-trained features. The hyperparameters of this convolutional layer can be set according to the compression ratio. In one example, if a compression ratio of 4 is used, the feature length before compression is... Then there are a total of The features of an image patch, when the compression rate is set to 4, result in a compressed feature length of . Its characteristic matrix size is .like Figure 4As shown, in order to achieve this compression process through a convolutional network, the kernel size K is set to 2, and the kernel stride S is also set to... This process compresses the data, resulting in compressed features after processing by the convolutional layer.

[0065] Step 4: Combine local attention mechanism and global attention mechanism to collaboratively update compressed features, so as to fuse local details and global context information to obtain high-level features.

[0066] Combination Figure 5 As shown, step 4 further includes:

[0067] Set the width and height dimensions as The compressed feature data is segmented into windows and uniformly cropped into... Square area of ​​size , will the Each region is denoted as and obtain the foreground area ;in, Indicates the height of the panoramic slice image. Indicates the width of the panoramic slice image. Indicates the size of the image patch. Indicates the kernel size;

[0068] For those in Input features of the region First, calculate the local attention. The calculation formula is:

[0069] ,

[0070] in, This represents a multi-head attention calculation method. To be simultaneously in Region and The input set of the region;

[0071] pass After the local attention calculation, local features are obtained. ;

[0072] For local features Recalculate global attention The calculation formula is:

[0073] ,

[0074] in, For being in The input set of the region; This refers to the high-level features obtained.

[0075] In one example, compressed features are updated collaboratively using a combination of local and global attention mechanisms, where multiple networks with local and global attention are stacked for computation. The network layers in each network module consist of, in sequence, a regularization layer, an attention layer, another regularization layer, and a linear layer with an activation function.

[0076] The difference between network modules with local attention and those with global attention lies in the implementation of the attention layer. Specifically, for local attention, the size is first... The compression features are used to divide the window into blocks, using a window size of [missing information]. Then, after window segmentation, we get There are 10 windows, each window has 10 windows. For each feature, a local attention layer computes self-attention separately for each window to capture local features. This achieves lower computational cost and a certain receptive field through a smaller self-attention computation range. For global attention, instead of dividing the compressed features into windows, self-attention is computed directly for all features. Global attention can capture the correlation between all features simultaneously, resulting in a larger receptive field but also higher computational cost. Considering computational efficiency and performance, the stacking method for multiple local and global attention modules is as follows: use 8 local attention modules followed by 1 global attention module to achieve a balance between computational cost and receptive field.

[0077] In addition, this example proposes a foreground / background mask self-attention module as the self-attention calculation module. This module first distinguishes the foreground and background regions defined in steps 1 to 3. Specifically, after compressing the basic feature map from step 1 into compressed features in step 3, it distinguishes whether the compressed features contain background region features. If the compressed feature is entirely composed of background regions, it is marked as a mask. In the self-attention calculation, features marked as masks are skipped. This self-attention calculation discards invalid features that are background regions and ensures that valid features are included in the calculation. The compressed features are processed through self-attention calculations of multiple modules to obtain high-level features.

[0078] Step 5: Input the high-level features into the feature fusion module, fuse them with the text features, and align them to the same semantic space to generate the final visual features for the large visual language model.

[0079] Combination Figure 5 As shown, step 5 further includes:

[0080] High-level features are compressed using convolutional layers to obtain visual features;

[0081] Obtain the foreground region of the tissue segmentation, compare the coordinates of each feature in the original pathological panoramic slice to see if the corresponding region is a tissue part. If the tissue area of ​​the corresponding region is less than the threshold, discard the feature, and rearrange all the remaining features into two-dimensional features to obtain the final visual features.

[0082] In one example, the feature fusion module uses the same structure as in step 3, employing a convolutional layer to further compress the high-level features obtained in step 4 to obtain visual features. These visual features are then concatenated with the text features from the large language model to achieve fusion. The feature length before compression is... There are a total of One feature, setting the compression rate in step 5. The compressed feature length is then Its characteristic matrix size is .

[0083] Subsequently, the foreground region of the tissue segmentation calculated in step 1 is obtained. The coordinates of each feature in the original image are compared to determine if the corresponding region is a tissue portion. If the tissue area of ​​the corresponding region is less than a threshold, the feature is discarded. All remaining features are then rearranged into two-dimensional features to obtain the final visual features. Assume the number of remaining features is... , dimension So, visual features have ,in It is the field of real numbers.

[0084] like Figure 1 As shown, the final visual features, after alignment training in the pathology visual module, can be accepted by the large language model. For the input question, the vectorization module encodes the text into token inputs that the model can accept, concatenates them with the visual features, and then decodes them through the large model Qwen2.5 to obtain the output. The output is encoded and needs to be decoded again through the vectorization module to obtain the Chinese answer.

[0085] In one example, the effectiveness of the method described in this invention for feature extraction was verified through a classification task on the publicly available colorectal datasets IMP-CRS and UniToPatho. The IMP-CRS dataset contains a training set of 4433 images and a test set of 900 images. Pathologists randomly selected 10 slides for local tissue segmentation labeling, which was used for pre-training in step 2. After training, the 900 images were used as the internal test set to test the predictive performance of features within the same feature domain. The UniToPatho dataset, containing 292 images, was used as the external test set to test the predictive performance of features with domain shifts. All experiments were conducted at 20x magnification, with labels set to normal and malignant, i.e., a benign / malignant classification task. The evaluation criteria used were accuracy, sensitivity, specificity, and F1 accuracy. Three models were used for ablation comparison experiments: Model 1 was the complete model in this invention; Model 2 was based on Model 1 with the pre-trained feature module removed in step 2, and the foreground / background mask self-attention module in step 4 replaced with a regular self-attention module; Model 3 was based on Model 1 with the feature compression module removed in step 3 and the feature fusion module removed in step 5. The experiments were conducted on a device with one A100 graphics card, using the Qwen2.5-vl-3B large language pedestal model. The ablation experiment results are shown in Table 1 below.

[0086] Table 1 Evaluation results of each model

[0087]

[0088] The experimental results show that Model 3, lacking an effective feature compression process, fails to function properly with conventional image processing methods, indicating that the feature compression steps in steps 3 and 5 play a crucial role in the practical feasibility of the model. Comparing Model 1 and Model 2, the pathological features extracted using the UNI base model in step 1 are more effective than traditionally computed pathological features, resulting in no significant difference in the internal test set IMP-CRS results, with an average difference of approximately 1-2 percentage points. Their performance within the training distribution is similar. However, the effectiveness of this feature is limited by the same feature distribution, leading to a significant performance gap on the external dataset UniToPatho. This is mainly due to the tissue segmentation pre-training process in step 2, which learns prior knowledge of benign and malignant tissues. Compared to the base model features relying solely on UNI image texture phenotypes, the pre-trained model with prior knowledge is more robust across different datasets. This robustness allows for better prediction performance in subsequent large-scale model components when encountering previously unseen domain offset data, demonstrating that step 2 of this invention plays a key role in the pathological visual features of the large model.

[0089] The method for feature extraction from panoramic pathological slide images for a large visual language model, as described in this invention, integrates a basic pathological model into a visual language model. This fully leverages the rich information in panoramic slide images and the performance of the large visual language model to provide precise support for clinical decision-making. Addressing the large size of panoramic slide images, this invention extracts features with low space consumption and high information content through a basic pathological module and feature compression, respectively. This solves the problem that traditional large visual models cannot handle the huge size of panoramic slide images, achieving efficient and feasible visual feature processing. By fusing tissue segmentation knowledge through a feature encoding module and a feature update module, the features are strengthened and updated. This invention discriminates between the foreground and background by extracting feature distributions with prior knowledge of tissue differences. Finally, through a feature fusion module, visual features that can be understood by the large language model are extracted. Combining the information processing capabilities of the large language model with the rich details of the panoramic pathological slide image, accurate predictions in the field of medical pathology are achieved.

Claims

1. A method for feature extraction from pathological panoramic slide images for large-scale visual language models, characterized in that, Includes the following steps: Step 1: Obtain the panoramic pathological slide image to be processed and perform preprocessing. Input the preprocessed panoramic pathological slide image into the pathological basic module to extract the basic feature map. Step 2: Process the basic feature map through the pre-trained feature encoding module to extract pre-trained features containing tissue segmentation semantics; Step 3: Compress the dimensions of the pre-trained features using convolutional layers to obtain compressed features; Step 4: Combine local attention mechanism and global attention mechanism to collaboratively update compressed features, so as to fuse local details and global context information to obtain high-level features; Step 5: Input the high-level features into the feature fusion module, fuse them with the text features, and align them to the same semantic space to generate the final visual features for the large visual language model. The feature encoding module is a convolutional multilayer perceptron, which contains convolutional kernels with a size of [missing information]. convolutional layers Three identical first linear layers Second linear layer Third linear layer and a SiLU activation function ; Using a convolutional multilayer perceptron to process the basic feature map of the input The calculation process is expressed as follows: ， in, These are pre-trained features; Step 4 includes: Set the width and height dimensions as The compressed feature data is segmented into windows and uniformly cropped into... Square area of ​​size , will the Each region is denoted as and obtain the foreground area ;in, Indicates the height of the panoramic slice image. Indicates the width of the panoramic slice image. Indicates the size of the image patch. Indicates the kernel size; For those in Input features of the region First, calculate the local attention. The calculation formula is: ， in, This represents a multi-head attention calculation method. To be simultaneously in Region and The input set of the region; pass After the local attention calculation, local features are obtained. ; For local features Recalculate global attention The calculation formula is: ， in, For being in The input set of the region; This refers to the high-level features obtained; Step 5 includes: High-level features are compressed using convolutional layers to obtain visual features; Obtain the foreground region of the tissue segmentation, compare the coordinates of each feature in the original pathological panoramic slice to see if the corresponding region is a tissue part. If the tissue area of ​​the corresponding region is less than the threshold, discard the feature, and rearrange all the remaining features into two-dimensional features to obtain the final visual features.

2. The method for feature extraction from pathological panoramic slide images for large-scale visual language models according to claim 1, characterized in that, The steps for acquiring and preprocessing panoramic pathological slide images include: First, the red, green, and blue color channels of the pathological panoramic slide image are converted into hue, saturation, and brightness color spaces. Then, threshold segmentation is performed based on the saturation and brightness channels. The threshold segmentation results are then dilated and eroded to remove free and tissue-bound blank areas, distinguishing between the foreground area with tissue and the blank background area. Finally, the entire pathological panoramic slide image is cropped into image blocks of the same size, and image blocks whose background area exceeds a set threshold are discarded.

3. The method for feature extraction from pathological panoramic slide images for large-scale visual language models according to claim 1, characterized in that, The pathology foundation module is a visual transformer pre-trained on a large pathology image dataset.

Images