Large model-based image analysis and diagnosis method and system for pathological images

Abstract

The present invention discloses a large-model-based image analysis and diagnosis method and system for pathological images, and relates to the field of image processing technology. Specifically disclosed is a large-model-based image analysis and diagnosis method for pathological images, including: normalizing multiple pathological images of multiple pathological image types and multiple cancer types, and extracting foregrounds of each pathological image; after normalizing the foreground of each pathological image, dividing the processed foreground of each pathological image into multiple patch image blocks; and performing cancer prediction training on multiple patch image blocks of each pathological image. The present invention can quickly locate the significant areas of pathological images, thereby achieving more efficient and accurate disease cancer prediction of pathological images.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a large model-based image analysis and diagnosis method and system for pathological images. Background Art

[0002] In the field of pathology, accurate pathological image analysis is crucial for the prevention, detection and diagnosis of diseases. Existing technical solutions mainly rely on traditional image processing technology and manual experience to analyze pathological images. These solutions usually include the following steps: 1) Preprocessing: De-noising, contrast enhancement and other operations are performed on the collected images to improve image quality. 2) Feature extraction, classification and segmentation machine learning methods: Through manually designed feature extraction algorithms such as edge detection and texture analysis, key information in the image is identified. Then traditional machine learning algorithms are used to classify and segment the image. 3) Deep learning methods: Apply a large amount of manual annotation and deep learning network training for image classification and segmentation to assist diagnosis.

[0003] Although existing machine learning and deep learning technologies can assist pathological diagnosis to a certain extent, they still have limitations such as limited performance, high R&D efficiency and low cost, low model generalization, and especially low migration ability. Specifically, the following problems are included: 1) Accuracy limitation: For machine learning, the manually designed feature extraction method has limited understanding of image content and is difficult to capture complex pathological changes. 2) Efficiency problem: For deep learning, it takes a lot of time and energy for doctors to manually annotate pathological images, and the algorithm's operating efficiency is also low during image preprocessing and feature extraction, making it difficult to cope with large-scale image data. 3) Generalization ability: Traditional model algorithms have weak generalization ability when facing images from different sources and conditions, and the consistency and reliability of diagnostic results are limited. 4) Migration ability: Existing methods cannot support multiple tasks; for each pathology-assisted diagnosis application of one cancer type, a corresponding AI diagnostic model needs to be developed. If 10 cancer applications are developed, 10 times the amount of data annotation and model development is required, which is time-consuming and labor-intensive.

[0004] In order to overcome these limitations, it is currently necessary to propose a large-model-based image analysis and diagnosis method for pathological images, which can quickly locate the salient areas of pathological images, thereby achieving more efficient and accurate disease and cancer prediction in pathological images. Summary of the invention

[0005] The technical problem to be solved by the present invention is how to quickly locate the significant areas of pathological images, so as to achieve more efficient and accurate disease and cancer prediction in pathological images. The purpose is to provide an image analysis and diagnosis method and system based on a large model for pathological images, which solves the above-mentioned problem.

[0006] The present invention is achieved through the following technical solutions:

[0007] A large-model-based image analysis and diagnosis method for pathological images includes: normalizing multiple pathological images of multiple pathological image types and multiple cancer types, and extracting the foreground of each pathological image; after normalizing the foreground of each pathological image, dividing the processed foreground of each pathological image into multiple patch image blocks; and performing cancer type prediction training on the multiple patch image blocks of each pathological image.

[0008] The foreground extraction of each pathological image includes:

[0009] Each pathological image is mapped to the HSV space, and after the median filter noise reduction process, the basic histogram of the image is calculated; the threshold of the basic histogram is calculated using an adaptive method to distinguish the foreground and background of each pathological image, and the foreground image is extracted.

[0010] The above process normalizes the foreground of each pathological image, including:

[0011] The statistical distribution of the color space is calculated for the foreground image, and the color space statistical distribution of the given image is linearly mapped with the standard color space distribution to establish a color-normalized image.

[0012] The above standard color space distribution is obtained through the following steps:

[0013] High-quality images of multiple pathological image types of each cancer type are manually selected, and standard image libraries are established according to different pathological image types. After magnification normalization processing, the foreground of each pathological image is extracted, and the color space distribution of the foreground of each pathological image is collected and merged to form the total color space distribution of each of the above-mentioned standard image libraries as the standard color space distribution of the corresponding pathological image type.

[0014] The foreground of each processed pathological image is divided into multiple small blocks, including:

[0015] The sliding window segmentation method is used to segment the processed pathological image foreground into multiple patch image blocks of fixed size.

[0016] The above cancer type prediction training is performed on multiple patch image blocks of each pathological image, including:

[0017] Image vectorization is performed on multiple patch image blocks of multiple pathological image types, and multiple patch image blocks of each cancer subclass are clustered according to multiple cancer subclasses of each cancer type. A self-supervised learning framework is used for the clustered multiple patch image blocks, and masked image modeling is performed through self-distillation to obtain local image features, and an online word segmenter is introduced for feature mining. The multiple patch image blocks and features selected after vectorization are used for cancer type prediction training.

[0018] The above-mentioned clustering process of the multiple patch image blocks of each cancer subclass according to the multiple cancer subclasses of each cancer type includes:

[0019] According to the multiple cancer subclasses preset for each cancer type, multiple typical images are selected respectively, and multiple patch image blocks are randomly sampled according to the multiple cancer subclasses of each cancer type; the average vector values ​​of the multiple typical images and the multiple patch image blocks of each cancer subclass of each cancer type are calculated as the category initial value of each cancer subclass cluster of each cancer type; the vector values ​​of the patch image blocks of each cancer type that do not belong to the cancer subclass are randomly generated as the center initial value of the cluster; wherein the number of the center initial values ​​is calculated by multiplying the sum of the mean and the variance of the number of cancer subclasses by the number of cancer types; the category initial value and the center initial value are the multiple patch image blocks selected after vectorization.

[0020] Large model-based image analysis and diagnosis system for pathological images, including:

[0021] Image processing unit: used for normalizing the magnification of multiple pathological images of multiple pathological image types and multiple cancer types, and performing foreground extraction on each pathological image;

[0022] Image segmentation unit: used for performing color space normalization processing on each pathological image foreground, and then segmenting each processed pathological image foreground into multiple patch image blocks;

[0023] Image training unit: used to perform cancer type prediction training on multiple patch image blocks of each pathological image.

[0024] An electronic device comprises a memory, a processor and a computer program running on the processor. When the processor executes the computer program, the steps of the large model-based image analysis and diagnosis method for pathological images are implemented.

[0025] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the large-model-based image analysis and diagnosis method for pathological images.

[0026] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0027] This application proposes a large-model-based image analysis and diagnosis method for pathological images. After foreground extraction of multiple pathological images of multiple pathological image types and multiple cancer types, the images are divided into multiple patch image blocks; and cancer type prediction training is performed on the multiple patch image blocks of each pathological image. Through the above technical solution, the significant parts of the pathological image can be quickly located, thereby effectively achieving more accurate pathological cancer type identification and classification. This solution has broad application prospects and can be used for automated processing and auxiliary diagnosis of pathological images, improving the accuracy and efficiency of pathological diagnosis. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] In order to more clearly illustrate the technical solutions of the exemplary embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments. It should be understood that the following drawings only illustrate certain embodiments of the present invention and should not be regarded as limiting the scope. For those of ordinary skill in the art, other relevant drawings can be obtained based on these drawings without creative work. In the drawings:

[0029] Figure 1 This is a schematic diagram of a large model-based image analysis and diagnosis method for pathological images according to an embodiment of the present application. DETAILED DESCRIPTION

[0030] In order to make the objectives, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with embodiments and drawings. The exemplary embodiments of the present invention and their description are only used to explain the present invention and are not intended to limit the present invention.

[0031] Example

[0032] like Figure 1 As shown, an embodiment of the present application provides a large-model-based image analysis and diagnosis method for pathological images, including: normalizing multiple pathological images of multiple pathological image types and multiple cancer types, and extracting the foreground of each pathological image; after normalizing the foreground of each pathological image, dividing the processed foreground of each pathological image into multiple patch image blocks; and performing cancer type prediction training on the multiple patch image blocks of each pathological image.

[0033] The foreground extraction of each pathological image includes:

[0034] Each pathological image is mapped to the HSV space, and after the median filter noise reduction process, the basic histogram of the image is calculated; the threshold of the basic histogram is calculated using an adaptive method to distinguish the foreground and background of each pathological image, and the foreground image is extracted.

[0035] The above process normalizes the foreground of each pathological image, including:

[0036] The statistical distribution of the color space is calculated for the foreground image, and the color space statistical distribution of the given image is linearly mapped with the standard color space distribution to establish a color-normalized image.

[0037] The above standard color space distribution is obtained through the following steps:

[0038] High-quality images of multiple pathological image types of each cancer type are manually selected, and standard image libraries are established according to different pathological image types. After magnification normalization processing, the foreground of each pathological image is extracted, and the color space distribution of the foreground of each pathological image is collected and merged to form the total color space distribution of each of the above-mentioned standard image libraries as the standard color space distribution of the corresponding pathological image type.

[0039] Pathological images usually have extremely high resolutions, such as 50000 x 50000. Different types of pathological images, such as whole slice pathological images (WSI) and tissue microarray pathological images (TMA), have different image sizes and shapes. Conventional AI algorithms are difficult to directly process such large images. In order to better process high-resolution pathological images, the images are divided into multiple small blocks (patches). In addition, the patch image blocks are used to construct a data set for training, and multiple patch image blocks are merged for prediction during inference. In addition, due to the different tissue distributions of the foreground and background of pathological images of different types of cancer, the foreground and background areas in the pathological images are unevenly distributed. Some patch image blocks may contain a large amount of background information, which has a negative interference effect on subsequent feature extraction and classification tasks. Therefore, in order to improve the training efficiency and accuracy of the model, the image is first normalized by magnification, and the foreground of the pathological image is extracted by calculating the foreground through the adaptive threshold method. Then, each pathological image is automatically normalized to the color or brightness space, and high-information value patch image blocks are automatically constructed for different types of pathological images for model training.

[0040] The specific steps of magnification normalization are as follows: Normalize the pathology image to 20 times, i.e. 20X. For the whole-slice pathology image (WSI image), select level 0. If the WSI magnification is 40X, it can be converted to 20X. If there is no magnification information, no scaling is performed. The tissue microarray pathology image (TMA image) and other types of pathology images are subjected to the same normalization operation.

[0041] The specific steps of foreground extraction of pathological images are as follows: first map the color image to HSV space for whole slice pathological images (WSI) or tissue microarray pathological images (TMA), use median filtering to reduce the noise of the image, and then calculate the basic histogram of the image. Adaptive methods such as OTSU and triangulation can be applied to the histogram to calculate the threshold T_obj of the pathological image, distinguish the foreground and background of the image, and use this threshold T_obj to calculate the foreground of the image to ensure that all calculations and analyses are performed on the effective foreground area.

[0042] The specific steps of color normalization of pathological images are as follows: Calculate the statistical distribution H_img of the RGB color space for the foreground area (the area with grayscale value higher than T_obj), linearly map the color space statistical distribution of the given image with the standard distribution H_STD, and establish a color normalized image. You can also select other color spaces or adopt nonlinear mapping and other change methods.

[0043] In order to establish the standard color space distribution H_STD, a batch of high-quality images of different pathological image types and multiple cancer types were manually selected to establish a standard image library, and the above-mentioned magnification normalization and pathological image foreground extraction steps were run. The color space distribution H_img of each image was collected and merged to form the total color space distribution H_STD corresponding to the standard image library, which was used as the standard color space distribution. Then, for each WSI, TMA pathological image was color normalized according to this step.

[0044] The foreground of each processed pathological image is divided into multiple small blocks, including:

[0045] The sliding window segmentation method is used to segment the processed pathological image foreground into multiple patch image blocks of fixed size.

[0046] The specific steps of sliding window segmentation patch image blocks are as follows:

[0047] a) Set the size of the sliding window (e.g. 256x256 pixels) and the sliding step size (e.g. 256 pixels).

[0048] b) The pathological image is divided according to the set sliding window size and step size to generate multiple patch image blocks.

[0049] c) A foreground area ratio threshold T_patch is set for the patch image block, such as 0.7. All patch image blocks are cut on the pathological image after foreground segmentation and color normalization. Patches with foreground ratios greater than the threshold T_patch are retained, and other patches are discarded to form the final training patch data set with high information value.

[0050] The above cancer type prediction training is performed on multiple patch image blocks of each pathological image, including:

[0051] Image vectorization is performed on multiple patch image blocks of multiple pathological image types, and multiple patch image blocks of each cancer subclass are clustered according to multiple cancer subclasses of each cancer type. A self-supervised learning framework is used for the clustered multiple patch image blocks, and masked image modeling is performed through self-distillation to obtain local image features, and an online word segmenter is introduced for feature mining. The multiple patch image blocks and features selected after vectorization are used for cancer type prediction training.

[0052] Self-supervised learning (SSL) is a learning method that does not require manually labeled data. By designing pre-training tasks, the model is trained on unlabeled data to learn the feature representation of the image, greatly reducing the cost of manual annotation and improving efficiency. Common SSL methods include contrastive learning, generative adversarial networks, etc. In an embodiment of the present application, based on the method framework of contrastive learning, the information of the pathological image is automatically mined to achieve self-supervised training of massive pathological images. This scheme adopts the iBOT (Image BERT Pre-Training with Online Tokenizer) self-supervised learning framework, and performs masked image modeling (MIM) by self-distillation. Some areas in the image are randomly masked, and the content of these areas is predicted by the model to automatically mine the high-order semantic information of the image. By introducing an online word segmenter, iBOT can perform BERT-style pre-training on images and learn richer image features.

[0053] The above-mentioned clustering process of the multiple patch image blocks of each cancer subclass according to the multiple cancer subclasses of each cancer type includes:

[0054] According to the multiple cancer subclasses preset for each cancer type, multiple typical images are selected respectively, and multiple patch image blocks are randomly sampled according to the multiple cancer subclasses of each cancer type; the average vector values ​​of the multiple typical images and the multiple patch image blocks of each cancer subclass of each cancer type are calculated as the category initial value of each cancer subclass cluster of each cancer type; the vector values ​​of the patch image blocks of each cancer type that do not belong to the cancer subclass are randomly generated as the center initial value of the cluster; wherein the number of the center initial values ​​is calculated by multiplying the sum of the mean and the variance of the number of cancer subclasses by the number of cancer types; the category initial value and the center initial value are the multiple patch image blocks selected after vectorization.

[0055] The core advantage of this model is that it can process multiple cancer pathology images and obtain a comprehensive and high-quality training data set, which ensures model training, multi-task application and generalization. This solution further optimizes the patch data set. In the training data preparation stage, clustering retrieval methods are used to establish a typical and comprehensive data set, ensuring the diversity of the data set and removing redundant data.

[0056] 1) In the data preprocessing stage, through methods such as adaptive threshold adjustment, foreground segmentation, color normalization, and high information value patch collection, the noise influence such as background patches is eliminated to construct a multi-category and high-quality patch dataset.

[0057] 2) In the self-supervised training stage, make full use of the pathological data of multiple cancer types, perform image vectorization and cluster retrieval on the patch dataset generated above to form sub-datasets containing different pathological features, further optimize and screen the dataset, ensure the diversity and comprehensiveness of the dataset, and improve the model training performance.

[0058] The number of clustering categories K is selected as the number of cancer types multiplied by the mean and variance of the number of multiple cancer sub-types, as follows:

[0059] K=N*(MEAN+SD);

[0060] N-number of cancer types, MEAN-mean number of multiple cancer subtypes, SD-variance of multiple cancer subtypes.

[0061] The initial values ​​of clustering categories are selected as follows: S1 typical images of each subclass of each cancer type are randomly selected (including cancer subclass identifiers), S2 patches are randomly sampled from the basic patch data set constructed for each typical image, and the visual model is used to calculate the average vector value of S1*S2 patches as the initial value of each category of clustering. Category centers are also randomly generated in the cancer type patch image blocks without subclasses, and finally K cluster initial values ​​are obtained.

[0062] In the clustered patch dataset, nearly equal number of patch image blocks are selected for each category, and highly similar patch image blocks are removed to establish a typical and comprehensive 80 million patch dataset.

[0063] 3) The above high-quality patch dataset is used for self-supervised training. The prediction results of the patch image blocks are obtained by randomly masking the area, analyzing the local features of the masked area and the overall category to achieve high-level semantic information mining for multiple cancer types.

[0064] Optionally, when using multiple patch image blocks selected after vectorization for cancer prediction training, weakly supervised learning, such as the multiple instance learning (MIL) method, can be used to effectively use data and labels for model training and prediction when label information is incomplete, inaccurate or insufficient. Multiple instance learning is suitable for processing samples containing multiple instances, such as multiple patch image blocks in pathological images. In the MIL framework, each sample consists of multiple instances, and the model composes the features of the overall sample by learning the features of each instance.

[0065] Since MIL training refers to limited label information from massive data and automatically learns to extract advanced features from complex multi-instance data to make more accurate predictions or classifications, effective preparation of the data set is a core element, which needs to cover positive and negative sample information. The specific steps are as follows:

[0066] 1) Bags and Instances: Each segmented patch image block is regarded as an instance. The patch set from the same pathological image is a bag. Each bag corresponds to an overall label, that is, the pathological image type or cancer subclass category, for training. For each pathological image, 30 patch instances are selected to form a bag.

[0067] 2) Bag dataset construction: Since the resolution of pathological images is very high, the images contain cancerous areas and non-cancerous areas. The sizes of cancerous areas on each image vary, and the characteristics of cancerous areas of different cancer types are different. Therefore, in the process of Bags construction, it is necessary to ensure that the Bags contain diverse and representative patch instances, especially positive instances, to fully characterize the pathological characteristics. This solution adopts grid random sampling and multi-weight sampling strategies to construct different bags for the same pathological image to improve the richness of the dataset.

[0068] Each pathological image can be divided into 5*6=30 uniform regions. In each region, 1 patch is randomly selected based on the previous patch data set, totaling 30 patches, to form a bag. Each image is randomly sampled multiple times to form multiple bags to ensure that high-quality patch instances of cancer areas are collected in the bag. In order to improve the accuracy and prior knowledge orientation of self-learning, a small number of pathological images are manually outlined for cancer areas. In this batch of pathological images, the grid random sampling and multi-weight sampling strategies are optimized to ensure that each bag randomly collects 1 instance, each bag randomly collects 2 instances, and so on until each bag randomly collects 5 instances. After the sampling strategy, a diverse bag data set is established.

[0069] 3) Data enhancement: To further improve the generalization ability of the model, data enhancement operations are performed on the instances in the training set, such as rotation, flipping, scaling, color jittering, etc., to increase the robustness of the model to image changes.

[0070] 4) Use the above data to train the MIL model to achieve more accurate prediction and classification.

[0071] Through the above technical solutions, fast and effective feature extraction of pathological images can be achieved for multiple cancer types and multiple downstream tasks, such as pathological cancer classification, whether pathological images have become cancerous, cancer subtype classification, cancer area segmentation, etc. This solution uses self-supervised learning and multi-instance learning methods to improve processing efficiency and accuracy. The introduction of the iBOT model makes feature extraction more efficient and can capture fine-grained features in the image. The use of the MIL model can effectively process and utilize the most influential patch samples or features in the pathological image to improve the accuracy of the task. The application of self-supervised learning methods can make full use of large-scale unlabeled data and improve the generalization ability of the model. This solution has broad application prospects and can be used for automated processing and auxiliary diagnosis of pathological images to improve the accuracy and efficiency of pathological diagnosis.

[0072] In summary, the embodiments of the present application provide a large-model-based image analysis and diagnosis method and system for pathological images, which extracts the foreground of multiple pathological images of multiple pathological image types and multiple cancer types, and then divides them into multiple patch image blocks; and performs cancer type prediction training on the multiple patch image blocks of each pathological image. Through the above technical solution, the significant parts of the pathological image can be quickly located, and through feature extraction, more accurate pathological cancer type identification and classification can be effectively achieved. This solution has broad application prospects and can be used for automated processing and auxiliary diagnosis of pathological images, improving the accuracy and efficiency of pathological diagnosis.

[0073] The present invention realizes the auxiliary diagnosis of multi-cancer pathological image analysis and multi-dimensional downstream tasks. It has significant technical effects in cancer classification, cancer area detection, liver cancer area segmentation, subtype classification and cell nucleus segmentation. For cancer classification, the traditional method relies on the experience and manual annotation of pathologists, which is time-consuming and error-prone. The present invention uses a deep learning model to classify pathological images into cancer types. It has been verified that there are 18 types of cancer classifications with an accuracy rate of 0.972. It can effectively distinguish different types of cancer, assist pathologists in diagnosis, and improve the accuracy and efficiency of diagnosis. At the same time, the classification results can also be used for cancer research and analysis to help researchers understand the characteristics and distribution of different cancers. In terms of cancer area detection, traditional methods are difficult to quickly screen a large number of pathological images, and the workload is heavy. The present invention uses a deep learning model to detect whether there is a cancer area in the image, with an accuracy rate of 0.968. It can quickly detect whether there is a cancer area in the image, reduce the workload of pathologists, improve work efficiency, and enable them to focus on more complex diagnostic tasks. For the segmentation of liver cancer cancer areas, the traditional method relies on manual annotation, which is time-consuming and difficult to accurately segment the cancer area. The present invention uses a semantic segmentation model to accurately segment the cancer area in the liver cancer image. The segmentation results can be used for subsequent detailed analysis and quantitative research, such as calculating the area and shape characteristics of the cancer area, etc., to improve the accuracy of the research. In terms of subtype classification, such as the distinction between the five subtypes of ovarian cancer, including HGSC, LGSC, CC, EC, and MC, it can accurately distinguish different ovarian cancer subtypes, with an average accuracy rate of 0.89, which can assist pathologists in diagnosis and improve the accuracy of diagnosis. At the same time, the classification results can also be used for the research and analysis of ovarian cancer.

[0074] The specific implementation methods described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific implementation method of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A large model-based image analysis and diagnosis method for pathological images, characterized in that: include: Normalizing multiple pathological images of multiple pathological image types and multiple cancer types, and performing foreground extraction on each pathological image; Each pathological image foreground is normalized, including: The statistical distribution of the color space is calculated for the foreground image, and the statistical distribution of the color space of the given image is linearly mapped with the standard color space distribution to establish a color normalized image; the standard color space distribution is obtained by the following steps: Manually select high-quality images of multiple pathological image types of each cancer type, establish standard image libraries according to different pathological image types, and perform magnification normalization processing, extract the foreground of each pathological image, collect the color space distribution of the foreground of each pathological image, merge to form the total color space distribution of each standard image library, as the standard color space distribution of the corresponding pathological image type; divide the processed foreground of each pathological image into multiple patch image blocks; perform cancer type prediction training on the multiple patch image blocks of each pathological image, including: Performing image vectorization on multiple patch image blocks of multiple pathological image types; clustering and processing multiple patch image blocks of each cancer subclass according to multiple cancer subclasses of each cancer type, including: Select multiple typical images according to multiple cancer subclasses preset for each cancer type, and randomly sample multiple patch image blocks according to multiple cancer subclasses of each cancer type; calculate the average vector values ​​of multiple typical images and multiple patch image blocks of each cancer subclass of each cancer type as the category initial value of each cancer subclass cluster of each cancer type; randomly generate the center initial value of the cluster from the vector values ​​of the patch image blocks that do not belong to the cancer subclass of each cancer type; wherein the number of the center initial values ​​is calculated by multiplying the sum of the mean and variance of the number of cancer subclasses by the number of cancer types; the category initial value and the center initial value are multiple patch image blocks selected after vectorization; The clustered multiple patch image blocks use a self-supervised learning framework, and masked image modeling is performed through self-distillation to obtain local image features, and an online word segmenter is introduced for feature mining; the multiple patch image blocks and features selected after vectorization are used for cancer prediction training.

2. The large model-based image analysis and diagnosis method for pathological images according to claim 1, characterized in that: The foreground extraction of each pathological image includes: Each pathological image is mapped to the HSV space, and after the median filter noise reduction process, the basic histogram of the image is calculated; the threshold of the basic histogram is calculated using an adaptive method to distinguish the foreground and background of each pathological image, and the foreground image is extracted.

3. The large model-based image analysis and diagnosis method for pathological images according to claim 1, characterized in that: The foreground of each processed pathological image is divided into a plurality of small blocks, including: The sliding window segmentation method is used to segment the processed pathological image foreground into multiple patch image blocks of fixed size.

4. A large model-based image analysis and diagnosis system for pathological images, characterized in that: include: Image processing unit: used for normalizing the magnification of multiple pathological images of multiple pathological image types and multiple cancer types, and performing foreground extraction on each pathological image; Image segmentation unit: used to normalize the color space of each pathological image foreground, including: The statistical distribution of the color space is calculated for the foreground image, and the statistical distribution of the color space of the given image is linearly mapped with the standard color space distribution to establish a color normalized image; the standard color space distribution is obtained by the following steps: Manually select high-quality images of multiple pathological image types of each cancer type, establish standard image libraries according to different pathological image types, and perform magnification normalization processing. After foreground extraction of each pathological image, collect the color space distribution of the foreground of each pathological image, merge to form the total color space distribution of each standard image library, and serve as the standard color space distribution of the corresponding pathological image type; divide the processed foreground of each pathological image into multiple patch image blocks; Image training unit: used to perform cancer type prediction training on multiple patch image blocks of each pathological image, including: Performing image vectorization on multiple patch image blocks of multiple pathological image types; clustering and processing multiple patch image blocks of each cancer subclass according to multiple cancer subclasses of each cancer type, including: Select multiple typical images according to multiple cancer subclasses preset for each cancer type, and randomly sample multiple patch image blocks according to multiple cancer subclasses of each cancer type; calculate the average vector values ​​of multiple typical images and multiple patch image blocks of each cancer subclass of each cancer type as the category initial value of each cancer subclass cluster of each cancer type; randomly generate the center initial value of the cluster from the vector values ​​of the patch image blocks that do not belong to the cancer subclass of each cancer type; wherein the number of the center initial values ​​is calculated by multiplying the sum of the mean and variance of the number of cancer subclasses by the number of cancer types; the category initial value and the center initial value are multiple patch image blocks selected after vectorization; The clustered multiple patch image blocks use a self-supervised learning framework, and masked image modeling is performed through self-distillation to obtain local image features, and an online word segmenter is introduced for feature mining; the multiple patch image blocks and features selected after vectorization are used for cancer prediction training.

5. An electronic device comprising a memory, a processor, and a computer program running on the processor, characterized in that: When the processor executes the computer program, the steps of the large model-based image analysis and diagnosis method for pathological images as described in any one of claims 1 to 3 are implemented.

6. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the large model-based image analysis and diagnosis method for pathological images as claimed in any one of claims 1 to 3 are implemented.

Images